LOCOFY Large Design Models -- Design to code conversion solution

Ashwati Vipin; Honey Mittal; Karan Shetti; Sohaib Muhammad

REVIEW 5 major objections 5 minor 7 references

LOCOFY Large Design Models -- Design to code conversion solution

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large Design Models, trained on designs rather than text, can convert UI mockups into accurate, reusable front-end code.

desk verdict A product announcement with one solid subcomponent evaluation, but the headline fidelity claim rests on an undefined metric and invisible baselines. read the letter →

arxiv 2507.16208 v1 pith:RSDWKYDW submitted 2025-07-22 cs.SE cs.AI

Sohaib Muhammad , Ashwati Vipin , Karan Shetti , Honey Mittal This is my paper

classification cs.SEcs.AI

keywords design-to-codelargedesignmodelsUItaggingcomponentextractionpreviewmatchscorefront-endcodegenerationmultimodaltrainingdeterministicinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Large Design Models (LDMs), a family of models trained specifically on design files and web pages rather than on text, and argues that this design-native training makes them better suited than general LLMs for converting UI designs into production-ready code. The system's training pipeline has three components: a Design Optimizer that restructures messy design files, a Tagging and feature-detection model that recognizes UI elements, and an Auto Components module that extracts repeated structures into reusable components. At inference, the pipeline does not generate code directly; it emits precise, interpretable instructions that a proprietary code engine turns into deterministic, reproducible output. On a test set of 1,000 real-world designs, 89.6% of screens scored above 95% on the authors' Preview Match Score, and comparative experiments report that LDMs outperform LLM baselines on node positioning, responsiveness, and reproducibility. If these results generalize, the practical consequence is a reliable, component-based alternative to prompt-dependent LLM code generation.

What carries the argument

The load-bearing mechanism is the deterministic inference pipeline, which separates 'understanding the design' from 'generating code.' Rather than asking a generative model to synthesize HTML/CSS directly, the LDM converts design metadata, layer structure, and visual context into a set of explicit, interpretable layout instructions that a deterministic code-generation engine executes identically every time. Supporting that pipeline are three trained components: the Design Optimizer, a supervised model that groups layers and restores best-practice structure such as Auto Layout; the Tagging and feature-detection model, an object-detection backbone pre-trained on UI-specific data under the 'Jasmine' strategy and fine-tuned to classify and group UI elements; and the Auto Components module, which finds repeated UI structures across screens and abstracts them into reusable components with automatically inferred props. The authors' evaluation instrument is the Preview Match Score, defined narratively as a node-level comparison of rendered output to the source design on width, height, and absolute $(x, y)$ coordinates, with 95% as the near-perfect threshold.

What would settle it

Run the same node-level comparison the paper describes—width, height, and absolute (x, y) coordinates of rendered nodes against source designs—on an independently assembled public set of 1,000 designs, with the LLM baseline given a fixed optimized prompt; if that baseline reaches a ≥95% match on more than a small handful of screens, the claim that LDMs are superior in node positioning and visual fidelity is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the design-to-code problem is better solved by models that ingest design structure natively than by text-trained or general multimodal LLMs. The paper calls this paradigm the Large Design Model (LDM) and describes it as a multimodal system trained on over one million web and design nodes. Its training pipeline combines a Design Optimizer (supervised on expert-annotated transformations via a gradient-boosted tree model), a UI tagging and feature-detection model built on an object-detection backbone pre-trained with the custom 'Jasmine' strategy, and an Auto Components module that detects repeated UI groups and abstracts them into reusable components with inferred props. The inference pipeline is deliberately deterministic: it converts a design into structured, interpretable instructions that a code-generation engine renders, so the same design always yields the same code. The authors' evaluation reports that 89.6% of 1,000 community-collected real-world screens exceeded a 95% Preview Match Score—a node-level comparison of width, height, and x,y coordinates—and that their tagging model reached macro-average F1 scores of 86.07% for small tags and 77.22% for large tags, while LLM baselines scored lower on positioning, responsiveness, and reproducibility.

Load-bearing premise

The load-bearing premise is that the proprietary Preview Match Score, applied to the authors' own 1,000-design community test set without independent validation, genuinely measures design-to-code quality; if the metric or the test designs are biased, the reported superiority of LDMs over LLMs collapses.

Editorial extensions

If this is right

Design-to-code conversion can be made deterministic: the same design file will produce identical code, which is what version control and professional workflows require.
Repeated UI patterns can be abstracted into single component definitions, so generated front-ends carry less redundancy and are easier to maintain.
Well-structured designs can be converted at production fidelity without a generative model re-rolling the layout each time, removing the prompt-sensitivity that affects LLM approaches.
The reported tagging F1 scores imply that interactive elements such as buttons, inputs, and headers can be recognized and grouped automatically, a prerequisite for generating semantic rather than purely visual code.
Because the inference step produces interpretable instructions, an engineer can audit and edit the intermediate representation before code is generated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: replace the self-defined Preview Match Score with independent human perceptual ratings or a standard pixel-level similarity measure on a public benchmark; the reported 89.6% figure is only as strong as that metric's validity.
If the deterministic instruction-based approach is the source of the reproducibility advantage, one could separate the 'understanding' and 'rendering' stages and reuse the instruction format to drive multiple target frameworks, an architectural consequence the paper only hints at.
The authors acknowledge that free-form and AI-generated designs perform poorly; this suggests the approach is most effective when design files already follow structured conventions, and a plausible next step is a normalization module that pre-processes arbitrary designs before optimization.
Given the paper's caveat that its model parameters are 'hundreds of millions' and may not qualify as truly large, one could test whether the reported preview match scores improve with dataset and parameter scale.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a product announcement, not a research paper. The Locofy pipeline is real and clearly does something useful, but the headline fidelity claim is untestable as written. The one genuinely informative section is tagging: Jasmine, their UI-pretrained YOLO variant, reaches macro F1 86.07 on small tags and 77.22 on large tags across 5,000 designs, versus 76.09/51.39 for baseline YOLO. Those numbers are concrete and suggest the UI-specific pretraining helps. The limitations section is also honest: free-form and AI-generated designs "do not work well," which is a real constraint and weakens the broad "superior solution" claim in the abstract.

The soft spot is load-bearing. Preview Match Score is never defined. We are told it compares width, height, and absolute x,y at node level, but not how nodes are matched, what tolerance applies, how coordinates are normalized, or how per-node differences become a per-screen score. Without that, the 89.6%-above-95% result cannot be checked or compared. The LLM comparisons are worse: the text says LLMs "consistently underperformed" with "significantly lower preview match scores," but no table or figure reports any LLM score. The qualitative figures are screenshots, not measurements. The test set of 1,000 community-collected designs has no inclusion or exclusion criteria, so the stated limitation on free-form designs raises a selection bias concern. Every end-to-end superiority claim rests on this undefined metric and invisible baselines.

The design optimizer and auto-components sections are descriptive, not experimental. The next-token prediction figure is unlabeled. So the paper is a strong product note but a weak scientific report.

If I were the editor, I would not desk reject outright: the tagging results are real, and the pipeline description is coherent enough that a serious referee could push the authors to define the metric, release baseline numbers, and describe the test set. But without those additions, the paper cannot be judged. Recommendation: send to peer review with a strong request for revision, or desk reject if the authors are not willing to supply the missing evidence. I would not cite the end-to-end numbers, but I would consider citing the tagging approach if it were fully specified.

Referee Report

5 major / 5 minor

Summary. The manuscript introduces 'Large Design Models' (LDMs), a set of proprietary, design-specific models and a deterministic inference pipeline for converting UI design files into front-end code. The proposed system comprises a Design Optimizer (XGBoost-based), a UI element tagging and feature detection model (YOLO backbone pretrained on UI-specific data, called Jasmine), an Auto Components module for reusable component extraction, and a deterministic code generation engine. The paper claims that on 1,000 community-collected real-world designs, 89.6% of screens achieved a Preview Match Score exceeding 95%, that LDMs outperform general-purpose and fine-tuned LLMs on node positioning, responsiveness, and reproducibility, and that the tagging model attains macro-average F1 scores up to 86.07% for small tags and 77.22% for large tags. The central contribution is an empirical superiority claim for the proprietary LDM system over LLM-based alternatives, supported by the 'novel' Preview Match Score metric.

Significance. If the central claims were properly supported, the work would be practically significant: a deterministic design-to-code pipeline with high visual fidelity and reusable component extraction could offer meaningful engineering value for front-end production workflows. The authors also deserve credit for building a large UI-specific pretraining corpus (over one million web and design nodes), for articulating a concrete modular architecture, and for making an explicit design choice to separate model outputs from a deterministic code generation engine, which is a sensible way to address reproducibility. However, the paper as submitted does not establish scientifically testable evidence for its headline claims. The Preview Match Score is not formally defined, no LLM baseline scores are reported numerically, the test sets and their construction are not described, and the limitations section itself concedes that free-form and AI-generated designs 'do not work well,' which raises serious concerns about selection bias. Consequently, the significance is currently more industrial than scientific, and the manuscript would need substantial new material to support its empirical assertions.

major comments (5)

[End-to-end design to code conversion accuracy] The Preview Match Score, the load-bearing metric of the paper, is never formally defined. The text states only that comparisons were made 'at a node-level granularity, assessing discrepancies in width, height, and absolute (x, y) coordinates,' but it does not specify how nodes are matched between the source design and rendered output, what tolerance is used for coordinate or dimension differences, how coordinates are normalized across viewports or devices, or how per-node discrepancies are aggregated into a per-screen score. Without this algorithmic definition, the headline result that 89.6% of 1,000 screens achieved a score above 95% cannot be checked, reproduced, or meaningfully compared across systems. The metric is also not validated against human judgment or an external benchmark. Please provide the full scoring algorithm and a validation study, or remove or reframe the quantitative claims.
[End-to-end design to code conversion accuracy] The comparative claims against LLMs are unsupported by any reported numbers. The text asserts that 'alternative LLMs consistently underperformed, exhibiting significantly lower preview match scores,' but no table, figure, or numerical baseline for any LLM is included anywhere in the manuscript. To make the comparison testable, the authors must report per-model preview match scores, sample sizes, and measures of variance, along with the exact evaluation protocol used for each baseline. At present, the superiority claim rests entirely on an unreported comparison.
[End-to-end design to code conversion accuracy / Limitations and Future Work] The test set of 1,000 designs is described only as 'real-world designs collected from the community,' with no inclusion or exclusion criteria, no distribution over design tools, screen types, or complexity levels, and no description of how ground truth was obtained. This omission is especially serious because the Limitations section concedes that 'free-form designs and designs generated by AI do not work well for code generation.' If the test suite was implicitly filtered to well-structured designs, the reported 89.6% figure would not generalize to the broad claim made in the abstract. Please provide a detailed description of the test-set construction, a breakdown of its properties, and, ideally, evaluation on a public benchmark such as the Design2Code benchmark cited as reference [7].
[Design Optimizer] The comparative evaluation of the Design Optimizer is only qualitative. The text reports that LLaMA, T5, Gemini 1.5 Pro, and ChatGPT-4o 'struggled to produce coherent outputs' as nesting increased from 2 to 3 layers, and that the LDM 'demonstrated strong performance,' but no quantitative metrics for node positioning accuracy, structural validity, or hierarchy reconstruction are given. There is also no description of the benchmark's composition or size. Please provide quantitative scores, error bars, and the full set of test inputs and outputs, or limit the claims to what the presented figures actually support.
[Tagging and feature detection performance] The tagging results in Table 2 report macro-average F1 scores but omit essential context: the number of tag categories in the 'small' and 'large' sets, per-class precision/recall/F1, the distribution of tags in the test set, and any measure of variance across runs or model checkpoints. The statement that the chosen model was tested 'across a wide range of 5,000 test designs' is not accompanied by a description of how those designs were selected or labeled. Without this information, the 86.07% and 77.22% F1 scores cannot be assessed for significance or generalizability.

minor comments (5)

[Abstract vs Conclusions] The abstract states that 89.6% of screens exceeded a 95% Preview Match Score, while the conclusion says '90% of screens achieving a >95% match score.' Please make the numbers consistent.
[Introduction] There is a typo in the first paragraph: 'Comb at these gaps' should read 'combat these gaps.' In addition, the manuscript uses 'LMMs' and 'LLMs' inconsistently to refer to multimodal language models; please settle on a single abbreviation.
[Tagging and feature detection performance] The prose referring to tagging results says 'see Table 1,' but the tagging table is labeled Table 2, while Table 1 describes the LLM fine-tuning setup. Please correct the cross-reference.
[LLM model finetuning] The fine-tuning experiment is described as evaluating 'cost per node,' but Table 1 lists only epochs, adapter size, and dataset size, with no compute cost or token conversion numbers. Please report the cost and token metrics in a table or define how they were calculated.
[Overall presentation] Figure 11 (next token prediction accuracy) lacks axis labels and a description of how the accuracy was computed. Please provide a caption with definitions, units, and model names. More generally, several figures (Figures 4-8) are presented as screenshots without detailed captions explaining the transformation being illustrated; adding explicit captions would improve reproducibility and accessibility.

Circularity Check

1 steps flagged · score 4.0 of 10

No equation-level circularity in the LDM pipeline; one supporting evaluation is circular because the tagging model was selected on the same test set used to report its F1 scores, and the central end-to-end superiority claim rests on an unspecified self-defined metric.

fitted input called prediction [Tagging and feature detection performance]
"Nevertheless, the best results were achieved with the post-processed Jasmine model, which reached macro-average F1-scores of 86.07% for small tags and 77.22% for large tags (Table 2). This model was selected as the final version for our tagging and feature detection pipeline. Across a wide range of 5,000 test designs, the chosen model demonstrated high precision, strong generalization, and consistent performance."

The post-processed Jasmine variant was selected because it scored highest on the 5,000-design evaluation, and those same test-design scores are then reported as evidence of the chosen model's precision and generalization. The reported F1 is therefore the selection criterion restated: choosing the variant that maximizes the score on a set and then citing that score as an independent prediction is statistically forced. This is a concrete circular step, though it concerns the tagging submodule rather than the central end-to-end design-to-code claim.

full rationale

The paper's main derivation chain is not circular in the equation-level sense: the LDM components are trained on in-house data and then evaluated on community/curated sets, and the Preview Match Score, though unspecified, is presented as a measurement rather than as an input that defines the predicted quantity. The single concrete circularity is the tagging model selection on the 5,000-design test set, which contaminates the reported F1 as an independent estimate. The larger concern is evaluability, not circularity: the Preview Match Score is never given a formula, no LLM baseline scores appear anywhere, and the test-set curation criteria are absent. Those issues undermine the strength of the central superiority claim but are correctness and reproducibility risks, not demonstrations that the result is equivalent to its inputs by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on proprietary datasets, expert-judged ground truth, a self-defined metric, and domain assumptions about design metadata. No scientific derivation or external benchmark is supplied. A single hand-selected threshold (95%) drives the headline fidelity number, while the trained model weights are undisclosed.

free parameters (1)

Preview Match Score success threshold = 95%
Chosen by the authors to define 'near-perfect match'; the reported headline (89.6% of screens above this threshold) depends on this hand-selected cutoff.

assumptions (5)

domain assumption Design metadata, layer structure, and visual context contain sufficient signal to infer UI semantics and layout behavior.
Invoked throughout the training and inference pipeline (e.g., 'LDM directly process design metadata, layer structures, and visual context'); if this is false, the whole approach cannot recover UI intent.
domain assumption Expert-annotated design transformations represent objectively correct 'best-practice' ground truth.
Design Optimiser approach states that expert designers annotated best-practice transformations; these annotations are treated as ground truth without inter-annotator agreement or objective correctness measure.
ad hoc to paper A Preview Match Score above 95% (computed on node width, height, and coordinates) indicates near-perfect visual fidelity.
The threshold is introduced in the evaluation section to declare success; no human rating or external benchmark validates that 95% node alignment equals production-ready fidelity.
domain assumption Deterministic output (same design to same code) is a sufficient basis for reliability claims.
Inference Pipeline asserts determinism is 'a critical requirement for reliability and version control,' but reliability of code quality is not separately measured.
domain assumption The LLM baselines were evaluated under fair and equivalent conditions.
No details are given about prompts, decoding parameters, or scoring for the LLM comparisons, so the superiority claim depends on assumed fairness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LOCOFY Large Design Models -- Design to code conversion solution." pith.science (2026). https://pith.science/paper/RSDWKYDW

@misc{pith2026250716208,
  author       = {Pith},
  title        = {Pith review of: LOCOFY Large Design Models -- Design to code conversion solution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RSDWKYDW}},
  note         = {Machine review of arXiv:2507.16208}
}

read the original abstract

Despite rapid advances in Large Language Models and Multimodal Large Language Models (LLMs), numerous challenges related to interpretability, scalability, resource requirements and repeatability remain, related to their application in the design-to-code space. To address this, we introduce the Large Design Models (LDMs) paradigm specifically trained on designs and webpages to enable seamless conversion from design-to-code. We have developed a training and inference pipeline by incorporating data engineering and appropriate model architecture modification. The training pipeline consists of the following: 1)Design Optimiser: developed using a proprietary ground truth dataset and addresses sub-optimal designs; 2)Tagging and feature detection: using pre-trained and fine-tuned models, this enables the accurate detection and classification of UI elements; and 3)Auto Components: extracts repeated UI structures into reusable components to enable creation of modular code, thus reducing redundancy while enhancing code reusability. In this manner, each model addresses distinct but key issues for design-to-code conversion. Separately, our inference pipeline processes real-world designs to produce precise and interpretable instructions for code generation and ensures reliability. Additionally, our models illustrated exceptional end-to-end design-to-code conversion accuracy using a novel preview match score metric. Comparative experiments indicated superior performance of LDMs against LLMs on accuracy of node positioning, responsiveness and reproducibility. Moreover, our custom-trained tagging and feature detection model demonstrated high precision and consistency in identifying UI elements across a wide sample of test designs. Thus, our proposed LDMs are a reliable and superior solution to understanding designs that subsequently enable the generation of efficient and reliable production-ready code.

Figures

Figures reproduced from arXiv: 2507.16208 by the authors.

**Figure 2.** LDM training pipeline Inference Pipeline The LDM is part of a deterministic inference pipeline( [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** LDM inference pipeline In the following sections, we detail the components of the LDM model architecture [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: ). This process starts with a step that groups layers (see [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Grouping of layers [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Auto layout enables designs to stay interactive at any form factor. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Tagging and Feature Detection Approach: We present a Tagging and Feature Detection model based on a YOLO backbone to address this issue. This model can identify individual UI features—such as buttons, input fields, and icons—and combine them into more complex structure…

**Figure 8.** Figure 8: Auto components for reusable elements: Components help maintain consistency and reduce redundancy by allowing users to reuse UI elements across multiple screens [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

**Figure 9.** Figure 9: Distribution of preview match scores across different screens [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

**Figure 10.** Figure 10: LLM vs LDM testing results In contrast, our LDM demonstrated strong performance across all complexity levels, consistently generating structurally accurate and hierarchically aligned outputs. We also assessed next-token prediction accuracy, which measures a model's ab…

**Figure 11.** Figure 11: Next token prediction accuracy Competing LLM-based models frequently encountered issues such as: ● Lack of component structure ● Poor responsiveness ● High mismatch scores during previews ● Low consistency and reproducibility in the generated output code In contrast, …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

7 extracted references · 1 canonical work pages

[7]

Design2Code: Benchmarking Multimodal Code Generation for Automated Front -End Engineering

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang. Design2Code: Benchmarking Multimodal Code Generation for Automated Front -End Engineering. arXiv:2403.03163 (2024)

arXiv 2024
[1]

Year’s Top Web Development Stats You Can’t Ignore

“Year’s Top Web Development Stats You Can’t Ignore”, https://www.pixelcrayons.com/blog/software-development/web-development-stats/

work page
[2]

A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage

Muhammad Usman Hadi, Qasem Al Tashi, Rizwan Qureshi, et al. A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage. TechRxiv. July 10, 2023. DOI: 10.36227/techrxiv.23589741.v1

doi:10.36227/techrxiv.23589741.v1 2023
[3]

2023d.Towards enhancing in - context learning for code generation.arXiv preprint arXiv:2303.17780 (2023)

Jia Li, Yunfei Zhao, Yongmin Li, Ge Li, and Zhi Jin. 2023d.Towards enhancing in - context learning for code generation.arXiv preprint arXiv:2303.17780 (2023)

arXiv 2023
[4]

2021.Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374 (2021)

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374 (2021)

arXiv 2021
[5]

Challenges and Applications of Large Language Models

Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy. Challenges and Applications of Large Language Models. arXiv:2307.10169 (2023)

arXiv 2023
[6]

Testing LLMs on Code Generation with Varying Levels of Prompt Specificity

Lincoln Murr, Morgan Grainger, David Gao. Testing LLMs on Code Generation with Varying Levels of Prompt Specificity. arXiv:2311.07599 (2023)

arXiv 2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [7]

Design2Code: Benchmarking Multimodal Code Generation for Automated Front -End Engineering

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang. Design2Code: Benchmarking Multimodal Code Generation for Automated Front -End Engineering. arXiv:2403.03163 (2024)

arXiv 2024

[2] [1]

Year’s Top Web Development Stats You Can’t Ignore

“Year’s Top Web Development Stats You Can’t Ignore”, https://www.pixelcrayons.com/blog/software-development/web-development-stats/

work page

[3] [2]

A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage

Muhammad Usman Hadi, Qasem Al Tashi, Rizwan Qureshi, et al. A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage. TechRxiv. July 10, 2023. DOI: 10.36227/techrxiv.23589741.v1

doi:10.36227/techrxiv.23589741.v1 2023

[4] [3]

2023d.Towards enhancing in - context learning for code generation.arXiv preprint arXiv:2303.17780 (2023)

Jia Li, Yunfei Zhao, Yongmin Li, Ge Li, and Zhi Jin. 2023d.Towards enhancing in - context learning for code generation.arXiv preprint arXiv:2303.17780 (2023)

arXiv 2023

[5] [4]

2021.Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374 (2021)

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374 (2021)

arXiv 2021

[6] [5]

Challenges and Applications of Large Language Models

Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy. Challenges and Applications of Large Language Models. arXiv:2307.10169 (2023)

arXiv 2023

[7] [6]

Testing LLMs on Code Generation with Varying Levels of Prompt Specificity

Lincoln Murr, Morgan Grainger, David Gao. Testing LLMs on Code Generation with Varying Levels of Prompt Specificity. arXiv:2311.07599 (2023)

arXiv 2023