Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FrontendBench is a 148-task benchmark that claims automatic sandbox-based scoring of LLM-generated front-end HTML agrees with expert human judges on 90.54% of evaluations, making automated grading a viable substitute for human review.

desk verdict A genuinely useful interactive benchmark design, undermined by an in-sample agreement statistic and implementation-specific tests that don't yet support the reliability claim. read the letter →

arxiv 2506.13832 v2 pith:RPGMKVZ6 submitted 2025-06-16 cs.SE cs.AI

Hongda Zhu , Yiwen Zhang , Bing Zhao , Jingzhe Ding , Siyao Liu , Tong Liu , Dandan Wang , Yanan Liu
show 1 more author
Zhaojian Li
This is my paper
classification cs.SEcs.AI
keywords front-endcodegenerationLLMbenchmarkautomaticevaluationsandboxexecutionhuman-LLMcollaborationinteractivewebcomponentsJestPuppeteer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that a benchmark built from 148 human- and LLM-constructed front-end tasks, each paired with an executable test script, can grade LLM-generated HTML automatically and reliably. It reports that the sandbox-based grading agrees with expert human evaluation on 90.54% of comparisons, and that the agreement holds across four different models. If true, this means expensive expert review can be replaced by automated checks for routine front-end code evaluation, and the reported pass rates (o3-mini at 83.11%, DeepSeek-R1 at 75.00%, Gemini-2.5-pro at 70.27%, DeepSeek-V3 at 66.89%) become a usable measure of current model capability on interactive front-end tasks.

What carries the argument

The load-bearing mechanism is the paired prompt-test_code dataset combined with a sandboxed automatic evaluation pipeline built on Node.js, Puppeteer, and Jest. Puppeteer launches a headless browser, simulates user interactions such as clicks and scrolling, and Jest assertions compare the resulting page state against the expected conditions; a task scores 1 only if all its assertions pass. The paper also uses an iterative human-in-the-loop refinement loop in which experts review disagreements and revise prompts or assertions until automatic and human scores converge.

What would settle it

Take a random sample of FrontendBench tasks and have human developers write fresh reference implementations from the prompts alone, without seeing the original test scripts; then run the benchmark's tests against those implementations. If many fresh implementations fail tests, or if human experts rate the tests as mismatched to the prompts, the dataset encodes generation conventions rather than user requirements.

Watch

Extended reading notes

Core claim

The central claim is that FrontendBench is a reliable and scalable benchmark for LLM front-end code generation because it combines fine-grained functional prompts with executable test scripts that run in a sandbox. The test scripts verify element presence, functional correctness, and interactive behavior in a headless browser, and a binary scoring rule awards a point only when every test case for an item passes. The paper reports a 90.54% average agreement rate between this automatic scoring and expert human judgment across four models, with per-model agreement rates of 87.83%, 89.86%, 91.89%, and 92.57%.

Load-bearing premise

The benchmark's value rests on the assumption that prompts and tests recovered from LLM-generated HTML describe what a human user actually wants; if the tests instead encode the generating model's output habits, the high agreement with human judges is agreement on a self-referential task set.

Editorial extensions

If this is right

  • Automatic sandbox evaluation can stand in for expert human review of front-end generation tasks at roughly 90% agreement, lowering the cost of large-scale model comparison.
  • Pass rates on FrontendBench separate current models: o3-mini leads at 83.11%, followed by DeepSeek-R1 at 75.00%, Gemini-2.5-pro at 70.27%, and DeepSeek-V3 at 66.89%.
  • Most models perform best on moderate-difficulty tasks and decline on the hardest level, while o3-mini's pass rate rises with difficulty, indicating different scaling of interactive reasoning ability.
  • The benchmark's per-level consistency rates identify where automatic evaluation is less trustworthy, such as Gemini-2.5-pro's 55.56% agreement on Level 1 and 66.67% on Level 5.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because each task's prompt and tests were reverse-engineered from LLM-generated HTML, the benchmark may reward conventions of the generating model rather than independent user intent; a reader should treat the 90.54% agreement as agreement on a self-consistent task set.
  • The same sandbox-and-assertion design could be turned into a reward signal for reinforcement-learning fine-tuning of front-end code models, using the binary per-task score as a training objective.
  • Adding screenshot-based or natural-language-only prompts would test whether the benchmark's DOM-id-heavy task format unfairly advantages models that imitate the generating LLM's markup style.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces FrontendBench, a benchmark of 148 prompt–test-case pairs for evaluating LLMs on front-end code generation, organized into five difficulty levels across application categories such as utilities, games, web interfaces, and data visualizations. The authors propose an automatic evaluation framework that runs model-generated HTML in a sandboxed Puppeteer/Jest environment and reports a 90.54% average agreement rate with expert human evaluations, based on the four models they benchmark (Gemini-2.5-pro, DeepSeek-R1, DeepSeek-V3, and o3-mini). They also report per-model pass rates and argue that the framework is reliable, scalable, and more comprehensive than existing front-end benchmarks.

Significance. If the reliability claim were properly supported, FrontendBench would fill a genuine gap: most prior front-end benchmarks rely on visual similarity rather than end-to-end interactive behavior, and the sandboxed execution approach with scripted user interactions is a reasonable design for scalable evaluation. The paper also contributes a concrete dataset of 148 tasks, a clearly described binary scoring protocol, and a useful comparison of current LLMs on interactive front-end tasks. However, the central reliability evidence is currently in-sample, and the test construction is tied to the generating model's DOM conventions, so the benchmark's practical validity is not yet established. The contribution is directionally valuable but needs substantial additional validation.

major comments (4)
  1. [§3.2 and Table 3] The reported 90.54% agreement rate is an in-sample statistic. Section 3.2 describes an iterative loop in which step (3) revises prompts and test assertions whenever automatic and human scores disagree, and step (4) repeats until 'a high agreement is achieved.' Table 3 then reports consistency rates on exactly the same four models that were used in that refinement loop. This measures goodness of fit to the tuning set, not out-of-sample reliability. The authors should provide a held-out evaluation, such as agreement on a set of new tasks or on models not used during test-script refinement, or a cross-validation scheme over the 148 items. Without such evidence, the headline claim in the abstract that the framework 'achieves a 90.54% agreement rate' is unsupported.
  2. [§3.1 step 3 and Appendix A.3] The test scripts are reverse-engineered from an LLM-generated reference implementation and assert on specific DOM identifiers and class names. Appendix A.3, for example, hard-codes '#category-select-id' and '.product-card'. A functionally correct page that uses different class names or a different DOM structure will fail these tests. This contradicts the Section 3.1 claim that the test scripts assess correctness 'regardless of its implementation.' Moreover, because the reference implementation is produced by an LLM, the tests may reward resemblance to the generating model's output conventions. The authors should refactor assertions to be behavioral and semantic (e.g., visible text, computed styles, interaction outcomes, or element roles) or provide evidence that multiple structurally distinct correct implementations pass the current test suite.
  3. [§5.1] The human evaluation procedure lacks essential measurement details. The paper does not report the number of human annotators, whether each item was scored by more than one annotator, or inter-rater reliability (e.g., Cohen's kappa or Krippendorff's alpha). Given that the 90.54% agreement rate is the principal evidence for the framework's reliability, the absence of these details makes it impossible to assess how much noise is in the human labels or how much of the human-machine agreement is attributable to chance.
  4. [Eq. (2) and Table 4] The consistency rates in Table 4 are computed on the same items and models used in the Section 3.2 refinement loop, so their variation across difficulty levels (e.g., Gemini-2.5-pro at 55.56% for Level 1 vs. 94.44% for Level 2) may reflect overfitting to particular item types rather than genuine difficulty-dependent reliability. The paper should report confidence intervals for the pass rates and consistency rates, and ideally report these metrics on a validation split that was not used for test-script tuning.
minor comments (5)
  1. [Figure 2 caption] The caption reads 'eutomatic evaluation method'; this should be 'automatic evaluation method'.
  2. [§5.2] The statement that 'the small variance among consistency rates suggests that the automatic evaluation results are generally reliable' is a non sequitur: low variance across models does not imply that the agreement rate reflects true reliability, especially when the same data were used to tune the tests.
  3. [Appendix A.2] In the example HTML, the id attribute is written as 'p ag e- co nta in er -i d' with spaces, which appears to be a typesetting artifact; please ensure the appendix code is reproduced verbatim so that the test assertions in Appendix A.3 are actually consistent with the shown HTML.
  4. [§3.3 and Table 2] The difficulty-level assignment is described only by coarse labels such as 'Simple page with basic interaction' (which accounts for 90 of the 148 items). The authors should clarify the criteria used to assign tasks to levels, since this classification is used in later per-level analyses.
  5. [Abstract and §6] The abstract states that 'Our data and code will be released soon,' but no repository, supplementary material, or release timeline is provided; please include at least a data/code availability statement or an anonymized link for review.

Circularity Check

2 steps flagged · score 6.0 of 10

90.54% agreement is in-sample: §3.2 tunes tests until human/auto agreement is high on the same four models, so the headline reliability claim is forced by the stopping rule; tasks also inherit DOM identifiers from an LLM-generated reference.

  1. fitted input called prediction [Section 3.2 steps (3)-(4); reported in Abstract and Section 5.2 / Eq. (2)]
    "Domain experts analyze discrepancies between human and machine scores, and resolve issues such as ambiguous prompt descriptions or misaligned test case assertions. (4) The revised data set is reintroduced into the evaluation pipeline and the above steps are repeated iteratively until a high agreement is achieved between the human and automatic evaluation results. ... This framework achieves a 90.54% agreement rate with expert human evaluations, demonstrating high reliability."

    The 90.54% figure is the output of a stopping rule: prompts and test assertions are revised until human and automatic scores agree, and the agreement is then measured on the same models (Gemini-2.5-pro, DeepSeek-R1, DeepSeek-V3, o3-mini) after that tuning. No held-out tasks or held-out models are reported. Thus the headline reliability statistic is an in-sample goodness-of-fit value produced by the construction loop, not independent evidence that the framework is reliable.

  2. self definitional [Section 3.1 step (3); Section 3.3; Appendix A.3]
    "This output is then used to reverse-engineer a detailed prompt and corresponding test code, ensuring consistency between specification and verification. Key DOM elements are explicitly identified (e.g., via id,class) and styled with fixed visual attributes. ... await page.select('#category-select-id','fund'); ... const productCards = await page.$$('.product-card'); expect(productCards.length).toBe(2);"

    Each test asserts the DOM ids and classes of a single LLM-generated reference implementation, and the prompt is reverse-engineered from that same reference. The paper claims tests assess correctness 'regardless of its implementation,' but by construction a functionally correct page using different class names or ids fails. Model pass rates therefore measure conformance to the reference generator's naming conventions as much as functional correctness, making the evaluation partially self-referential.

full rationale

FrontendBench's central reliability claim is circular in a quotable and specific way. Section 3.2 describes an iterative loop in which prompts and test assertions are revised until human and automatic scores agree, and the resulting 90.54% agreement rate is then presented in the Abstract and Section 5.2 as evidence of 'high reliability.' Because the test suite was optimized to elicit that agreement on the very models whose consistency is reported, the number is an in-sample stopping-rule outcome, not an out-of-sample validation. The benchmark tasks are also self-referential in construction: prompts and DOM-id-specific test scripts are reverse-engineered from an LLM-generated reference (Section 3.1 step 3), so pass/fail partially encodes resemblance to that reference rather than implementation-independent functional correctness. This is partial, not total, circularity: the per-model pass rates are genuine sandbox executions, and no load-bearing self-citation chain is present. The score of 6 reflects that the headline reliability claim reduces by construction to the tuning loop, while other benchmark content retains independent empirical value.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on five domain assumptions: human evaluation is a valid ground truth, reverse-engineered tasks preserve real user intent, headless browser interaction represents real usage, 148 tasks are representative, and the expert reviewers are qualified. There are no fitted numeric free parameters and no invented entities.

assumptions (5)
  • domain assumption Human expert evaluation, defined by the rubric in Section 3.2, is a valid ground truth for front-end code quality.
    The automatic framework's reliability is measured against this human rubric; if the rubric is wrong, the 90.54% agreement rate does not establish validity.
  • domain assumption LLM-generated reference HTML and the test code reverse-engineered from it correctly capture intended user-facing functionality.
    Section 3.1, step 3 relies on this to build prompt-test pairs; if tests encode implementation details rather than user intent, the benchmark is biased.
  • domain assumption Puppeteer headless browser interactions faithfully represent real user interactions across mainstream browsers.
    The automatic evaluation (Section 4) uses simulated clicks, typing, and scrolling in a headless browser as a proxy for real usage.
  • domain assumption 148 tasks across five categories are representative of real front-end development work.
    Generalization from the benchmark to practical front-end ability depends on this representativeness, which is asserted rather than demonstrated.
  • domain assumption Reviewers with at least one year of software development experience are qualified to judge visual and functional quality.
    Section 3.2 defines expert reviewers this way, but the paper does not report inter-rater agreement or qualifications beyond years of experience.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation." pith.science (2026). https://pith.science/paper/RPGMKVZ6

@misc{pith2026250613832,
  author       = {Pith},
  title        = {Pith review of: FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RPGMKVZ6}},
  note         = {Machine review of arXiv:2506.13832}
}
read the original abstract

Large Language Models (LLMs) have made significant strides in front-end code generation. However, existing benchmarks exhibit several critical limitations: many tasks are overly simplistic, test cases often lack rigor, and end-to-end validation is absent. These issues hinder the accurate assessment of model performance. To address these challenges, we present FrontendBench, a benchmark co-developed by humans and LLMs. FrontendBench categorizes tasks based on code functionality and incorporates interactive test scenarios, enabling a more comprehensive and practical evaluation of front-end code generation capabilities. The benchmark comprises 148 meticulously crafted prompt-test case pairs spanning five levels of web components, from basic UI elements to complex interactive features. Each task reflects realistic front-end development challenges. Furthermore, we introduce an automatic evaluation framework that executes generated code within a sandbox environment and assesses outcomes using predefined test scripts. This framework achieves a 90.54% agreement rate with expert human evaluations, demonstrating high reliability. We benchmark several state-of-the-art LLMs on FrontendBench and observe substantial performance disparities in handling real-world front-end tasks. These results highlight FrontendBench as a reliable and scalable benchmark, supporting consistent multimodal evaluation and providing a robust foundation for future research in front-end code generation. Our data and code will be released soon.

Figures

Figures reproduced from arXiv: 2506.13832 by the authors.

Figure 1
Figure 1. Construction process and examples of FrontendBench. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Front-end code generation eutomatic evaluation method. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Rendered output of the paper airplane element. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Encoding behavior with different input types. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The screenshot of the financial wealth management [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation

    cs.SE 2026-08 conditional novelty 6.0 of 10

    On a new 1,440-screenshot fill-in-the-blank benchmark, five frontier multimodal models default to repeated UI patterns instead of visual deviations, with mean bias rates from 69.8% on card widths to 80.2% on font sizes.

  2. LH-Bench: Skill-Grounded Evaluation of Long-Horizon Agents on Subjective Enterprise Tasks

    cs.AI 2026-03 conditional novelty 6.0 of 10

    Expert-authored dual-use skills make LLM judges more reliable on subjective long-horizon enterprise agent tasks than LLM-authored rubrics, with human preferences confirming the main ranking boundary.

  3. GameDevBench: Evaluating Agentic Capabilities Through Game Development

    cs.AI 2026-02 conditional novelty 6.0 of 10

    A new 132-task Godot benchmark shows frontier AI agents solve only about 54.5% of game-development tasks, with visual feedback giving consistent but modest gains.

Reference graph

Works this paper leans on

28 extracted references · 5 canonical work pages · cited by 3 Pith papers

  1. [1]

    Rajas Agashe, Srinivasan Iyer, and Luke Zettlemoyer. 2019. JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), Kentaro Inui, Jing J...

    work page doi:10.18653/v1/d19-1546 2019

  2. [2]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models.arXiv preprint arXiv:2108.07732(2021)

    arXiv 2021

  3. [3]

    Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps- Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, Arjun Guha, Michael Greenberg, and Abhinav Jangda. 2023. MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation.IEEE Transactions on Software Engineering49, 7 (2023...

    arXiv 2023

  4. [4]

    Linzheng Chai, Shukai Liu, Jian Yang, Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, Zekun Wang, Boyang Wang, Xianjie Wu, Bing Wang, Tongliang Li, Liqun Yang, Sufeng Duan, and Zhoujun Li. 2024. McEval: Massively Multilingual Code Evaluation. arXiv:2406.07436 [cs.PL] https: //arxiv.org/abs/2406.07436

    arXiv 2024

  5. [5]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374(2021)

    arXiv 2021

  6. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    arXiv 2025

  7. [7]

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. 2021. Mea- suring coding challenge competence with apps.arXiv preprint arXiv:2105.09938 (2021)

    arXiv 2021

  8. [8]

    Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2018. Mapping Language to Code in Programmatic Context. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, Brussels, Belgium, 1643...

    doi:10.18653/v1/d18- 2018

Show all 28 references
  1. [9]

    Kush Jain, Gabriel Synnaeve, and Baptiste Rozière. 2025. TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark. arXiv:2410.00752 [cs.SE] https://arxiv.org/abs/2410.00752

    2025 arXiv

  2. [10]

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Live- CodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv:2403.07974 [cs.SE] https://arxiv.org/...

    2024 arXiv

  3. [11]

    Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conversion of web screenshots into html code with the websight dataset.arXiv preprint arXiv:2403.09029(2024). FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation , ,

    2024 arXiv

  4. [12]

    Jia Li, Ge Li, Xuanming Zhang, Yihong Dong, and Zhi Jin. 2024. EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repos- itories. arXiv:2404.00599 [cs.CL] https://arxiv.org/abs/2404.00599

    2024 arXiv

  5. [13]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode.Science378, 6624 (2022), 1092–1097

    2022

  6. [14]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437(2024)

    2024 arXiv

  7. [15]

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Am- brosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong (YIMING), Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu F...

    2021

  8. [16]

    Doan, Nam V

    Dung Nguyen Manh, Thang Phan Chau, Nam Le Hai, Thong T. Doan, Nam V. Nguyen, Quang Pham, and Nghi D. Q. Bui. 2025. CodeMMLU: A Multi-Task Bench- mark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs. arXiv:2410.01999 [cs.SE] https://arxiv.org/abs/2410.01999

    2025 arXiv

  9. [17]

    OpenAI. 2025. OpenAI o3 and o4-mini System Card. https://openai.com/index/o3- o4-mini-system-card

    2025

  10. [18]

    Qiwei Peng, Yekun Chai, and Xuhong Li. 2024. HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization. arXiv:2402.16694 [cs.CL] https://arxiv.org/abs/2402.16694

    2024 arXiv

  11. [19]

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang

  12. [20]

    Xiaoshuai Song, Muxi Diao, Guanting Dong, Zhengyang Wang, Yujia Fu, Runqi Qiao, Zhexu Wang, Dayuan Fu, Huangxuan Wu, Bin Liang, Weihao Zeng, Yejie Wang, Zhuoma GongQue, Jianing Yu, Qiuna Tan, and Weiran Xu. 2025. CS-Bench: A Comprehensive Benchmark for Large Language Models to...

    2025 arXiv

  13. [21]

    Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. 2023. PDEBENCH: AN EXTENSIVE BENCHMARK FOR SCI- ENTIFIC MACHINE LEARNING. InICLR 2023 Workshop on Physics for Machine Learning. https://openreview.net/fo...

    2023

  14. [22]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    2023 arXiv

  15. [23]

    Weixi Tong and Tianyi Zhang. 2024. CodeJudge: Evaluating Code Generation with Large Language Models.arXiv preprint arXiv:2410.02184(2024)

    2024 arXiv

  16. [24]

    Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, et al

  17. [25]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models. arXiv:2304.10592 [cs.CV] https://arxiv.org/abs/2304.10592

    2023 arXiv

  18. [26]

    Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Frame- work for Multimodal LLMs.arXiv preprint arXiv:2406.20098(2024)

    2024 arXiv

  19. [28]

    Qiming Zhu, Jialun Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, and Shing- Chi Cheung. 2024. DOMAINEVAL: An Auto-Constructed Benchmark for Multi- Domain Code Generation. arXiv:2408.13204 [cs.AI] https://arxiv.org/abs/2408. 13204

    2024 arXiv

  20. [2024]

    Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering.arXiv preprint arXiv:2403.03163(2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.