Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Design knowledge kept in a living repository graph is the highest-leverage fix for multi-iteration front-end code generation by LLMs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:24 UTC pith:LSNUH4KJ

load-bearing objection Solid systems paper: a design-memory KG plus plan/patch/sandbox loop beats Web-Agent and burns far fewer tokens than general coding agents; the “design knowledge is fundamental” claim is a bit oversold. the 3 major comments →

arxiv 2607.10621 v1 pith:LSNUH4KJ submitted 2026-07-12 cs.SE

WebDesignIter: Co-Evolving Design Knowledge for Repository-Level Front-End Code Generation

Zheng Pei , Mingwei Liu , Zhenxi Chen , Zihao Wang , Yanlin Wang This is my paper
classification cs.SE
keywords code generationfront-end developmentrepository-level codedesign knowledgeknowledge graphincremental developmentLLM coding agentsWeb-Bench
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Front-end repositories grow through many small, cross-file changes. Coding agents tuned for single-shot tasks lose architectural context, break earlier work, and produce code that is hard to maintain. This paper argues that the missing piece is design knowledge—module responsibilities, structural constraints, and historical design decisions—and shows how to keep that knowledge synchronized with the code in a persistent graph. The proposed system plans from that graph, applies small diff patches, validates them in a sandbox, and updates the graph after each cycle. On a multi-task front-end benchmark it raises pass rates over prior agents while using far fewer tokens, and ablations show design knowledge matters more than retrieval, patching, or verification alone.

Core claim

Embedding design knowledge—architectural principles, module responsibilities, and structural constraints—into a persistent knowledge graph that co-evolves with the repository yields more correct, maintainable, and token-efficient multi-iteration front-end generation than code-only retrieval or general agentic loops. On Web-Bench, stripping design knowledge drops Pass@1 by 11.40 points, more than removing code-graph retrieval, patch generation, or sandbox verification, and the full system posts the highest Pass@1 and Pass@2 while using 25–30× fewer input tokens than general-purpose coding agents.

What carries the argument

WebAppArchKG, a persistent knowledge graph that fuses repository structure with historical design, overview design summaries, and cross-file component links, driving WebDesignIter’s two stages: design-informed planning (implementation plan plus tests from the graph) and design-aware generation (priority-ordered diff patches with syntax repair, sandbox validation, rule-based refactoring, then graph update).

Load-bearing premise

The paper treats success on Web-Bench’s fifty multi-task projects as a sufficient stand-in for real long-running front-end engineering, so that design-knowledge gains will transfer beyond that suite.

What would settle it

Rebuild the same planning, patching, and sandbox pipeline but replace design fields with empty or random architectural summaries; if Pass@1 and Pass@2 on Web-Bench still match the full system, the claim that design knowledge is the decisive component fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multi-iteration front-end agents should maintain an explicit, updatable design layer rather than re-inferring architecture from raw files each turn.
  • Diff-based patches guided by design knowledge cut regression failures more than full-file rewrites.
  • Token cost of repository-level front-end generation can fall by an order of magnitude when context comes from a structured design graph instead of broad agentic exploration.
  • Automated refactoring of mixed-language spaghetti files improves both maintainability metrics and the quality of design knowledge later steps consume.
  • For this task class, investing in design knowledge can outweigh further gains from retrieval or verification alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Co-evolving design graphs may help other multi-file domains where regressions, not single-file correctness, dominate failure modes.
  • Because design summaries are written by the same models that generate code, long task chains may drift if those summaries stop tracking true architectural intent.
  • Pass@1/Pass@2 alone may understate maintainability gains that appear as shorter files and fewer regression errors.
  • Generating the implementation plan and its tests in one step may be a general way to reduce plan–test misalignment in agent pipelines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes WebDesignIter, a two-stage framework for incremental repository-level front-end code generation built around WebAppArchKG, a persistent knowledge graph that stores repository structure, historical design records, and LLM-written overview summaries. Design-informed planning retrieves this context to produce an implementation plan and test scripts; design-aware generation applies priority-ordered diff patches with AST syntax repair and Docker sandbox validation, then updates the graph and optionally refactors heterogeneous fragments. On Web-Bench (50 projects × 20 sequential tasks), the system improves over Web-Agent by average +7.98 Pass@1 / +9.55 Pass@2 across nine foundation models (Table I), and in RQ4 it beats Claude Code, OpenHands, SWE-Agent, and Codex CLI on every reported model while using roughly 25–30× fewer input tokens (Table IV). An ablation on Claude 4 Sonnet (Table II) ranks components by Pass@1 drop: Design −11.40, Code Graph −8.40, Patch −3.50, Sandbox −3.30. The authors conclude that design knowledge is the most impactful component and yields a fundamentally more efficient and reliable path to repository-level front-end generation.

Significance. If the causal story holds, the work is significant for software-engineering agents: it shows that a structured, co-evolving memory of design and repository state can beat both a specialized front-end agent and strong general agentic loops on an incremental multi-framework benchmark while cutting token cost by an order of magnitude. Strengths that should be credited include the multi-model head-to-head vs Web-Agent (Table I), the agent comparison with explicit token accounting (Table IV), the maintainability metrics (avg file length, error-type shift in Fig. 8 / Table III), and the promised public code/data release. Even under a weaker reading—that structured multi-turn memory plus patch/sandbox discipline is what helps—the efficiency result is practically useful for front-end repository agents.

major comments (3)
  1. [Table II; Method §B; Abstract; Finding 2] Central claim vs. what Table II actually removes. Abstract, Finding 2, and Conclusion state that “design knowledge” (architectural principles, module responsibilities, structural constraints) is the most impactful component because w/o Design drops Pass@1 by 11.40 points. Method §B and Fig. 3 show that the Design fields retrieved for planning are Historical Design (prior task texts), Repo Structure (directory tree), and Overview Design (LLM-generated file summaries written back after successful sandbox runs). These are structured multi-turn memory and self-summaries, not independently curated architectural principles. The large ablation drop is therefore consistent with loss of task history / RAG rather than absence of design knowledge per se. Please either (a) reframe claims and the ranking language to match the operational definition, or (b) add an analysis that isolates architectural
  2. [Table II; RQ2] The load-bearing ranking of components rests on a single-model ablation (Claude 4 Sonnet only). RQ1 reports gains across nine models, but the claim that Design > Code Graph > Patch > Sandbox is not shown to be stable across models. At minimum, repeat the four ablations on one additional strong model from Table I (e.g., Gemini 2.5 Pro) and report whether the ordering and the ~11-point Design drop hold; otherwise qualify the causal ranking as model-specific.
  3. [Method §B; Repo-Verification; Evaluation metrics] Test scripts used for sandbox Pass/Fail are generated by the same LLM in the Design-informed Planning stage (Method §B, “Design Knowledge-Guided Test Script Generation”), synchronously with the plan. Pass@k therefore measures agreement with model-authored tests as much as external functional correctness. Clarify whether Web-Bench also scores against fixed benchmark oracles, and report agreement between generated tests and any gold tests, or run a sensitivity check with held-out/human tests on a subset of projects. Without this, the reliability claim is partly circular.
minor comments (6)
  1. [Abstract; Finding 4; Table IV] Abstract says “consuming 2530 fewer input tokens”; body and Finding 4 say 25–30× fewer. Fix the abstract typo and align all efficiency statements with Table IV units (K tokens).
  2. [Fig. 6] Fig. 6 caption and axis labels are partially garbled (Unicode private-use sequences). Replace with clean text labels so the per-project boxplots are readable.
  3. [Fig. 2; Method] Inconsistent naming: WebAppArchKG vs WebProject-ArchKG (Fig. 2) vs WebAppArchKG elsewhere. Standardize.
  4. [Table I] Table I reports Avg. Δ only for WebDesignIter; absolute averages for both systems would make the +7.98 / +9.55 figures easier to audit.
  5. [Fig. 8; RQ3] RQ3 error-type percentages (Fig. 8) appear to be over unresolved failures only; state the denominator (N failures) and whether counts are pooled across models or for one model.
  6. [Introduction; Method] Minor prose issues: missing spaces after periods (“However,these methods”), “WebDesig-nIter” line break, and “2530” vs “25–30×” already noted. A pass for copy-editing would help.

Circularity Check

0 steps flagged

Empirical systems paper on an external benchmark; no derivation that reduces a claimed prediction to its own fitted inputs by construction.

full rationale

WebDesignIter is an engineering/systems paper: it builds a two-stage agent with a persistent knowledge graph, measures Pass@1/Pass@2 on the external Web-Bench suite (50 projects × 20 sequential tasks), and compares against Web-Agent and third-party coding agents under the same protocol. The central quantitative claims (average +9.55 Pass@2 over baselines; 11.40-point Pass@1 drop when Design is ablated on Claude 4 Sonnet; 25–30× fewer input tokens than agentic loops) are empirical outcomes, not first-principles predictions derived from parameters fitted to the same target. There is no uniqueness theorem, no self-citation that forces the result, and no renaming of a known closed-form law. The only mild self-referential element is that Overview Design / Historical Design fields written back into WebAppArchKG after successful runs are LLM summaries of prior work and then re-injected as planning context—so the system can reinforce its own memory. That is ordinary multi-turn agent memory, not a circular derivation: success is still scored by independent sandbox tests and benchmark oracles, not by agreement with those summaries. Component ranking and the word “fundamentally” may over-interpret a single-model ablation and a synthetic suite (correctness/external-validity concerns), but they do not make the reported pass rates tautological. Score 1 for that minor self-reinforcing memory loop; steps empty of load-bearing circular reductions.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 2 invented entities

Load-bearing content is mostly domain assumptions about front-end evolution and benchmark validity, plus engineered artifacts (KG schema, refactoring rules, patch format). There is no fitted physical constant; free parameters are pipeline knobs. Invented entities are the system components themselves, with independent evidence only via the reported benchmark experiments.

free parameters (2)
  • syntax_repair_retry_limit = 2 failed repairs before full regeneration
    Repo-Verification invokes the LLM to repair patches and falls back to full-file generation after more than two failed repairs; this threshold is a hand-chosen control that affects whether patches are accepted.
  • implementation_plan_priority_and_dependence_fields
    Planning emits ordered file create/modify steps with priority and dependence; how the LLM assigns these is unconstrained and shapes generation order without a learned or validated priority model.
axioms (4)
  • domain assumption Front-end repositories evolve incrementally with heterogeneous cross-file dependencies, and explicit design knowledge (module responsibilities, structural constraints) is necessary to avoid regressions and spaghetti code.
    Stated in Abstract and Introduction as the motivation for WebAppArchKG; underpins why design knowledge should dominate ablations.
  • domain assumption Web-Bench (50 projects, 20 sequential tasks, Vue/Angular/Tailwind etc.) is a valid stand-in for realistic repository-level front-end workflows for comparing agents.
    All effectiveness, ablation, and cost claims are measured only on Web-Bench (Evaluation §A).
  • ad hoc to paper Rule-based extraction of embedded heterogeneous fragments (e.g., JS in HTML) plus LLM-written Historical/Overview Design fields produce a design memory that remains useful and non-drifted across sequential tasks.
    Architectural Evolution via Refactoring and Design Knowledge update steps (Method §A, §D) assume these updates improve maintainability without systematic error accumulation.
  • domain assumption Pass@1 and Pass@2 under the benchmark protocol, plus average file length, adequately measure functional success and maintainability.
    Primary metrics in RQ1–RQ4; maintainability analysis uses avg file length as proxy (Table III).
invented entities (2)
  • WebAppArchKG independent evidence
    purpose: Persistent graph fusing code blocks, cross-file links, Historical Design, Repo Structure, and Overview Design so planning and generation share evolving architectural memory.
    Core postulated artifact of the paper; schema in Fig. 3; no existence outside this system.
  • WebDesignIter two-stage pipeline (design-informed planning + design-aware generation with patch/sandbox loop) independent evidence
    purpose: Operationalize design knowledge for incremental front-end repository code generation and verification.
    The proposed method; evidence is only the reported Web-Bench experiments.

pith-pipeline@v1.1.0-grok45 · 23723 in / 3549 out tokens · 44861 ms · 2026-07-14T10:24:09.567583+00:00 · methodology

0 comments
read the original abstract

Front-end development accumulates change after change at the repository level, weaving complex cross-file dependencies that current LLM coding agents tuned for single-shot tasks cannot reliably track across multiple iterations, leading to functional regressions and code that resists maintenance. We argue the missing piece is design knowledge: architectural principles, module responsibilities, and structural constraints that developers lean on to keep code readable, maintainable, and evolvable as a system scales. To operationalize this, we propose WebDesignIter, a framework built around a persistent knowledge graph (WebAppArchKG) that fuses repository structure with design knowledge and keeps both in sync across development cycles. WebDesignIter works in two stages: design-informed planning pulls historical context and architectural overviews from WebAppArchKG to produce an implementation plan with corresponding test scripts, and design-aware generation executes that plan through targeted diff-based patches, validated by sandbox execution and automatic syntax repair. On Web-Bench, WebDesignIter delivers an average Pass@2 gain of 9.55 percentage points across nine foundation models over existing baselines. More importantly, WebDesignIter outperforms every general-purpose coding agent Claude Code, OpenHands, SWE-Agent, Codex CLI on every model configuration, posting the highest Pass@1 and Pass@2 while consuming 2530 fewer input tokens. Ablation singles out design knowledge as the most impactful component: stripping it drops Pass@1 by 11.40 percentage points, a degradation far larger than removing code-graph retrieval, patch-based generation, or sandbox verification, confirming that design knowledge provides a fundamentally more efficient and reliable path to repository-level code generation.

Figures

Figures reproduced from arXiv: 2607.10621 by Mingwei Liu, Yanlin Wang, Zheng Pei, Zhenxi Chen, Zihao Wang.

Figure 1
Figure 1. Figure 1: The impact of modularized output formats and model output formats on readability and maintainability Recent benchmarks, such as Web-Bench [19], target in￾cremental, multi-task, and repository-level front-end develop￾ment settings and systematically expose the complexity and challenges of these scenarios by simulating realistic software development workflows. On this benchmark, Web-Agent [19] achieves only … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of WebDesignIter. files or HTML snippets within TypeScript files. Using a rule￾based extraction strategy, these fragments are separated into new files, while the original files are updated to maintain consistency. This refactoring step evolves the architectural representation by improving modular boundaries, reducing file complexity, and preserving consistency within the dependency structure encod… view at source ↗
Figure 3
Figure 3. Figure 3: Schema of WebAppArchKG. import { Component } from '@angular/core' import { HeaderComponent } from './components/header/header.component'; import { MainComponent } from './components/main/main.component'; @Component({ selector: 'app-root', imports: [HeaderComponent, MainComponent], template: `……`, styles: `……`, }) export class AppComponent { title = 'angular' } Code Content import { Component } from '@angul… view at source ↗
Figure 4
Figure 4. Figure 4: An Example of Code Content Segmentation into Blocks. included in the prompt to construct a well-founded contextual representation, preventing the model from generating fabri￾cated outputs due to incomplete knowledge of repository-level dependencies. Based on this enriched contextual grounding, the system generates a corresponding set of Design File Patches for each high-level design action, ensuring that t… view at source ↗
Figure 5
Figure 5. Figure 5: Impact of WebAppArchKG–Based Retrieval on Generated Code Correctness. the test scripts produced during the high-level system design phase are executed to ensure that the generated implementation meets user requirements. Upon successful validation, the system performs a refac￾toring scan of the applied code, and both the refactored and unrefactored files are fed into WebAppArchKG for node expansion and depe… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of task completion per project between Web-Agent and our method (WebDesignIter) across multiple models. Each boxplot summa￾rizes the distribution of completed tasks (pass@2 20) across all projects within a model, with Web-Agent depicted in blue and WebDesignIter in orange. Individual project results are overlaid as jittered points to illustrate variability. Result. As shown in Table I, WebDesign… view at source ↗
Figure 7
Figure 7. Figure 7: A Case Study on Patch Strategy and Code Generation. Finding 2: Every component of WebDesignIter—Design, Code Graph, Patch, and Sandbox—contributes measur￾ably to overall performance. The Design module and Code Graph are the most impactful, while Patch and Sandbox provide complementary benefits in consistency and verification. D. RQ3: Robustness and Maintainability Design. WebDesignIter leverages explicit d… view at source ↗
Figure 8
Figure 8. Figure 8: Radar chart of error type distributions for Web-Agent and the WebDesignIter method. Finding 3: WebDesignIter shifts the dominant failure mode from regression and dependency errors toward higher-level functional and logical challenges, while pro￾ducing more concise code. This indicates that the primary performance bottleneck moves from architectural consis￾tency to the base model’s reasoning capabilities. E… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 2 canonical work pages

  1. [1]

    Exploring the latest front-end development trends,

    R. Stefanova, “Exploring the latest front-end development trends,” 2024

    2024

  2. [2]

    Static components dependency graph detection with evaluation metrics in React.js projects,

    Kurant, “Static components dependency graph detection with evaluation metrics in React.js projects,” Krakw, Poland, Oct. 2025, pp. 127–134. [Online]. Available: https://annals-csis.org/V olume 45/ drp/5264.html

    2025

  3. [3]

    Programs, life cycles, and laws of software evolution,

    M. M. Lehman, “Programs, life cycles, and laws of software evolution,” Proceedings of the IEEE, vol. 68, no. 9, pp. 1060–1076, 2005

    2005

  4. [4]

    Soen-101: Code generation by emulating software process models using large language model agents (2024),

    F. Lin, D. Kim, and C. Tse-Husn, “Soen-101: Code generation by emulating software process models using large language model agents (2024),”arXiv preprint arXiv:2403.15852

    Pith/arXiv arXiv 2024

  5. [5]

    Morepair: Teaching llms to repair code via multi- objective fine-tuning,

    B. Yang, H. Tian, J. Ren, H. Zhang, J. Klein, T. F. Bissyand ´e, C. L. Goues, and S. Jin, “Morepair: Teaching llms to repair code via multi- objective fine-tuning,”arXiv preprint arXiv:2404.12636, 2024

    arXiv 2024

  6. [6]

    When llms meet api documentation: Can retrieval augmentation aid code generation just as it helps developers?

    J. Chen, S. Chen, J. Cao, J. Shen, and S.-C. Cheung, “When llms meet api documentation: Can retrieval augmentation aid code generation just as it helps developers?”arXiv preprint arXiv:2503.15231, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Automatically translating c to rust,

    J. Hong and S. Ryu, “Automatically translating c to rust,”Communica- tions of the ACM, vol. 68, no. 11, pp. 58–65, 2025

    2025

  8. [8]

    Fine tuning large language model for secure code generation,

    J. Li, A. Sangalay, C. Cheng, Y . Tian, and J. Yang, “Fine tuning large language model for secure code generation,” inProceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering, 2024, pp. 86–90

    2024

  9. [9]

    Stall+: Boosting llm-based repository-level code completion with static analysis,

    J. Liu, Y . Chen, M. Liu, X. Peng, and Y . Lou, “Stall+: Boosting llm-based repository-level code completion with static analysis,”arXiv preprint arXiv:2406.10018, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Empirical and sustainability aspects of software engineering research in the era of large language models: A reflection,

    D. Williams, M. Hort, M. Kechagia, A. Aleti, J. Petke, and F. Sarro, “Empirical and sustainability aspects of software engineering research in the era of large language models: A reflection,” 2026. [Online]. Available: https://arxiv.org/abs/2510.26538

    arXiv 2026

  11. [11]

    Intention-based gui test migration for mobile apps using large language models,

    S. Cao, M. Pan, Y . Lan, and X. Li, “Intention-based gui test migration for mobile apps using large language models,”Proc. ACM Softw. Eng., vol. 2, no. ISSTA, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3728978

    doi:10.1145/3728978 2025

  12. [12]

    Methods and techniques of agentic software engineering: A systematic literature review,

    N. Otoum and N. Elkhalili, “Methods and techniques of agentic software engineering: A systematic literature review,”IEEE Access, vol. 14, pp. 7443–7465, 2026

    2026

  13. [13]

    Vibe coding vs. agentic coding: Fundamentals and practical implications of agentic ai,

    R. Sapkota, K. I. Roumeliotis, and M. Karkee, “Vibe coding vs. agentic coding: Fundamentals and practical implications of agentic ai,” 2025. [Online]. Available: https://arxiv.org/abs/2505.19443

    Pith/arXiv arXiv 2025

  14. [14]

    Mllm-based ui2code automation guided by ui layout information,

    F. Wu, C. Gao, S. Li, X.-C. Wen, and Q. Liao, “Mllm-based ui2code automation guided by ui layout information,”Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 1123–1145, 2025

    2025

  15. [15]

    How artificial intelligence can improve web development and testing,

    A. Stocco, “How artificial intelligence can improve web development and testing,” inCompanion of the 3rd International Conference on Art, Science, and Engineering of Programming, ser. Programming ’19. New York, NY , USA: ACM, Apr. 2019, pp. 1–13. [Online]. Available: http://doi.acm.org/10.1145/3328433.3328447

    work page doi:10.1145/3328433.3328447 2019

  16. [16]

    Designcoder: Hierarchy-aware and self-correcting ui code generation with large language models,

    Y . Chen, S. Ding, Y . Zhang, W. Chen, J. Du, L. Sun, and L. Chen, “Designcoder: Hierarchy-aware and self-correcting ui code generation with large language models,” 2025. [Online]. Available: https://arxiv.org/abs/2506.13663

    Pith/arXiv arXiv 2025

  17. [17]

    Unlocking the conversion of web screenshots into html code with the websight dataset,

    H. Laurenc ¸on, L. Tronchon, and V . Sanh, “Unlocking the conversion of web screenshots into html code with the websight dataset,”arXiv preprint arXiv:2403.09029, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Automatically generating ui code from screenshot: A divide-and- conquer-based approach,

    Y . Wan, C. Wang, Y . Dong, W. Wang, S. Li, Y . Huo, and M. R. Lyu, “Automatically generating ui code from screenshot: A divide-and- conquer-based approach,”arXiv preprint arXiv:2406.16386, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Web-bench: A llm code benchmark based on web standards and frameworks,

    K. Xu, Y . Mao, X. Guan, and Z. Feng, “Web-bench: A llm code benchmark based on web standards and frameworks,” 2025. [Online]. Available: https://arxiv.org/abs/2505.07473

    Pith/arXiv arXiv 2025

  20. [20]

    Software architecture optimization methods: A systematic literature review,

    A. Aleti, B. Buhnova, L. Grunske, A. Koziolek, and I. Meedeniya, “Software architecture optimization methods: A systematic literature review,”IEEE Transactions on Software Engineering, vol. 39, no. 5, pp. 658–683, 2013

    2013

  21. [21]

    Codes: Natural language to code repository via multi-layer sketch,

    D. Zan, A. Yu, W. Liu, D. Chen, B. Shen, Y . Yao, W. Li, X. Chen, Y . Gong, B. Guanet al., “Codes: Natural language to code repository via multi-layer sketch,”ACM Transactions on Software Engineering and Methodology, 2024

    2024

  22. [22]

    RepoCoder: Repository-level code completion through iterative retrieval and generation,

    F. Zhang, B. Chen, Y . Zhang, J. Keung, J. Liu, D. Zan, Y . Mao, J.-G. Lou, and W. Chen, “RepoCoder: Repository-level code completion through iterative retrieval and generation,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguist...

    2023

  23. [23]

    Demystifying llm-based software engineering agents,

    C. S. Xia, Y . Deng, S. Dunn, and L. Zhang, “Demystifying llm-based software engineering agents,”Proc. ACM Softw. Eng., vol. 2, no. FSE, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3715754

    doi:10.1145/3715754 2025

  24. [24]

    Swe-agent: Agent-computer interfaces enable automated soft- ware engineering,

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: Agent-computer interfaces enable automated soft- ware engineering,”Advances in Neural Information Processing Systems, vol. 37, pp. 50 528–50 652, 2024

    2024

  25. [25]

    Chain of agents: Large language models collaborating on long-context tasks,

    Y . Zhang, R. Sun, Y . Chen, T. Pfister, R. Zhang, and S. Arik, “Chain of agents: Large language models collaborating on long-context tasks,”Ad- vances in Neural Information Processing Systems, vol. 37, pp. 132 208– 132 237, 2024

    2024

  26. [26]

    Uicopilot: Automating ui synthesis via hierarchical code generation from webpage designs,

    Y . Gui, Y . Wan, Z. Li, Z. Zhang, D. Chen, H. Zhang, Y . Su, B. Chen, X. Zhou, W. Jiang, and X. Zhang, “Uicopilot: Automating ui synthesis via hierarchical code generation from webpage designs,” inProceedings of the ACM on Web Conference 2025, ser. WWW ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 18461855. [Online]. Available: h...

    doi:10.1145/3696410.3714891 2025

  27. [27]

    Starcoder 2 and the stack v2: The next generation,

    A. L. et al., “Starcoder 2 and the stack v2: The next generation,” 2024. [Online]. Available: https://arxiv.org/abs/2402.19173

    Pith/arXiv arXiv 2024

  28. [28]

    Beyond functional correctness: Investigating coding style inconsistencies in large language models,

    Y . Wang, T. Jiang, M. Liu, J. Chen, M. Mao, X. Liu, Y . Ma, and Z. Zheng, “Beyond functional correctness: Investigating coding style inconsistencies in large language models,”Proc. ACM Softw. Eng., vol. 2, no. FSE, Jun. 2025. [Online]. Available: https: //doi.org/10.1145/3715749

    work page doi:10.1145/3715749 2025

  29. [29]

    Evaluating large language models trained on code,

    M. Chen, “Evaluating large language models trained on code,”arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  30. [30]

    Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation,

    X. Du, M. Liu, K. Wang, H. Wang, J. Liu, Y . Chen, J. Feng, C. Sha, X. Peng, and Y . Lou, “Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation,”arXiv preprint arXiv:2308.01861, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Code graph model (cgm): A graph-integrated large language model for repository-level software engineering tasks,

    H. Tao, Y . Zhang, Z. Tang, H. Peng, X. Zhu, B. Liu, Y . Yang, Z. Zhang, Z. Xu, H. Zhanget al., “Code graph model (cgm): A graph-integrated large language model for repository-level software engineering tasks,” arXiv preprint arXiv:2505.16901, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Contested: Consistency-aided tested code generation with llm,

    J. Dong, J. Sun, W. Zhang, J. S. Dong, and D. Hao, “Contested: Consistency-aided tested code generation with llm,”Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 596–617, 2025

    2025

  33. [33]

    Repomincoder: Improving repository-level code generation based on information loss screening,

    Y . Li, E. Shi, D. Zheng, K. Duan, J. Chen, and Y . Wang, “Repomincoder: Improving repository-level code generation based on information loss screening,” inProceedings of the 15th Asia-Pacific Symposium on Internetware, 2024, pp. 229–238

    2024

  34. [34]

    Stile: a tool for parallel execution of e2e web test scripts,

    D. Olianas, M. Leotta, F. Ricca, M. Biagiola, and P. Tonella, “Stile: a tool for parallel execution of e2e web test scripts,” in2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, pp. 460–465

    2021

  35. [35]

    Webvia: A web-based vision-language agentic framework for interactive and verifiable ui-to-code generation,

    M. Xu, Z. Yang, W. Hong, L. Pan, X. Fan, Y . Wang, X. Gu, B. Xu, and J. Tang, “Webvia: A web-based vision-language agentic framework for interactive and verifiable ui-to-code generation,” 2025. [Online]. Available: https://arxiv.org/abs/2511.06251

    arXiv 2025

  36. [36]

    Understanding software vulnerabilities related to archi- tectural security tactics: An empirical investigation of chromium, php and thunderbird,

    J. C. Santos, A. Peruma, M. Mirakhorli, M. Galstery, J. V . Vidal, and A. Sejfia, “Understanding software vulnerabilities related to archi- tectural security tactics: An empirical investigation of chromium, php and thunderbird,” in2017 IEEE International Conference on Software Architecture (ICSA). IEEE, 2017, pp. 69–78

    2017

  37. [37]

    Winograd,Bringing design to software

    T. Winograd,Bringing design to software. ACM, 1996

    1996

  38. [38]

    Graphcoder: Enhancing repository-level code completion via coarse-to-fine retrieval based on code context graph,

    W. Liu, A. Yu, D. Zan, B. Shen, W. Zhang, H. Zhao, Z. Jin, and Q. Wang, “Graphcoder: Enhancing repository-level code completion via coarse-to-fine retrieval based on code context graph,” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, 2024, pp. 570–581

    2024