Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Coding agents often edit and submit before they have enough repository evidence; this paper introduces an evidence-conditioned execution layer that gates such commitments and shows, on 500 real-world issues, a 4.8–11.8 percentage-point Pass

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 00:17 UTC pith:NDA7SJ4G

load-bearing objection Genuinely useful execution-layer idea with a clean controlled eval; needs code/data and a bit more rigor before I'd trust the numbers fully. the 4 major comments →

arxiv 2607.28815 v1 pith:NDA7SJ4G submitted 2026-07-30 cs.SE

Preventing Premature Commitment in Coding Agents with an Evidence-Conditioned Execution Layer

Yisen Xu , Chenglin Li , Zehao Wang , Jinqiu Yang , Tse-Hsun Chen This is my paper
classification cs.SE
keywords coding agentspremature commitmentevidence-conditioned executionexecution gatingruntime enforcementevidence specificationtoken efficiencyissue resolution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Many coding agents fail not because they generate bad patches but because they commit too early: they edit a function or submit a patch before inspecting the callers, related implementations, tests, or failure conditions that would justify the change. This paper proposes ECLoop, an execution layer that sits between the agent and the repository, compiles a task-specific evidence specification from the issue, tracks which conditions the agent's runtime trajectory has satisfied, and postpones any edit or final submission whose required evidence is missing. The central claim is that this simple, model- and scaffold-agnostic intervention raises resolution rates on a 500-instance evaluation by 4.8–11.8 percentage points across two models and two scaffolds, while reducing token consumption by up to 12.1%. If correct, it offers a lightweight, orthogonal fix for premature commitment that needs no retraining and no change to the agent's action-selection policy.

Core claim

On the paper's own terms, the discovery is that the decision 'what action should the agent take next' and the decision 'is the agent ready to commit to this edit or submission' can and should be separated. ECLoop does this by compiling, once per task, a set of grounded evidence conditions (each tied to a concrete program entity and a required observable event pattern), parsing the runtime trajectory into observable events, and deterministically checking which conditions are satisfied. A global evidence gap guides further investigation, while an action-specific gap gates each proposed commitment; a commitment proceeds only when its gap is empty. The paper argues that this mechanism recovers a

What carries the argument

The central mechanism is the pair of an evidence specification and a two-level evidence gap. Each condition φᵢ = ⟨bᵢ, vᵢ, Rᵢ, satᵢ⟩ names a commitment action type, a concrete program entity, an event pattern that must appear in the trajectory, and a deterministic satisfaction predicate evaluated against the runtime evidence state. The global evidence gap (all unsatisfied conditions) is rendered into the agent's context as guidance; the action-specific gap (unsatisfied conditions that apply to the proposed action) is the gate: a commitment proceeds only if it is empty. The same structured conditions are evaluated at every step, which the paper argues beats compressing the remaining evidence i

Load-bearing premise

The load-bearing premise is that a deterministic check on parseable trajectory events can stand in for 'the agent has understood the relevant evidence' — plus the corollary that conditions that cannot be grounded to concrete repository entities are safely droppable.

What would settle it

Find or construct a set of issues where the correct patch requires understanding that is not directly observable from any single command output (e.g., a subtle interaction between two functions that the agent never reads together). If ECLoop shows no gain over baseline on this subset — while still gaining on other subsets — the benefit comes from forcing generic extra exploration rather than from verifying task-specific evidence. A simpler check: on a sample of 100 issues, record for every held action the specific condition that was unmet, and have independent annotators judge whether that con

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • ECLoop's gains hold across two language models and two agent scaffolds, suggesting evidence-gating is a portable layer rather than a fix for one specific agent.
  • The gate does not just add checks; it reduces total token use, because holding unsupported actions shortens unproductive trajectories.
  • Structured per-condition evidence tracking outperforms an equivalent natural-language summary of what is missing, arguing for machine-checkable evidence states in agent execution.
  • Post hoc self-refinement (having the model critique its own patch) does not recover from premature commitment, while pre-execution gating does; this redirects effort from correction to prevention.
  • Each of ECLoop's three operations — guidance, evidence-state update, and commitment check — contributes independently, so the approach is modular and can be adopted incrementally.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same evidence-gating idea could apply beyond code repair: any agent taking consequential actions (making a purchase, issuing a command, publishing a conclusion) could be required to demonstrate task-specific evidence first.
  • Because ECLoop's conditions are grounded through AST, call-graph, and class-hierarchy resolution, its coverage is bounded by static analysis; a dynamic variant that learns evidence requirements from execution traces or from the agent's own successful trajectories might handle weakly-typed or reflective code better.
  • The regressions ECLoop incurs (on roughly 9–16 previously-correct instances per configuration, out of 500) suggest a tunable risk: a learned or adaptive hold budget, or a confidence-aware release, could reduce those collateral failures.
  • The paper's shared-model limitation implies that using a stronger or independent model for evidence compilation and satisfaction assessment could widen the gap between ECLoop and baseline — an inexpensive experiment since the evidence operations are already isolated from the main agent.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ECLoop proposes an execution layer for LLM coding agents that, once per task, compiles an evidence specification Cq from the issue and repository, grounds conditions to concrete program entities, deterministically evaluates satisfaction against parsed trajectory events, and gates commitment actions (edits and final submission) until the action-specific evidence gap is empty, while feeding the global gap back as guidance. The empirical section reports Pass@1 on all 500 SWE-bench Verified instances for GPT-5-mini and MiniMax-M2.5 on mini-swe-agent v2 and Codex CLI, with improvements of 4.8–11.8 percentage points, statistically significant under exact McNemar tests, plus token and cost reductions. Ablations decompose the contributions of guidance, the commitment check, and the evidence-state update, and compare structured conditions against natural-language summaries.

Significance. The contribution is potentially significant: it isolates a distinct failure mode (premature commitment), proposes a lightweight intervention orthogonal to the agent's model and scaffold, and evaluates on a large paired benchmark. The controlled comparison is a notable strength: baseline and ECLoop share model, prompt, tools, and scaffold, and the deterministic satisfaction checks are an attempt to avoid relying on the model's self-report. The full 500-instance evaluation across two models and two scaffolds, with token and cost accounting, strengthens the empirical claims. However, the central mechanism's validity rests on the assumption that observable trajectory events (file views, command executions) are a faithful proxy for evidence actually acquired, and the paper's own Limitations section concedes gaps in this area. If the proxy is validated and the statistical robustness is improved, this could be a solid contribution to software engineering AI agents.

major comments (4)
  1. [§3.1, Eq. (3); §3.2–3.3, Eqs. (4)–(10)] The gate's entire signal is sat_i(Zt), computed from parsed command/output events rather than from any check that the model processed or understood the evidence. The paper concedes 'the agent may still misinterpret that evidence' (§3.2) and that conditions that cannot be grounded to concrete repository entities are removed (Eq. 3). Thus the mechanism enforces an observable-investigation proxy, not demonstrated understanding. The reported gains on well-specified SWE-bench issues may not transfer to vague real-world issues where critical conditions are dropped and premature commitments pass unchecked. Please provide a direct validation of the proxy, e.g., human annotation comparing trajectory events with actual evidence acquired, or a variant where satisfaction is verified by a stronger/independent model; also quantify how often Eq. (3) drops conditions on SWE-bench and on a noisier issue
  2. [§4, Tables 1–2; §4.1] Pass@1 is reported as single-run point estimates without confidence intervals or multi-seed variance. The exact McNemar tests establish paired-difference significance for that one run, but the magnitude claims (4.8–11.8 pp) and RRU values are functions of a single stochastic trajectory per instance. Given model sampling temperature, a rerun could shift these numbers, and the smaller gains (4.8 and 5.0 pp) are especially vulnerable. Please report bootstrap confidence intervals or multiple seeds for at least the GPT-5-mini configurations, and report per-instance variance.
  3. [§3.3, end; §4.1–4.2] The hold budget is a free parameter fixed at three with no sensitivity analysis, yet the paper attributes the 9–16 regressions to budget exhaustion. The operating point determines how often the fallback releases under-supported actions and thus directly affects the accuracy/efficiency trade-off. Please vary the budget (e.g., 1, 2, 3, 5) and report Pass@1, regression counts, and token usage. This is needed to establish that the gains are not an artifact of this particular choice.
  4. [§4.4, Table 4] The ablation is conducted on a fixed 100-instance subset with a single run and no significance testing. Differences of 5–10 pp among configurations are within plausible sampling noise at n=100. Please report confidence intervals or multiple runs, and ideally run the ablation on the full 500 instances to align with the main claim.
minor comments (5)
  1. [Abstract; §4.3, Table 3] 'No additional inference cost' is potentially misleading because Table 3 includes the per-task specification-compilation call. State explicitly that reported totals include this call and clarify what 'inference cost' excludes.
  2. [§3.1, Eq. (2)] The notation vt denotes the concrete program entity, but some conditions are resolved only when the agent proposes an action; the formal definition of Ct and the timing of grounding should be clarified.
  3. [§3.3] The behavior of a 'held' edit is underspecified: is the proposed action queued, discarded, or revised? How does the fallback interact with the trajectory? Please specify.
  4. [Figure 1] The figure caption and surrounding text are difficult to parse. Please redraw the diagram and make the caption self-contained.
  5. [Table 4] The 'spec→natural language' row is only described as replacing the structured specification. Clarify whether the natural-language summary is also used for satisfaction checking or only for guidance.

Circularity Check

0 steps flagged

No significant circularity: the central claim is an external benchmark result, the gate is deterministic over trajectory events, and the one self-citation is not load-bearing.

full rationale

ECLoop's empirical claim is evaluated on all 500 SWE-bench Verified instances against held-out tests, not derived from its own equations or from a fitted parameter. The evidence specification Cq is generated by the same LLM that drives the agent, but satisfaction is deterministic: Eq. (2)-(5) compute sat_i(Zt) from parsed trajectory events, and Eqs. (9)-(10) gate by whether the action-specific gap is empty. Thus the reported Pass@1 improvement is not equivalent by construction to the input specification. The only self-citation (Guo et al. 2026a in Section 3.1) supports a representation choice (structured specifications), not the central premise, and is not load-bearing. The acknowledged limitations—'Dependence on issue quality' and 'Shared-model evidence operations'—are genuine validity concerns: the gate may rest on a proxy for evidence and may miss conditions on vague issues, and the same model both proposes conditions and drives the agent. But these are threats to generalization and construct validity, not circularity: no step in the derivation chain makes the output equal to its input. The paper is therefore self-contained against an external benchmark for its central claim.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The core premise is that evidence conditions can be compiled from the issue, grounded to concrete entities via repository structure, and evaluated deterministically from parsed trajectory events. These are engineering assumptions with acknowledged failure modes, not mathematical axioms. No new physical or natural-kind entities are posited.

free parameters (1)
  • hold_budget = 3 (fixed)
    Commitment actions are postponed at most three times for the same target before an audited fallback releases them. No sensitivity analysis is reported, and regressions in §4.1 are attributed to exhaustion of this budget.
axioms (4)
  • domain assumption Observed trajectory events faithfully represent the evidence the agent has gathered.
    Eqs. (4)-(5) define Zt = Observe(τt) and sat_i(Zt); a viewed location or executed diagnostic counts as satisfying inspection/reproduction conditions. The paper concedes recorded evidence may still be misinterpreted.
  • ad hoc to paper Conditions that cannot be grounded to concrete repository entities can be safely dropped.
    Eq. (3) returns ⊥ for ungroundable conditions and they are removed from the specification. If key evidence references cannot be resolved, the gate cannot enforce them.
  • domain assumption Repository structure (AST, call graph, class hierarchy) resolves all relevant entities referenced by conditions.
    Section 3.1 uses these analyses to resolve abstract roles like 'the function to be edited' and 'its callers'; incomplete call graphs or dynamic dispatch would mis-ground conditions.
  • domain assumption SWE-bench Verified Pass@1 is a valid proxy for coding-agent repair performance.
    Standard benchmark assumption; the paper uses all 500 instances and exact McNemar tests for significance.

pith-pipeline@v1.3.0-alltime-deepseek · 11993 in / 10904 out tokens · 119279 ms · 2026-08-03T00:17:34.516947+00:00 · methodology

0 comments
read the original abstract

LLM-based coding agents often edit source code or submit patches before examining enough repository evidence to justify the change, a failure pattern we call premature commitment. We present ECLoop, an execution layer that interposes between the agent and the repository to enforce evidence-conditioned execution. For each task, ECLoop uses the issue description and repository structure to compile a set of conditions specifying what the agent should observe before each type of code modification or patch submission. During execution, ECLoop tracks which conditions the agent's runtime trajectory has satisfied and postpones any proposed action whose required conditions remain unmet. Evaluated on all 500 instances of SWE-bench Verified with two language models and two agent scaffolds, ECLoop raises Pass@1 by 4.8-11.8 percentage points without model retraining or scaffold changes. Ablation experiments show that each of ECLoop's three operations contributes distinct value and that structured evidence conditions outperform an equivalent natural-language summary. These gains come at no additional inference cost: by redirecting the agent before it pursues unsupported actions, ECLoop lowers average token consumption by up to 12.1%.

Figures

Figures reproduced from arXiv: 2607.28815 by Chenglin Li, Jinqiu Yang, Tse-Hsun Chen, Yisen Xu, Zehao Wang.

Figure 1
Figure 1. Figure 1: ECLoop overview. ECLoop interposes between the coding agent (left) and the repository: (1) compiling evidence conditions from the issue once per task, (2) gating commitment actions whose action-specific evidence gap remains nonempty, and (3) returning the global evidence gap as guidance until all conditions are met. Information-gathering actions proceed freely. task-specific evidence needed to justify an e… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 5 canonical work pages · 3 internal anchors

  1. [1]

    Manning and Hinrich Sch

    Christopher D. Manning and Hinrich Sch. Foundations of statistical natural language processing , publisher =. 1999 , isbn =

    1999

  2. [2]

    Psychometrika , year =

    McNemar, Quinn , title =. Psychometrika , year =. doi:10.1007/BF02295996 , url =

    doi:10.1007/bf02295996

  3. [3]

    CoRR , volume =

    Rakshith Vasudev and Melisa Russak and Dan Bikel and Waseem AlShikh , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.03338 , eprinttype =. 2602.03338 , timestamp =

    doi:10.48550/arxiv.2602.03338 2026

  4. [4]

    CoRR , volume =

    Chung. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.09252 , eprinttype =. 2605.09252 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.09252 2026

  5. [5]

    CoRR , volume =

    Hao Han and Jin Xie and Xuehao Ma and Weiquan Zhu and Ziyao Zhang and ZhiLiang Long and Hongkai Chen and Qingwen Ye , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.14820 , eprinttype =. 2604.14820 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.14820 2026

  6. [6]

    AgentPRM: Process Reward Models for

    Zhiheng Xi and Chenyang Liao and Guanyu Li and Zhihao Zhang and Wenxiang Chen and Binghai Wang and Senjie Jin and Yuhao Zhou and Jian Guan and Wei Wu and Tao Ji and Tao Gui and Qi Zhang and Xuanjing Huang , editor =. AgentPRM: Process Reward Models for. Proceedings of the. 2026 , url =. doi:10.1145/3774904.3792551 , timestamp =

    arXiv 2026

  7. [7]

    From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

    Yuhao Sun and Jiacheng Zhang and Shaanan Cohney and Zhexin Zhang and Feng Liu and Xingliang Yuan , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.05805 , eprinttype =. 2606.05805 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.05805 2026

  8. [8]

    NeMo Guardrails:

    Traian Rebedea and Razvan Dinu and Makesh Narsimhan Sreedhar and Christopher Parisien and Jonathan Cohen , editor =. NeMo Guardrails:. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,. 2023 , url =. doi:10.18653/V1/2023.EMNLP-DEMO.40 , timestamp =

    doi:10.18653/v1/2023.emnlp-demo.40 2023

  9. [9]

    2025 , eprint=

    MI9: An Integrated Runtime Governance Framework for Agentic AI , author=. 2025 , eprint=

    2025

  10. [10]

    GuardAgent: Safeguard

    Zhen Xiang and Linzhi Zheng and Yanjie Li and Junyuan Hong and Qinbin Li and Han Xie and Jiawei Zhang and Zidi Xiong and Chulin Xie and Carl Yang and Dawn Song and Bo Li , editor =. GuardAgent: Safeguard. Forty-second International Conference on Machine Learning,. 2025 , url =

    2025

  11. [11]

    Weidi Luo and Shenghong Dai and Xiaogeng Liu and Suman Banerjee and Huan Sun and Muhao Chen and Chaowei Xiao , editor =. AGrail:. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),. 2025 , url =. doi:10.18653/V1/2025.ACL-LONG.399 , timestamp =

    doi:10.18653/v1/2025.acl-long.399 2025

  12. [12]

    Le , title =

    Lesly Miculicich and Mihir Parmar and Hamid Palangi and Krishnamurthy Dj Dvijotham and Mirko Montanari and Tomas Pfister and Long T. Le , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2510.05156 , eprinttype =. 2510.05156 , timestamp =

    doi:10.48550/arxiv.2510.05156 2025

  13. [13]

    2026 , eprint=

    ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety , author=. 2026 , eprint=

    2026

  14. [14]

    CoRR , volume =

    Adharsh Kamath and Sishen Zhang and Calvin Xu and Shubham Ugare and Gagandeep Singh and Sasa Misailovic , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.23738 , eprinttype =. 2512.23738 , timestamp =

    doi:10.48550/arxiv.2512.23738 2025

  15. [15]

    2026 , eprint=

    ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports , author=. 2026 , eprint=

    2026

  16. [16]

    Reproduction Test Generation for Java SWE Issues

    Toufique Ahmed and Jatin Ganhotra and Avraham Shinnar and Martin Hirzel , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.04320 , eprinttype =. 2605.04320 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.04320 2026

  17. [17]

    Otter: Generating Tests from Issues to Validate

    Toufique Ahmed and Jatin Ganhotra and Rangeet Pan and Avraham Shinnar and Saurabh Sinha and Martin Hirzel , editor =. Otter: Generating Tests from Issues to Validate. Forty-second International Conference on Machine Learning,. 2025 , url =

    2025

  18. [18]

    SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents , booktitle =

    Niels M. SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents , booktitle =. 2024 , url =

    2024

  19. [19]

    2026 , eprint=

    SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests , author=. 2026 , eprint=

    2026

  20. [20]

    Abstain and Validate:

    Jos. Abstain and Validate:. Proceedings of the. 2026 , url =. doi:10.1145/3786583.3786858 , timestamp =

    arXiv 2026

  21. [21]

    CoRR , volume =

    Anvith Pabba and Simin Chen and Alex Mathai and Anindya Chakraborty and Baishakhi Ray , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2510.03588 , eprinttype =. 2510.03588 , timestamp =

    doi:10.48550/arxiv.2510.03588 2025

  22. [22]

    CoRR , volume =

    Shaoqiu Zhang and Yuhang Wang and Jialiang Liang and Yuling Shi and Wenhao Zeng and Maoquan Wang and Shilin He and Ningyuan Xu and Siyu Ye and Kai Cai and Xiaodong Gu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.07297 , eprinttype =. 2606.07297 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2606.07297 2026

  23. [23]

    Coding Agents Don't Know When to Act

    Thibaud Gloaguen and Niels M. Coding Agents Don't Know When to Act , journal =. 2026 , url =. doi:10.48550/ARXIV.2605.07769 , eprinttype =. 2605.07769 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.07769 2026

  24. [24]

    Self-Refine: Iterative Refinement with Self-Feedback , booktitle =

    Aman Madaan and Niket Tandon and Prakhar Gupta and Skyler Hallinan and Luyu Gao and Sarah Wiegreffe and Uri Alon and Nouha Dziri and Shrimai Prabhumoye and Yiming Yang and Shashank Gupta and Bodhisattwa Prasad Majumder and Katherine Hermann and Sean Welleck and Amir Yazdanbakhsh and Peter Clark , editor =. Self-Refine: Iterative Refinement with Self-Feedb...

    2023

  25. [25]

    CoRR , volume =

    Qianru Meng and Xiao Zhang and Zhaochun Ren and Joost Visser , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.27238 , eprinttype =. 2605.27238 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.27238 2026

  26. [26]

    Reflexion: language agents with verbal reinforcement learning , booktitle =

    Noah Shinn and Federico Cassano and Ashwin Gopinath and Karthik Narasimhan and Shunyu Yao , editor =. Reflexion: language agents with verbal reinforcement learning , booktitle =. 2023 , url =

    2023

  27. [27]

    Weld , title =

    Daniel S. Weld , title =. 1994 , url =. doi:10.1609/AIMAG.V15I4.1109 , timestamp =

    work page doi:10.1609/aimag.v15i4.1109 1994

  28. [28]

    Narasimhan and Yuan Cao , title =

    Shunyu Yao and Jeffrey Zhao and Dian Yu and Nan Du and Izhak Shafran and Karthik R. Narasimhan and Yuan Cao , title =. The Eleventh International Conference on Learning Representations,. 2023 , url =

    2023

  29. [29]

    Devanbu and Michael Pradel , title =

    Islem Bouzenia and Premkumar T. Devanbu and Michael Pradel , title =. 47th. 2025 , url =. doi:10.1109/ICSE55347.2025.00157 , timestamp =

    arXiv 2025

  30. [30]

    Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H

    Xingyao Wang and Boxuan Li and Yufan Song and Frank F. Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H. Tran and Fuqiang Li and Ren Ma and Mingzhang Zheng and Bill Qian and Yanjun Shao and Niklas Muennighoff and Yizhe Zhang and Binyuan Hui and Junyang Lin , title =. The Thirteenth Internation...

    2025

  31. [31]

    2026 , eprint=

    Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study , author=. 2026 , eprint=

    2026

  32. [32]

    CoRR , volume =

    Oorja Majgaonkar and Zhiwei Fei and Xiang Li and Federica Sarro and He Ye , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2511.00197 , eprinttype =. 2511.00197 , timestamp =

    doi:10.48550/arxiv.2511.00197 2025

  33. [33]

    Chunqiu Steven Xia and Yinlin Deng and Soren Dunn and Lingming Zhang , title =. Proc. 2025 , url =. doi:10.1145/3715754 , timestamp =

    doi:10.1145/3715754 2025

  34. [34]

    AutoCodeRover: Autonomous Program Improvement , booktitle =

    Yuntong Zhang and Haifeng Ruan and Zhiyu Fan and Abhik Roychoudhury , editor =. AutoCodeRover: Autonomous Program Improvement , booktitle =. 2024 , url =. doi:10.1145/3650212.3680384 , timestamp =

    arXiv 2024

  35. [35]

    Agent-SAMA: State-Aware Mobile Assistant , booktitle =

    Linqiang Guo and Wei Liu and Yi Wen Heng and Tse. Agent-SAMA: State-Aware Mobile Assistant , booktitle =. 2026 , url =. doi:10.1609/AAAI.V40I35.40187 , timestamp =

    work page doi:10.1609/aaai.v40i35.40187 2026

  36. [36]

    Jimenez and Alexander Wettig and Kilian Lieret and Shunyu Yao and Karthik Narasimhan and Ofir Press , editor =

    John Yang and Carlos E. Jimenez and Alexander Wettig and Kilian Lieret and Shunyu Yao and Karthik Narasimhan and Ofir Press , editor =. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering , booktitle =. 2024 , url =

    2024

  37. [37]

    2026 , howpublished =

    Minimax , title =. 2026 , howpublished =

    2026

  38. [38]

    2026 , howpublished =

    OpenAI , title =. 2026 , howpublished =

    2026

  39. [39]

    CoRR , volume =

    OpenAI , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2601.03267 , eprinttype =. 2601.03267 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.03267 2026

  40. [40]

    Poskitt and Jun Sun , title =

    Haoyu Wang and Christopher M. Poskitt and Jun Sun , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2503.18666 , eprinttype =. 2503.18666 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.18666 2025

  41. [41]

    2024 , url=

    Carlos E Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik R Narasimhan , booktitle=. 2024 , url=

    2024

  42. [42]

    and Narasimhan, Karthik , year =

    Chowdhury, Neil and Yang, John and Jimenez, Carlos E. and Narasimhan, Karthik , year =. 2412.12345 , archivePrefix =

    Pith/arXiv arXiv

  43. [43]

    and Wettig, Alexander and Lieret, Kilian and Yao, Shunyu and Narasimhan, Karthik and Press, Ofir , title =

    Yang, John and Jimenez, Carlos E. and Wettig, Alexander and Lieret, Kilian and Yao, Shunyu and Narasimhan, Karthik and Press, Ofir , title =. Advances in Neural Information Processing Systems

  44. [44]

    Gauthier, Paul , year =. Aider:

  45. [45]

    IEEE Transactions on Software Engineering , volume =

    Gazzola, Luca and Micucci, Daniela and Mariani, Leonardo , title =. IEEE Transactions on Software Engineering , volume =

  46. [46]

    IEEE Transactions on Software Engineering , volume =

    Le Goues, Claire and Nguyen, ThanhVu and Forrest, Stephanie and Weimer, Westley , title =. IEEE Transactions on Software Engineering , volume =

  47. [47]

    Proceedings of the International Symposium on Software Testing and Analysis , year =

    Xia, Chunqiu Steven and Zhang, Lingming , title =. Proceedings of the International Symposium on Software Testing and Analysis , year =

  48. [48]

    Computer , volume =

    Meyer, Bertrand , title =. Computer , volume =

  49. [49]

    Journal of Logic and Algebraic Programming , volume =

    Leucker, Martin and Schallhart, Christian , title =. Journal of Logic and Algebraic Programming , volume =

  50. [50]

    2025 , howpublished =

    Codex , author =. 2025 , howpublished =

    2025

  51. [51]

    2025 , howpublished =

    Claude Code , author =. 2025 , howpublished =

    2025

  52. [52]

    2210.03629 , archivePrefix =

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , year =. 2210.03629 , archivePrefix =

    Pith/arXiv arXiv

  53. [53]

    2407.16741 , archivePrefix =

    Wang, Xingyao and others , year =. 2407.16741 , archivePrefix =

    Pith/arXiv arXiv

  54. [54]

    2024 , eprint =

    Large Language Monkeys: Scaling Inference Compute with Repeated Sampling , author =. 2024 , eprint =

    2024

  55. [55]

    Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , year =. Scaling. 2408.03314 , archivePrefix =

    Pith/arXiv arXiv

  56. [56]

    and Krauser, Edward W

    Agrawal, Gail and Horgan, Joseph R. and Krauser, Edward W. and London, Saul A. , title =. Proceedings of the Conference on Software Maintenance , pages =

  57. [57]

    2404.05427 , archivePrefix =

    Zhang, Yuntong and Ruan, Haifeng and Fan, Zhiyu and Roychoudhury, Abhik , year =. 2404.05427 , archivePrefix =

    Pith/arXiv arXiv

  58. [58]

    , title =

    Horty, John F. , title =. Oxford University Press , year =

  59. [59]

    Competition-Level Code Generation with

    Li, Yujia and others , year =. Competition-Level Code Generation with. 2203.07814 , archivePrefix =

    Pith/arXiv arXiv