Pith. sign in

REVIEW 3 major objections 4 minor 8 cited by

Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that reinforcement learning from verifiable rewards can be made effective for software-engineering agents by adding agent guidance, lifting pass@1 on SWE-Bench Verified from 9.4% to 22.4% for a 72B open model, with the…

desk verdict Useful empirical recipe, but the 'guidance is critical' claim is compromised by gold-patch leakage; needs an oracle-free ablation before the mechanism is credible. read the letter →

arxiv 2506.11425 v2 pith:KRCLUOGP submitted 2025-06-13 cs.CL cs.AI

Jeff Da , Clinton Wang , Xiang Deng , Yuntao Ma , Nikhil Barhate , Sean Hendryx This is my paper
classification cs.CLcs.AI
keywords reinforcementlearningfromverifiablerewardssoftwareengineeringagentsagentguidancedirectpreferenceoptimizationSWE-BenchVerifiedtest-timerewardmodelcoderepairsparse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that reinforcement learning from verifiable rewards (RLVR), which works well for math and competitive coding, fails in agentic settings because unit-test rewards are too sparse across long multi-step trajectories. To fix this, the authors introduce Agent-RLVR, a training loop in which the agent first attempts a task, then reattempts it with 'agent guidance' — hints generated from the gold patch, the stack trace, and file locations — and the policy is updated by DPO on passing-versus-failing trajectory pairs. The reported result is that this raises pass@1 on SWE-Bench Verified from 9.4% to 22.4% for Qwen-2.5-72B-Instruct, and that guidance itself is a critical component, improving pass@1 by 2.6 points and pass@32 by 4.2 points over the same loop without guidance. The same guided RLVR data also trains a reward model that ranks 32 patches and pushes pass@1 to 27.8%. If correct, the method shows that a mid-size open model, trained on just 817 environments with verifiable rewards and oracle-derived hints, can approach agents trained on orders of magnitude more data.

What carries the argument

The central mechanism is agent guidance, a set of three informational cues — a plan, environment feedback, and environment interaction (file/location hints) — generated by an external LLM from the gold patch and the failing trajectory, and appended to the prompt when the agent reattempts a task during training. The framework combines this with an RLVR loop: initial rollouts are graded by unit tests, failed tasks are reattempted with guidance, passing trajectories are paired with failing ones for DPO, and 20% of positive trajectories seed an SFT reference policy. Guidance's role is to increase the number of successful rollouts (from 16.9% to 20.3%) and reduce reward sparsity, giving the DPO learner informative preference pairs rather than a near-empty success set.

What would settle it

Train the same Agent-RLVR loop with guidance generated from a deliberately corrupted patch, or from the problem statement alone with the reference patch withheld, and compare pass@1 on SWE-Bench Verified: if the score falls back toward the 19.8% no-guidance level, the gold-patch content is the load-bearing ingredient. A complementary check is to evaluate the trained agent on a fresh set of issues whose reference patches were never available during training and confirm the gain persists.

Watch

Extended reading notes

Core claim

Agent-RLVR is a framework for making RLVR effective in agentic software engineering. The paper's central claim is that sparse environment rewards can be made dense enough for RL training by injecting agent guidance during trajectory rollouts: before the agent reattempts a failed task, an external LLM produces a plan, feedback on the environment error, and file/location hints from the reference patch and the failing stack trace. These guided trajectories, paired with unguided failures, become DPO preference data, and a short SFT phase on successful trajectories initializes the reference policy. The authors report that this pipeline lifts Qwen-2.5-72B-Instruct from 9.4% to 22.4% pass@1 on SWE-Bench Verified, that the guidance component is responsible for a meaningful part of the gain (22.4% vs 19.8% at pass@1; 38.4% vs 34.2% at pass@32), and that the guided data trains a reward model which, ranking 32 sampled patches, reaches 27.8% pass@1. The guidance is used only during training; at test time inference is unchanged.

Load-bearing premise

The guidance generator sees the reference (correct) patch for every training task, and the hints it writes from that patch are what teach the policy behaviors that transfer to test time, where no hints are given; if the policy instead latches onto hint-shaped instructions or gold-patch-specific cues, the reported gains would not generalize to new issues.

Editorial extensions

If this is right

  • A 72B open-weights model trained on 817 environments with verifiable rewards plus guidance reaches 22.4% pass@1 on SWE-Bench Verified, roughly matching or exceeding prior open models trained with far more data and specialized scaffolds.
  • The guidance-vs-no-guidance gap grows with pass@k (22.4% vs 19.8% at pass@1; 38.4% vs 34.2% at pass@32), implying guidance improves both accuracy and generation diversity, not just the greedy trajectory.
  • The same RLVR dataset doubles as training data for a test-time reward model; ranking 32 generated patches raises pass@1 to 27.8%, so guided RLVR data has reuse value beyond policy training.
  • Because the scaffold is simplified (localization and repair only, no expensive test-generation selection step), the gains are attributable to the training method rather than to a stronger agent harness.
  • Performance scales with model size (14B: 18.0%, 32B: 21.6%, 72B: 22.4%), suggesting the method's benefit compounds with base-model capability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If gold-patch-derived hints are what make guidance work, a natural extension is to generate guidance from the agent's own successful trajectories or from repeated environment feedback alone, which would let the method apply to tasks with no reference patch (e.g., newly filed issues).
  • The same reattempt-with-hints loop should transfer to other agentic domains with verifiable outcomes — web navigation, tool use, or computer use — where reward sparsity is the primary obstacle to RLVR.
  • The reward model trained on guided data could be used as a dense reward signal in an online RL iteration (e.g., GRPO), potentially amplifying the offline gains beyond the current 22.4%–27.8% range.
  • The widening guidance gap at higher pass@k suggests a measurable signature: guidance-trained models should produce more diverse patch sets. A simple test is to measure pairwise patch dissimilarity in the top-k generations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Agent-RLVR, a framework for training software engineering agents with reinforcement learning from verifiable rewards (unit tests) augmented by 'agent guidance' hints generated by an external LLM. The training loop first samples trajectories, then for failed tasks generates three types of guidance (plan, environment feedback, environment interaction) and reattempts; preference pairs from correct/incorrect trajectories are used for SFT and DPO. On SWE-Bench Verified, Agent-RLVR improves pass@1 from 9.4% to 22.4% for Qwen-2.5-72B-Instruct, and a reward model trained on the same data further boosts pass@1 to 27.8%. The authors claim guidance is a critical component, evidenced by a guidance vs no-guidance gap (22.4% vs 19.8% pass@1 and widening at higher pass@k).

Significance. If the empirical claims hold, the paper demonstrates a practical and sample-efficient recipe (817 training environments) for turning a mid-size open-weight model into a substantially better SWE agent using only environment rewards plus auxiliary hint data, and shows the same RLVR trajectories are reusable for test-time reward modeling. The evaluation is on an external benchmark with standard deviations and multiple ablations (guidance vs no guidance, SFT baseline, model-size scaling), which is a strength. However, the central attribution of the gains to the guidance mechanism is weakened by the fact that the guidance is generated with access to the reference patch, so the 'guidance' condition includes oracle-derived solution information. The distinction is material to the claim that the method transfers to test-time inference.

major comments (3)
  1. [Section 2.2.2, Figure 3] The guidance generation prompt explicitly includes the reference patch (PATCH: {patch}), and the environment-interaction hint is described as pointing to the correct file and patch location. Consequently, every guidance-augmented trajectory in the training set is conditioned on a hint that encodes the correct answer. The ablation in Section 3.2 (Figure 2) compares guidance vs no-guidance DPO, but the guidance condition differs not only in the presence of a teaching signal but also in the presence of gold-patch information. This confound means the reported +2.6 point pass@1 difference (22.4% vs 19.8%) and the widening pass@k gap may reflect distillation of the answer key rather than transferable guidance. To support the paper's central claim, the authors should include an ablation in which guidance is generated without access to the reference patch (e.g., using only the problem statement and the agent's own previous trajectory and environment feedback), or otherwise control for the information content of the hint.
  2. [Algorithm 1, lines 16-21] The DPO pairs added for guided trajectories are (d_i, t'_i, t_i), where t'_i is generated under the guidance-augmented prompt (d_i, g_i) and t_i is the original unguided trajectory. The DPO loss in Eq. (2) assumes y_w and y_l are responses to the same input x; here the winning response was sampled from a different conditional distribution than the losing response. The manuscript does not explain whether the loss is computed with x=d_i (ignoring the guidance) or x=(d_i,g_i), and if the former, the preference is not a valid comparison of two responses to the same prompt. This mismatch could bias the policy update, and the paper should clarify the exact pairing and justify it, or modify the algorithm to pair trajectories generated under the same prompt.
  3. [Section 3.2, Table 4] The paper's own SFT ablation shows that training with guidance-augmented trajectories hurts SFT performance substantially (16.8% vs 20.8% pass@1), which the authors attribute to overfitting to the guidance prompt. This result underscores the train/test prompt mismatch that arises because guidance is present only during training. Without an experiment that varies the guidance information content or that explicitly tests the model's sensitivity to hint-like prompts at inference, the reader cannot distinguish between a transferable guidance effect and a spurious dependence on the oracle-derived hint format. Table 2's claim that guidance 'reduces reward landscape sparsity' is similarly a direct consequence of adding the answer to the prompt, rather than evidence that the environment reward is made less sparse.
minor comments (4)
  1. [Section 2.2.3] In the dataset description, the total is given as 817 environments, but the sum of the stated components (593 SWE-Gym + 219 self-collected) is 812; please reconcile this numerical inconsistency.
  2. [Section 5.1] In the SWE-Agent description, 'iteract' should be 'interact'.
  3. [Throughout] The notation for pass@k is inconsistent (PASS@1, pass@1, PASS@k); please unify capitalization for clarity.
  4. [Section 3.2] The description of the reward model says it is trained on the same RLVR data, but it is not stated whether guidance-augmented trajectories are included and whether the same oracle-patch concern applies; a brief clarification would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all reported pass@1 numbers are produced by an external benchmark (SWE-Bench Verified), and the guidance-vs-no-guidance comparison, while confounded by gold-patch access during training, is an empirical result rather than a construction.

full rationale

The paper contains no mathematical derivation whose output is equivalent to its input by construction. The trained models are evaluated with the held-out, human-validated SWE-Bench Verified harness, and the final rewards are external unit tests, so the headline pass@1 gains (9.4% to 22.4%, 27.8% with the reward model) are not determined by the training inputs. It is true that guidance generation has access to the reference patch (Section 2.2.2; Figure 3's PATCH: {patch}), and that DPO pairs use trajectories generated under that guidance (Algorithm 1, lines 16-20), creating a train/test prompt mismatch and a legitimate oracle-leakage confound for the 'guidance is critical' claim (19.8% vs 22.4%). However, that is an experimental-validity concern, not a circularity: the test-time inference removes the hint, the benchmark is not the training set, and the SFT ablation (Table 4) shows that training on guidance-augmented data can even hurt, so the RLVR guidance result is not tautologically forced. The self-citations (refs 8, 17, 27) are used only for motivation, contamination caveats, and related-work context; none is a load-bearing justification or an imported uniqueness/ansatz result. Under the strict rule that circularity requires an exhibited reduction of the claimed prediction to its own inputs, no such step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. The free parameters are standard training hyperparameters, and the axioms are domain assumptions about the evaluation and the guidance mechanism. The main conceptual addition, 'agent guidance', is a training procedure rather than an invented entity.

free parameters (4)
  • DPO beta (beta) = not reported
    KL penalty coefficient in the DPO objective (Eq. 2); chosen without reporting a value, but the result is not sensitive to a specific fitted constant.
  • SFT learning rate = 1e-5
    Learning rate for instruct-tuning; the paper states all models are instruct-tuned for five epochs with this rate, and the SFT baseline was tuned via grid search.
  • DPO learning rate = 1e-6
    Learning rate for one epoch of DPO training; a standard hyperparameter, not a fit to the central claim.
  • Reward model training steps = 500
    Number of training steps for the reward model, chosen empirically and reported in Section 3.1.
assumptions (5)
  • domain assumption Unit tests in the training set and SWE-Bench Verified are a reliable proxy for patch correctness.
    The entire reward signal is based on passing unit tests, which may not capture all correctness aspects but is the standard for SWE benchmarks.
  • domain assumption The reference patch used in guidance generation is available and accurately resolves the issue.
    Guidance generation in Figure 3 uses PATCH: {patch}, so the method requires the gold patch for every training task.
  • domain assumption The training environments (SWE-Gym and self-collected) do not overlap with SWE-Bench Verified repositories.
    The authors state they are careful to avoid overlap, but this is a claim about dataset construction that cannot be fully verified from the paper.
  • domain assumption The external LLM (claude-3-7-sonnet) produces guidance that improves agent success without teaching test-time-unavailable cues.
    The effectiveness of the method rests on the guidance being useful and not causing overfitting to hint-like instructions.
  • standard math Standard DPO/RLVR objective (Eq. 2) is a valid optimization target.
    The paper uses the standard DPO loss from Rafailov et al.; this is a known result, not a new derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards." pith.science (2026). https://pith.science/paper/KRCLUOGP

@misc{pith2026250611425,
  author       = {Pith},
  title        = {Pith review of: Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KRCLUOGP}},
  note         = {Machine review of arXiv:2506.11425}
}
read the original abstract

Reinforcement Learning from Verifiable Rewards (RLVR) has been widely adopted as the de facto method for enhancing the reasoning capabilities of large language models and has demonstrated notable success in verifiable domains like math and competitive programming tasks. However, the efficacy of RLVR diminishes significantly when applied to agentic environments. These settings, characterized by multi-step, complex problem solving, lead to high failure rates even for frontier LLMs, as the reward landscape is too sparse for effective model training via conventional RLVR. In this work, we introduce Agent-RLVR, a framework that makes RLVR effective in challenging agentic settings, with an initial focus on software engineering tasks. Inspired by human pedagogy, Agent-RLVR introduces agent guidance, a mechanism that actively steers the agent towards successful trajectories by leveraging diverse informational cues. These cues, ranging from high-level strategic plans to dynamic feedback on the agent's errors and environmental interactions, emulate a teacher's guidance, enabling the agent to navigate difficult solution spaces and promotes active self-improvement via additional environment exploration. In the Agent-RLVR training loop, agents first attempt to solve tasks to produce initial trajectories, which are then validated by unit tests and supplemented with agent guidance. Agents then reattempt with guidance, and the agent policy is updated with RLVR based on the rewards of these guided trajectories. Agent-RLVR elevates the pass@1 performance of Qwen-2.5-72B-Instruct from 9.4% to 22.4% on SWE-Bench Verified. We find that our guidance-augmented RLVR data is additionally useful for test-time reward model training, shown by further boosting pass@1 to 27.8%. Agent-RLVR lays the groundwork for training agents with RLVR in complex, real-world environments where conventional RL methods struggle.

Figures

Figures reproduced from arXiv: 2506.11425 by the authors.

Figure 1
Figure 1. Agent-RLVR is a framework for training agents with RLVR using environment feedback and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. We compare PASS@k (k = 2[0,5]) between the guidance and non-guidance ver￾sions of Agent-RLVR. We find that the guidance￾augmented method performance is higher at PASS@1 and that the performance gap increases at higher PASS@k, suggesting that the guidance￾trained model is able to sample more diverse and accurate generations. Metric Value Without Guidance # of Successful Rollouts (avg) 138 (16.9%) Empty Patch % 8.71% … view at source ↗
Figure 3
Figure 3. Guidance generation prompt. repo refers to the name of the repo, problem_statement is the task statement, patch is the reference patch, and stacktrace_hint is the stacktrace from the previous attempt. In addition, please carefully consider the following hint, which includes a suggested path(s) to file(s) that may be problematic. ### Hint ### {hint} ### Please solve the problem starting from the very beginning. Use t… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Example of including guidance during trajectory generation, in which the above prompt is [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

    cs.AI 2026-05 conditional novelty 7.0 of 10

    Masked diffusion language models, not larger autoregressive LLMs, are the better building block for text-based world models in agentic RL, improving rollout fidelity, diversity, and downstream task success.

  2. Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

    cs.AI 2026-07 accept novelty 6.0 of 10

    Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.

  3. Fate of Secondary Droplets Produced by High-speed Raindrops Interacting with a Liquid Pool

    physics.flu-dyn 2026-04 unverdicted novelty 6.0 of 10

    Secondary droplet size distribution from raindrop-liquid pool impacts scales as N_d(r_s) ∝ r_s^{-5/2} and collapses onto one curve when normalized by surface tension and raindrop diameter.

  4. SERA: Soft-Verified Efficient Repository Agents

    cs.CL 2026-01 conditional novelty 6.0 of 10

    SERA trains competitive coding agents via supervised finetuning on synthetic trajectories verified only by patch-overlap recall, cutting data-generation cost by roughly two orders of magnitude.

  5. SWE-IF: Aligning Code Evaluation with Human Preference

    cs.CL 2025-10 conditional novelty 6.0 of 10

    A blend of functional correctness and instruction-following predicts human preference for code LLMs better than pass@k alone.

  6. Trading Human Curation for Synthetic Augmentation in RLVR

    cs.LG 2026-06 unverdicted novelty 5.0 of 10

    Gated synthetic augmentations of a 10-task human base substitute for ~87 extra human RLVR tasks on aggregate held-out pass@1, with cost-adjusted trade rate ρ_cost in [1.4×, 11.6×].

  7. Self-Evolving Coding Agents

    cs.SE 2026-08 conditional novelty 4.0 of 10

    A systematic survey that defines self-evolving coding agents, organizes systems into a five-category object-centered taxonomy, and analyzes when and on what evidence these agents evolve.

  8. ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning

    cs.AI 2026-07 conditional novelty 4.0 of 10

    An episode-level variant of GRPO (ESPO) improves personalized GUI-agent reasoning on the 102-episode SmartSpot benchmark, outperforming step-wise and outcome-only training baselines.

Reference graph

Works this paper leans on

29 extracted references · 3 canonical work pages · cited by 8 Pith papers

  1. [9]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? ArXiv, abs/2310.06770, 2023

    arXiv 2023

  2. [1]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiaoling Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bing-Li Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Dama...

    arXiv 2025

  3. [2]

    Jimenez, John Yang, Leyton Ho, Tejal Patwardhan, Kevin Liu, and Aleksander Madry

    Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, Carlos E. Jimenez, John Yang, Leyton Ho, Tejal Patwardhan, Kevin Liu, and Aleksander Madry. Introducing SWE-bench verified, 2024

    work page 2024

  4. [3]

    Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul Francis Christiano

    Nisan Stiennon, Ouyang Long, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul Francis Christiano. Learning to summarize with human feedback. InNeural Information Processing Systems, 2020

    work page 2020

  5. [4]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with h...

    arXiv 2022

  6. [5]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. ArXiv, abs/2305.18290, 2023

    arXiv 2023

  7. [6]

    Cai, Michael Terry, Quoc V

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V . Le, and Charles Sutton. Program synthesis with large language models.ArXiv, abs/2108.07732, 2021

    arXiv 2021

  8. [7]

    Measur- ing coding challenge competence with apps.ArXiv, abs/2105.09938, 2021

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Xiaodong Song, and Jacob Steinhardt. Measur- ing coding challenge competence with apps.ArXiv, abs/2105.09938, 2021

    arXiv 2021

Show all 29 references
  1. [8]

    Hendryx, Summer Yue, and Hugh Zhang

    Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean M. Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves llm search for code generation.ArXiv, abs/2409.03733, 2024. 10

    2024 arXiv

  2. [10]

    Training software engineering agents and verifiers with swe-gym.ArXiv, abs/2412.21139, 2024

    Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with swe-gym.ArXiv, abs/2412.21139, 2024

    2024 arXiv

  3. [11]

    STar: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. STar: Bootstrapping reasoning with reasoning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022

    2022

  4. [12]

    Doucet, Orhan Firat, and Nando de Freitas

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alexa Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, A. Doucet, Orhan Firat, and Nando de Freitas. Reinforced self-training (rest) for language modelin...

    2023 arXiv

  5. [13]

    Agentless: Demystifying llm-based software engineering agents.ArXiv, abs/2407.01489, 2024

    Chun Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying llm-based software engineering agents.ArXiv, abs/2407.01489, 2024

    2024 arXiv

  6. [14]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brenna...

    2024

  7. [15]

    Swe- fixer: Training open-source llms for effective and efficient github issue resolution.ArXiv, abs/2501.05040, 2025

    Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. Swe- fixer: Training open-source llms for effective and efficient github issue resolution.ArXiv, abs/2501.05040, 2025

    2025 arXiv

  8. [16]

    Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution.ArXiv, abs/2502.18449, 2025

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriele Synnaeve, Rishabh Singh, and Sida Wang. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution.ArXiv, abs/2502.18449, 2025

    2025 arXiv

  9. [17]

    Hendryx, Russell Kaplan, Michele Lunati, and Summer Yue

    Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, Sean M. Hendryx, Russell Kaplan, Michele Lunati, and Summer Yue. A careful examination of large language model performance on grade school arithmetic.ArXiv...

    2024 arXiv

  10. [18]

    Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A

    Nathan Lambert, Jacob Daniel Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Validad Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris ...

    2024 arXiv

  11. [19]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bing-Li Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dong-Li Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li...

    2024 arXiv

  12. [20]

    Chiu, Claire Cardie, Matthias Gall’e, and Alexander M

    Wenting Zhao, Nan Jiang, Celine Lee, Justin T. Chiu, Claire Cardie, Matthias Gall’e, and Alexander M. Rush. Commit0: Library generation from scratch.ArXiv, abs/2412.01769, 2024

    2024 arXiv

  13. [21]

    Jimenez, Alexander Wettig, Kilian Adriano Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Adriano Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering.ArXiv, abs/2405.15793, 2024

    2024 arXiv

  14. [22]

    React: Synergizing reasoning and acting in language models.ArXiv, abs/2210.03629, 2022

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models.ArXiv, abs/2210.03629, 2022

    2022 arXiv

  15. [23]

    Proximal policy optimization algorithms.ArXiv, abs/1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.ArXiv, abs/1707.06347, 2017

    2017 arXiv

  16. [24]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y . K. Li, Yu Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.ArXiv, abs/2402.03300, 2024

    2024 arXiv

  17. [25]

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.ArXiv, abs/2404.07972, 2024

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for op...

    2024 arXiv

  18. [26]

    Mind2web: Towards a generalist agent for the web.ArXiv, abs/2306.06070, 2023

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web.ArXiv, abs/2306.06070, 2023

    2023 arXiv

  19. [27]

    Vaskar Nath, Pranav Raja, Claire Yoon, and Sean M. Hendryx. Toolcomp: A multi-tool reasoning & process supervision benchmark.ArXiv, abs/2501.01290, 2025

    2025 arXiv

  20. [28]

    Ui-tars: Pioneering automated gui interaction with native agents.ArXiv, abs/2501.12326, 2025

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng, C...

    2025 arXiv

  21. [29]

    Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning.ArXiv, abs/2411.02337, 2024

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Xinyue Yang, Jiadai Sun, Yu Yang, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning.ArXiv, abs/2411.02337, 2024. 1...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.