Pith. sign in

REVIEW 4 major objections 3 minor 82 references

TaPR turns per-turn hidden-test pass ratios into dense rewards, improving multi-turn Pass@3 by 2.44pp.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:50 UTC pith:MONTFCUE

load-bearing objection A well-run controlled comparison showing dense per-turn pass-ratio rewards change where multi-turn code RL gains appear, but the headline dissociation rests on a 3-problem shift at a best-observed checkpoint and needs selection-aware analysis. the 4 major comments →

arxiv 2608.00494 v1 pith:MONTFCUE submitted 2026-08-01 cs.AI

TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation

Aofan Liu , Jingxiang Meng , Fangxin Liu , Yongbiao Chen This is my paper
classification cs.AI
keywords multi-turn code generationreinforcement learningdense rewardtest-pass ratioGRPOself-repairPass@1/Pass@3 dissociationexecution feedback
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multi-turn code agents get execution feedback and should learn to repair, but standard RL rewards and metrics look only at final correctness, so first-shot talent and feedback-driven repair are tangled. TaPR replaces the sparse final-answer reward with a dense per-turn reward equal to the change in the fraction of hidden tests passed, plus a completion bonus. On 219 LiveCodeBench problems across six models, the selected TaPR checkpoints raise pooled three-turn success (Pass@3) by 2.44 percentage points, with the 7B/8B slice up 3.31 points. The paper's key empirical claim is a dissociation: under matched training, TaPR leaves first-attempt accuracy (Pass@1) identical to baseline while improving three-turn repair, showing the gain comes from feedback-conditioned refinement rather than stronger initial generation. This matters because it gives researchers a way to train and measure self-repair as a distinct capability.

Core claim

The central claim is that decomposing code-generation RL rewards by turn—rewarding the change in hidden-test pass ratio ΔPt = Pt − P_{t−1} at each turn, plus a completion bonus 0.5·𝟙[Pt=1]—creates a different optimization trajectory from outcome-only GRPO while leaving first-shot generation untouched. The paper formalizes the Pass@1/Pass@3 dissociation: on the Hard subset of LCB-219, untrained baseline, GRPO, and TaPR all solve 7/61 problems on the first attempt, yet three-turn success is 7/61, 8/61, and 10/61 respectively. Because the difference appears only when execution feedback is available across turns, the paper interprets it as improved feedback-conditioned repair, not improved gener

What carries the argument

The mechanism is a dense per-turn reward built from the execution oracle. The policy samples a program at each turn; the test runner returns a binary pass vector over hidden tests, and Pt is the fraction passed. The reward at turn t is ΔPt = Pt − P_{t−1} scaled by λΔ=1.0 plus λdone=0.5 when Pt=1. Because ΔPt telescopes over an episode, the undiscounted return equals final pass ratio PT, so lost progress is not credited; the reward locates credit on the specific revision that changed the pass fraction. The paired protocol—evaluating the same checkpoints on the same 219 problem identifiers under a standardized three-turn interaction—converts the dissociation into a testable quantity: Pass@1 me

Load-bearing premise

The paper's Discussion concedes that with one binary hidden test the dense pass-ratio reward Pt collapses to outcome reward; the claimed separation of repair from generation therefore presupposes that APPS/LCB test suites are granular and non-redundant, and that the added Pass@3 successes are local repairs rather than broad regenerations.

What would settle it

Run the matched Qwen3-8B ablation on a variant of LCB-219 where every problem has exactly one hidden test (or all tests perfectly correlated). Under TaPR, Pt becomes binary, so the dense reward collapses to outcome reward; if TaPR still beats GRPO on Hard Pass@3, the gain is not from pass-ratio density. Separately, log token-level edits between turn 1 and turn 3 on the 10/61 Hard successes: if most are near-total regenerations rather than local repairs, the 'repair competence' interpretation would need revision.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Code RL can be trained to improve feedback-conditioned repair without changing initial generation quality, since Pass@1 stayed flat while Pass@3 rose.
  • Evaluations of multi-turn code agents should report Pass@1 and Pass@3 together; each alone hides part of the effect (GRPO closed most of the pooled gap but not the Hard-subset repair gap).
  • Dense execution rewards give nonzero learning signal in all early steps, whereas outcome-only GRPO had zero reward in 40% of the first ten steps, implying faster early credit assignment within a fixed budget.
  • The reward is most useful in the middle regime where initial programs are incorrect but pass some tests; already-solved or all-fail cases give the same signal as sparse reward.
  • Public comparisons must match benchmark release, split, sampling budget, and turn count; heterogeneous protocols cannot be read as a controlled leaderboard.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the Pass@1/Pass@3 dissociation protocol is a general probe for any multi-turn task with a deterministic oracle (math answer checking, web navigation, tool use), not just code; it asks whether training changed first-attempt competence or repair competence.
  • Editorial inference: TaPR's reward design makes test-suite quality part of the training signal; redundant or correlated tests could let the policy inflate Pt without learning a generally correct solution, so one testable extension is measuring sensitivity to test redundancy by ablating the test set.
  • Editorial inference: the method should combine cleanly with verifier-based or self-verification approaches—dense pass-ratio reward supplies learning signal while a learned verifier could handle cases without executable tests.
  • Editorial inference: the transient peak at step 300 followed by regression at step 500 suggests early stopping or checkpoint selection becomes a first-class decision under dense rewards; a testable extension is whether KL or reward normalization can sustain the gain.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes TaPR (Test-aware Policy Refinement), a multi-turn RL framework for code generation in which the reward at each turn is a dense function of the change in hidden-test pass ratio, r_t = lambda_delta (P_t - P_{t-1}) + lambda_done * 1[P_t = 1], plus an optional completion bonus. The method is trained with GRPO on 500 APPS problems and evaluated on 219 LiveCodeBench problems (LCB-219) under a three-turn protocol. The main claims are: (i) pooled Pass@3 improves by +2.44 pp over six paired model evaluations (p = 8.30e-5, one-sided exact test), with a larger +3.31 pp gain in a predefined 7B/8B slice; (ii) on a matched Qwen3-8B ablation, TaPR provides denser early reward and gradient signal than outcome-only GRPO; (iii) the authors report a Pass@1/Pass@3 dissociation, where Hard-subset Pass@1 stays at 7/61 for baseline, GRPO, and TaPR, while Hard Pass@3 differs (7/61, 8/61, 10/61). The paper also contains a careful, explicit set of limitations, including the statement that reported gains come from the highest observed checkpoint and that paired tests do not quantify checkpoint-selection uncertainty.

Significance. If the central empirical claims survive scrutiny, the reward decomposition is a useful and simple contribution: it converts deterministic execution feedback into a per-turn credit-assignment signal, and the proposed Pass@1/Pass@3 reporting protocol would let the community separate first-shot generation quality from feedback-conditioned repair. The authors are unusually candid about their own limitations, including non-monotonic training curves, the inability to distinguish local repair from broad regeneration, and the dependence of the method on test granularity. The controlled matched ablation, the paired problem-level analysis, and the explicit disclosure of checkpoint-selection bias are strengths. However, the headline quantitative claims and the dissociation are currently expressed through best-observed checkpoints and a matched comparison at the apparent peak of the TaPR training curve, which is exactly the setting where selection noise can manufacture spurious effects. The paper's framework is valuable, but the reported effect sizes need a selection-corrected analysis before the claims can be accepted.

major comments (4)
  1. [§Experimental Setup, 'Checkpoint reporting and paired analysis'] The paper explicitly states that 'We report the highest observed Pass@3 checkpoint among saved checkpoints' and that paired tests 'do not quantify checkpoint-selection uncertainty.' This is a load-bearing issue. The headline +2.44 pp pooled gain and the Hard-subset dissociation (7/61 vs 8/61 vs 10/61) are maxima over a non-monotonic training curve (Fig. 2: Qwen3-8B returns to baseline by step 500). For 61 binary Hard problems, a difference of 2-3 successes can easily arise from selecting the best of many checkpoints under a null effect. The paper should either pre-specify a checkpoint selection rule, report a null distribution over maximum-over-checkpoints differences, or show that the result is robust across all checkpoints, not just the best one.
  2. [§Abstract and §Training Trajectory and Overfitting Dynamics] The abstract claims TaPR 'reaches a higher Hard-subset peak than outcome-only GRPO within the tested budget,' but the only Hard-subset comparison shown is at step 300 (Table 3). GRPO's full Hard Pass@3 trajectory is not reported, so it is unknown whether GRPO's own peak occurs at a different step and equals or exceeds the TaPR peak. The claim should be supported by plotting both methods' Hard Pass@3 across all checkpoints, or removed if only the step-300 comparison is intended.
  3. [§Ablation, Table 3] The matched step-300 ablation is not an independent comparison point: step 300 is selected because it is the apparent TaPR peak (Fig. 2). Outcome-only GRPO is reported at the same step, but GRPO 'nearly matches pooled Pass@3 by step 300' according to the abstract. Comparing at the peaking step of one method and not the other biases the Hard-subset dissociation in TaPR's favor. Please provide the full checkpoint curves for both methods and analyze the difference either at a pre-registered step or with a selection-aware correction.
  4. [§Discussion, 'Scope of the dissociation'] The paper's own Discussion correctly notes that with one binary test P_t reduces to outcome reward, and that redundant tests can overstate progress. This is a direct limitation on Eq. (1) and Eq. (3), but it is not merely a scope condition for future work: the APPS training and LCB evaluation protocols determine whether the reported Pass@3 gains reflect general repair competence or optimization toward the specific test suites. The authors should provide per-problem test counts or a sensitivity analysis on problems with more granular tests. As written, the central mechanism is only as strong as the assumption that APPS/LCB tests are granular and non-redundant, which is plausible but not demonstrated.
minor comments (3)
  1. [Table 1 caption] The table reports 'Result Δpp' from the lowest displayed untrained checkpoint within a block, even when the public baseline uses a different benchmark release, split, or protocol. The body text explains this clearly, but the caption should make it impossible to misread a TaPR row as a controlled gain against the public method.
  2. [§Observations, bullet 2] The text says GRPO has a '13× lower mean and standard deviation' and '10× lower early-training gradient norm'; Table 2 reports 0.160 vs 0.0125 (12.8×) and 0.030 vs 0.0030 (10×). The rounded numbers are consistent, but the prose should state these are rounded values to avoid apparent mismatch.
  3. [§Experimental Setup, 'Datasets and Evaluation Protocol'] The APPS-300 held-out result is reported only as aggregate counts because per-problem paired outcomes were not retained. This is an honest disclosure, but since the secondary in-domain check cannot be included in the paired statistical analysis, it would be helpful to state explicitly that no inferential claim is made from that result.

Circularity Check

0 steps flagged

No circularity: TaPR is an empirical RL comparison; reward/evaluation alignment is intentional design, not a self-referential derivation.

full rationale

TaPR's derivation chain is self-contained. The dense reward (Eq. 3) is defined from the per-turn pass ratio P_t (Eq. 1) and its difference (Eq. 2), and the telescoping identity (Eq. 4) is a mathematical simplification of that definition, not an input assumed to equal the headline result. The headline Pass@3 gain is measured on held-out LCB-219, while training rewards are computed on APPS trajectories; the same test oracle is not used for both fitting and prediction. The alignment between the training reward and the evaluation metric is the intended mechanism of the method, not a fitted parameter renamed as a prediction. The paper explicitly discloses the places where circularity-like concerns could hide: 'We report the highest observed Pass@3 checkpoint among saved checkpoints... paired tests quantify problem-level uncertainty but not checkpoint-selection uncertainty' (Checkpoint reporting section), and 'It does not prove that each later success is a local repair: feedback may trigger broad regeneration' (Scope of the dissociation paragraph). These are acknowledged empirical validity limitations, not constructional circularity. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or conceptual entities. Its free parameters are hand-set reward weights and a checkpoint-selection rule, and its unstated premises are the informativeness of hidden-test pass ratios and the interpretability of Pass@3 as repair competence.

free parameters (4)
  • lambda_delta (progress reward weight) = 1.0
    Hand-set in Eq. (3); no sensitivity analysis reported, yet it scales all partial-progress credit.
  • lambda_done (completion bonus weight) = 0.5
    Hand-set in Eq. (3); changes the relative incentive for full correctness vs partial progress.
  • Checkpoint selection rule = highest Pass@3 among saved checkpoints
    The reported gains use the best checkpoint observed on the evaluation set, a hand-chosen selection rule that inflates estimates and is not an independently pre-registered choice.
  • KL penalty beta = 0.04
    Chosen from the conservative configuration; affects how far the policy can drift from the reference model.
axioms (4)
  • standard math Sum of per-turn pass-ratio improvements telescopes to the final pass ratio: sum(Delta P_t) = P_T (Eq. 4).
    Used to argue the dense reward preserves final functional quality; standard algebra.
  • domain assumption Hidden-test pass ratio is a meaningful measure of partial correctness.
    The whole method relies on P_t carrying useful signal; the paper itself notes the limit when tests are degenerate (Discussion).
  • domain assumption Three-turn Pass@3 under the LCB-219 protocol measures feedback-conditioned repair rather than mere resampling.
    The paper acknowledges in 'Scope of the dissociation' that feedback may trigger broad regeneration, so Pass@3 gain is not proven to be local repair.
  • domain assumption APPS-500 training distribution transfers to LiveCodeBench evaluation.
    The method is evaluated out-of-distribution on LCB-219; if APPS and LCB draw on different problem styles, the transfer claim weakens.

pith-pipeline@v1.3.0-alltime-deepseek · 10950 in / 14820 out tokens · 142577 ms · 2026-08-05T00:50:21.350869+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation." pith.science (2026). https://pith.science/paper/MONTFCUE

@misc{pith2026260800494,
  author       = {Pith},
  title        = {Pith review of: TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MONTFCUE}},
  note         = {Machine review of arXiv:2608.00494}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multi-turn code agents rely on execution feedback to repair incorrect programs, yet standard reinforcement learning paradigms optimize and evaluate policy performance primarily using single-shot outcome rewards. This misalignment conflates initial code generation with feedback-driven refinement, discards granular execution signals across intermediate turns, and fails to evaluate whether the policy actually acquires self-repair capabilities. We propose Test-aware Policy Refinement (TaPR), a framework that transforms execution feedback into a dense per-turn test-pass-ratio reward under a consistent multi-turn interaction protocol. Across six models on 219 code-generation problems from LiveCodeBench, TaPR improves the pooled three-turn success rate (Pass@3) by 2.44 percentage points. In the predefined 7B/8B high-headroom slice, pooled accuracy increases from 30.25% to 33.56% (+3.31 pp), with 42 improvements and 13 regressions in paired trials. On a matched Qwen3-8B ablation, the dense reward supplies nonzero feedback in all of the first ten steps and reaches a higher Hard-subset peak than outcome-only GRPO within the tested budget, although GRPO nearly matches pooled Pass@3 by step 300. Our primary contribution is a reward-decomposition framework and a turn-aware evaluation protocol that decouple first-shot generation quality from multi-turn repair competence.

Figures

Figures reproduced from arXiv: 2608.00494 by Aofan Liu, Fangxin Liu, Jingxiang Meng, Yongbiao Chen.

Figure 1
Figure 1. Figure 1: Overview of Test-aware Policy Refinement (TaPR). The upper path illustrates multi-turn code generation where [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Qwen3-8B pass@1 and pass@3 on LCB-219 across checkpoints. Dashed lines denote the untrained checkpoint and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Ablation overlay on Qwen3-8B: TaPR vs. GRPO trajectories on LCB-219 across training steps. TaPR reaches its [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 61 canonical work pages · 2 internal anchors

  1. [1]

    2024 , note =

    Kumar, Aviral and others , title =. 2024 , note =

    2024

  2. [2]

    2025 , note =

    Chen, Zhipeng and others , title =. 2025 , note =

    2025

  3. [3]

    2021 , note =

    Chen, Mark and others , title =. 2021 , note =

    work page 2021

  4. [4]

    and Sutton, Charles , title =

    Austin, Jacob and Odena, Augustus and Nye, Maxwell and Bosma, Maarten and Michalewski, Henryk and Dohan, David and Jiang, Ellen and Cai, Carrie and Terry, Michael and Le, Quoc V. and Sutton, Charles , title =. 2021 , note =

    work page 2021

  5. [5]

    2021 , note =

    Hendrycks, Dan and Basart, Steven and Kadavath, Saurav and Mazeika, Mantas and Arora, Akul and Guo, Ethan and Burns, Collin and Puranik, Samir and He, Horace and Song, Dawn and Steinhardt, Jacob , title =. 2021 , note =

    work page 2021

  6. [6]

    2022 , note =

    Li, Yujia and others , title =. 2022 , note =

    work page 2022

  7. [7]

    2022 , note =

    Lai, Yuhang and Li, Chengxi and Wang, Yiming and Zhang, Tianyi and Zhong, Ruiqi and Zettlemoyer, Luke and Yih, Scott Wen-tau and Fried, Daniel and Wang, Sida and Yu, Tao , title =. 2022 , note =

    work page 2022

  8. [8]

    International Conference on Learning Representations , year =

    Zhuo, Terry Yue and others , title =. International Conference on Learning Representations , year =

    work page

  9. [9]

    International Conference on Learning Representations , year =

    Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion , title =. International Conference on Learning Representations , year =

    work page

  10. [10]

    2025 , note =

    Ekbote, Chanakya and Lingam, Vijay and Sanghavi, Sujay and Huan, Jun and Omidvar-Tehrani, Behrooz and Deoras, Anoop and Soatto, Stefano , title =. 2025 , note =

    work page 2025

  11. [11]

    and Zhao, Wenting and Choudhury, Sanjiban , title =

    Jain, Arnav Kumar and Gonzalez-Pumariega, Gonzalo and Chen, Wayne and Rush, Alexander M. and Zhao, Wenting and Choudhury, Sanjiban , title =. Proceedings of the International Conference on Machine Learning , year =

    work page

  12. [12]

    2025 , note =

    Cho, Jeonghun and Kang, Deokhyung and Kim, Hyounghun and Lee, Gary Geunbae , title =. 2025 , note =

    work page 2025

  13. [13]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , year =

    Zhang, Kechi and Li, Zhuo and Li, Jia and Li, Ge and Jin, Zhi , title =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , year =

    work page

  14. [14]

    Findings of the Association for Computational Linguistics: NAACL 2024 , year =

    Wang, Xingyao and Peng, Hao and Jabbarvand, Reyhaneh and Ji, Heng , title =. Findings of the Association for Computational Linguistics: NAACL 2024 , year =

    work page 2024

  15. [15]

    Findings of the Association for Computational Linguistics: ACL 2025 , year =

    Li, Qingyao and Dai, Xinyi and Li, Xiangyang and Zhang, Weinan and Wang, Yasheng and Tang, Ruiming and Yu, Yong , title =. Findings of the Association for Computational Linguistics: ACL 2025 , year =

    work page 2025

  16. [16]

    2025 , note =

    Jin, Yiyang and Xu, Kunzhao and Li, Hang and Han, Xueting and Zhou, Yanmin and Li, Cheng and Bai, Jing , title =. 2025 , note =

    work page 2025

  17. [17]

    2026 , note =

    Tang, Zitian and Zhang, Xu and Yuan, Jianbo and Zou, Yang and Gunjal, Varad and Jiang, Songyao and Modolo, Davide , title =. 2026 , note =

    work page 2026

  18. [18]

    2026 , note =

    Modecrua, Wachiravit and Kaewtawee, Krittanon and Pachtrachai, Krittin and Kraisingkorn, Touchapon , title =. 2026 , note =

    work page 2026

  19. [19]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , year =

    Wei, Zhepei and Yao, Wenlin and Liu, Yao and Zhang, Weizhi and Lu, Qin and Qiu, Liang and Yu, Changlong and Xu, Puyang and Zhang, Chao and Yin, Bing and Yun, Hyokun and Li, Lihong , title =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , year =

    work page

  20. [20]

    Gehring, Jonas and others , title =

    work page

  21. [21]

    2024 , note =

    Shao, Zhihong and others , title =. 2024 , note =

    work page 2024

  22. [22]

    2017 , note =

    Schulman, John and Wolski, Filip and Dhariwal, Prafulla and Radford, Alec and Klimov, Oleg , title =. 2017 , note =

    work page 2017

  23. [23]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Chen, Weizhu , title =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Chen, Weizhu , title =. 2021 , note =

    work page 2021

  24. [24]

    Advances in Neural Information Processing Systems , year =

    Le, Hung and others , title =. Advances in Neural Information Processing Systems , year =

    work page

  25. [25]

    2023 , note =

    Lightman, Hunter and others , title =. 2023 , note =

    work page 2023

  26. [26]

    Proceedings of the Annual Meeting of the Association for Computational Linguistics , year =

    Wang, Peiyi and others , title =. Proceedings of the Annual Meeting of the Association for Computational Linguistics , year =

    work page

  27. [27]

    Advances in Neural Information Processing Systems , year =

    Madaan, Aman and others , title =. Advances in Neural Information Processing Systems , year =

    work page

  28. [28]

    Advances in Neural Information Processing Systems , year =

    Shinn, Noah and others , title =. Advances in Neural Information Processing Systems , year =

    work page

  29. [29]

    Transactions of the Association for Computational Linguistics , year =

    Kamoi, Ryo and others , title =. Transactions of the Association for Computational Linguistics , year =

    work page

  30. [30]

    2026 , note =

    Liu, Aofan and Meng, Jingxiang , title =. 2026 , note =

    work page 2026

  31. [31]

    IEEE International Conference on Acoustics, Speech and Signal Processing , year =

    Wang, Bin and others , title =. IEEE International Conference on Acoustics, Speech and Signal Processing , year =

    work page

  32. [32]

    2025 , note =

    Liu, Aofan and others , title =. 2025 , note =

    work page 2025

  33. [33]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , year =

    Liu, Aofan and others , title =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , year =

    work page

  34. [34]

    2025 , note =

    Liu, Wei and Peng, Chao and Gao, Pengfei and Liu, Aofan and Zhang, Wei and Zhao, Haiyan and Jin, Zhi , title =. 2025 , note =

    work page 2025

  35. [35]

    2025 , note =

    Yue, Yang and others , title =. 2025 , note =

    work page 2025

  36. [36]

    2024 , note =

    Yang, An and others , title =. 2024 , note =

    work page 2024

  37. [37]

    2024 , note =

    Hui, Binyuan and others , title =. 2024 , note =

    work page 2024

  38. [38]

    2025 , note =

    Yang, An and others , title =. 2025 , note =

    work page 2025

  39. [39]

    2024 , note =

    Dubey, Abhimanyu and others , title =. 2024 , note =

    work page 2024

  40. [40]

    2025 , note =

    Hu, Hanxu and Zhang, Xingxing and Vamvas, Jannis and Sennrich, Rico and Wei, Furu , title =. 2025 , note =

    work page 2025

  41. [41]

    2025 , note =

    Lin, Zi and Shen, Sheng and Kulikov, Ilia and Shang, Jingbo and Weston, Jason and Nie, Yixin , title =. 2025 , note =

    work page 2025

  42. [42]

    2026 , note =

    Chen, Ziru and Chen, Dongdong and Jin, Ruinan and Liang, Yingbin and Xie, Yujia and Sun, Huan , title =. 2026 , note =

    work page 2026

  43. [43]

    2026 , note =

    Wu, Jie and Li, Haoling and Zhang, Xin and Guo, Jiani and Luo, Jane and Liu, Steven and Huang, Yangyu and Chu, Ruihang and Li, Scarlett and Yang, Yujiu , title =. 2026 , note =

    work page 2026

  44. [44]

    2026 , howpublished =

    2026

  45. [45]

    Findings of the Association for Computational Linguistics: EMNLP , year =

    Li, Dacheng and Cao, Shiyi and Cao, Chengkun and Li, Xiuyu and Tan, Shangyin and Keutzer, Kurt and Xing, Jiarong and Gonzalez, Joseph and Stoica, Ion , title =. Findings of the Association for Computational Linguistics: EMNLP , year =

    work page

  46. [46]

    2026 , note =

    Cai, Jianfeng and Zhu, Jinhua and Sun, Ruopei and Zhao, Kangwen and Xue, Dongyun and Feng, Mingxiao and Zhou, Wengang and Li, Houqiang , title =. 2026 , note =

    work page 2026

  47. [47]

    2025 , note =

    Zhao, Xueliang and Wu, Wei and Guan, Jian and Gong, Zhuocheng and Kong, Lingpeng , title =. 2025 , note =

    work page 2025

  48. [48]

    2026 , note =

    Lin, ChunYen and Nhiem, Tran Van and Liao, I-Bin and Li, Yung-Hui and Chou, Hsin-Hung , title =. 2026 , note =

    work page 2026

  49. [49]

    Findings of the European Chapter of the Association for Computational Linguistics , year =

    Lee, Woongkyu and Cho, Junhee and Choi, Jungwook , title =. Findings of the European Chapter of the Association for Computational Linguistics , year =

    work page

  50. [50]

    V.; and Sutton, C

    Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.; Terry, M.; Le, Q. V.; and Sutton, C. 2021. Program Synthesis with Large Language Models. ArXiv:2108.07732

    Pith/arXiv arXiv 2021

  51. [51]

    Chen, M.; et al. 2021. Evaluating Large Language Models Trained on Code. ArXiv:2107.03374

    Pith/arXiv arXiv 2021

  52. [52]

    Chen, Z.; Chen, D.; Jin, R.; Liang, Y.; Xie, Y.; and Sun, H. 2026. Bridging Online and Offline RL : Contextual Bandit Learning for Multi-Turn Code Generation. ArXiv:2602.03806

    arXiv 2026

  53. [53]

    Chen, Z.; et al. 2025. Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models. ArXiv:2508.10751

    Pith/arXiv arXiv 2025

  54. [54]

    Cho, J.; Kang, D.; Kim, H.; and Lee, G. G. 2025. Self-Correcting Code Generation Using Small Language Models. ArXiv:2505.23060

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  55. [55]

    Ekbote, C.; Lingam, V.; Sanghavi, S.; Huan, J.; Omidvar-Tehrani, B.; Deoras, A.; and Soatto, S. 2025. MURPHY : Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation. ArXiv:2511.07833

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  56. [56]

    Gehring, J.; et al. 2025. RLEF : Grounding Code LLMs in Execution Feedback with Reinforcement Learning

    work page 2025

  57. [57]

    Hendrycks, D.; Basart, S.; Kadavath, S.; Mazeika, M.; Arora, A.; Guo, E.; Burns, C.; Puranik, S.; He, H.; Song, D.; and Steinhardt, J. 2021. Measuring Coding Challenge Competence with APPS . ArXiv:2105.09938

    Pith/arXiv arXiv 2021

  58. [58]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; and Chen, W. 2021. LoRA : Low-Rank Adaptation of Large Language Models. ArXiv:2106.09685

    Pith/arXiv arXiv 2021

  59. [59]

    Hu, H.; Zhang, X.; Vamvas, J.; Sennrich, R.; and Wei, F. 2025. QueST : Incentivizing LLMs to Generate Difficult Problems. ArXiv:2510.17715

    arXiv 2025

  60. [60]

    K.; Gonzalez-Pumariega, G.; Chen, W.; Rush, A

    Jain, A. K.; Gonzalez-Pumariega, G.; Chen, W.; Rush, A. M.; Zhao, W.; and Choudhury, S. 2025 a . Multi-Turn Code Generation Through Single-Step Rewards. In Proceedings of the International Conference on Machine Learning, 26700--26716

    work page 2025

  61. [61]

    Jain, N.; Han, K.; Gu, A.; Li, W.-D.; Yan, F.; Zhang, T.; Wang, S.; Solar-Lezama, A.; Sen, K.; and Stoica, I. 2025 b . LiveCodeBench : Holistic and Contamination Free Evaluation of Large Language Models for Code. In International Conference on Learning Representations. ArXiv:2403.07974

    Pith/arXiv arXiv 2025

  62. [62]

    Jin, Y.; Xu, K.; Li, H.; Han, X.; Zhou, Y.; Li, C.; and Bai, J. 2025. ReVeal : Self-Evolving Code Agents via Reliable Self-Verification. ArXiv:2506.11442

    arXiv 2025

  63. [63]

    Kamoi, R.; et al. 2024. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs . Transactions of the Association for Computational Linguistics

    work page 2024

  64. [64]

    Kumar, A.; et al. 2024. Training Language Models to Self-Correct via Reinforcement Learning ( SCoRe ). ArXiv:2409.12917

    Pith/arXiv arXiv 2024

  65. [65]

    Le, H.; et al. 2022. CodeRL : Mastering Code Generation Through Pretrained Models and Deep Reinforcement Learning. In Advances in Neural Information Processing Systems

    work page 2022

  66. [66]

    Lee, W.; Cho, J.; and Choi, J. 2026. MapCoder-Lite : Squeezing Multi-Agent Coding into a Single Small LLM . In Findings of the European Chapter of the Association for Computational Linguistics. ArXiv:2509.17489

    work page arXiv 2026

  67. [67]

    Li, Q.; Dai, X.; Li, X.; Zhang, W.; Wang, Y.; Tang, R.; and Yu, Y. 2025. CodePRM : Execution Feedback-Enhanced Process Reward Model for Code Generation. In Findings of the Association for Computational Linguistics: ACL 2025, 8169--8182

    work page 2025

  68. [68]

    Lightman, H.; et al. 2023. Let's Verify Step by Step. ArXiv:2305.20050

    Pith/arXiv arXiv 2023

  69. [69]

    V.; Liao, I.-B.; Li, Y.-H.; and Chou, H.-H

    Lin, C.; Nhiem, T. V.; Liao, I.-B.; Li, Y.-H.; and Chou, H.-H. 2026. MACS-Coder : A Multi-Agent Coding Framework for Small LMs ---From Fast Thinking to Deep Planning. ICLR 2026 Conference withdrawn submission

    work page 2026

  70. [70]

    Lin, Z.; Shen, S.; Kulikov, I.; Shang, J.; Weston, J.; and Nie, Y. 2025. Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation. ArXiv:2502.14948

    arXiv 2025

  71. [71]

    Madaan, A.; et al. 2023. Self-Refine : Iterative Refinement with Self-Feedback. In Advances in Neural Information Processing Systems

    work page 2023

  72. [72]

    Modecrua, W.; Kaewtawee, K.; Pachtrachai, K.; and Kraisingkorn, T. 2026. Multi-Turn RL for Tool-Calling Agents with Iterative Reward Calibration. ArXiv:2604.02869

    Pith/arXiv arXiv 2026

  73. [73]

    Shao, Z.; et al. 2024. DeepSeekMath : Pushing the Limits of Mathematical Reasoning in Open Language Models. ArXiv:2402.03300

    Pith/arXiv arXiv 2024

  74. [74]

    Shinn, N.; et al. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. In Advances in Neural Information Processing Systems

    work page 2023

  75. [75]

    Tang, Z.; Zhang, X.; Yuan, J.; Zou, Y.; Gunjal, V.; Jiang, S.; and Modolo, D. 2026. MM-ReCoder : Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction. ArXiv:2604.01600

    work page arXiv 2026

  76. [76]

    Wang, P.; et al. 2024 a . Math-Shepherd : Verify and Reinforce LLMs Step-by-Step Without Human Annotations. In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    work page 2024

  77. [77]

    Wang, X.; Peng, H.; Jabbarvand, R.; and Ji, H. 2024 b . LETI : Learning to Generate from Textual Interactions. In Findings of the Association for Computational Linguistics: NAACL 2024, 223--239

    work page 2024

  78. [78]

    Wei, Z.; Yao, W.; Liu, Y.; Zhang, W.; Lu, Q.; Qiu, L.; Yu, C.; Xu, P.; Zhang, C.; Yin, B.; Yun, H.; and Li, L. 2025. WebAgent-R1 : Training Web Agents via End-to-End Multi-Turn Reinforcement Learning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. ArXiv:2505.16421

    arXiv 2025

  79. [79]

    Wu, J.; Li, H.; Zhang, X.; Guo, J.; Luo, J.; Liu, S.; Huang, Y.; Chu, R.; Li, S.; and Yang, Y. 2026. X-Coder : Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests. ArXiv:2601.06953

    arXiv 2026

  80. [80]

    Yue, Y.; et al. 2025. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? ArXiv:2504.13837

    Pith/arXiv arXiv 2025

Showing first 80 references.