Pith. sign in

REVIEW 3 major objections 5 minor 35 references

CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CRScore++ argues that grounding RL rewards in linters and AI feedback improves code review and transfers across languages.

desk verdict Sensible recipe, closed-loop evaluation—worth a strong referee but not a citation until the judge is decontaminated. read the letter →

arxiv 2506.00296 v1 pith:JB3SLP4S submitted 2025-05-30 cs.SE

Manav Nitin Kapadnis , Atharva Naik , Carolyn Rose This is my paper
classification cs.SE
keywords codereviewreinforcementlearningAIfeedbackverifiablerewardsstaticanalysissmellscross-languagegeneralizationdirectpreferenceoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to show that code-review comment generation, an open-ended natural-language task, can be improved with reinforcement learning when the reward combines verifiable static-analysis signals with subjective LLM judgment. It introduces CRScore++, a two-stage pipeline in which a stronger teacher writes tool-grounded review demonstrations for a weaker student to imitate, then scores the student's candidate reviews and fine-tunes the student with direct preference optimization. The headline empirical claim is that models trained only on Python learn general code-quality analysis that transfers to Java and JavaScript, with comprehensiveness up about 56 percent relative over zero-shot for the 3B model. This matters because code review is hard to reward automatically, and a single trained reviewer that works across languages could replace per-language review systems.

What carries the argument

The load-bearing object is the hybrid reward: static-analysis facts from linters and code-smell detectors are injected into the teacher's chain-of-thought, and the same tool-grounded reasoning shapes the pseudo-reference topic lists that the judge uses to score reviews. This makes the unstructured task of writing a good review partially checkable, because the tools anchor comprehensiveness and relevance in concrete findings while the LLM judge covers aspects the tools miss, such as readability, security, performance, and design patterns. The training sequence does the remaining work: supervised fine-tuning establishes tool-aware behavior, and direct preference optimization, an RL-style alignment method that trains the student to favor preferred reviews, sharpens coverage of the tool-identified issues.

What would settle it

Have independent human raters score the same generated reviews on comprehensiveness, conciseness, and relevance without being shown the LLM-generated topic list, on the same Python, Java, and JavaScript samples used in the paper, and check whether Stage 2 still outranks zero-shot; if it does not, the reported gains are an artifact of aligning to the judge's own topic lists.

Watch

Extended reading notes

Core claim

CRScore++ turns CRScore, a code-review evaluation metric that blends LLM judgment with linter and code-smell detector outputs, into a training signal for generating reviews. Stage 1, knowledge distillation via demonstration, feeds Python diffs augmented with Ruff and PyScent outputs to GPT-4o-mini, which produces step-by-step analysis and a final review; Qwen2.5-Coder 3B and 7B are supervised on these demonstrations. Stage 2, preference optimization, uses the same teacher to generate a topics-to-be-covered pseudo-reference for each diff, scores twenty sampled candidate reviews per diff on comprehensiveness, conciseness, and relevance, and runs DPO on preference pairs separated by at least two points. The paper reports that CRScore++ raises comprehensiveness and relevance over zero-shot and over plain SFT at a small conciseness cost, and that Python-only models retain most of that quality on Java and JavaScript when evaluated with language-specific tools and the same LLM judge.

Load-bearing premise

The load-bearing premise is that the judge model's scores, reached by comparing a review with an expected-topics list that the same judge writes, are a valid and unbiased measure of review quality both when used as the training reward and when used as the final evaluation metric.

Editorial extensions

If this is right

  • A model trained once on Python data can be deployed to review Java and JavaScript without language-specific fine-tuning, so polyglot repositories need only one review model.
  • Practitioners can choose SFT models when they want concise comments and DPO models when they want maximum coverage, since Stage 2 trades conciseness for breadth.
  • Training on tool-grounded demonstrations teaches small models to incorporate linter and code-smell facts accurately, which should reduce hallucinated or off-topic review claims.
  • The same partial-verifier reward recipe could apply to other natural-language software-engineering tasks lacking a single correct output, such as API documentation or vulnerability reports.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a direct ablation that removes the verifier-generated topic lists would isolate whether the DPO gains come from static-analysis grounding or from the judge's own preference style.
  • Beyond the paper, swapping the judge for an independently trained open-source reward model would test whether the reported improvements and cross-language transfer are tied to one proprietary evaluator.
  • Beyond the paper, the comprehensiveness-conciseness trade-off implies a product split, with SFT for inline comments and DPO for pre-merge summaries, which the paper does not evaluate.
  • Beyond the paper, the cross-language claim would be sharpened by testing against a functionally distinct paradigm, such as Scala, to see whether the transferred skill is code-quality reasoning rather than object-oriented familiarity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CRScore++, a two-stage training pipeline for code review comment generation: first, supervised fine-tuning (SFT) on tool-augmented demonstrations generated by GPT-4o-mini from CodeReviewer Python samples with Ruff and PyScent outputs; second, direct preference optimization (DPO) where candidate reviews are scored by GPT-4o-mini against pseudo-reference "topics to be covered" lists that GPT-4o-mini itself generates. The authors claim that this approach improves comprehensiveness and relevance over zero-shot and baseline SFT in Python and, without any language-specific fine-tuning, achieves nearly equivalent performance on Java and JavaScript. The evaluation relies on GPT-4o-mini as judge for the main tables, with a small human study presented as the only independent check.

Significance. If the results were validated by an evaluation that is not contaminated by the training signal, the framework would be a useful contribution: it addresses a real problem (RL feedback for unstructured NLG in code review), integrates static-analysis tools as partial verifiers, and explores cross-language transfer of code-quality reasoning. The paper is also transparent in describing its limitations, and Appendix C reports Wilcoxon tests, which is a step beyond reporting means alone. However, the central claim currently rests on a circular evaluation loop: the same model (GPT-4o-mini) generates the SFT demonstrations, the pseudo-references used as DPO reward criteria, and the final evaluation scores against those same pseudo-references. The human study, the only independent evidence, is small and internally inconsistent. The cross-language generalization claim is therefore not yet established.

major comments (3)
  1. [Section 3.2, Section 3.3, Appendix B.1] The automated evaluation is circular with respect to the training signal. GPT-4o-mini generates the 'topics to be covered' pseudo-references (Section 3.3), scores the DPO candidate reviews against those topics to create preference pairs (Section 3.2), and then scores the final reviews against the same pseudo-references using the same judge (Appendix B.1). Since the SFT demonstrations are also produced by GPT-4o-mini (Appendix A.2), the reported improvements in Tables 1 and 2 could reflect the student learning to imitate GPT-4o-mini's topic-selection style and length/coverage preferences rather than an improvement in human-valued review quality. The tool outputs are only additional prompt context; there is no objective check that a generated review actually cites a real linter or code-smell finding. Table 3, which also uses GPT-4o-mini as judge, does not break this loop. The authors should re-evaluate using a different judge (e.g., a different model or human ratings without pseudo-references), or demonstrate that the pseudo-references are objectively derived from tool outputs and that the judge's scores correlate with an independent measure.
  2. [Section 5.4, Appendix B.3, Appendix B.7] The human study, which is the only independent check on the automated metric, contains material internal inconsistencies that need to be resolved. Section 5.4 states there were 100 code changes with 33 per language, but Appendix B.3 reports 33 Python, 34 Java, and 34 JavaScript instances, totaling 101. Section 5.4 reports inter-annotator agreement of 0.65, while Appendices B.3 and B.7 report 0.7. Section 5.4 refers to 'Human raters' without a count, Appendix B.3 says four experienced developers, and Appendix B.7 says five graduate students. Additionally, Appendix B.5 states that human raters were shown the GPT-4o-mini-generated 'topics to be covered' list, so even the human evaluation inherits the pseudo-reference frame. The authors should provide one consistent set of numbers, clarify the annotator count, and either re-run the human study without revealing the machine-generated topics or analyze how the topics influenced the ratings.
  3. [Tables 1 and 2] Tables 1 and 2 report only rounded point estimates for comprehensiveness, conciseness, and relevance, with no standard deviations, confidence intervals, or per-instance distributions. The Wilcoxon tests in Appendix C suggest that variance is available, but the main tables do not convey it. Without a measure of uncertainty, the claim that Stage 2 models 'consistently' achieve the best scores across languages, and the claim of 'nearly equivalent' cross-language performance, cannot be evaluated. The authors should report bootstrapped intervals or standard errors, and state the number of test instances each score is based on.
minor comments (5)
  1. [Section 3 and Section 1] The paper is inconsistent about the number of stages: the Introduction describes a two-stage framework, while Section 3 says 'three-stage methodology' and Figure 1 shows a three-stage pipeline; the authors should reconcile this terminology.
  2. [Appendix B.6] There is a typo in 'employes' in the first sentence of Appendix B.6; it should be 'employs'.
  3. [Appendix C, Tables 11-13] The formatting of the p-value and statistic columns in Tables 11-13 makes the values difficult to read, with entries such as 'p¡0.017.50e+06'; these should be separated clearly and the significance level stated consistently.
  4. [Section 4.3, Table 3] The text states that coverage improves from zero-shot to SFT and DPO stages, but in Table 3 the 3B Stage 2 coverage for Python (1.62) and Java (1.72) is lower than the corresponding Stage 1 values (1.90 and 1.88); the discussion should account for this pattern or justify a different reading of the table.
  5. [Section 3.2, Eq. (1)] Equation (1) defines the DPO loss but the notation is incomplete: the expectation variable and the policy conditioning on x are not specified clearly; a proper definition would help reproducibility.

Circularity Check

3 steps flagged · score 6.0 of 10

The DPO reward and the final evaluation are the same GPT-4o-mini judge scoring against GPT-4o-mini-generated pseudo-references, so reported in-domain gains report the training objective; the pseudo-reference validity rests partly on a CRScore self-citation, and the human study inherits the same topic list.

  1. fitted input called prediction [Section 3.2 (Direct Preference Optimization) and Section 3.3 (Cross-language Evaluation); Appendix B.3]
    "These are then scored by GPT4o-mini, which generates a step-by-step analysis and final review scores over a scale of 5 points on comprehensiveness, conciseness, and relevance metric. We then create preference pairs (yw,yl) with Δscore≥ 2 for alignment training of DPO model. ... First, the GPT-4o Mini creates a comprehensive list of topics considered crucial to evaluate a model ... The process creates pseudo ground-truths for evaluating reviews generated by the models."

    The DPO stage selects candidate reviews that increase GPT-4o-mini's scores, and Tables 1 and 2 then report increases on exactly those scores from the same model. Because the judge's rubric defines comprehensiveness as covering the judge's own Topics-to-be-Covered list (Appendix B.1), the reported in-domain improvement is the training objective itself, not an independent prediction. The evaluation is therefore the fitted reward function called a result.

  2. self citation load bearing [Section 1 (Introduction) and Section 2.1 (Related Work); reference Naik et al., 2024 (CRScore)]
    "the teacher model critiques the student model's review comments by first generating a list of expected review points (pseudo-references generated by CRScore (Naik et al., 2024)), incorporating tool outputs ... CRScore (Naik et al., 2024) shows that integrating static analysis tools with LLMs and using pseudo-references can lead to a more accurate signal of review quality."

    The validity of GPT-4o-mini-generated pseudo-references as ground truth is the load-bearing premise for both the DPO reward and the final evaluation. That premise is supported by a citation to CRScore, which shares two authors with the present paper (Naik and Rosé). The paper's claimed mitigation, the human study, does not independently establish pseudo-reference validity because raters were shown the same GPT-4o-mini-generated topic list (Appendix B.5).

1 more flagged steps
  1. other [Appendix B.5 (Codebook for Rating Code Review Quality)]
    "Unlike CRScore which focused on pseudo-references, our CRScore++ evaluation focused directly on the quality of the review comments in relation to a predefined list of "topics to be covered" for each code change. These topics were systematically generated by GPT-4o-mini ... Each rater was provided with the original code diff, the generated "topics to be covered" list, and access to the complete source files before and after the changes."

    The human study is presented as the external check on the automated judge, but it uses the same GPT-4o-mini-generated topic list that defines the DPO reward and the automatic metric. Raters' comprehensiveness judgments therefore measure coverage of the judge's own pseudo-references, so the human evaluation inherits the same closed loop rather than breaking it.

full rationale

The paper's central claim that CRScore++ improves code review quality is evaluated with GPT-4o-mini scores against GPT-4o-mini-generated pseudo-references, and the same judge was used to create the DPO preference pairs. In-domain gains in comprehensiveness and relevance are thus reports of the very objective the model was optimized for; this is a fitted-input-called-prediction loop rather than an independent measurement. The paper's own Limitations section acknowledges that "the use of GPT-4o-mini as an evaluation metric relies on proprietary infrastructure that can have hidden biases," but the stated mitigation (cross-checking with human evaluation) is weakened because Appendix B.5 shows that human raters were given the GPT-4o-mini-generated topics. The validity of the pseudo-reference ground truth is also grounded in a citation to CRScore (Naik et al., 2024), which shares two authors with this paper, making the self-citation load-bearing. The cross-language generalization claim is not arithmetically forced because the DPO training data are Python-only, but it is still measured by the same judge and the same pseudo-reference mechanism, so it inherits the same concern. A modest external anchor exists in the human ratings and the tool outputs fed into the prompts, so the circularity is partial rather than total.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on treating GPT-4o-mini as a trustworthy judge and on the faithfulness of static analysis outputs. Neither is independently verified beyond a small human study, and the judge is used on both sides of the training and evaluation loop.

free parameters (4)
  • DPO preference margin threshold (delta >= 2) = 2 (on a 1-5 scale)
    Hand-chosen cutoff for constructing preference pairs; affects which candidates are selected as wins or losses, but is not fitted to a benchmark.
  • DPO beta (regularization coefficient) = 0.1
    Chosen by hand (Section 3.2) to balance preference adherence against KL regularization.
  • Number of sampled candidates per instance = 20
    Hand-chosen; more candidates would improve pair quality at higher compute cost (Section 3.2).
  • SFT and DPO training epochs = 2
    Ad hoc choice; no tuning curve is reported.
assumptions (5)
  • domain assumption GPT-4o-mini's LLM-as-a-judge scores are valid proxies for code review quality along comprehensiveness, conciseness, and relevance.
    Used in Section 3.2 for DPO rewards and Section 3.3 for final evaluation; no independent validation beyond a small human study.
  • domain assumption Pseudo-references ('topics to be covered') generated by GPT-4o-mini are exhaustive and unbiased ground truths for review evaluation.
    Used in Section 3.3 and Appendix B.6; if these lists are incomplete or biased, the scores are biased.
  • domain assumption Static analysis outputs (Ruff, PyScent, PMD, JSNOSE) can be treated as verifiable facts about code quality.
    Section 3.1 and Evaluation; static analyzers are known to have false positives and negatives, but the paper treats them as ground truth.
  • standard math DPO loss (Rafailov et al. 2024) is a valid alignment objective for this setting.
    Equation 1 is taken from prior work; the paper does not re-derive or validate the objective.
  • domain assumption The CodeReviewer dataset split is representative of real code review tasks.
    All training and evaluation data comes from CodeReviewer (Section 3.1); no analysis of dataset bias is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review." pith.science (2026). https://pith.science/paper/JB3SLP4S

@misc{pith2026250600296,
  author       = {Pith},
  title        = {Pith review of: CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JB3SLP4S}},
  note         = {Machine review of arXiv:2506.00296}
}
read the original abstract

Reinforcement learning (RL) to improve code review comment generation requires handling unstructured outputs, making reinforcement learning (RL) feedback challenging. The two main RL approaches, namely RL with Verifiable Feedback (RLVR) and RL with AI Feedback (RLAIF), offer trade-offs: RLVR provides reliable feedback for structured tasks like code generation, while RLAIF works for unstructured outputs but is subjective. We bridge this gap with CRScore++, an RL framework that leverages both LLM-based subjective feedback and verifiable signals for training. Extending CRScore, a code review evaluation metric integrating LLMs with verifiers like linters and code smell detectors, CRScore++ transforms these signals into training rewards. We show that CRScore++ improves a weaker student model through a combination of supervised fine-tuning and RL critique from a stronger teacher model, thus enabling generalization to novel programming languages.

Figures

Figures reproduced from arXiv: 2506.00296 by the authors.

Figure 1
Figure 1. Three-stage training pipeline: (A) Tool-augmented distillation, (B) Preference ranking, (C) [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Evaluation framework combining static verification with LLM-as-a-judge [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 5 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, K...

    arXiv 2022

  4. [4]

    Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2022. http://arxiv.org/abs/2207.10397 Codet: Code generation with generated tests

    arXiv 2022

  5. [5]

    Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang, and Lin Yan. 2025. http://arxiv.org/abs/2410.17621 Process supervision-guided policy optimization for code generation

    arXiv 2025

  6. [6]

    Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Xuanjing Huang, and Tao Gui. 2024. http://arxiv.org/abs/2402.01391 Stepcoder: Improve code generation with reinforcement learning from compiler feedback

    arXiv 2024

  7. [7]

    Amin Milani Fard and Ali Mesbah. 2013. https://doi.org/10.1109/SCAM.2013.6648192 Jsnose: Detecting javascript code smells . In 2013 IEEE 13th International Working Conference on Source Code Analysis and Manipulation (SCAM), pages 116--125

    work page arXiv 2013

  8. [8]

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025. http://arxiv.org/abs/2504.11536 Retool: Reinforcement learning for strategic tool use in llms

    arXiv 2025

Show all 35 references
  1. [9]

    Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. 2025. Synthetic data generation & multi-step rl for reasoning & tool use. arXiv preprint arXiv:2504.04736

    2025 arXiv

  2. [11]

    Junda He, Jieke Shi, Terry Yue Zhuo, Christoph Treude, Jiamou Sun, Zhenchang Xing, Xiaoning Du, and David Lo. 2025 b . From code to courtroom: Llms as the new software judges. arXiv preprint arXiv:2503.02246

    2025 arXiv

  3. [12]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  4. [15]

    Imen Jaoua, Oussama Ben Sghaier, and Houari Sahraoui. 2025 c . Combining large language models with static analyzers for code review generation. arXiv preprint arXiv:2502.06633

    2025 arXiv

  5. [16]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2017. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization

    2017 arXiv

  6. [17]

    Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. 2022. http://arxiv.org/abs/2207.01780 Coderl: Mastering code generation through pretrained models and deep reinforcement learning

    2022 arXiv

  7. [18]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024. http://arxiv.org/abs/2309.00267 Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with...

    2024 arXiv

  8. [19]

    Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Xiang Wang, Bowen Yu, Binyuan Hui, Junyang Lin, and Dayiheng Liu. 2025. http://arxiv.org/abs/2503.04625 Start: Self-taught reasoner with tools

    2025 arXiv

  9. [20]

    Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, and Neel Sundaresan. 2022. http://arxiv.org/abs/2203.09095 Automating code review activities by large-scale pre-training

    2022 arXiv

  10. [21]

    Hong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, and Christoph Treude. 2025. Codereviewqa: The code review comprehension assessment for large language models. arXiv preprint arXiv:2503.16167

    2025 arXiv

  11. [22]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net

    2019

  12. [23]

    Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. 2024. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215

    2024 arXiv

  13. [24]

    Youssef Mroueh. 2025. http://arxiv.org/abs/2503.06639 Reinforcement learning with verifiable rewards: Grpo's effective loss, dynamics, and success amplification

    2025

  14. [25]

    Atharva Naik, Marcus Alenius, Daniel Fried, and Carolyn Rose. 2024. http://arxiv.org/abs/2409.19801 Crscore: Grounding automated evaluation of code review comments in code claims and smells

    2024 arXiv

  15. [26]

    OpenAI . 2024. https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/ Gpt-4o mini: advancing cost-efficient intelligence

    2024

  16. [27]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  17. [28]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  18. [29]

    Fabiano Pecorelli, Savanna Lujan, Valentina Lenarduzzi, Fabio Palomba, and Andrea De Lucia. 2022. On the adequacy of static analysis warnings with respect to code smell prediction. Empir. Softw. Eng., 27(3):64

    2022

  19. [30]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. http://arxiv.org/abs/2305.18290 Direct preference optimization: Your language model is secretly a reward model

    2024 arXiv

  20. [31]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. http://arxiv.org/abs/2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    2024 arXiv

  21. [32]

    Archit Sharma, Sedrick Scott Keh, Eric Mitchell, Chelsea Finn, Kushal Arora, and Thomas Kollar. 2024. A critical evaluation of ai feedback for aligning large language models. Advances in Neural Information Processing Systems, 37:29166--29190

    2024

  22. [33]

    Tushar Sharma. 2024. https://doi.org/10.1145/3643991.3644881 Multi-faceted code smell detection at scale using designitejava 2.0 . In Proceedings of the 21st International Conference on Mining Software Repositories, MSR '24, page 284–288, New York, NY, USA. Association for Com...

    2024

  23. [34]

    Tushar Sharma, Maria Kechagia, Stefanos Georgiou, Rohit Tiwari, Indira Vats, Hadi Moazen, and Federica Sarro. 2022. http://arxiv.org/abs/2110.09610 A survey on machine learning techniques for source code analysis

    2022 arXiv

  24. [35]

    Parshin Shojaee, Aneesh Jain, Sindhu Tipirneni, and Chandan K. Reddy. 2023. http://arxiv.org/abs/2301.13816 Execution-based code generation using deep reinforcement learning

    2023 arXiv

  25. [36]

    Rosalia Tufano, Simone Masiero, Antonio Mastropaolo, Luca Pascarella, Denys Poshyvanyk, and Gabriele Bavota. 2022. Using pre-trained models to boost code review automation. In Proceedings of the 44th international conference on software engineering, pages 2291--2302

    2022

  26. [38]

    Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, and Xin Xia. 2025 b . Can llms replace human evaluators? an empirical study of llm-as-a-judge in software engineering. arXiv preprint arXiv:2502.06193

    2025 arXiv

  27. [39]

    Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. 2023. Unifying the perspectives of nlp and software engineering: A survey on language models for code. arXiv preprint arXiv:2311.07989

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.