Pith. sign in

REVIEW 5 major objections 5 minor 92 references

Leveraging Reward Models for Guiding Code Review Comment Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that reinforcement learning with a reward based on a downstream code-refinement task improves code review comment generation beyond supervised fine-tuning and the current state of the art.

desk verdict Plausible first RL recipe for code review comment generation, but the BLEU-centric evaluation and an internally inconsistent LLM-judge study need fixing before the main claims can be trusted. read the letter →

arxiv 2506.04464 v1 pith:RK2BUOEG submitted 2025-06-04 cs.SE cs.LG

Oussama Ben Sghaier , Rosalia Tufano , Gabriele Bavota , Houari Sahraoui This is my paper · ORCID
classification cs.SEcs.LG
keywords codereviewcommentgenerationreinforcementlearningrewardmodelCrystalBLEUrefinementLLMfine-tuningautomated
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CoRAL, a framework that applies reinforcement learning to the task of generating code review comments. Its central claim is that fine-tuning a code language model with a reward signal tied to a downstream task improves the comments it produces: comments are rewarded when they are semantically similar to real human comments, or when they allow a separate code-refinement model to produce the correct revised code. The paper reports that every reward strategy tested improves over supervised fine-tuning, and that the best strategy, using CrystalBLEU, a code-aware similarity metric, to score the refined code, raises median BLEU from 7.05 to 8.67 and outperforms the DISCOREV approach. The motivation is that actionable, implementable review comments are more useful than fluent-sounding but vague ones, and that a reward based on successful refinement captures that usefulness.

What carries the argument

The load-bearing object is the reward function used in the reinforcement-learning fine-tuning stage. In the best-performing variant, the reward is computed by feeding the generated comment, together with the original code diff, into a code-refinement model and measuring how close the refined code it produces is to the actual revised code, using CrystalBLEU; a second variant uses the refinement model's loss as the reward. A separate reward strategy uses SBERT cosine similarity between the generated comment and the human-written comment. A KL-divergence penalty anchors the policy to the fine-tuned model to prevent degenerate text generation.

What would settle it

A human study in which experienced developers rate the usefulness of comments generated by CoRAL_crystal against the supervised baseline; if their preferences do not favor CoRAL despite its higher BLEU, the central claim is refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that reinforcement learning with a subsequent-task reward consistently helps review comment generation, regardless of the specific reward function adopted. CoRAL starts from a CodeLlama-7B model supervised-fine-tuned to produce a review comment from a code diff, then applies proximal policy optimization to maximize a reward. The reward is either the SBERT cosine similarity between the generated comment and the human-written comment, or the performance of a separately fine-tuned CodeLlama refinement model that tries to implement the generated comment, measured either by its loss or by CrystalBLEU. The CrystalBLEU variant achieves the highest median BLEU of 8.67 versus 7.05 for the supervised model and 7.51 for the comparison baseline, with statistically significant differences and large effect sizes, and in a language-model judge comparison CoRAL's comments are preferred over the baseline in 70 percent of pairs and over the comparison baseline in 55 percent.

Load-bearing premise

That BLEU scores are a valid stand-in for whether a review comment is useful to a developer.

Editorial extensions

If this is right

  • Reinforcement-learning fine-tuning with a subsequent-task reward improves review comment generation over supervised fine-tuning alone.
  • CrystalBLEU as a reward signal outperforms both semantic-similarity and loss-based rewards for comment generation.
  • The best CoRAL variant outperforms the comparison baseline, DISCOREV, on BLEU and in language-model-judged usefulness.
  • Reward-based training makes generated comments more likely to lead to correct code refinement by an automated model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is whether comments that earn high refinement-model rewards also help human developers act on them; if not, the reward may be exploiting the refinement model rather than measuring comment quality.
  • CoRAL's recipe suggests a general principle: reward a generative model by whether its output lets a downstream model succeed, which could transfer to other tasks with a subsequent execution step, such as generating documentation from code or bug reports from failing tests.
  • A follow-up could test whether the refinement-model reward transfers across different base language models, since both the generator and the reward model are CodeLlama-7B in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper presents CoRAL, a reinforcement-learning framework for code review comment generation. CoRAL first supervised fine-tunes CodeLlama-7B on comment generation, then applies PPO with one of three reward models: SBERT semantic similarity, the loss of a code-refinement model, or CrystalBLEU of the refined code. Using the 176,616-review-round dataset of Li et al., the paper answers four RQs: whether RL helps (RQ1), which reward is best (RQ2), whether CoRAL beats DISCOREV (RQ3), and whether judges find CoRAL comments more useful (RQ4). The main quantitative evidence is BLEU on the test set; RQ4 uses an LLM-as-judge with a human sanity check.

Significance. If the results were fully validated, CoRAL would be a useful contribution as the first RL-based approach for review comment generation, and the idea of rewarding subsequent-task correctness is compelling. The paper's strengths include a public replication package, reuse of an established split, re-training DISCOREV with CodeLlama-7B for a fairer comparison, statistical testing with effect sizes, and a human agreement check (Cohen's kappa 0.62) for the judge. However, the central claims currently rest on BLEU, an unvalidated metric for short review comments, and the judge experiment has unresolved inconsistencies, so the evidence base is not yet solid enough to accept the claimed superiority.

major comments (5)
  1. [§5.4 and §6.3] Section 5.4 states that the o3-mini judgment task 'was run for all 13,104 code reviews part of our test set,' while Section 6.3 and Figure 6 say each comparison was conducted on 1,000 comment pairs. Please reconcile these numbers, describe the sampling if 1,000 pairs were used, and report results for the full test set. As written, RQ4's usefulness claim is not reproducible or interpretable.
  2. [§5.4 and Table 1] The judge is called o3-mini in Section 5.4 but GPT-4 in the Introduction and Conclusion. The prompt in Table 1 also hardcodes 'Given the following Java file,' although Section 5.1 says the dataset is in nine languages. State which model was actually used and adapt the prompt to the code language; otherwise the judge results cannot be taken as a valid usefulness assessment.
  3. [§5.4 and Table 2] Table 2 reports negative Cliff's delta values for all CoRAL-vs-baseline comparisons (e.g., -0.851 for CoRAL_crystal vs CodeLlama_sft), but the text interprets these as CoRAL being better, and RQ3 reports a positive 0.76 for CoRAL_crystal vs DISCOREV. Under the standard convention, negative delta means the second group tends to be larger, which contradicts the reported medians (8.67 vs 7.05). Please state the sign convention or correct the computations; this is load-bearing for RQ1-RQ3.
  4. [§5.4, §6, and §9] The quantitative superiority claims in RQ1-RQ3 rely entirely on BLEU as a measure of review comment quality. BLEU is designed for machine translation and is a weak proxy for short, free-form comments; the paper itself calls for task-specific metrics in Section 9. To support the superiority claim, please add evidence that BLEU correlates with comment usefulness (e.g., a human or consistent LLM-judge study on a representative sample), or report additional metrics.
  5. [§6.1 and Table 3] The statement that 'RL always helps to boost the performance of the model, independently from the specific reward function adopted' goes beyond the evidence: the study covers one base LLM, one dataset, and one task, and the loss-based reward changes only from 0.69 to 0.68 in Table 3 without significance testing. Additionally, CoRAL_semantic vs CoRAL_loss is not significant (p=0.941 in Table 2). Please restrict the claim to the studied settings.
minor comments (5)
  1. [§1, §5.4, §9] The judge model is referred to as GPT-4 in the Introduction and Conclusion but as o3-mini in Section 5.4; please align the terminology throughout the paper.
  2. [Table 3] Table 3 contains a typo: 'CystalBLEU' should be 'CrystalBLEU.'
  3. [References] Reference [77] gives the first author as 'Tufan' while the text and other references use 'Tufano'; please correct the citation.
  4. [CCS Concepts] The CCS Concepts block still contains the ACM placeholder text 'Do Not Use This Code'; replace it with actual concepts.
  5. [Figure 4] Figure 4 boxplots lack axis labels and a caption describing what the whiskers and outliers represent; please add them.

Circularity Check

1 steps flagged · score 4.0 of 10

One supporting result (Table 3 reward increase) is circular because it reports the very objective that RL maximizes, but the central BLEU-based comparisons and DISCOREV baseline are independent.

  1. self definitional [Section 6.1, Table 3 and Equations 2-4]
    "The improved effectiveness of the model in executing the required task is also demonstrated by the increase in the reward provided to it during the RL-based training. Table 3 shows the initial and final reward provided to the CodeLlama model, with the initial one being the average reward (across all test set instances) assigned to CodeLlama_sft (i.e., the CodeLlama model which underwent standard fine-tuning without any RL step) and the final one being the same metric computed after the RL-based training."

    The final reward is produced by the same reward function R that the RL step is trained to maximize: Equation 2 sets r = r_theta - lambda r_KL and PPO updates the policy to maximize r. Thus reporting that the reward rises (or the refinement loss falls) after RL training is reporting movement of the training objective, not independent evidence of comment quality. The paper explicitly uses this increase to 'demonstrate improved effectiveness,' which is circular by construction. The step is supporting, not the sole basis of RQ1/RQ2: the central comparisons are BLEU scores on the test set, which are external to the reward.

full rationale

CoRAL's derivation chain is mostly self-contained. The RL variants are trained on the same dataset but evaluated with BLEU, an external metric not identical to the SBERT, loss, or CrystalBLEU rewards, so the RQ1/RQ2 claim that RL improves BLEU is not forced by construction. The comparison against DISCOREV uses the authors' prior work as a baseline, but it is retrained under CodeLlama-7B and evaluated with the same external BLEU metric, so this self-citation is not load-bearing. The only genuine circular step is the Table 3 presentation of reward/loss evolution as evidence of effectiveness: that quantity is the optimization objective itself. The LLM-as-judge inconsistencies (sample size, judge identity, hardcoded Java prompt) and the Cliff's delta sign inconsistencies are evaluation-validity issues, not circularity. Score 4 reflects one partial, non-central circular step while the main empirical claim retains independent content.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper's claims rest on standard ML assumptions and on the validity of its evaluation metrics. It introduces no invented entities and only one unreported free parameter, the KL penalty coefficient lambda.

free parameters (1)
  • KL penalty coefficient lambda
    In Eq. (2), the balance between reward and KL penalty is set by lambda, but its numeric value is never reported in the paper.
assumptions (4)
  • domain assumption BLEU is a valid measure of comment quality for code review.
    The primary quantitative evaluation uses BLEU to compare generated comments with human comments, but BLEU is designed for machine translation and may not capture the quality of short, free-form comments.
  • domain assumption o3-mini's usefulness judgments align with human judgments.
    The RQ4 evaluation relies on an LLM-as-judge approach, with a sanity check on 100 pairs showing substantial kappa, but the main results depend heavily on this assumption.
  • domain assumption The subsequent task reward, CrystalBLEU or loss, is a meaningful indicator of comment usefulness.
    The reward design assumes that a comment enabling better code refinement is a useful comment, which is stated in Section 4.3 but not directly validated.
  • domain assumption PPO with a fixed reward model and KL penalty yields stable and improved language model behavior.
    The RL fine-tuning relies on PPO and a KL penalty to prevent divergence, which is standard practice but not re-validated in this context.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Reward Models for Guiding Code Review Comment Generation." pith.science (2026). https://pith.science/paper/RK2BUOEG

@misc{pith2026250604464,
  author       = {Pith},
  title        = {Pith review of: Leveraging Reward Models for Guiding Code Review Comment Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RK2BUOEG}},
  note         = {Machine review of arXiv:2506.04464}
}
read the original abstract

Code review is a crucial component of modern software development, involving the evaluation of code quality, providing feedback on potential issues, and refining the code to address identified problems. Despite these benefits, code review can be rather time consuming, and influenced by subjectivity and human factors. For these reasons, techniques to (partially) automate the code review process have been proposed in the literature. Among those, the ones exploiting deep learning (DL) are able to tackle the generative aspect of code review, by commenting on a given code as a human reviewer would do (i.e., comment generation task) or by automatically implementing code changes required to address a reviewer's comment (i.e., code refinement task). In this paper, we introduce CoRAL, a deep learning framework automating review comment generation by exploiting reinforcement learning with a reward mechanism considering both the semantics of the generated comments as well as their usefulness as input for other models automating the code refinement task. The core idea is that if the DL model generates comments that are semantically similar to the expected ones or can be successfully implemented by a second model specialized in code refinement, these comments are likely to be meaningful and useful, thus deserving a high reward in the reinforcement learning framework. We present both quantitative and qualitative comparisons between the comments generated by CoRAL and those produced by the latest baseline techniques, highlighting the effectiveness and superiority of our approach.

Figures

Figures reproduced from arXiv: 2506.04464 by the authors.

Figure 1
Figure 1. CoRAL: framework overview 4.2 Supervised fine-tuning In the first step of our framework, we perform supervised finetuning of a large language model (LLM) specifically for the task of review comment generation. This involves training the LLM on a curated dataset that includes pairs of submitted code changes and their corresponding review comments. By exposing the model to a diverse array of examples, it learns to gen… view at source ↗
Figure 2
Figure 2. Reward models design The loss function in LLMs is traditionally designed to evaluate the exact correspondence between predicted outputs and the ground truth. Nonetheless, a multitude of other characteristics contribute to the quality of predictions, specifically review comments. Semantic similarity, for example, is an important characteristic as it allows the predicted comment to diverge lexically from the ground tr… view at source ↗
Figure 3
Figure 3. Detailed fine-tuning phase with reinforcement learning [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: BLEU score distribution across the test set for the different models [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Agreement rates between human and o3-mini. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Results of OpenAI o3-mini judgments. A Win indicates that o3-mini preferred CoRAL comment; a Loss means the baseline [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 45 canonical work pages

  1. [1]

    2000. PMD. https://pmd.github.io/

    2000

  2. [2]

    FindBugs

    2005. FindBugs. https://findbugs.sourceforge.net/

    2005

  3. [3]

    Amr Abo-eleneen, Ahammed Palliyali, and Cagatay Catal. 2023. The role of Reinforcement Learning in software testing. Information and Software Technology 164 (2023), 107325. https://doi.org/10.1016/j.infsof.2023.107325

    arXiv 2023

  4. [4]

    Frank Ackerman, Lynne S

    A. Frank Ackerman, Lynne S. Buchwald, and Frank H. Lewski. 1989. Software inspections: an effective verification process. IEEE software 6, 3 (1989), 31–36

    1989

  5. [5]

    David Adamo, Md Khorrom Khan, Sreedevi Koppula, and Renée Bryce. 2018. Reinforcement learning for android gui testing. In Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation . 2–8

    2018

  6. [6]

    Hamidreza Ahmadi, Mehrdad Ashtiani, Mohammad Abdollahi Azgomi, and Raana Saheb-Nassagh. 2022. A DQN-based agent for automatic software refactoring. Information and Software Technology 147 (2022), 106893

    2022

  7. [7]

    Areeg Ahmed, Shahira Azab, and Yasser Abdelhamid. 2023. Source-Code Generation Using Deep Learning: A Survey. In Progress in Artificial Intelligence, Nuno Moniz, Zita Vale, José Cascalho, Catarina Silva, and Raquel Sebastião (Eds.). Springer Nature Switzerland, Cham, 467–482

    2023

  8. [8]

    Open AI. 2019. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680 (2019)

    arXiv 2019

Show all 92 references
  1. [9]

    Wisam Haitham Abbood Al-Zubaidi, Patanamon Thongtanunam, Hoa Khanh Dam, Chakkrit Tantithamthavorn, and Aditya Ghose. 2020. Workload- Aware Reviewer Recommendation Using a Multi-Objective Search-Based Approach. InProceedings of the 16th ACM International Conference on Predictiv...

    2020

  2. [10]

    Ariyurek, A

    S. Ariyurek, A. Betin-Can, and E. Surer. 2019. Automated Video Game Testing Using Synthetic and Human-Like Agents. IEEE Transactions on Games (2019), 1–1. https://doi.org/10.1109/TG.2019.2947597 Manuscript submitted to ACM Leveraging Reward Models for Guiding Code Review Comme...

    2019

  3. [11]

    Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman. 2016. Managing technical debt in software engineering (Dagstuhl seminar 16162). In Dagstuhl Reports, Vol. 6. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik

    2016

  4. [12]

    Mojtaba Bagherzadeh, Nafiseh Kahani, and Lionel Briand. 2022. Reinforcement Learning for Test Case Prioritization. IEEE Transactions on Software Engineering 48, 8 (2022), 2836–2856. https://doi.org/10.1109/TSE.2021.3070549

    2022

  5. [13]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022)

    2022 arXiv

  6. [14]

    Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. 2020. Emergent Tool Use From Multi-Agent Autocurricula. ArXiv abs/1909.07528 (2020)

    2020 arXiv

  7. [15]

    Vipin Balachandran. 2013. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In 2013 35th International Conference on Software Engineering (ICSE) . IEEE, 931–940

    2013

  8. [16]

    Angela Barriga, Lawrence Mandow, José Luis Pérez de la Cruz, Adrian Rutle, Rogardt Heldal, and Ludovico Iovino. 2020. A comparative study of reinforcement learning techniques to repair models. In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Enginee...

    2020

  9. [17]

    Andrew G Barto. 2021. Reinforcement learning: An introduction. by richard’s sutton. SIAM Rev 6, 2 (2021), 423

    2021

  10. [18]

    Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1 (1995), 289–300

    1995

  11. [19]

    Bergdahl, C

    J. Bergdahl, C. Gordillo, K. Tollmar, and L. Gisslén. 2020. Augmenting Automated Game Testing with Deep Reinforcement Learning. In 2020 IEEE Conference on Games (CoG) . 600–603. https://doi.org/10.1109/CoG47356.2020.9231552

    2020

  12. [20]

    Pavol Bielik, Veselin Raychev, and Martin Vechev. 2017. Learning a static analyzer from data. In International Conference on Computer Aided Verification. Springer, 233–253

    2017

  13. [21]

    Amiangshu Bosu and Jeffrey C Carver. 2013. Impact of peer code review on peer impression formation: A survey. In 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement . IEEE, 133–142

    2013

  14. [22]

    Santo Carino and James H Andrews. 2015. Dynamically testing GUIs using ant colony optimization (t). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 138–148

    2015

  15. [23]

    Haoyang Chen, Botong Xu, and Kaiyang Zhong. 2024. Enhancing Software Effort Estimation through Reinforcement Learning-based Project Management-Oriented Feature Selection. arXiv preprint arXiv:2403.16749 (2024)

    2024 arXiv

  16. [24]

    Ting-Rui Chiang, Yi-Pei Chen, Yi-Ting Yeh, and Graham Neubig. 2021. Breaking down multilingual machine translation. arXiv preprint arXiv:2110.08130 (2021)

    2021 arXiv

  17. [25]

    Hyun Jae Cho and Madhur Behl. 2020. Towards automated safety coverage and testing for autonomous vehicles with reinforcement learning. arXiv preprint arXiv:2005.13976 (2020)

    2020 arXiv

  18. [26]

    Moataz Chouchen, Ali Ouni, Mohamed Wiem Mkaouer, Raula Gaikovina Kula, and Katsuro Inoue. 2021. WhoReview: A multi-objective search-based approach for code reviewers recommendation in modern code review. Applied Soft Computing 100 (2021), 106908

    2021

  19. [27]

    Eliane Collins, Arilo Neto, Auri Vincenzi, and José Maldonado. 2021. Deep reinforcement learning based android application gui testing. In Proceedings of the XXXV Brazilian Symposium on Software Engineering . 186–194

    2021

  20. [28]

    Aryaz Eghbali and Michael Pradel. 2022. CrystalBLEU: precisely and efficiently measuring the similarity of code. InProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering . 1–12

    2022

  21. [29]

    Stephen G Eick, Todd L Graves, Alan F Karr, J Steve Marron, and Audris Mockus. 2001. Does code decay? assessing the evidence from change management data. IEEE Transactions on Software Engineering 27, 1 (2001), 1–12

    2001

  22. [30]

    Morten W Fagerland and Leiv Sandvik. 2009. The wilcoxon–mann–whitney test under scrutiny. Statistics in medicine 28, 10 (2009), 1487–1497

    2009

  23. [31]

    Shuo Feng, Haowei Sun, Xintao Yan, Haojie Zhu, Zhengxia Zou, Shengyin Shen, and Henry X Liu. 2023. Dense reinforcement learning for safety validation of autonomous vehicles. Nature 615, 7953 (2023), 620–627

    2023

  24. [32]

    Martin Fowler and Matthew Foemmel. 2006. Continuous integration. https://martinfowler.com/articles/continuousIntegration.html

    2006

  25. [33]

    Leo Gao, John Schulman, and Jacob Hilton. 2023. Scaling laws for reward model overoptimization. In International Conference on Machine Learning . PMLR, 10835–10866

    2023

  26. [34]

    Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375 (2022)

    2022 arXiv

  27. [35]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144

    2020

  28. [36]

    Anshul Gupta and Neel Sundaresan. 2018. Intelligent code reviews using deep learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18) Deep Learning Day

    2018

  29. [37]

    Bakhta Haouari, Rania Mzid, and Olfa Mosbahi. 2023. On the Use of Reinforcement Learning for Real-Time System Design and Refactoring. In Intelligent Systems Design and Applications , Ajith Abraham, Sabri Pllana, Gabriella Casalino, Kun Ma, and Anu Bajaj (Eds.). Springer Nature...

    2023

  30. [38]

    Luke Harries, Rebekah Storan Clarke, Timothy Chapman, Swamy VPLN Nallamalli, Levent Ozgur, Shuktika Jain, Alex Leung, Steve Lim, Aaron Dietrich, José Miguel Hernández-Lobato, et al. 2020. Drift: Deep reinforcement learning for functional software testing.arXiv preprint arXiv:2...

    2020 arXiv

  31. [39]

    Matteo Hessel, Joseph Modayil, H. V. Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Gheshlaghi Azar, and David Silver. 2018. Rainbow: Combining Improvements in Deep Reinforcement Learning. In AAAI

    2018

  32. [40]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  33. [41]

    Yang Hong, Chakkrit Tantithamthavorn, Patanamon Thongtanunam, and Aldeida Aleti. 2022. Commentfinder: a simpler, faster, more accurate code review comments recommendation. In Proceedings of the 30th ACM joint European software engineering conference and symposium on the founda...

    2022

  34. [42]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  35. [43]

    James M Joyce. 2011. Kullback-leibler divergence. In International encyclopedia of statistical science . Springer, 720–722

    2011

  36. [44]

    Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun. 2021. Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation. arXiv preprint arXiv:2105.08919 (2021)

    2021 arXiv

  37. [45]

    Mark Koren, Saud Alsaif, Ritchie Lee, and Mykel J Kochenderfer. 2018. Adaptive stress testing for autonomous vehicles. In 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1–7

    2018

  38. [46]

    J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174

    1977

  39. [47]

    Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems 35 (2022), 21314–21328

    2022

  40. [48]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv e-prints (2023)

    2023

  41. [49]

    Sun-Ro Lee, Min-Jae Heo, Chan-Gun Lee, Milhan Kim, and Gaeul Jeong. 2017. Applying Deep Learning Based Automatic Bug Triager to Industrial Projects. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). 926–9...

    2017

  42. [50]

    Ruiyin Li, Peng Liang, and Paris Avgeriou. 2023. Code reviewer recommendation for architecture violations: An exploratory study. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering . 42–51

    2023

  43. [51]

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models

    2023

  44. [52]

    Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, et al. 2022. Automating code review activities by large-scale pre-training. In Proceedings of the 30th ACM Joint European Software Engineering Conf...

    2022

  45. [53]

    Zhixing Li, Yue Yu, Gang Yin, Tao Wang, Qiang Fan, and Huaimin Wang. 2017. Automatic Classification of Review Comments in Pull-based Development Model.. In SEKE. 572–577

    2017

  46. [54]

    Guillermo Macbeth, Eugenia Razumiejczyk, and Rubén Daniel Ledesma. 2011. Cliff’s Delta Calculator: A non-parametric effect size program for two groups of observations. Universitas Psychologica 10, 2 (2011), 545–555

    2011

  47. [55]

    Leonardo Mariani, Mauro Pezze, Oliviero Riganelli, and Mauro Santoro. 2012. Autoblacktest: Automatic black-box testing of interactive applications. In 2012 IEEE fifth international conference on software testing, verification and validation . IEEE, 81–90

    2012

  48. [56]

    Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2014. The impact of code review coverage and code review participation on software quality: A case study of the Qt, VTK, and ITK projects. In 11th working conference on mining software repositories . 192–201

    2014

  49. [57]

    Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering 21, 5 (2016), 2146–2189

    2016

  50. [58]

    Riedmiller

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. ArXiv abs/1312.5602 (2013)

    2013 arXiv

  51. [59]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charlie Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015

  52. [60]

    Rodrigo Morales, Shane McIntosh, and Foutse Khomh. 2015. Do code review practices impact design quality? A case study of the Qt, VTK, and ITK projects. In 2015 IEEE 22nd international conference on software analysis, evolution, and reengineering (SANER) . IEEE, 171–180

    2015

  53. [61]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  54. [62]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics . 311–318

    2002

  55. [63]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  56. [64]

    Replication Package [n. d.]. Replication Package. https://github.com/OussamaSghaier/RL4CR

  57. [65]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al

  58. [66]

    Caitlin Sadowski, Jeffrey Van Gogh, Ciera Jaspan, Emma Soderberg, and Collin Winter. 2015. Tricorder: Building a program analysis ecosystem. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering , Vol. 1. IEEE, 598–608. Manuscript submitted to ACM Leverag...

    2015

  59. [67]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  60. [68]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)

    2015 arXiv

  61. [69]

    Oussama Ben Sghaier and Houari Sahraoui. 2023. A Multi-Step Learning Approach to Assist Code Review. In2023 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) . IEEE

    2023

  62. [70]

    Oussama Ben Sghaier and Houari Sahraoui. 2024. Improving the Learning of Code Review Successive Tasks with Cross-Task Knowledge Distillation. arXiv preprint arXiv:2402.02063 (2024)

    2024 arXiv

  63. [71]

    Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5 (2017), 3909–3943

    2017

  64. [72]

    Jing Kai Siow, Cuiyun Gao, Lingling Fan, Sen Chen, and Yang Liu. 2020. Core: Automating review recommendation for code changes. In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 284–295

    2020

  65. [73]

    Nataša Sukur, Nemanja Milošević, Doni Pracner, and Zoran Budimac. 2024. Automated program improvement with reinforcement learning and graph neural networks. Soft Computing 28, 3 (2024), 2593–2604

    2024

  66. [74]

    Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken-ichi Matsumoto. 2015. Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In2015 IEEE 22nd International C...

    2015

  67. [75]

    Ahmed Tlili and Salim Chikhi. 2021. Risks analyzing and management in software project management using fuzzy cognitive maps with reinforcement learning. Informatica 45, 1 (2021)

    2021

  68. [76]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  69. [77]

    Rosalia Tufan, Luca Pascarella, Michele Tufanoy, Denys Poshyvanykz, and Gabriele Bavota. 2021. Towards automating code review activities. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . 163–174

    2021

  70. [78]

    Rosalia Tufano, Simone Masiero, Antonio Mastropaolo, Luca Pascarella, Denys Poshyvanyk, and Gabriele Bavota. 2022. Using pre-trained models to boost code review automation. arXiv preprint arXiv:2201.06850 (2022)

    2022 arXiv

  71. [79]

    Rosalia Tufano, Simone Scalabrino, Luca Pascarella, Emad Aghajani, Rocco Oliveto, and Gabriele Bavota. 2022. Using reinforcement learning for load testing of video games. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (IC...

    2022

  72. [80]

    Georgiev, A

    Oriol Vinyals, Timo Ewalds, Sergey Bartunov, P. Georgiev, A. S. Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, J. Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, S. Petersen, K. Simonyan, T. Schaul, H. V. Hasselt, D. Silver, T. Lillicrap, Kevin Calder...

    2017 arXiv

  73. [81]

    Thi Anh Tuyet Vuong and Shingo Takada. 2018. A reinforcement learning based approach to automated testing of android applications. InProceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation . 31–37

    2018

  74. [82]

    Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S. Yu. 2018. Improving automatic source code summarization via deep reinforcement learning. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE ’18) . 3...

    2018

  75. [83]

    Huanting Wang, Zhanyong Tang, Cheng Zhang, Jiaqi Zhao, Chris Cummins, Hugh Leather, and Zheng Wang. 2022. Automating reinforcement learning architecture design for code optimization. In Proceedings of the 31st ACM SIGPLAN International Conference on Compiler Construction . 129...

    2022

  76. [84]

    Yu, and Guandong Xu

    Wenhua Wang, Yuqun Zhang, Yulei Sui, Yao Wan, Zhou Zhao, Jian Wu, Philip S. Yu, and Guandong Xu. 2022. Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention. IEEE Transactions on Software Engineering 48, 1 (2022), 102–119. https://doi.org/10.1109...

    2022

  77. [85]

    Martin Weyssow, Aton Kamanda, and Houari Sahraoui. 2024. CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences. arXiv preprint arXiv:2403.09032 (2024)

    2024 arXiv

  78. [86]

    Yuechen Wu, Yingfeng Chen, Xiaofei Xie, Bing Yu, Changjie Fan, and Lei Ma. 2020. Regression Testing of Massively Multiplayer Online Role-Playing Games. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 692–696

    2020

  79. [87]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al

  80. [88]

    Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yingfeng Chen, and Changjie Fan. 2019. Wuji: Automatic online combat game testing using evolutionary deep reinforcement learning. In2019 34th IEEE/ACM International Conference on Automat...

    2019

  81. [89]

    Ming Zhu, Karthik Suresh, and Chandan K Reddy. 2022. Multilingual code snippets training for program translation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11783–11790

    2022

  82. [90]

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019). Manuscript submitted to ACM 24 Ben Sghaier et al. Rece...

    2019 arXiv

  83. [2023]

    arXiv preprint arXiv:2308.12950 (2023)

    Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023)

    2023 arXiv

  84. [2024]

    Advances in Neural Information Processing Systems 36 (2024)

    Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems 36 (2024)

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.