Pith. sign in

REVIEW 3 major objections 4 minor 159 references

Selective LLM reformulation of code-review comments yields comments that are both clearer and more human-like, improving automated generation and code refinement.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Selective LLM reformulation guided by high-quality exemplars yields cleaner, more diverse code-review datasets that improve downstream comment generation and code refinement.

T0 review reviewed 2026-07-13 challenge →

load-bearing objection Solid, usable dataset-curation paper for code-review automation; CuREV+ is a real methodological step, with residual LLM-judge circularity that is acknowledged and only partly mitigated. the 3 major comments →

arxiv 2607.09524 v1 pith:S6KTJL5O submitted 2026-07-10 cs.SE

Balancing Usefulness and Naturalness: An LLM-based Curation Pipeline for Code Review Comments

Oussama Ben Sghaier , Martin Weyssow , Houari Sahraoui This is my paper
classification cs.SE
keywords code reviewdataset curationlarge language modelscomment generationcode refinementdiversityLLM-as-a-judgesoftware maintenance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing public code-review datasets are noisy: comments are often vague, verbose, uncivil, or formulaic, so models trained on them produce weak feedback. This paper shows that two LLM-based curation pipelines can fix that. The first pipeline (CuREV) rewrites every retained comment for clarity, conciseness and civility; the second (CuREV+) keeps already-good human comments untouched and rewrites only the weak ones, using the good ones as in-context style exemplars. Both raise average clarity from ~6.9 to ~9 and eliminate uncivil language, but CuREV+ also restores lexical and stylistic diversity that pure rewriting destroys. Models trained or prompted with the curated comments produce better review text (BLEU rises from 7.7 to ~11) and more accurate code changes (CodeBLEU 0.36 o0.49). The central claim is therefore practical: dataset quality, not just model size, is the bottleneck for automated code review, and a selective, exemplar-guided rewrite is enough to remove that bottleneck while keeping comments natural.

Core claim

A selective, exemplar-guided curation pipeline (CuREV+) that preserves high-quality human comments and rewrites only low-quality ones produces review comments that are simultaneously clearer, more concise, fully civil, and stylistically diverse; models trained or prompted on this data measurably outperform those trained on the raw original dataset for both comment generation and code refinement.

What carries the argument

The dual-pipeline curation method: first an LLM-as-judge scores every comment on type/nature/civility/relevance/clarity/conciseness; then CuREV+ keeps comments above threshold unchanged and uses them as rotating in-context exemplars while the same LLM rewrites only the remaining low-quality comments.

Load-bearing premise

That a single large language model (with two secondary models for cross-check) can reliably score the six quality dimensions and that the chosen numeric cut-offs cleanly separate useful from useless comments.

What would settle it

Train identical models on the original Li et al. 2022 dataset versus CuREV+ and measure whether the claimed gains in BLEU (comment generation) and CodeBLEU/Exact-Match (code refinement) disappear, or run a human study showing that CuREV+ comments are judged less natural or less useful than the original comments.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Future code-review models can be trained on CuREV+ (or similarly curated data) and expect higher generation quality without larger architectures.
  • The same selective-rewrite pattern can be applied to other noisy software-engineering text corpora (issue reports, commit messages, forum answers).
  • Preserving a core of authentic high-quality human comments while rewriting the rest is a practical recipe for balancing standardization and naturalness in any instruction-tuning dataset.
  • Automated quality scoring of review comments becomes a reusable pre-processing step rather than an after-the-fact evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same pipeline were run with multiple independent LLMs as a jury rather than a single judge, residual self-preference bias in reformulation style might shrink further.
  • The diversity metrics (lower self-BLEU, richer prefixes) suggest CuREV+ could reduce mode collapse when used for reinforcement-learning-from-human-feedback of review agents.
  • Because the method already separates relevance from form, it could be extended to multi-language review datasets without requiring new quality rubrics for each language.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces two LLM-based curation pipelines for large-scale code-review comment datasets (Li et al. 2022). CuREV filters low-relevance comments and reformulates all remaining ones with Llama-3.1-70B for clarity, conciseness and civility while preserving intent. CuREV+ first partitions into high- and low-quality subsets via the same multi-dimensional evaluation framework (Type/Nature/Civility + Relevance/Clarity/Conciseness scores), retains high-quality human comments unchanged, and reformulates only the low-quality ones using randomly sampled high-quality comments as in-context exemplars. Both curated datasets (size 170 718) are re-scored with the framework, analysed for lexical/stylistic diversity (TTR, n-gram diversity, Self-BLEU, entropy, prefix n-grams), and used to fine-tune or prompt DeepSeek-Coder-6.7B on comment generation (BLEU) and code refinement (CodeBLEU + Exact Match). Results claim substantial quality lifts (clarity ~6.89 o8.95, conciseness 7.71 o8.53, 100 % civil), higher diversity for CuREV+, and consistent downstream gains (BLEU 7.71 o11.05–11.26; CodeBLEU 0.36 o0.44–0.49; EM 408 o445–463).

Significance. If the claims hold, the work supplies two publicly released, higher-quality training resources (CuREV / CuREV+) that measurably improve two core automated code-review tasks and offers a practical selective-curation recipe that trades off standardisation against natural stylistic diversity. Strengths that raise confidence include the open HuggingFace/GitHub/Zenodo artefacts, the multi-LLM cross-validation (Llama + MiniMax + GPT-5-nano Spearman correlations), the human sanity check (Cohen’s κ 0.64–1.0, quadratic weighted near-perfect), the threshold sensitivity table, BERTScore semantic-preservation numbers, and the end-to-end task improvements obtained with a different model family (DeepSeek). These elements make the contribution immediately usable by the SE community and provide a concrete template for future dataset-curation studies.

major comments (3)
  1. [§3.3–3.5, §4.1, Tables 5–6] Sections 3.3–3.5 and 4.1: the identical LLM-as-a-Judge framework (and largely the same model family) is used both to decide which comments are “poor” and therefore rewritten and later to quantify the quality gains of the rewritten set. Although a 100-sample human check and two secondary LLMs are supplied, the absolute score calibration remains weak (especially conciseness Spearman ρ≈0.34) and the human sample is not stratified by language or type. This circularity is load-bearing for the central quality claims; an independent larger human rating or an alternative non-LLM quality metric on a held-out stratified sample is needed to confirm that the reported lifts (clarity +30 %, conciseness +10.6 %) are not artefacts of the judge’s own stylistic preferences.
  2. [§6.1, Tables 15–16] Section 6.1 and Tables 15–16: CuREV’s BLEU gain (7.71 o11.26) coincides with extreme lexical homogenisation (“consider” appears as the opening token of ~126 k comments). The paper itself documents this formulaic pattern; consequently a non-negligible fraction of the n-gram improvement may be surface-form rather than semantic. Reporting BERTScore (or another embedding-based metric) between generated and reference comments, or an ablation that normalises for the dominant prefix, is required before the claim that “curated comments enable more accurate review-comment generation” can be fully accepted.
  3. [§6.2, §7.1] Section 6.2 / RQ5 and §7.1: usefulness is operationalised solely by CodeBLEU and Exact Match of the refined code. While these are reasonable automatic proxies, the manuscript acknowledges the absence of any developer perception or actionability study. Because the title and abstract emphasise “usefulness”, a modest human evaluation (even on a few hundred comments) or a clearer discussion of the proxy’s limitations is necessary for the usefulness claim to stand.
minor comments (4)
  1. [Fig. 3 / §3.6] Figure 3 reports Refactoring 69.7 % while the accompanying text (p. 9) states 80.07 %. Because Type is multi-label the percentages need not sum to 100, yet the two numbers should be reconciled or the multi-label nature made explicit in the figure caption.
  2. [Table 10] Table 10 and the surrounding prose mix absolute scores with relative percentages; a single consistent reporting style (e.g., absolute + Δ) would improve readability.
  3. [Tables 4, 8, 9] The prompt excerpts in Tables 4, 8 and 9 are helpful but truncated; the full prompts should be placed in an appendix or the replication package with a clear pointer.
  4. [throughout] A few minor typographical inconsistencies appear (e.g., “prescreptive”, “mmmm welcome…”). A final proof-reading pass is recommended.

Circularity Check

2 steps flagged

Mild circularity: same LLM-as-judge framework and criteria used both to select/reformulate comments and to claim quality gains; downstream BLEU/CodeBLEU remain external.

specific steps
  1. self definitional [Section 4.1 (CuREV/CuREV+ pipelines) + Section 4.2 (Results) + Table 10]
    "we re-evaluated the curated review comments using the same evaluation schema as was applied to the original dataset. ... Both curation strategies strongly enhanced clarity, with average scores rising from 6.89 in the original dataset to 8.96 in CuREV and 8.95 in CuREV+. ... CuREV+ raised the average conciseness score to 8.53"

    The reformulation prompt instructs the LLM to improve precisely the three scored attributes (clarity, conciseness, civility). The identical LLM-as-judge and scoring rubric are then reapplied to claim large gains on those same attributes. The reported quality lifts are therefore partly definitional rather than independently measured.

  2. fitted input called prediction [Section 4.1 (threshold selection) + Section 4.4 (Sensitivity Analysis) + Table 13]
    "we rely on the relevance scores obtained from the initial evaluation ... and set a threshold of 4. ... A comment is labeled as low-quality if it is uncivil or if either its clarity or conciseness score falls below the threshold of 7. ... ⋆ denotes the configuration used in our study."

    Thresholds that partition 'good' vs 'poor' (and therefore decide which comments are rewritten) are chosen from the same LLM scores that later serve as the success metric. Sensitivity analysis shows the chosen cut-offs produce the desired 54 % high-quality split, but the quality gains remain conditioned on those fitted cut-offs.

full rationale

The paper's central quality claims rest on re-scoring CuREV/CuREV+ with the identical evaluation framework (Type/Nature/Civility + Relevance/Clarity/Conciseness 1-10) and the same Llama-3.1-70B judge that drove filtering and reformulation. Reformulation prompts explicitly target clarity, conciseness and civility; re-evaluation then reports large lifts on exactly those axes (clarity 6.89 o8.95/8.96, conciseness 7.71 o8.05/8.53, civility 98.8% o100%). This is partially by construction. Mitigations exist and keep the score moderate: (1) a 100-sample dual-author human sanity check with high Cohen's/weighted kappa, (2) cross-architecture Spearman correlations on a 1 668-sample subset with MiniMax-M2.5 and GPT-5-nano, (3) BERTScore semantic-preservation numbers, and (4) fully external downstream metrics (BLEU on comment generation, CodeBLEU/Exact Match on code refinement) that do not reuse the quality scores. No self-definitional equations, no uniqueness theorems imported from the authors, and no fitted parameters renamed as predictions. The circularity is therefore limited to the intrinsic quality scores and does not force the utility claims.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

The central empirical claims rest on a small set of hand-chosen numeric thresholds, the assumption that a single strong LLM can serve as a reliable multi-dimensional judge, and the newly introduced evaluation taxonomy itself. No free physical constants; the free parameters are purely methodological cut-offs.

free parameters (3)
  • relevance_threshold = 4
    Comments scoring below 4 are discarded; sensitivity table shows the partition is stable only in a narrow band around this value.
  • quality_threshold_clarity_conciseness = 7
    Comments with clarity or conciseness < 7 (or uncivil) are labeled low-quality and rewritten; chosen by inspection rather than optimized against an external objective.
  • number_of_in_context_exemplars = 10
    Exactly 10 high-quality comments are sampled per rewrite; no ablation of this hyper-parameter is reported.
axioms (3)
  • domain assumption A sufficiently capable LLM (Llama-3.1-70B) can act as a reliable multi-dimensional judge of review-comment quality, with agreement comparable to human–human agreement.
    Stated in Section 3.3 and supported by a 100-sample sanity check; still an unproved general claim about LLM-as-judge reliability.
  • domain assumption Reformulation that preserves “core message or intent” does not alter the technical correctness or actionability of a review comment.
    Assumed throughout the reformulation prompts (Tables 8–9) and checked only via BERTScore F1 ≈ 0.90.
  • ad hoc to paper The six-dimensional taxonomy (Type, Nature, Civility, Relevance, Clarity, Conciseness) exhaustively captures review-comment quality for the purpose of dataset curation.
    Introduced in Section 3.2; derived from prior literature but the precise multi-label combination and 1–10 scales are paper-specific.
invented entities (2)
  • CuREV / CuREV+ curated datasets independent evidence
    purpose: Provide cleaner training corpora for code-review automation models.
    New public artifacts released with the paper; their utility is demonstrated only inside the paper’s own experiments.
  • Selective exemplar-guided reformulation pipeline no independent evidence
    purpose: Balance quality improvement against stylistic homogenization.
    The core methodological contribution; no external independent validation yet.

reviewed 2026-07-13 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Balancing Usefulness and Naturalness: An LLM-based Curation Pipeline for Code Review Comments." pith.science (2026). https://pith.science/paper/S6KTJL5O

@misc{pith2026260709524,
  author       = {Pith},
  title        = {Pith review of: Balancing Usefulness and Naturalness: An LLM-based Curation Pipeline for Code Review Comments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S6KTJL5O}},
  note         = {Machine review of arXiv:2607.09524}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Code review is a cornerstone of software development, where reviewers provide feedback through written comments to ensure code quality, maintainability, and correctness. The effectiveness of this process hinges on the quality of review comments. As large language models (LLMs) gain traction in automating code review tasks, the utility of these systems is directly limited by the quality of the datasets on which they are trained. Unfortunately, existing code review datasets are often noisy, inconsistent, or poorly structured, which hinders the ability of LLMs to learn to generate accurate, helpful, and human-like review comments. To overcome these limitations, we propose two different curation pipelines designed to improve both the quality and the utility of large-scale code review datasets. In the first pipeline, all review comments are systematically reformulated by an LLM to improve their clarity, conciseness, and civility while preserving their semantic intent. The curated dataset resulting from this approach, called CuREV, offers cleaner, higher-quality, and easier-to-learn-from comments that lead to measurable improvements in downstream automation tasks, namely review comment generation and code refinement. Building on this, we propose an improved pipeline, guided by high-quality exemplars, that enhances the realism and diversity of curated review comments. This method first separates the dataset into high-quality and low-quality reviews, based on a systematic quality assessment using an evaluation framework. High-quality comments are preserved in their original form and further used as in-context exemplars to inspire the reformulation of low-quality comments. By varying the exemplars provided, the reformulated comments are not only clearer and more actionable but also exhibit a broader range of writing styles, making them more realistic and human-like.

Figures

Figures reproduced from arXiv: 2607.09524 by Houari Sahraoui, Martin Weyssow, Oussama Ben Sghaier.

Figure 1
Figure 1. Figure 1: Overview of the proposed methodology. We begin by assessing the quality of the original code review [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evaluation framework for categorizing and scoring review comments. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of the different categories across the original dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of scoring criteria on the original dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the two curation pipelines. CuREV combines filtering and reformulation of poor-quality comments, while CuREV+ enhances this process by incorporating selection and in-context examples from high-quality reviews to guide reformulation. 4 Curated Code Review Datasets: CuREV and CuREV+ In this section, we first recall the curation pipeline proposed in our previous work (Sghaier et al., 2025), which … view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of the clarity and conciseness scoring criteria across curated datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

159 extracted references · 3 canonical work pages

  1. [1]

    arXiv preprint arXiv:2402.02063 , year=

    Improving the Learning of Code Review Successive Tasks with Cross-Task Knowledge Distillation , author=. arXiv preprint arXiv:2402.02063 , year=

    Pith/arXiv arXiv

  2. [2]

    2006 , howpublished=

    Continuous integration , author=. 2006 , howpublished=

    2006

  3. [3]

    2007 , publisher=

    Continuous integration: improving software quality and reducing risk , author=. 2007 , publisher=

    2007

  4. [4]

    2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) , pages=

    LLaMA-Reviewer: Advancing code review automation with large language models through parameter-efficient fine-tuning , author=. 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) , pages=. 2023 , organization=

    2023

  5. [5]

    arXiv preprint arXiv:2312.15698 , year=

    Repairllama: Efficient representations and fine-tuned adapters for program repair , author=. arXiv preprint arXiv:2312.15698 , year=

    Pith/arXiv arXiv

  6. [6]

    ACM Transactions on Software Engineering and Methodology , year=

    Large language models for software engineering: A systematic literature review , author=. ACM Transactions on Software Engineering and Methodology , year=

  7. [7]

    arXiv preprint arXiv:2308.10462 , year=

    Exploring parameter-efficient fine-tuning techniques for code generation with large language models , author=. arXiv preprint arXiv:2308.10462 , year=

    Pith/arXiv arXiv

  8. [8]

    ACM Transactions on Intelligent Systems and Technology , volume=

    A survey on evaluation of large language models , author=. ACM Transactions on Intelligent Systems and Technology , volume=. 2024 , publisher=

    2024

  9. [9]

    arXiv preprint arXiv:2304.14317 , year=

    ICE-Score: Instructing Large Language Models to Evaluate Code , author=. arXiv preprint arXiv:2304.14317 , year=

    Pith/arXiv arXiv

  10. [10]

    Advances in Neural Information Processing Systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in Neural Information Processing Systems , volume=

  11. [11]

    Expectations, outcomes, and challenges of modern code review , year=

    Bacchelli, Alberto and Bird, Christian , booktitle=. Expectations, outcomes, and challenges of modern code review , year=

  12. [12]

    The impact of code review coverage and code review participation on software quality: A case study of the

    McIntosh, Shane and Kamei, Yasutaka and Adams, Bram and Hassan, Ahmed E , booktitle=. The impact of code review coverage and code review participation on software quality: A case study of the

  13. [13]

    Empirical Software Engineering , volume=

    An empirical study of the impact of modern code review practices on software quality , author=. Empirical Software Engineering , volume=. 2016 , publisher=

    2016

  14. [14]

    IEEE software , volume=

    Software inspections: an effective verification process , author=. IEEE software , volume=. 1989 , publisher=

    1989

  15. [15]

    Software inspections and the industrial production of software , author=. Proc. of a symposium on Software validation: inspection-testing-verification-alternatives , pages=

  16. [16]

    International Conference on Computer Aided Verification , pages=

    Learning a static analyzer from data , author=. International Conference on Computer Aided Verification , pages=. 2017 , organization=

    2017

  17. [17]

    2015 IEEE/ACM 37th IEEE International Conference on Software Engineering , volume=

    Tricorder: Building a program analysis ecosystem , author=. 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering , volume=. 2015 , organization=

    2015

  18. [18]

    Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18) Deep Learning Day , year=

    Intelligent code reviews using deep learning , author=. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18) Deep Learning Day , year=

  19. [19]

    arXiv preprint arXiv:2201.06850 , year=

    Using pre-trained models to boost code review automation , author=. arXiv preprint arXiv:2201.06850 , year=

    Pith/arXiv arXiv

  20. [20]

    2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , pages=

    Towards automating code review activities , author=. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , pages=

    2021

  21. [21]

    arXiv preprint arXiv:1810.04805 , year=

    BERT: Pre-training of deep bidirectional transformers for language understanding , author=. arXiv preprint arXiv:1810.04805 , year=

    Pith/arXiv arXiv

  22. [22]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  23. [23]

    arXiv preprint arXiv:1907.11692 , year=

    RoBERTa: A robustly optimized BERT pretraining approach , author=. arXiv preprint arXiv:1907.11692 , year=

    Pith/arXiv arXiv 1907

  24. [24]

    arXiv preprint arXiv:2002.08155 , year=

    CodeBERT: A pre-trained model for programming and natural languages , author=. arXiv preprint arXiv:2002.08155 , year=

    Pith/arXiv arXiv 2002

  25. [25]

    Guo, Daya and Ren, Shuo and Lu, Shuai and Feng, Zhangyin and Tang, Duyu and Liu, Shujie and Zhou, Long and Duan, Nan and Svyatkovskiy, Alexey and Fu, Shengyu and others , journal=

  26. [26]

    Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc , journal=

  27. [27]

    2021 , school=

    Reducing unintended bias in text classification using multitask learning , author=. 2021 , school=

    2021

  28. [28]

    2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER) , pages=

    Core: Automating review recommendation for code changes , author=. 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER) , pages=. 2020 , organization=

    2020

  29. [29]

    arXiv preprint arXiv:1910.10683 , year=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. arXiv preprint arXiv:1910.10683 , year=

    Pith/arXiv arXiv 1910

  30. [30]

    Human factors , volume=

    Intelligent agent transparency in human--agent teaming for Multi-UxV management , author=. Human factors , volume=. 2016 , publisher=

    2016

  31. [31]

    2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI , pages=

    Improving robot controller transparency through autonomous policy explanation , author=. 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI , pages=. 2017 , organization=

    2017

  32. [32]

    2014 , institution=

    Situation awareness-based agent transparency , author=. 2014 , institution=

    2014

  33. [33]

    2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pages=

    Four eyes are better than two: On the impact of code reviews on software quality , author=. 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pages=. 2015 , organization=

    2015

  34. [34]

    Software pioneers , pages=

    Design and code inspections to reduce errors in program development , author=. Software pioneers , pages=. 2002 , publisher=

    2002

  35. [35]

    Do code review practices impact design quality? A case study of the

    Morales, Rodrigo and McIntosh, Shane and Khomh, Foutse , booktitle=. Do code review practices impact design quality? A case study of the. 2015 , organization=

    2015

  36. [36]

    Software Engineering Journal , volume=

    Industrial perspective on static analysis , author=. Software Engineering Journal , volume=. 1995 , publisher=

    1995

  37. [37]

    IEEE Transactions on Software Engineering , volume=

    What types of defects are really discovered in code reviews? , author=. IEEE Transactions on Software Engineering , volume=. 2008 , publisher=

    2008

  38. [38]

    IEEE Transactions on Software Engineering , volume=

    Does code decay? assessing the evidence from change management data , author=. IEEE Transactions on Software Engineering , volume=. 2001 , publisher=

    2001

  39. [39]

    arXiv preprint arXiv:1604.01450 , year=

    Does technical debt lead to the rejection of pull requests? , author=. arXiv preprint arXiv:1604.01450 , year=

    Pith/arXiv arXiv

  40. [40]

    Managing technical debt in software engineering (

    Avgeriou, Paris and Kruchten, Philippe and Ozkaya, Ipek and Seaman, Carolyn , booktitle=. Managing technical debt in software engineering (. 2016 , organization=

    2016

  41. [41]

    IEEE Access , volume=

    Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices , author=. IEEE Access , volume=. 2017 , publisher=

    2017

  42. [42]

    Advances in neural information processing systems , volume=

    XLNet: Generalized autoregressive pretraining for language understanding , author=. Advances in neural information processing systems , volume=

  43. [43]

    OpenAI blog , volume=

    Language models are unsupervised multitask learners , author=. OpenAI blog , volume=

  44. [44]

    Ensemble machine learning , pages=

    Ensemble learning , author=. Ensemble machine learning , pages=. 2012 , publisher=

    2012

  45. [45]

    arXiv preprint arXiv:1508.07909 , year=

    Neural machine translation of rare words with subword units , author=. arXiv preprint arXiv:1508.07909 , year=

    Pith/arXiv arXiv

  46. [46]

    Artificial intelligence review , volume=

    A perspective view and survey of meta-learning , author=. Artificial intelligence review , volume=. 2002 , publisher=

    2002

  47. [47]

    Automated Machine Learning , pages=

    Meta-learning , author=. Automated Machine Learning , pages=. 2019 , publisher=

    2019

  48. [48]

    IEEE Circuits and systems magazine , volume=

    Ensemble based systems in decision making , author=. IEEE Circuits and systems magazine , volume=. 2006 , publisher=

    2006

  49. [49]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Automatic code review by learning the revision of source code , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  50. [50]

    Applied Soft Computing , volume=

    WhoReview: A multi-objective search-based approach for code reviewers recommendation in modern code review , author=. Applied Soft Computing , volume=. 2021 , publisher=

    2021

  51. [51]

    Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

    Towards automating code review at scale , author=. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

  52. [52]

    Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

    Automating code review activities by large-scale pre-training , author=. Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

  53. [53]

    arXiv preprint arXiv:2109.00859 , year=

    Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation , author=. arXiv preprint arXiv:2109.00859 , year=

    Pith/arXiv arXiv

  54. [54]

    Acm sigplan notices , volume=

    Finding bugs is easy , author=. Acm sigplan notices , volume=. 2004 , publisher=

    2004

  55. [55]

    Proceedings of the 19th international symposium on Software testing and analysis , pages=

    The google findbugs fixit , author=. Proceedings of the 19th international symposium on Software testing and analysis , pages=

  56. [56]

    Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

    Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

  57. [57]

    arXiv preprint arXiv:1503.02531 , year=

    Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

    Pith/arXiv arXiv

  58. [58]

    2016 , eprint=

    Sequence-Level Knowledge Distillation , author=. 2016 , eprint=

    2016

  59. [59]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Ye, Han-Jia and Lu, Su and Zhan, De-Chuan , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

  60. [60]

    CKD: Cross-Task Knowledge Distillation for Text-to-Image Synthesis , year=

    Yuan, Mingkuan and Peng, Yuxin , journal=. CKD: Cross-Task Knowledge Distillation for Text-to-Image Synthesis , year=

  61. [61]

    2022 , eprint=

    Prototype-guided Cross-task Knowledge Distillation for Large-scale Models , author=. 2022 , eprint=

    2022

  62. [62]

    International encyclopedia of statistical science , pages=

    Kullback-leibler divergence , author=. International encyclopedia of statistical science , pages=. 2011 , publisher=

    2011

  63. [63]

    arXiv preprint arXiv:2105.08919 , year=

    Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation , author=. arXiv preprint arXiv:2105.08919 , year=

    Pith/arXiv arXiv

  64. [64]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Distilling cross-task knowledge via relationship matching , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  65. [65]

    IEEE Transactions on Multimedia , volume=

    CKD: Cross-task knowledge distillation for text-to-image synthesis , author=. IEEE Transactions on Multimedia , volume=. 2019 , publisher=

    2019

  66. [66]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Cross-task knowledge distillation in multi-task recommendation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  67. [67]

    arXiv preprint arXiv:2212.13180 , year=

    Prototype-guided Cross-task Knowledge Distillation for Large-scale Models , author=. arXiv preprint arXiv:2212.13180 , year=

    Pith/arXiv arXiv

  68. [68]

    Communications of the ACM , volume=

    Generative adversarial networks , author=. Communications of the ACM , volume=. 2020 , publisher=

    2020

  69. [69]

    arXiv preprint arXiv:2206.11719 , year=

    AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models , author=. arXiv preprint arXiv:2206.11719 , year=

    Pith/arXiv arXiv

  70. [70]

    2023 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) , year=

    A Multi-Step Learning Approach to Assist Code Review , author=. 2023 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) , year=

    2023

  71. [71]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Multilingual code snippets training for program translation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  72. [72]

    arXiv preprint arXiv:2110.08130 , year=

    Breaking down multilingual machine translation , author=. arXiv preprint arXiv:2110.08130 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2009.10297 , year=

    Codebleu: a method for automatic evaluation of code synthesis , author=. arXiv preprint arXiv:2009.10297 , year=

    Pith/arXiv arXiv 2009

  74. [74]

    doi:10.5281/zenodo.10676741 , url =

    Ben Sghaier, Oussama , title =. doi:10.5281/zenodo.10676741 , url =

    work page doi:10.5281/zenodo.10676741

  75. [75]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  76. [76]

    International Conference on Machine Learning , pages=

    Scaling laws for reward model overoptimization , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  77. [77]

    arXiv preprint arXiv:2209.14375 , year=

    Improving alignment of dialogue agents via targeted human judgements , author=. arXiv preprint arXiv:2209.14375 , year=

    Pith/arXiv arXiv

  78. [78]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    Pith/arXiv arXiv

  79. [79]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  80. [80]

    2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , pages=

    Impact of peer code review on peer impression formation: A survey , author=. 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , pages=. 2013 , organization=

    2013

Showing first 80 references.

This paper was first reviewed by grok-4.5 on July 13, 2026.