Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Intention is All You Need: Refining Your Code from Your Intention

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that code refinement improves when the reviewer's comment is first converted into a templated intention and the revised code is generated from that intention, reaching up to 66% exact-match accuracy versus about 54% for…

desk verdict Promising decomposition of code refinement into intent extraction and intent-guided generation, but the RAG evaluation leaks ground-truth test examples into the prompt, making the headline numbers untrustworthy. read the letter →

arxiv 2502.08172 v1 pith:YOGINBPZ submitted 2025-02-12 cs.SE

Qi Guo , Xiaofei Xie , Shangqing Liu , Ming Hu , Xiaohong Li , Lei Bu This is my paper · ORCID
classification cs.SE
keywords coderefinementintentionextractionlargelanguagemodelsreviewpromptengineeringretrieval-augmentedgenerationexactmatchdatacleaning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that code refinement — turning a code-review comment into revised code — fails when models are asked to jump straight from the comment to the code, because review comments are often vague, noisy, and mixed with explanations and pleasantries. The proposed remedy is to split the task into two stages: first extract a structured intention from the comment (for example, "change word A to B", "revert the last change", or "delete these lines"), then generate the revised code from that intention instead of from the raw comment. The authors test this framework on a code-review dataset with five large language models and report that the decomposition raises exact-match accuracy for every model, with GPT-4o going from 54.08% with the best end-to-end prompting baseline to 64.77% with the framework. They also report 79% accuracy in extracting the correct intention and show that intention-based checks clean low-quality refinement data more precisely (90.73% precision) than comment-based checks (79.38%). A sympathetic reader would take the central claim to be that the comment-to-intention step is the bottleneck, and making that step explicit and structured is what drives better refinement.

What carries the argument

The load-bearing mechanism is the intention template: a constrained natural-language schema into which every review comment is mapped before code generation. Three categories and eight subcategories partition reviewer intent — explicit code-block insertions, reversions of the last diff hunk, and six general patterns covering word-level and code-level changes and deletions, both single- and multi-line. The templates do the work because they convert an open-ended comment into a fixed-format instruction with named slots, so the revision generator operates on slot values rather than on raw comment prose. A complementary mechanism is the rule-based post-processing that repairs LLM outputs by enforcing inclusion of suggestion code, preservation of untouched lines, and comment consistency, plus the deterministic reversion path that directly undoes the last diff hunk when a reversion intention is detected.

What would settle it

Re-run the same five-model comparison on the full unfiltered 2,000-sample set, or on a fresh unfiltered sample from the same dataset, and check whether the intention-based framework's exact-match advantage over the best end-to-end baseline survives; if the advantage shrinks toward zero on the unfiltered data, the manual filtering was doing the work.

Watch

Extended reading notes

Core claim

The central discovery is that the semantic gap between review comments and revised code can be bridged by an intermediate representation the paper calls the intention: a short, templated statement of the modification the reviewer wants. The framework defines three categories covering eight intention types. Explicit code suggestions are comments that include a suggestion code block, detected by regular expressions. Reversion suggestions convey that the previous modification should be undone; these are identified with an LLM that consults the last code diff hunk and matches the comment against paraphrased reversion patterns. Everything else falls into general suggestions, which are mapped by an LLM onto six templates: word-level change, word-level deletion, code-level change, code-level deletion, multi-line deletion, and multi-line change. The extracted intention, together with the original code and the review line, is then fed to an LLM that produces the revised code, and rule-based post-processing repairs the output by checking that suggested code appears verbatim, untouched lines remain unchanged, and comment consistency is preserved. The paper reports that this two-stage pipeline outperforms end-to-end LLM prompting across all five tested models, and that the rule-based reversion branch alone contributes gains of 57 to 76 percentage points on reversion samples.

Load-bearing premise

The gains are measured on 1,337 manually filtered samples from which 663 low-quality cases were removed; if that filtering disproportionately removes cases where end-to-end methods would have done relatively better, the reported improvements could be inflated.

Editorial extensions

If this is right

  • If the central claim holds, code-refinement systems should route every review comment through an explicit intention-extraction step instead of prompting an LLM to rewrite code end-to-end.
  • The rule-based reversion component contributes the largest per-category gains, so making reversible changes a deterministic revert of the last diff hunk is the most reliable part of the pipeline.
  • Because weak models sometimes extract wrong intentions that then misguide generation, the framework implies separating the intention extractor from the generator and using the strongest available model for extraction.
  • The same intention signal can be used to filter noisy training data: intention-based verification reaches 90.73% precision versus 79.38% for comment-based verification, which would improve the quality of future code-refinement datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would expect the framework's advantage to concentrate in the reversion branch, which injects diff-hunk knowledge that generic prompts do not carry; ablating that branch in a controlled study would separate the value of structured input from the value of intention semantics.
  • The six general-suggestion templates are still coarse, so a natural extension is to refine them into executable operations such as rename, extract-method, or change-argument-order, letting deterministic tools or smaller models handle each one.
  • A testable extension suggested by the 'GPT-4o intentions' column is to feed high-quality extracted intentions to small open-weights models and measure whether the performance gap against GPT-4o narrows more than with end-to-end prompting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes an intention-based code refinement framework that decomposes the comment-to-code task into two stages: (1) extracting a structured 'intention' from the review comment using a taxonomy of three categories (explicit code suggestion, reversion suggestion, general suggestion, with six general templates) via a hybrid rule/LLM pipeline, and (2) generating revised code from the extracted intention using LLMs with simple, RAG, or self-generated prompting. The authors supplement the CodeReview dataset with ReviewLine and LastCodeDiffHunk, manually filter 2,000 samples to 1,337 'clean' cases with inter-annotator agreement of kappa=0.719, and evaluate five LLMs. They report 79% intention-extraction accuracy (GPT4o) and up to 66% exact-match (EM) on code refinement, outperforming end-to-end LLM baselines (e.g., GPT4o 54.08% best baseline vs. 64.77% with the framework), and an intention-based data-cleaning method with 81.45% accuracy and 90.73% precision.

Significance. If the results survive a leakage-free re-evaluation, the two-stage decomposition is a practical and model-agnostic contribution: the reversion-suggestion branch is rule-based and shows large, interpretable gains; the taxonomy and prompt templates are described in enough detail to reproduce; and the manual annotation effort (Cohen's kappa 0.719) is a credible basis for the cleaned set. The paper also makes a falsifiable prediction about intention quality (higher-quality intentions improve downstream EM). Current confidence is limited by the internal RAG test-set leakage, the under-specified filtering protocol for baselines, and the absence of statistical reliability measures.

major comments (4)
  1. [IV-B / V-B] The RAG retrieval database is built from the same evaluation set. Section IV-B states the database is constructed from the 2,000 RQ1 samples, and Section IV-A says 663 are removed as invalid, leaving 1,337. Table I identifies this 1,337-sample set as the 'All clean data' test set for RQ1, and RQ2 is evaluated on the same clean data (Table II). For each test case, the RAG prompt retrieves three BM25-similar examples from this database, and these examples contain the ground-truth RevisedCode. Excluding only the test case itself does not prevent leakage: the model observes ground-truth revisions from the test distribution, selected for similarity, before generating its answer. This inflates the absolute EM numbers (e.g., 64.77% for GPT4o) and weakens the conclusion that 'RAG is stable.' Section VI-B's data-leakage discussion addresses possible LLM training-data leakage only, not this internal test-set leakage. Please rebuild the retrieval database from a split disjoint from the evaluation set (e.g., the rest of the CodeReview dataset) and re-report both absolute and relative results.
  2. [IV-A / V-B] The filtering step and the exact evaluation set used by the baselines are under-specified. Section IV-A says 663 of 2,000 samples were excluded because 'the revised code was unrelated to the review comments,' and Section V-B then compares the intention framework with LLM baselines. The paper never states explicitly that every baseline row in Tables II and III is computed on the same 1,337 cleaned samples. If baselines were evaluated on a different subset (e.g., the original 2,000), the comparison is not head-to-head. Please state the exact evaluation set for every row and report results on the full 2,000-sample set as a sensitivity check, so the reader can judge whether the filtering drives the reported gains.
  3. [V-B / Tables II-III] No statistical reliability evidence is reported. All EM scores in Tables II-V appear to be single runs with unspecified sampling parameters; differences between prompt strategies are often small (e.g., GPT4o RAG 64.77 vs. Self-generated 65.97 in Table III). Without confidence intervals, bootstrap or repeated runs, or a significance test, it is impossible to tell whether the 2-5 point differences reflect real improvements or sampling noise. Please report the number of runs, temperature, and variance or intervals for at least the main comparisons.
  4. [V-A / Table I] The intention-accuracy metric is permissive for overlapping categories. Section V-A notes that 20% of Reversion Suggestions also fall under General Suggestions and that 'predicting the intention as either category is considered correct.' That rule inflates the headline 78.61%/79% accuracy, because a model that predicts only the general category receives credit for those reversion cases. Please report a strict template-level matching accuracy alongside the tolerant metric, and describe exactly how the manual assessment of correctness was performed (e.g., whether placeholders in the six templates had to match the human-annotated spans).
minor comments (5)
  1. [Fig. 3] The RAG prompt includes the phrase 'Let's think step by step' whereas the Simple Prompt does not; this confounds retrieval augmentation with chain-of-thought prompting. Please separate these factors or justify the design choice.
  2. [Table II] The heading 'Tufuno COT' is a typo for 'Tufano COT'; please ensure all prompt names are consistent with the descriptions in Section IV-B.
  3. [IV-A] The random selection of 2,000 samples is not accompanied by a random seed or the exact sampling procedure; adding this would improve reproducibility.
  4. [II-C] The supplementary website [31] is cited for detailed prompts, but providing the prompts and annotation instructions in an appendix would make the paper more self-contained.
  5. [V-B] The paper dismisses BLEU and CodeBLEU as not useful for this task, but a supplementary table with these metrics would help readers calibrate the strict EM results, especially for the general-suggestion category.

Circularity Check

1 steps flagged · score 6.0 of 10

RAG evaluation is partially circular: the retrieval database is the same 1,337-sample clean test pool, so RAG prompts see ground-truth RevisedCode from the test set; this contaminates the headline RAG numbers, though the Simple-Prompt comparison gives the core framework some independent support.

  1. other [Section IV-B (RQ2), 'RAG Prompt' paragraph; Table I (All clean data #Samples 1337)]
    "The retrieval database is constructed from the dataset used in RQ1, specifically the 2,000 randomly selected samples from the CodeReview dataset. However, only 1,337 of these samples are included in the database, as the remaining 663 cases exhibit low-quality refinements that do not align well with the review comments. ... If the test data is included in the retrieval results, it is excluded and replaced with another sample."

    Table I reports the RQ1 evaluation set as 'All clean data' with '#Samples 1337', and Section V-A says '1,337 out of a total of 2,000 identified as valid.' The RAG value field 'includes OriginalCode, Intention, ReviewLine, and RevisedCode', so each retrieved exemplar carries the ground-truth RevisedCode. Since the test data itself is in the same 1,337-sample pool (only itself is replaced when retrieved), every RAG test case is preceded by up to three labeled test-set revisions selected by BM25 similarity, then exact-matched against that same pool. The RAG-based 'prediction' is therefore partly constructed from the target labels. The paper's Threats to Validity acknowledges only LLM training-data leakage, not this internal test-set leakage.

full rationale

The intention taxonomy and two-phase decomposition are not circular: the templates come from a separate 1,100-commit preliminary study, and the Simple-Prompt results (e.g., GPT4o 64.10 with the framework vs. 46.60 for the baseline) do not depend on the retrieval database. However, the RAG-based evaluation in RQ2 is contaminated by construction. Section IV-B states that the retrieval database is the RQ1 2,000-sample set filtered to the same 1,337 'clean' samples reported in Table I, and that each test case retrieves three samples while only excluding itself. The retrieved exemplars' values include RevisedCode, the ground-truth target. Thus every RAG test case is preceded by up to three labeled test-set revisions selected by BM25 similarity, and the resulting exact-match scores are not a clean measure of refinement ability. The 'RAG is stable' conclusion and the absolute 'up to 66%' headline are affected. The Threats to Validity section only discusses LLM training-data leakage, not this internal test-set leakage. The central decomposition claim is not forced by definition, so this is partial circularity rather than full reduction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The framework relies on a hand-built taxonomy of intentions and on the assumption that the filtered CodeReviewer dataset and reconstructed diff hunks are faithful. These are domain assumptions rather than fitted parameters, but the taxonomy granularity and RAG shot count are free choices that affect results.

free parameters (2)
  • Intention taxonomy granularity = 3 categories, 8 subcategories
    Hand-designed from a preliminary study of 1,100 commits; the taxonomy determines what intentions can be extracted and thus the ceiling of the approach.
  • Number of retrieved examples in RAG prompt = 3
    Set to 3-shot without reported ablation; retrieval-augmented generation performance is sensitive to this hyperparameter.
assumptions (4)
  • domain assumption The CodeReviewer dataset ground-truth RevisedCode correctly reflects reviewer intent in the 1,337 filtered samples.
    Used as the evaluation gold standard throughout RQ2-RQ4 (Sections IV and V).
  • domain assumption Human annotation of intention correctness (Cohen's kappa 0.719) is a reliable proxy for true reviewer intentions.
    Used to measure intention extraction accuracy (Section V-A).
  • domain assumption The GitHub API-based reconstruction of LastCodeDiffHunk yields the exact previous code modification.
    Used to define reversion suggestions and to rule-based revert changes (Section II-C).
  • domain assumption LLMs fill the predefined intention templates faithfully enough to guide revision generation.
    The whole framework depends on LLMs mapping comments to the hand-defined templates (Section III-B).
invented entities (1)
  • Intention template
    purpose: Structured intermediate representation of reviewer intent used to guide code generation.
    This is a paper-defined construct; no external falsifiable handle, but it is evaluated indirectly through downstream refinement accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intention is All You Need: Refining Your Code from Your Intention." pith.science (2026). https://pith.science/paper/YOGINBPZ

@misc{pith2026250208172,
  author       = {Pith},
  title        = {Pith review of: Intention is All You Need: Refining Your Code from Your Intention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YOGINBPZ}},
  note         = {Machine review of arXiv:2502.08172}
}
read the original abstract

Code refinement aims to enhance existing code by addressing issues, refactoring, and optimizing to improve quality and meet specific requirements. As software projects scale in size and complexity, the traditional iterative exchange between reviewers and developers becomes increasingly burdensome. While recent deep learning techniques have been explored to accelerate this process, their performance remains limited, primarily due to challenges in accurately understanding reviewers' intents. This paper proposes an intention-based code refinement technique that enhances the conventional comment-to-code process by explicitly extracting reviewer intentions from the comments. Our approach consists of two key phases: Intention Extraction and Intention Guided Revision Generation. Intention Extraction categorizes comments using predefined templates, while Intention Guided Revision Generation employs large language models (LLMs) to generate revised code based on these defined intentions. Three categories with eight subcategories are designed for comment transformation, which is followed by a hybrid approach that combines rule-based and LLM-based classifiers for accurate classification. Extensive experiments with five LLMs (GPT4o, GPT3.5, DeepSeekV2, DeepSeek7B, CodeQwen7B) under different prompting settings demonstrate that our approach achieves 79% accuracy in intention extraction and up to 66% in code refinement generation. Our results highlight the potential of our approach in enhancing data quality and improving the efficiency of code refinement.

Figures

Figures reproduced from arXiv: 2502.08172 by the authors.

Figure 1
Figure 1. The framework of our intention-based code refinement. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The prompt used for extracting general intention. • Expressing an opinion: You still need this code. • Expressing an opinion: Change the code back. • Raising a question: Why delete this code? • Raising a question: Why did you do this? • Giving a suggestion: You should add another piece of code. Our preliminary study shows that the first five types cover the main deletion-related reversions, while the last one indica… view at source ↗
Figure 3
Figure 3. The format of the used prompt in the generation. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The used prompt format for different tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding

    cs.HC 2025-05 conditional novelty 7.0 of 10

    HiLDe, a code completion UI that exposes and lets users override the LLM's token-level choices, reduced security vulnerabilities in generated code compared to a baseline assistant in a within-subjects study of 18 programmers.

Reference graph

Works this paper leans on

49 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Expectations, outcomes, and challenges of modern code review,

    A. Bacchelli and C. Bird, “Expectations, outcomes, and challenges of modern code review,” in2013 35th International Conference on Software Engineering (ICSE). IEEE, 2013, pp. 712–721

    2013

  2. [2]

    Convergent contemporary software peer review practices,

    P. C. Rigby and C. Bird, “Convergent contemporary software peer review practices,” in Proceedings of the 2013 9th joint meeting on foundations of software engineering , 2013, pp. 202–212

    2013

  3. [3]

    A history of software inspections,

    M. Fagan, “A history of software inspections,” Software pioneers: contributions to software engineering , pp. 562–573, 2002

    work page 2002

  4. [4]

    The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects,

    S. McIntosh, Y . Kamei, B. Adams, and A. E. Hassan, “The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects,” in Proceedings of the 11th working conference on mining software repositories, 2014, pp. 192–201

    work page 2014

  5. [5]

    Code review survey study,

    L. Autio, “Code review survey study,” https://dev.solita.fi/2022/01/20/ code-reviews.html, 2022

    work page 2022

  6. [6]

    Modern code review: a case study at google,

    C. Sadowski, E. S ¨oderberg, L. Church, M. Sipko, and A. Bacchelli, “Modern code review: a case study at google,” inProceedings of the 40th international conference on software engineering: Software engineering in practice, 2018, pp. 181–190

    2018

  7. [7]

    Code reviews do not find bugs. how the current code review best practice slows us down,

    J. Czerwonka, M. Greiler, and J. Tilford, “Code reviews do not find bugs. how the current code review best practice slows us down,” in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 2. IEEE, 2015, pp. 27–28

    work page 2015

  8. [8]

    Four eyes are better than two: On the impact of code reviews on software quality,

    G. Bavota and B. Russo, “Four eyes are better than two: On the impact of code reviews on software quality,” in 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2015, pp. 81–90

    work page 2015

Show all 49 references
  1. [9]

    Impact of peer code review on peer impres- sion formation: A survey,

    A. Bosu and J. C. Carver, “Impact of peer code review on peer impres- sion formation: A survey,” in 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 2013, pp. 133–142

    2013

  2. [10]

    Peer review on open-source software projects: Parameters, statistical models, and theory,

    P. C. Rigby, D. M. German, L. Cowen, and M.-A. Storey, “Peer review on open-source software projects: Parameters, statistical models, and theory,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 23, no. 4, pp. 1–33, 2014

    2014

  3. [11]

    Characteristics of useful code reviews: An empirical study at microsoft,

    A. Bosu, M. Greiler, and C. Bird, “Characteristics of useful code reviews: An empirical study at microsoft,” in 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories . IEEE, 2015, pp. 146–156

    2015

  4. [12]

    Mining the modern code review repositories: A dataset of people, process and product,

    X. Yang, R. G. Kula, N. Yoshida, and H. Iida, “Mining the modern code review repositories: A dataset of people, process and product,” in Proceedings of the 13th international conference on mining software repositories, 2016, pp. 460–463

    2016

  5. [13]

    On learning meaningful code changes via neural machine translation,

    M. Tufano, J. Pantiuchina, C. Watson, G. Bavota, and D. Poshyvanyk, “On learning meaningful code changes via neural machine translation,” in 2019 IEEE/ACM 41st International Conference on Software Engi- neering (ICSE). IEEE, 2019, pp. 25–36

    2019

  6. [14]

    Towards automating code review activities,

    R. Tufano, L. Pascarella, M. Tufano, D. Poshyvanyk, and G. Bavota, “Towards automating code review activities,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021, pp. 163–174

    2021

  7. [15]

    Using pre-trained models to boost code review automa- tion,

    R. Tufano, S. Masiero, A. Mastropaolo, L. Pascarella, D. Poshyvanyk, and G. Bavota, “Using pre-trained models to boost code review automa- tion,” in Proceedings of the 44th international conference on software engineering, 2022, pp. 2291–2302

    2022

  8. [16]

    Automating code review activities by large-scale pre-training,

    Z. Li, S. Lu, D. Guo, N. Duan, S. Jannu, G. Jenks, D. Majumder, J. Green, A. Svyatkovskiy, S. Fu et al. , “Automating code review activities by large-scale pre-training,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundat...

    2022

  9. [17]

    Exploring the potential of chatgpt in automated code refinement: An empirical study,

    Q. Guo, J. Cao, X. Xie, S. Liu, X. Li, B. Chen, and X. Peng, “Exploring the potential of chatgpt in automated code refinement: An empirical study,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , 2024, pp. 1–13

    2024

  10. [18]

    Fine-tuning and prompt en- gineering for large language models-based code review automation,

    C. Pornprasit and C. Tantithamthavorn, “Fine-tuning and prompt en- gineering for large language models-based code review automation,” Information and Software Technology , p. 107523, 2024

    2024

  11. [19]

    Gpt-3.5 for code review automation: How do few-shot learning, prompt design, and model fine-tuning impact their performance?

    ——, “Gpt-3.5 for code review automation: How do few-shot learning, prompt design, and model fine-tuning impact their performance?” arXiv preprint arXiv:2402.00905, 2024

    2024 arXiv

  12. [20]

    Enhancing code intelligence tasks with chatgpt,

    K. Yang, X. Mao, S. Wang, T. Zhang, B. Lin, Y . Wang, Y . Qin, Z. Zhang, and X. Mao, “Enhancing code intelligence tasks with chatgpt,” arXiv preprint arXiv:2312.15202, 2023

    2023 arXiv

  13. [21]

    Aicodereview: Advancing code quality with ai-enhanced reviews,

    Y . Almeida, D. Albuquerque, E. Dantas Filho, F. Muniz, K. de Farias Santos, M. Perkusich, H. Almeida, and A. Perkusich, “Aicodereview: Advancing code quality with ai-enhanced reviews,” SoftwareX, vol. 26, p. 101677, 2024

    2024

  14. [22]

    Exploring the capabilities of llms for code change related tasks,

    L. Fan, J. Liu, Z. Liu, D. Lo, X. Xia, and S. Li, “Exploring the capabilities of llms for code change related tasks,” arXiv preprint arXiv:2407.02824, 2024

    2024 arXiv

  15. [23]

    On the use of chatgpt for code review,

    M. Watanabe, Y . Kashiwa, B. Lin, T. Hirao, K. Yamaguchi, and H. Iida, “On the use of chatgpt for code review,” arXiv preprint arXiv , 2024

    2024

  16. [24]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

    2022

  17. [25]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  18. [26]

    Code review automation: strengths and weaknesses of the state of the art,

    R. Tufano, O. Dabi ´c, A. Mastropaolo, M. Ciniselli, and G. Bavota, “Code review automation: strengths and weaknesses of the state of the art,” IEEE Transactions on Software Engineering , 2024

    2024

  19. [27]

    Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence,

    Q. Zhu, D. Guo, Z. Shao, D. Yang, P. Wang, R. Xu, Y . Wu, Y . Li, H. Gao, S. Ma et al., “Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence,” arXiv preprint arXiv:2406.11931 , 2024

    2024 arXiv

  20. [28]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence,

    D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y . Wu, Y . Li et al., “Deepseek-coder: When the large language model meets programming–the rise of code intelligence,” arXiv preprint arXiv:2401.14196, 2024

    2024 arXiv

  21. [29]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang et al. , “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  22. [30]

    What hap- pens in my code reviews? an investigation on automatically classifying review changes,

    E. Fregnan, F. Petrulio, L. Di Geronimo, and A. Bacchelli, “What hap- pens in my code reviews? an investigation on automatically classifying review changes,” Empirical Software Engineering, vol. 27, no. 4, p. 89, 2022

    2022

  23. [31]

    Intentsion code refinement,

    IntentionWebsite, “Intentsion code refinement,” https://sites.google.com/ view/intentioncoderefinement, 2024

    2024

  24. [32]

    CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,

    Y . Wang, W. Wang, S. Joty, and S. C. Hoi, “CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Online and Punta Cana, Dominican ...

    2021

  25. [33]

    Sentencepiece: A simple and language inde- pendent subword tokenizer and detokenizer for neural text processing,

    T. Kudo and J. Richardson, “Sentencepiece: A simple and language inde- pendent subword tokenizer and detokenizer for neural text processing,” arXiv preprint arXiv:1808.06226 , 2018

    2018 arXiv

  26. [34]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

    2022

  27. [35]

    Self-consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint arXiv:2203.11171 , 2022

    2022 arXiv

  28. [36]

    Large language models as analogical reasoners,

    M. Yasunaga, X. Chen, Y . Li, P. Pasupat, J. Leskovec, P. Liang, E. H. Chi, and D. Zhou, “Large language models as analogical reasoners,” arXiv preprint arXiv:2310.01714 , 2023

    2023 arXiv

  29. [37]

    Autotrans- form: Automated code transformation to support modern code review process,

    P. Thongtanunam, C. Pornprasit, and C. Tantithamthavorn, “Autotrans- form: Automated code transformation to support modern code review process,” in Proceedings of the 44th international conference on software engineering, 2022, pp. 237–248

    2022

  30. [38]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  31. [39]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  32. [40]

    Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,

    Y . Wang, W. Wang, S. Joty, and S. C. Hoi, “Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” arXiv preprint arXiv:2109.00859 , 2021

    2021 arXiv

  33. [41]

    Specgen: Automated generation of formal program specifications via large language models,

    L. Ma, S. Liu, Y . Li, X. Xie, and L. Bu, “Specgen: Automated generation of formal program specifications via large language models,” arXiv preprint arXiv:2401.08807, 2024

    2024 arXiv

  34. [42]

    Speceval: Evaluating code comprehension in large language models via program specifica- tions,

    L. Ma, S. Liu, L. Bu, S. Li, Y . Wang, and Y . Liu, “Speceval: Evaluating code comprehension in large language models via program specifica- tions,” arXiv preprint arXiv:2409.12866 , 2024

    2024 arXiv

  35. [43]

    Contrastrepair: Enhancing conversation-based automated program repair via contrastive test case pairs,

    J. Kong, M. Cheng, X. Xie, S. Liu, X. Du, and Q. Guo, “Contrastrepair: Enhancing conversation-based automated program repair via contrastive test case pairs,” arXiv preprint arXiv:2403.01971 , 2024

    2024

  36. [44]

    Ft2ra: A fine-tuning-inspired approach to retrieval-augmented code completion,

    Q. Guo, X. Li, X. Xie, S. Liu, Z. Tang, R. Feng, J. Wang, J. Ge, and L. Bu, “Ft2ra: A fine-tuning-inspired approach to retrieval-augmented code completion,” in Proceedings of the 33rd ACM SIGSOFT Interna- tional Symposium on Software Testing and Analysis, 2024, pp. 313–324

    2024

  37. [45]

    Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt,

    C. S. Xia and L. Zhang, “Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt,” arXiv preprint arXiv:2304.00385, 2023

    2023 arXiv

  38. [46]

    Com- mentfinder: a simpler, faster, more accurate code review comments recommendation,

    Y . Hong, C. Tantithamthavorn, P. Thongtanunam, and A. Aleti, “Com- mentfinder: a simpler, faster, more accurate code review comments recommendation,” in Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software eng...

    2022

  39. [47]

    Chatgptblog,

    OpenAI, “Chatgptblog,” https://openai.com/blog/chatgpt, 2023

    2023

  40. [48]

    Code review quality: How developers see it,

    O. Kononenko, O. Baysal, and M. W. Godfrey, “Code review quality: How developers see it,” in Proceedings of the 38th international con- ference on software engineering , 2016, pp. 1028–1038

    2016

  41. [49]

    Information needs in contemporary code review,

    L. Pascarella, D. Spadini, F. Palomba, M. Bruntink, and A. Bacchelli, “Information needs in contemporary code review,” Proceedings of the ACM on human-computer interaction, vol. 2, no. CSCW, pp. 1–27, 2018

    2018

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.