Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws

Andres Neyem; Juan J. Alonso-Lecaros; Juan Pablo Sandoval Alcocer; Leonardo Centellas-Claros

REVIEW 4 major objections 4 minor 25 references

Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes a pipeline that reverse-engineers vulnerability-fixing commits into synthetic security review comments and anticipates that fine-tuning code review models on them will improve their precision on real security-critical…

desk verdict A well-written proposal for a synthetic vulnerability-review dataset, but it has no results yet and a treatable leakage risk in the generation design. read the letter →

arxiv 2504.16310 v1 pith:TY4KY2XE submitted 2025-04-22 cs.SE

Leonardo Centellas-Claros , Juan J. Alonso-Lecaros , Juan Pablo Sandoval Alcocer , Andres Neyem This is my paper · ORCID

classification cs.SE

keywords codereviewautomationcommentslargelanguagemodelssyntheticdatasetgenerationvulnerabilitydetectionsecurity-focusedfine-tuningcode-to-comment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to address the scarcity of security-focused code review data: mine vulnerability-fixing commits from mature open-source Java projects, then use large language models to write the review comment that could plausibly have prompted each commit. The central assertion, stated as an anticipation rather than a completed result, is that fine-tuning existing code review models on this synthetic dataset will improve their performance on real security reviews. The authors plan to test multiple LLMs and prompt strategies, filter the best combination by human review, generate a large dataset, and evaluate fine-tuned models on a small set of real security-related review comments. If the claim holds, automated code review tools could learn to flag security flaws despite the near-absence of real vulnerability comments in existing training data.

What carries the argument

The load-bearing object is the reverse-engineering prompt: a prompt that feeds an LLM the diff and commit message of a post-fix commit and asks it to write the review comment that would have led to that change. This construction matters because it converts one abundant type of data, vulnerability-fixing commits, into the scarce type of data, security-focused review comments, without manual annotation at scale. The prompt pipeline is paired with a security-keyword filter, a human precision screen over 1,200 generated comments (four LLMs across three prompt strategies on 100 sampled commits), and a fine-tuning comparison against baseline models on a real security review test set.

What would settle it

The claim would be falsified if a model fine-tuned on the synthetic dataset performs no better than its untuned baseline on the paper's real, manually verified security review test set. A reader could check this by taking the filtered security-related review comments from the test partition of the existing large review dataset and comparing the two models under the paper's metrics, such as BLEU-4 and manual judgments of semantic equivalence and applicability.

Watch

Extended reading notes

Core claim

The paper's central claim is that a vulnerability-fixing commit carries enough information to reverse-engineer the human review that triggered it. Given only the diff hunk and the commit message of a commit that patched a vulnerability, an LLM can produce a coherent review comment that names the flaw and could have prompted the fix. The authors argue this matters because real security-related review comments are a tiny fraction of available review datasets, and they report early evidence that the gap is large: in a dataset of 20,000 review comments, only 614 were security-related. Their proposed pipeline filters candidate commits by security keywords, refines the keyword list through two rounds of human sampling, screens generated comments for coherence and plausibility, and then fine-tunes three existing code review models on the best-generated corpus. The anticipated outcome is that these fine-tuned models achieve higher-precision vulnerability comments on real, unseen security reviews than the original models.

Load-bearing premise

The pipeline assumes that a vulnerability-fixing commit's diff and commit message contain enough information to reconstruct the review comment that would have prompted the commit, so synthetic comments can stand in for real reviewer feedback.

Editorial extensions

If this is right

If fine-tuning works, automated code review systems could gain a capability they currently lack: reliably commenting on security-critical flaws rather than general code quality issues.
The pipeline would turn one of the most common forms of software data, commit histories, into scarce review labels, reducing dependence on manually collected security review datasets.
The same reverse-engineering approach could be retargeted to other underrepresented review types, such as performance or accessibility concerns, wherever commits encode the fix.
The planned real-world evaluation would provide the first direct estimate of how much LLM-generated security review data transfers to genuine reviewer comments.
The two-stage selection process, human screening of 1,200 generated comments before mass generation, offers a reusable recipe for choosing an LLM and prompt strategy for synthetic dataset construction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The transfer claim could be tested more sharply on the subset of vulnerability-fixing commits that were actually preceded by a review discussion: if the synthetic comment differs systematically from the real review that prompted the fix, the reverse-engineering premise is weaker than the paper assumes.
If the generated comments mostly identify the vulnerability class rather than the vulnerable line or the exact condition, fine-tuning may improve topic-level relevance without improving line-level localization that reviewers need.
The keyword-filtered real test set may share the blind spots of the keyword list, so measured transfer could overstate real-world performance; a stronger test would include vulnerabilities found by independent security reports.
The method's success likely depends on commit-message quality; commits with terse or generic messages may produce plausible-sounding but disconnected reviews, which suggests a cheap quality gate: compare synthetic comments against those generated from randomized message text.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a proposal, not a research result. The paper describes a plan to generate synthetic vulnerability-focused code reviews by reverse-engineering fixing commits, and gives some early corpus counts. It is well written, honest about being a plan, and the idea is worth pursuing. But there is no dataset, no trained model, no evaluation, so the central claim is untested. For a normal peer-reviewed venue, I would desk reject it as incomplete.

What is genuinely new: using vulnerability-fixing commits as the source for synthetic review comments is an angle I haven't seen. The methodology is laid out in reasonable detail, and the early numbers (35,950 candidate commits after filtering) suggest the raw material exists. The authors also acknowledge the subjectivity of their evaluation and the Java-only scope.

The soft spots are real. The generation prompt gives the LLM the post-fix diff and asks for a review that could have prompted that fix. That means the LLM sees the solution before writing the comment. Generated reviews will naturally describe what the patch changed, not how a reviewer would have spotted the flaw before the fix. Fine-tuning on those comments will teach a model to paraphrase diffs, and that skill will not transfer to real reviews where the fixed version isn't in the input. The stress-test note is right that this is a load-bearing issue, and the threats-to-validity section doesn't mention it.

The planned evaluation is thin: 43 and 63 real samples, and the test set is filtered by the same keyword list used to build the training data. That is not a reliable basis for claiming improvement. BLEU-4 is also a weak metric here, though the manual evaluation is welcome. None of this is fatal to the idea. A careful data-generation design that withholds the fix (e.g., generating from the pre-fix state or from the vulnerability alone) and a larger real test set would make the study meaningful.

Who this is for: someone working on automated code review or synthetic training data may want to read it as a short position piece. I wouldn't cite it as a result.

Recommendation: this is not ready for peer review as a full paper. It could be a workshop contribution or a "research plan" paper if the venue permits that. I would tell the authors to run the experiments and come back with the dataset and evaluation.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes a method for constructing a synthetic dataset of vulnerability-focused code review comments. The pipeline selects Java projects and single-file commits, filters commits with an iteratively refined security keyword list, prompts several LLMs (GPT-4o, Claude 3.5 Sonnet, Flan-T5, Qwen 2.5) with zero-shot, chain-of-thought, and self-reflection prompts to generate review comments from commit diffs and messages, and then evaluates the combinations manually. The resulting dataset would be used to fine-tune three code-to-comment models (CodeReviewer, a Lin et al. variant, and GPT-4o prompting) and evaluated on a filtered test subset of an existing real dataset using BLEU-4 and manual metrics. The paper reports only early findings: counts of candidate repositories (5,973), mined commits (3,827,517), keyword-filtered commits (43,131), and commits after test-file exclusion (35,950). It contains no generated synthetic reviews, no precision measurements, no fine-tuning experiments, and no evaluation on real review comments; RQ1 and RQ2 are posed but not answered.

Significance. If the proposed approach worked, it could address a recognized limitation in automated code review research: the scarcity of security-related review comments in existing training datasets. The methodology is thoughtfully structured, and the plan to validate generated reviews through independent manual assessment with Cohen's Kappa is commendable. The authors are transparent that their central claim is an anticipation. However, as a journal submission the manuscript is currently a research proposal rather than a completed empirical study, and no evidence is provided that synthetic reviews of this kind improve model precision. The approach also rests on a load-bearing assumption—that a vulnerability-fixing diff and its commit message contain enough information to reconstruct a plausible pre-fix review comment—which is plausible but untested and potentially problematic. The topic is relevant, but the current contribution is limited by the absence of results and by the unaddressed risk of answer leakage.

major comments (4)

[Section IV and overall] The paper presents no completed experiments: Section IV reports only repository and commit counts, and the abstract's statement that 'the synthetic dataset will improve the performance of the original code review models' is explicitly an anticipation. There are no generated synthetic reviews, no manual precision results, no fine-tuning runs, and no comparison on real review comments. Consequently, neither RQ1 nor RQ2 is answered, and the title's promise of 'improving automated secure code reviews' is not supported by any empirical evidence. This is a proposal, not a completed study.
[Section III.B.5] The prompt design introduces a train/test leakage risk. The prompt provides the post-fix diff and commit message and asks for a review comment that 'could have led to making said commit,' meaning the corrected code is fully visible to the LLM. The generated comments can therefore be reverse-engineered paraphrases of the fix rather than diagnoses written from a reviewer's pre-fix perspective. When these comments are used as fine-tuning targets, the model may learn to reproduce or describe the fixed code; at inference, a submitted diff under review does not contain a fixed version, so such comments would not transfer. Section V's threats-to-validity discussion does not acknowledge this leakage. The authors should either condition generation on the pre-fix code, mask the fix, or otherwise demonstrate that the generated comments are not simply echoes of the corrected diff.
[Section III.C.3] The planned real-world test set is too small to support a meaningful claim of improvement: the authors report only 43 and 63 potentially security-related samples for the code-to-comment and code & comment-to-code tasks, respectively. With samples in the tens, BLEU-4 differences and manual evaluation scores will have very wide confidence intervals, and the manual review of all generated outputs could easily be influenced by a handful of cases. The paper should either substantially enlarge the curated test set from additional sources, or provide confidence intervals, effect sizes, and an explicit analysis of how the small sample limits the conclusions.
[Section III.B.3 and Section III.C.3] The same security keyword list is used both to select the training commits (Section III.B.3) and to filter the real-world evaluation set (Section III.C.3). This creates a circularity risk: a model fine-tuned on comments generated from keyword-matched commits may only need to recognize those keywords to appear successful on a test set filtered by the same list, rather than to understand vulnerabilities generally. Additionally, the test set will not represent the full distribution of real security reviews, which often require context beyond a keyword match. The manuscript should use an independently constructed test set, report the degree of keyword overlap, and evaluate robustness to keyword choice.

minor comments (4)

[Section II.A] There are language errors in the opening of the related work section, including 'This sections summarizes' and 'effectiveness of this models'; these should be corrected to 'This section' and 'these models.'
[Section II.C] The related work on artificial dataset generation is thin and does not explicitly discuss prior attempts to generate synthetic code review comments or to reverse-engineer commits into review feedback. The novelty claim in Section II.C would be easier to assess if the authors compared their approach with existing data augmentation work for code review datasets.
[Section III.B.3] The plan to retain only keywords with precision above 75% and to 'adjust this threshold depending on the number of commits remaining' is a potential source of selection bias. The manuscript should report the final keyword list, the precision of each keyword, and the effect of threshold changes on dataset size.
[General] Since the contribution is a dataset, the paper should include a data availability statement with a repository or DOI for the generated dataset, the exact prompts, the model versions, and the manual evaluation annotations. None of these artifacts are currently provided.

Circularity Check

1 steps flagged · score 2.0 of 10

The central RQ2 claim is an untested empirical prediction, not a circular derivation; the only notable circularity is a self-referential accuracy criterion for RQ1.

self definitional [Section III.B.5 (Prompt Design) and Section III.B.6 (Prompt/LLM Evaluation)]
"Using the commit message and diff information from each commit, the goal is to design a prompt that instructs the model to generate a review comment that could plausibly have prompted the creation of the commit, relying solely on this data. ... To determine suitability, each author will assess (1) whether the generated reviews are coherent, (2) whether they address the identified vulnerability, and (3) whether they could plausibly prompt the corresponding commit."

The generation target and the evaluation criterion are defined by the same construct: a review 'that could plausibly have prompted the creation of the commit' is generated from the post-fix diff, and the same 'could plausibly prompt the corresponding commit' standard is used to judge its quality. A comment that restates the vulnerability fixed in the diff satisfies both the prompt and the criterion, so the Step 6 precision measure primarily checks instruction-following rather than independent review quality. This loop is load-bearing for choosing the LLM/prompt combination that produces the synthetic dataset, although the planned RQ2 evaluation on real ground-truth reviews remains an independent empirical test.

full rationale

This is a proposal/plan paper rather than a derivation chain: it contains no equations, no fitted parameters renamed as predictions, and no uniqueness theorems imported from the authors' prior work. The references are external, not self-citations, and the central claim in the Abstract and Section I is an explicit anticipation that the synthetic dataset will improve code-review models; that claim is an untested empirical hypothesis, not something forced by construction. The one circular element is internal to RQ1: the success criterion for a synthetic review is the same 'could plausibly prompt the commit' construct used to generate it, so the reported precision will partly measure adherence to the prompt rather than independent reviewer quality. The shared security-keyword list used both to select training commits (Section III.B.3) and to filter the real test set (Section III.C.3) is a plausible confound for the planned RQ2 evaluation, but because the test ground truth is real review data and the models' outputs are not constructed from those keywords, this does not make the improvement prediction true by construction. Overall, the central claim retains independent content, so the circularity score is low.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The study relies on unverified data-quality assumptions: that keyword-based filtering identifies real vulnerability-fixing commits, that post-fix diffs plus commit messages contain enough signal to reconstruct the missing review comment, and that synthetic reviews are suitable for fine-tuning. None of these assumptions is tested in this preprint.

assumptions (4)

domain assumption Security keyword list from Alfadel et al. identifies vulnerability-fixing commits with high precision.
Section III.B.3 relies on keyword filtering to select vulnerability commits, with a planned precision threshold of 75%; the list may miss security-relevant commits not captured by the keywords.
domain assumption A review comment can be reverse-engineered from the post-fix diff and commit message.
Section III.B.5 prompt design assumes the diff and message contain enough signal to reconstruct the missing review comment that would have prompted the commit.
domain assumption Excluding changes to files whose names contain 'test' removes noise without losing security-relevant data.
Section IV applies this exclusion post hoc, reducing the candidate set from 43,131 to 35,950; test-related security code could be discarded.
domain assumption Synthetic reviews can serve as effective fine-tuning data for code review models.
This is the central hypothesis of RQ2 in Section III.C, stated as an expectation and not yet tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws." pith.science (2026). https://pith.science/paper/TY4KY2XE

@misc{pith2026250416310,
  author       = {Pith},
  title        = {Pith review of: Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TY4KY2XE}},
  note         = {Machine review of arXiv:2504.16310}
}

read the original abstract

Automation of code reviews using AI models has garnered substantial attention in the software engineering community as a strategy to reduce the cost and effort associated with traditional peer review processes. These models are typically trained on extensive datasets of real-world code reviews that address diverse software development concerns, including testing, refactoring, bug fixes, performance optimization, and maintainability improvements. However, a notable limitation of these datasets is the under representation of code vulnerabilities, critical flaws that pose significant security risks, with security-focused reviews comprising a small fraction of the data. This scarcity of vulnerability-specific data restricts the effectiveness of AI models in identifying and commenting on security-critical code. To address this issue, we propose the creation of a synthetic dataset consisting of vulnerability-focused reviews that specifically comment on security flaws. Our approach leverages Large Language Models (LLMs) to generate human-like code review comments for vulnerabilities, using insights derived from code differences and commit messages. To evaluate the usefulness of the generated synthetic dataset, we plan to use it to fine-tune three existing code review models. We anticipate that the synthetic dataset will improve the performance of the original code review models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 9 canonical work pages

[1]

Mi ning code review data to understand waiting times between accept ance and merging: an empirical analysis,

G. Kudrjavets, A. Kumar, N. Nagappan, and A. Rastogi, “Mi ning code review data to understand waiting times between accept ance and merging: an empirical analysis,” in Proceedings of the 19th International Conference on Mining Software Repositories , ser. MSR ’22. New Y ork, NY , USA: Association for Computing Machinery, 2022, p . 579–590. [Online]. Avail...

arXiv 2022
[2]

Impact of peer code review on pee r impression formation: A survey,

A. Bosu and J. C. Carver, “Impact of peer code review on pee r impression formation: A survey,” in 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measureme nt, 2013, pp. 133–142

work page 2013
[4]

LLaMA- Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efﬁcient Fine-Tuning ,

J. Lu, L. Y u, X. Li, L. Y ang, and C. Zuo, “ LLaMA- Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efﬁcient Fine-Tuning , ” in 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) . Los Alamitos, CA, USA: IEEE Computer Society, Oct. 2023, pp. 647–658. [Online]. Availa ble: https://doi...

arXiv 2023
[5]

D-act: Towards diff-aware code transformation for code re view under a time-wise evaluation,

C. Pornprasit, C. Tantithamthavorn, P . Thongtanunam, a nd C. Chen, “D-act: Towards diff-aware code transformation for code re view under a time-wise evaluation,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , 2023, pp. 296–307

work page 2023
[6]

Code review automation: Strengths and weaknesses of the st ate of the art,

R. Tufano, O. Dabi´ c, A. Mastropaolo, M. Ciniselli, and G . Bavota, “Code review automation: Strengths and weaknesses of the st ate of the art,” IEEE Trans. Softw. Eng. , vol. 50, no. 2, p. 338–353, Jan. 2024. [Online]. Available: https://doi.org/10.1109/TSE.2023 .3348172

doi:10.1109/tse.2023 2024
[7]

Codee ditor: Learning to edit source code with pre-trained models,

J. Li, G. Li, Z. Li, Z. Jin, X. Hu, K. Zhang, and Z. Fu, “Codee ditor: Learning to edit source code with pre-trained models,” ACM Trans. Softw. Eng. Methodol. , vol. 32, no. 6, Sep. 2023. [Online]. Available: https://doi.org/10.1145/3597207

doi:10.1145/3597207 2023
[8]

Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ,

J. Y u, L. Fu, P . Liang, A. Tahir, and M. Shahin, “ Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ,” in 2023 ACM/IEEE International Symposium on Empirical Software Engineerin g and Measurement (ESEM) . Los Alamitos, CA, USA: IEEE Computer Society, Oct. 2023, pp. 1–12. [Online]. Available : https://doi.ieeecomputer...

arXiv 2023
[9]

Software security during mode rn code review: the developer’s perspective,

L. Braz and A. Bacchelli, “Software security during mode rn code review: the developer’s perspective,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Sympos ium on the F oundations of Software Engineering , ser. ESEC/FSE 2022. New Y ork, NY , USA: Association for Computing Machinery, 2022, p . 810–821. [Online]. Availabl...

arXiv 2022

Show all 25 references

[10]

Automa ting code review activities by large-scale pre-training,

Z. Li, S. Lu, D. Guo, N. Duan, S. Jannu, G. Jenks, D. Majumd er, J. Green, A. Svyatkovskiy, S. Fu, and N. Sundaresan, “Automa ting code review activities by large-scale pre-training,” in Proceedings of the 30th ACM Joint European Software Engineering Confere nce and Symposium ...

2022
[11]

Using pre-trained models to boost code revie w automation,

R. Tufano, S. Masiero, A. Mastropaolo, L. Pascarella, D . Poshyvanyk, and G. Bavota, “Using pre-trained models to boost code revie w automation,” in Proceedings of the 44th International Conference on Software Engineering , ser. ICSE ’22. New Y ork, NY , USA: Association for C...

2022
[12]

Improving automated code reviews: Learning from experien ce,

H. Y . Lin, P . Thongtanunam, C. Treude, and W. Charoenwet , “Improving automated code reviews: Learning from experien ce,” in Proceedings of the 21st International Conference on Mining Software Repositories, ser. MSR ’24. New Y ork, NY , USA: Association for Computing Machine...

2024
[13]

Towards automating code review activities,

R. Tufano, L. Pascarella, M. Tufano, D. Poshyvanyk, and G. Bavota, “Towards automating code review activities,” in Proceedings of the 43rd International Conference on Software Engineering , ser. ICSE ’21. IEEE Press, 2021, p. 163–174. [Online]. Available : https://doi.org/10.1...

2021
[14]

Exploring the potential of chatgpt in automated code reﬁne ment: An empirical study,

Q. Guo, J. Cao, X. Xie, S. Liu, X. Li, B. Chen, and X. Peng, “Exploring the potential of chatgpt in automated code reﬁne ment: An empirical study,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , ser. ICSE ’24. New Y ork, NY , USA: Associ...

2024
[15]

Curated email-based code reviews datasets,

M. Liang, W. Charoenwet, and P . Thongtanunam, “Curated email-based code reviews datasets,” in Proceedings of the 21st International Conference on Mining Software Repositories , ser. MSR ’24. New Y ork, NY , USA: Association for Computing Machinery, 2024, p . 294–298. [Online]...

2024
[16]

Synthetic dialogue dataset generation using llm agents,

Y . Abdullin, D. Molla-Aliod, B. Ofoghi, J. Y earwood, an d Q. Li, “Synthetic dialogue dataset generation using llm agents,” 2024. [Online]. Available: https://arxiv.org/abs/2401.17461

2024 arXiv
[17]

Llm based generation of item-description for recommendation system,

A. Acharya, B. Singh, and N. Onoe, “Llm based generation of item-description for recommendation system,” in Proceedings of the 17th ACM Conference on Recommender Systems , ser. RecSys ’23. New Y ork, NY , USA: Association for Computing Machinery, 2023, p. 1204–1207. [Online]. ...

2023
[18]

Unnatu ral instructions: Tuning language models with (almost) no huma n labor,

O. Honovich, T. Scialom, O. Levy, and T. Schick, “Unnatu ral instructions: Tuning language models with (almost) no huma n labor,” in Proceedings of the 61st Annual Meeting of the Association fo r Computational Linguistics (V olume 1: Long Papers) , A. Rogers, J. Boyd-Graber, a...

2023
[19]

Zerogen: Efﬁcient zero-shot learning via dataset generat ion,

J. Y e, J. Gao, Q. Li, H. Xu, J. Feng, Z. Wu, T. Y u, and L. Kong , “Zerogen: Efﬁcient zero-shot learning via dataset generat ion,” 2022. [Online]. Available: https://arxiv.org/abs/2202.07922

2022 arXiv
[20]

Large language model as attributed training data generator: a tale of diversity and bias,

Y . Y u, Y . Zhuang, J. Zhang, Y . Meng, A. Ratner, R. Krishna, J. Shen, and C. Zhang, “Large language model as attributed training data generator: a tale of diversity and bias,” in Proceedings of the 37th International Conference on Neural Information Processing Systems , ser...

2024
[21]

Why security defects g o unnoticed during code reviews? a case-control study of the chromium os project,

R. Paul, A. K. Turzo, and A. Bosu, “Why security defects g o unnoticed during code reviews? a case-control study of the chromium os project,” in 2021 IEEE/ACM 43rd International Conference on Software En gi- neering (ICSE) , 2021, pp. 1373–1385

2021
[22]

Characteristics of the vulnerable code chang es identiﬁed through peer code review,

A. Bosu, “Characteristics of the vulnerable code chang es identiﬁed through peer code review,” in Companion Proceedings of the 36th International Conference on Software Engineering , ser. ICSE Companion 2014. New Y ork, NY , USA: Association for Computing Machinery, 2014, p. 7...

2014
[23]

Empirical analysis of security-related cod e reviews in npm packages,

M. Alfadel, N. A. Nagy, D. E. Costa, R. Abdalkareem, and E. Shihab, “Empirical analysis of security-related cod e reviews in npm packages,” Journal of Systems and Software, vol. 203, p. 111752, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S01641...

2023
[24]

Language models are few-shot learners,

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P . D hariwal, A. Neelakantan, P . Shyam, G. Sastry, A. Askell, S. Agarwal, A . Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Zieg ler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gr...

2020
[25]

Chain-of-thought prompting elicits r easoning in large language models,

J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xi a, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits r easoning in large language models,” in Proceedings of the 36th International Conference on Neural Information Processing Systems , ser. NIPS ’...

2024
[26]

Self-reﬂection in llm agents: Effects on problem-solving performance,

M. Renze and E. Guven, “Self-reﬂection in llm agents: Effects on problem-solving performance,” 2024. [Online]. Available: https://arxiv.org/abs/2405.06682

2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Mi ning code review data to understand waiting times between accept ance and merging: an empirical analysis,

G. Kudrjavets, A. Kumar, N. Nagappan, and A. Rastogi, “Mi ning code review data to understand waiting times between accept ance and merging: an empirical analysis,” in Proceedings of the 19th International Conference on Mining Software Repositories , ser. MSR ’22. New Y ork, NY , USA: Association for Computing Machinery, 2022, p . 579–590. [Online]. Avail...

arXiv 2022

[2] [2]

Impact of peer code review on pee r impression formation: A survey,

A. Bosu and J. C. Carver, “Impact of peer code review on pee r impression formation: A survey,” in 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measureme nt, 2013, pp. 133–142

work page 2013

[3] [4]

LLaMA- Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efﬁcient Fine-Tuning ,

J. Lu, L. Y u, X. Li, L. Y ang, and C. Zuo, “ LLaMA- Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efﬁcient Fine-Tuning , ” in 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) . Los Alamitos, CA, USA: IEEE Computer Society, Oct. 2023, pp. 647–658. [Online]. Availa ble: https://doi...

arXiv 2023

[4] [5]

D-act: Towards diff-aware code transformation for code re view under a time-wise evaluation,

C. Pornprasit, C. Tantithamthavorn, P . Thongtanunam, a nd C. Chen, “D-act: Towards diff-aware code transformation for code re view under a time-wise evaluation,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , 2023, pp. 296–307

work page 2023

[5] [6]

Code review automation: Strengths and weaknesses of the st ate of the art,

R. Tufano, O. Dabi´ c, A. Mastropaolo, M. Ciniselli, and G . Bavota, “Code review automation: Strengths and weaknesses of the st ate of the art,” IEEE Trans. Softw. Eng. , vol. 50, no. 2, p. 338–353, Jan. 2024. [Online]. Available: https://doi.org/10.1109/TSE.2023 .3348172

doi:10.1109/tse.2023 2024

[6] [7]

Codee ditor: Learning to edit source code with pre-trained models,

J. Li, G. Li, Z. Li, Z. Jin, X. Hu, K. Zhang, and Z. Fu, “Codee ditor: Learning to edit source code with pre-trained models,” ACM Trans. Softw. Eng. Methodol. , vol. 32, no. 6, Sep. 2023. [Online]. Available: https://doi.org/10.1145/3597207

doi:10.1145/3597207 2023

[7] [8]

Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ,

J. Y u, L. Fu, P . Liang, A. Tahir, and M. Shahin, “ Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ,” in 2023 ACM/IEEE International Symposium on Empirical Software Engineerin g and Measurement (ESEM) . Los Alamitos, CA, USA: IEEE Computer Society, Oct. 2023, pp. 1–12. [Online]. Available : https://doi.ieeecomputer...

arXiv 2023

[8] [9]

Software security during mode rn code review: the developer’s perspective,

L. Braz and A. Bacchelli, “Software security during mode rn code review: the developer’s perspective,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Sympos ium on the F oundations of Software Engineering , ser. ESEC/FSE 2022. New Y ork, NY , USA: Association for Computing Machinery, 2022, p . 810–821. [Online]. Availabl...

arXiv 2022