Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey maps 58 studies of LLM-based vulnerability detection and concludes that the field is strong on isolated code snippets but weak on real-world, repository-level vulnerabilities.

desk verdict A genuinely useful survey of LLM-based vulnerability detection, but its quantitative claims rest on a non-reproducible 58-paper sample and the 'first comprehensive' framing is not supported. read the letter →

arxiv 2502.07049 v2 pith:7FDJDUIL submitted 2025-02-10 cs.CR cs.AI

Ze Sheng , Zhicheng Chen , Shuning Gu , Heqing Huang , Guofei Gu , Jeff Huang This is my paper · ORCID
classification cs.CRcs.AI
keywords largelanguagemodelsvulnerabilitydetectionsoftwaresecuritysurveypromptengineeringfine-tuningbenchmarksdatasets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a systematic map of how large language models are used for software vulnerability detection, based on 58 studies selected from roughly 500 screened works. It argues that the field has shifted toward large decoder-only models, that most research targets C/C++ at function or file level, and that the main blockers to real-world use are dataset quality, repository-level scope, and model explainability. A sympathetic reader would care because this organizing pass tells practitioners which architectures, prompting techniques, and fine-tuning strategies are worth building on, and where the open problems actually lie.

What carries the argument

The machinery carrying the argument is the survey's classification scheme: three LLM architecture families (encoder-only, encoder-decoder, decoder-only), three technique families (code data preprocessing, prompt engineering, and fine-tuning), and a dataset taxonomy by granularity (function-level, file-level, commit-level, and repository- or application-level). These categories are applied to the 58 selected papers to produce the reported percentages and the gap analysis. The four research questions act as the ordering device that turns individual studies into comparable evidence.

What would settle it

Re-run the screening process with a formal protocol and a second annotator on the same venue set and time window, then check whether the headline distributions survive: 50% of studies targeting C/C++, 65% of fine-tuning experiments using decoder-only models, 41.3% using code-processing techniques, and the claim that over 95% of studies report code complexity as a hindrance. If a systematic re-screen finds materially different proportions, the survey's descriptive conclusions reflect selection rather than the field.

Watch

Extended reading notes

Core claim

On its own terms, this paper claims to be the first systematic survey dedicated to understanding LLMs in vulnerability detection. It organizes 58 selected papers around four research questions: which LLMs are applied, which benchmarks and metrics evaluate them, which techniques (code preprocessing, prompt engineering, fine-tuning) are used, and what challenges remain. Its reported findings include that decoder-only models account for 65% of fine-tuning experiments, that C/C++ is the target language in 50% of studies, that 41.3% of studies use code-processing techniques, and that more than 95% of studies find code complexity hinders detection. The central claim is that these patterns give a reliable picture of the field's strengths (high accuracy on isolated function-level benchmarks) and weaknesses (poor handling of cross-file, repository-level, and logical vulnerabilities), and that this picture identifies dataset scarcity as the primary barrier to progress.

Load-bearing premise

The survey's conclusions stand on the assumption that the 58 selected papers fairly represent the larger population of LLM vulnerability-detection research from 2019 to 2024; if that sample is skewed, every reported percentage and gap claim inherits the skew.

Editorial extensions

If this is right

  • If the map is right, teams building detectors should expect the largest gains from fine-tuning large decoder-only models rather than from code-graph preprocessing alone.
  • Researchers evaluating detectors should distrust function-level F1 scores as evidence of real-world readiness, since the survey finds those benchmarks overstate performance relative to repository-level settings.
  • Dataset builders have a concrete target: a large, leakage-controlled, repository-level benchmark with annotations for call sequences and control flow would fill the field's most cited gap.
  • Language coverage should expand beyond C/C++ and Solidity; Java's enterprise and Android footprint is underserved relative to its CVE volume.
  • The gap between high F1 scores on narrow datasets and poor cross-file detection suggests that vulnerability reproduction should become a standard part of evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported percentages (50% C/C++, 65% decoder-only among fine-tuning studies, 41.3% using code-processing techniques) describe the selected corpus, not necessarily the whole literature; the survey's own iterative keyword-based screening means the numbers are directional, not census-level.
  • If the field follows the survey's suggested direction, benchmarks should migrate from function-level accuracy to repository-level tasks with exploit-based validation, a migration that is testable by tracking whether new datasets adopt call-sequence and control-flow annotations.
  • The emphasis on vulnerability reproduction gestures toward a plausible evaluation standard: a detector's output counts as a true positive only when a proof-of-concept input can be generated. This is an extension the authors raise in Challenge 1 rather than a claim they prove.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper surveys LLM-based software vulnerability detection, reporting a selection of 58 highly relevant papers from roughly 500--600 screened works over a two-month period. It organizes the literature around four research questions: which LLMs are applied, which benchmarks/datasets/metrics are used, which techniques (preprocessing, prompt engineering, fine-tuning) are employed, and what challenges and future directions exist. The central claim, stated in Section 1, is that this is the first comprehensive survey of LLM strengths and weaknesses in vulnerability detection, with quantitative findings such as C/C++ dominance (50% of studies), decoder-only models in 65% of fine-tuning experiments, and code processing techniques in 41.3% of studies. The paper also maintains a GitHub repository of selected papers.

Significance. A rigorous survey of this fast-moving area would be genuinely valuable, and the manuscript has useful raw material: a reasonable three-way taxonomy of LLM architectures, a broad catalog of datasets in Table 4, a table of fine-tuning studies with reported F1 scores, and an explicit list of open challenges. The qualitative synthesis and many of the identified gaps are plausible and consistent with the literature I know. However, the paper's distinctive contribution is its quantitative synthesis, and that part is not currently reliable: the selection protocol is not reproducible, several percentages are internally inconsistent or lack denominators, and the 'first comprehensive survey' claim is difficult to sustain. These issues are local and fixable, so I view this as a strong revision candidate rather than a rejection.

major comments (4)
  1. [Section 2.1] The paper selection procedure is not reproducible. Section 2.1 describes an iterative keyword search over a two-month period, screening approximately 500--600 papers, but gives no date-stamped search strings, no explicit inclusion/exclusion criteria, no screening tool, no inter-rater reliability check, and no list of excluded papers. The Limitations section (Section 4) acknowledges that terminology variation 'may lead to oversights in initial searches.' This matters because every descriptive statistic in the paper (e.g., 50% C/C++, 65% fine-tuning with decoder-only models, 41.3% code processing) rests on the representativeness of the 58-paper sample. Please add a fully reproducible protocol: search strings, databases, screening stages with counts, inclusion/exclusion rules, and the list of the 58 included studies.
  2. [Section 3.3 and Findings I--II] Internal count inconsistencies undermine the quantitative synthesis. Section 3.3 says the language distribution is based on '56 selected papers,' while Section 1 and Section 3.2 say 58. Finding I reports C/C++ at 50% of studies, but Finding II says C/C++ dominates at 'around 60% coverage'; these numbers cannot both describe the same corpus without further explanation. Additionally, Challenge 1 states that 'approximately 40 studies (83%)' focus on isolated code snippets, but 40 of 58 is 69%, not 83%. Please reconcile all reported counts and state the denominator for every percentage.
  3. [Answer to RQ1 and Section 3.4] Several headline percentages are given without raw counts or temporal definitions. The Answer to RQ1 reports that encoder-only models dominate non-fine-tuning studies at 72.4% and decoder-only models account for 65% of fine-tuning experiments, but no denominators are provided. Finding III's 41.3% of studies employing code processing techniques and Finding IV's '100% of recent studies adopting CoT' are similarly unsupported; 'recent studies' is left undefined. Because these findings are presented as the paper's systematic contribution, please provide the counts behind each percentage or convert the claims to clearly labeled qualitative statements.
  4. [Section 1 and Section 2.2] The claim that this is 'the first comprehensive survey focused on understanding the strengths and weaknesses of LLMs in vulnerability detection' is overstated given the related work the paper itself cites. Section 2.2 names Zhou et al. [140], a survey specifically on LLM vulnerability detection and repair, and Xu et al. [124], a systematic review of LLMs for cyber security. Please either substantiate the novelty claim with concrete differentiators from these and any other recent surveys, or soften it to 'one of' or 'a focused survey.'
minor comments (4)
  1. [Section 2.2] There is a typo: 'cybersecutiry' should be 'cybersecurity.'
  2. [Section 3.4.1] The text says 'Fig. 8 illustrates the principle of RAG,' but Figure 8 is captioned as an example of an AST; the RAG figure reference appears to be incorrect.
  3. [Section 3.3.1] Section 3.3.1 is an empty subsection heading titled 'Evaluation Metrics'; the material appears in Sections 3.3.2 and 3.3.3, and the efficiency metrics mentioned in the Answer to RQ2 are not given a corresponding subsection.
  4. [References] References [143] and [144] appear to designate the same paper (Zhou, Zhang, and Lo 2024) under two different numbers; please deduplicate.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey's findings are descriptive summaries of an external literature, with no fitted prediction or self-citation chain that reduces to its inputs.

full rationale

This is a literature survey rather than a derivational or predictive work. Its central claims are descriptive summaries of 58 selected papers: language distributions, model-usage percentages, technique frequencies, and research gaps. None of these findings is obtained by fitting a parameter to a subset of data and then predicting a closely related quantity; the percentages are counts over the selected corpus, and the paper does not present them as independent predictions. The formal definitions in Section 2.4 (binary detection, classification, severity prediction) are notational conventions, not results whose conclusions are defined by their inputs. The taxonomy in Section 3.2 is adopted from an external source (Pan et al. [94]), so it is not a self-citation chain. The paper's Limitations section acknowledges that terminology variation 'may lead to oversights in initial searches,' which is a transparency statement about coverage rather than a circular step. Internal inconsistencies such as the 56-paper count in Section 3.3 versus 58 papers elsewhere, or the differing C/C++ percentages in Finding I and Finding II, are correctness and reproducibility concerns about the quantitative synthesis, but they do not make the survey circular: the conclusions are not used to justify the selection, and the selection is not defined in terms of the conclusions. No load-bearing step reduces to the paper's own inputs by construction, and no self-citation is used to justify a forced choice. Honest non-finding is therefore appropriate.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey introduces no fitted parameters, no new entities, and no mathematical derivations. It relies on three domain assumptions: the adopted model taxonomy, the representativeness of the 58-paper sample, and the reliability of public vulnerability databases. These assumptions are load-bearing because the survey's statistics and gap analysis depend on them.

assumptions (3)
  • domain assumption The taxonomy of LLMs into encoder-only, encoder-decoder, and decoder-only models, taken from Pan et al. [94], is a valid organizing framework for vulnerability detection studies.
    Section 3.2 uses this taxonomy to classify all 58 studies and to compute usage percentages. If the taxonomy misclassifies some models, the reported statistics such as 67.1% decoder-only usage would be distorted.
  • domain assumption The 58 selected papers are representative of the broader literature on LLM-based vulnerability detection.
    Section 2.1 describes an iterative keyword search over two months without a formal inclusion protocol. The survey's overall findings and gap analysis depend on this sample being representative.
  • domain assumption The CVE, CWE, CVSS, and NVD databases accurately enumerate vulnerability types and severities.
    Section 2.4.1 uses these databases to describe domain knowledge and to motivate language-specific vulnerability distributions, such as the dominance of memory-related vulnerabilities in C/C++.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights." pith.science (2026). https://pith.science/paper/7FDJDUIL

@misc{pith2026250207049,
  author       = {Pith},
  title        = {Pith review of: LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7FDJDUIL}},
  note         = {Machine review of arXiv:2502.07049}
}
read the original abstract

Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection, addressing critical challenges in the security domain. Traditional methods, such as static and dynamic analysis, often falter due to inefficiencies, high false positive rates, and the growing complexity of modern software systems. By leveraging their ability to analyze code structures, identify patterns, and generate repair suggestions, LLMs, exemplified by models like GPT, BERT, and CodeBERT, present a novel and scalable approach to mitigating vulnerabilities. This paper provides a detailed survey of LLMs in vulnerability detection. It examines key aspects, including model architectures, application methods, target languages, fine-tuning strategies, datasets, and evaluation metrics. We also analyze the scope of current research problems, highlighting the strengths and weaknesses of existing approaches. Further, we address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis. Based on these findings, we propose solutions for issues like dataset scalability, model interpretability, and applications in low-resource scenarios. Our contributions are threefold: (1) a systematic review of how LLMs are applied in vulnerability detection; (2) an analysis of shared patterns and differences across studies, with a unified framework for understanding the field; and (3) a summary of key challenges and future research directions. This work provides valuable insights for advancing LLM-based vulnerability detection. We also maintain and regularly update latest selected paper on https://github.com/OwenSanzas/LLM-For-Vulnerability-Detection

Figures

Figures reproduced from arXiv: 2502.07049 by the authors.

Figure 1
Figure 1. Complaints and Financial Losses from 2018-2023 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Survey Structure & Research Selection IEEE Transactions on Software Engineering. Then we searched by extracting key terms such as "vulnerability detection," "LLM," "large language model," and "AI" from papers published in conferences and journals. Using these keywords, we conducted iterative searches every three weeks, refining the selection over time. Over a two-month period, we screened approximately 500-600 paper… view at source ↗
Figure 3
Figure 3. An Example Workflow of Vuln. Detection, Classification and Severity Prediction [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Research Questions 3 RESEARCH RESULTS 3.1 Overview Our analysis shows that research on LLM-based vulnerability detection mainly focuses on C/C++, Java, and Solidity. Each language has unique challenges and research priorities. Studies on C/C++ focus on memory-related v…
Figure 5
Figure 5. Figure 5: Distribution of Target Programming Languages in LLM-based Vulnerability Detection Research [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: The test case 501129 of Juliet C/C++ 1.3 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: An Example Illustrating AST [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 10
Figure 10. Figure 10: An Instance of Multi-level Prompting for [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 12
Figure 12. Figure 12: Challenges and Potential Directions • Vulnerability Repair: While many studies discussed vulnerability repair, practical imple￾mentation in real-world projects remains challenging [143]. Successful vulnerability repair in production environments must meet several crit…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection

    cs.CR 2026-07 conditional novelty 6.0 of 10

    DREA improves repository-level vulnerability detection by coupling an LLM planner that forms hypotheses with a cheap local explorer that gathers cross-file evidence, lifting paired accuracy from 19-26% to 30-42% at mu...

  2. LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs

    cs.CR 2025-05 conditional novelty 5.0 of 10

    Linear probe accuracy on simple code metrics can guide layer pruning and roughly predict post-fine-tuning vulnerability detection performance, but several headline numbers in the abstract do not match the paper's own tables.

Reference graph

Works this paper leans on

149 extracted references · 41 canonical work pages · cited by 2 Pith papers

  1. [140]

    Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2024. Large Language Model for Vulnerability Detection and Repair: Literature Review and Roadmap. arXiv preprint arXiv:2404.02525 (2024)

    arXiv 2024

  2. [144]

    Xin Zhou, Ting Zhang, and David Lo. 2024. Large Language Model for Vulnerability Detection: Emerging Results and Future Directions. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (Lisbon, Portugal) (ICSE-NIER’24). Association for Computing Machinery, New York, NY, USA, 47–51

    work page 2024

  3. [124]

    HanXiang Xu, ShenAo Wang, Ningke Li, Yanjie Zhao, Kai Chen, Kailong Wang, Yang Liu, Ting Yu, and HaoYu Wang

    work page

  4. [1]

    Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Unified Pre-training for Program Understanding and Generation. arXiv preprint arXiv:2103.06333 (2021)

    arXiv 2021

  5. [2]

    Md Tauseef Alam, Raju Halder, and Abyayananda Maiti. 2024. Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities. arXiv preprint arXiv:2409.10574 (2024)

    work page Pith review arXiv 2024

  6. [3]

    Virendra Ashiwal, Soeren Finster, and Abdallah Dawoud. 2024. LLM-based Vulnerability Sourcing from Unstructured Data. In 2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 634–641. , Vol. 1, No. 1, Article . Publication date: February 2025. LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights 27

    2024

  7. [4]

    Ömer Aslan, Semih Serkant Aktuğ, Merve Ozkan-Okay, Abdullah Asim Yilmaz, and Erdal Akin. 2023. A Comprehensive Review of Cyber Security Vulnerabilities, Threats, Attacks, and Solutions. Electronics 12, 6 (2023)

    2023

  8. [5]

    Syafiq Al Atiiq, Christian Gehrmann, Kevin Dahlén, and Karim Khalil. 2024. From generalist to specialist: Exploring cwe-specific vulnerability detection. arXiv preprint arXiv:2408.02329 (2024)

    arXiv 2024

Show all 149 references
  1. [6]

    Guru Bhandari, Amara Naseer, and Leon Moonen. 2021. CVEfixes: automated collection of vulnerabilities and their fixes from open-source software. InProceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering (Athens, Greece) ...

    2021

  2. [7]

    Biagio Boi, Christian Esposito, and Sokjoon Lee. 2024. Smart Contract Vulnerability Detection: The Role of Large Language Model (LLM). ACM SIGAPP Applied Computing Review 24, 2 (2024), 19–29

    2024

  3. [8]

    Biagio Boi, Christian Esposito, and Sokjoon Lee. 2024. VulnHunt-GPT: a Smart Contract vulnerabilities detector based on OpenAI chatGPT. In Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing (SAC ’24). Association for Computing Machinery, New York, NY, USA, 1517–1524

    2024

  4. [9]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey W...

    2020

  5. [10]

    Díaz Ferreyra

    Quang-Cuong Bui, Riccardo Scandariato, and Nicolás E. Díaz Ferreyra. 2022. Vul4J: a dataset of reproducible Java vulnerabilities geared towards the study of program repair techniques. In Proceedings of the 19th International Conference on Mining Software Repositories (Pittsbur...

    2022

  6. [11]

    Marcel Böhme, Ezekiel Olamide Soremekun, Sudipta Chattopadhyay, Emamurho Juliet Ugherughe, and Andreas Zeller. 2017. How Developers Debug Software — The DBGBENCH Dataset. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). 244–246

    2017

  7. [12]

    Daipeng Cao and W. Jun. 2024. LLM-CloudSec: Large Language Model Empowered Automatic and Deep Vulnerability Analysis for Intelligent Clouds. InIEEE INFOCOM 2024-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 1–6

    2024

  8. [13]

    Di Cao, Yong Liao, and Xiuwei Shang. 2024. RealVul: Can We Detect Vulnerabilities in Web Applications with LLM? arXiv preprint arXiv:2410.07573 (2024)

    2024 arXiv

  9. [14]

    naturalizing

    Saikat Chakraborty, Toufique Ahmed, Yangruibo Ding, Premkumar T. Devanbu, and Baishakhi Ray. 2022. Nat- Gen: generative pre-training by “naturalizing” source code. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of...

    2022

  10. [15]

    Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2022. Deep Learning Based Vulnerability Detection: Are We There Yet? IEEE Transactions on Software Engineering 48, 9 (2022), 3280–3296

    2022

  11. [16]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    2021 arXiv

  12. [17]

    Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, and David Wagner. 2023. DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. In Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses ...

    2023

  13. [18]

    Brian Chess and Gary McGraw. 2004. Static analysis for security. IEEE security & privacy 2, 6 (2004), 76–79

    2004

  14. [19]

    Min-Je Choi, Sehun Jeong, Hakjoo Oh, and Jaegul Choo. 2017. End-to-end prediction of buffer overruns from raw source code via neural memory networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Melbourne, Australia) (IJCAI’17). AAAI Pr...

    2017

  15. [20]

    CVE Numbering Authorities (CNAs). 2024. https://www.cve.org/programorganization/cnas

    2024

  16. [21]

    Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, et al. 2023. Auggpt: Leveraging chatgpt for text data augmentation. arXiv preprint arXiv:2302.13007 (2023)

    2023 arXiv

  17. [22]

    DARPA and ARPA-H. 2024. Artificial Intelligence Cyber Challenge (AIxCC). https://aicyberchallenge.com/ [Accessed: 01-24-2025]

    2024

  18. [23]

    Xiao Deng, Fuyao Duan, Rui Xie, Wei Ye, and Shikun Zhang. 2024. Improving Long-Tail Vulnerability Detection Through Data Augmentation Based on Large Language Models. In 2024 IEEE International Conference on Software , Vol. 1, No. 1, Article . Publication date: February 2025. 2...

    2024

  19. [24]

    Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. 2025. Vulnerability Detection with Code Language Models: How Far Are We? . In 2025 IEEE/ACM 47th International Conference on Software Engin...

    2025

  20. [25]

    Xiaohu Du, Ming Wen, Jiahao Zhu, Zifan Xie, Bin Ji, Huijun Liu, Xuanhua Shi, and Hai Jin. 2024. Generalization- Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning. arXiv preprint arXiv:2406.03718 (2024)

    2024 arXiv

  21. [26]

    Xueying Du, Geng Zheng, Kaixin Wang, Jiayi Feng, Wentai Deng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2024. Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG. arXiv preprint arXiv:2406.11147 (2024)

    2024 arXiv

  22. [27]

    Adanma Cecilia Eberendu, Valentine Ikechukwu Udegbe, Edmond Onwubiko Ezennorom, Anita Chinonso Ibegbulam, Titus Ifeanyi Chinebu, et al . 2022. A systematic literature review of software vulnerability detection. European Journal of Computer Science and Information Technology 10...

    2022

  23. [28]

    Jiahao Fan, Yi Li, Shaohua Wang, and Tien N. Nguyen. 2020. A C/C++ Code Vulnerability Dataset with Code Changes and CVE Summaries. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computin...

    2020

  24. [29]

    Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. 2024. Llm agents can autonomously exploit one-day vulnerabilities. arXiv preprint arXiv:2404.08144 (2024)

    2024 arXiv

  25. [30]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP ...

    2020

  26. [31]

    Ferreira, Pedro Cruz, Thomas Durieux, and Rui Abreu

    João F. Ferreira, Pedro Cruz, Thomas Durieux, and Rui Abreu. 2021. SmartBugs: a framework to analyze solidity smart contracts. InProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (Virtual Event, Australia) (ASE ’20). Association for Co...

    2021

  27. [32]

    NSA Center for Assured Software. 2024. Software Assurance Reference Dataset (SARD): Juliet C/C++ 1.3. https: //samate.nist.gov/SARD/test-suites/112 Accessed: November 10, 2024

    2024

  28. [33]

    NSA Center for Assured Software. 2024. Software Assurance Reference Dataset (SARD): Juliet Java 1.3. https: //samate.nist.gov/SARD/test-suites/111 Accessed: November 10, 2024

    2024

  29. [34]

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. arXiv preprint arXiv:2204.05999 (2022)

    2022 arXiv

  30. [35]

    Michael Fu, Chakkrit Kla Tantithamthavorn, Van Nguyen, and Trung Le. 2023. Chatgpt for vulnerability detection, classification, and repair: How far are we?. In 2023 30th Asia-Pacific Software Engineering Conference (APSEC). IEEE, 632–636

    2023

  31. [36]

    Duck, Ruyi Ji, Yingfei Xiong, and Abhik Roychoudhury

    Xiang Gao, Bo Wang, Gregory J. Duck, Ruyi Ji, Yingfei Xiong, and Abhik Roychoudhury. 2021. Beyond Tests: Program Vulnerability Repair via Crash Constraint Extraction. ACM Trans. Softw. Eng. Methodol. 30, 2, Article 14 (Feb. 2021), 27 pages

    2021

  32. [37]

    Zeyu Gao, Hao Wang, Yuchen Zhou, Wenyu Zhu, and Chao Zhang. 2023. How far have we gone in vulnerability detection using large language models. arXiv preprint arXiv:2311.12420 (2023)

    2023 arXiv

  33. [38]

    Asem Ghaleb and Karthik Pattabiraman. 2020. How effective are smart contract analysis tools? evaluating smart contract static analysis tools using bug injection. InProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual Event, USA)...

    2020

  34. [39]

    Rikhiya Ghosh, Oladimeji Farri, Hans-Martin von Stockhausen, Martin Schmitt, and George Marica Vasile. 2024. CVE-LLM: Automatic vulnerability evaluation in medical device industry using large language models.arXiv preprint arXiv:2407.14640 (2024)

    2024 arXiv

  35. [40]

    GitHub. 2023. GitHub Copilot. https://github.com/features/copilot Accessed: 2024-11-14

    2023

  36. [41]

    José Gonçalves, Tiago Dias, Eva Maia, and Isabel Praça. 2024. Scope: Evaluating llms for software vulnerability detection. arXiv preprint arXiv:2407.14372 (2024)

    2024 arXiv

  37. [42]

    Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. UniXcoder: Unified Cross-Modal Pre- training for Code Representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Smaranda Muresan,...

    2022

  38. [43]

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svy- atkovskiy, Shengyu Fu, et al. 2020. Graphcodebert: Pre-training code representations with data flow. arXiv preprint , Vol. 1, No. 1, Article . Publication date: February 2025. ...

    2020 arXiv

  39. [44]

    Yuejun Guo, Constantinos Patsakis, Qiang Hu, Qiang Tang, and Fran Casino. 2024. Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection. In Computer Security – ESORICS 2024, Joaquin Garcia-Alfaro, Rafał Kozik, Michał Choraś, and Sokratis Katsik...

    2024

  40. [45]

    Hazim Hanif and Sergio Maffeis. 2022. VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection. In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8

    2022

  41. [46]

    Jean Haurogné, Nihala Basheer, and Shareeful Islam. [n. d.]. Advanced Vulnerability Detection Using Llm with Transparency Obligation Practice Towards Trustworthy Ai. Available at SSRN 4925500 ([n. d.])

  42. [47]

    Jean Haurogné, Nihala Basheer, and Shareeful Islam. 2024. Vulnerability detection using BERT based LLM model with transparency obligation practice towards trustworthy AI. Machine Learning with Applications 18 (2024), 100598

    2024

  43. [48]

    Ahmad Hazimeh, Adrian Herrera, and Mathias Payer. 2020. Magma: A Ground-Truth Fuzzing Benchmark. Proc. ACM Meas. Anal. Comput. Syst. 4, 3, Article 49 (Nov. 2020), 29 pages

    2020

  44. [49]

    Jingxuan He and Martin Vechev. 2023. Large Language Models for Code: Security Hardening and Adversarial Testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (Copenhagen, Denmark) (CCS ’23). Association for Computing Machinery, New Y...

    2023

  45. [50]

    Junda He, Xin Zhou, Bowen Xu, Ting Zhang, Kisub Kim, Zhou Yang, Ferdian Thung, Ivana Clairine Irsan, and David Lo. 2024. Representation learning for stack overflow posts: How far are we? ACM Transactions on Software Engineering and Methodology 33, 3 (2024), 1–24

    2024

  46. [51]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large Language Models for Software Engineering: A Systematic Literature Review. ACM Trans. Softw. Eng. Methodol. (Sept. 2024). Just Accepted

    2024

  47. [52]

    Sihao Hu, Tiansheng Huang, Fatih İlhan, Selim Furkan Tekin, and Ling Liu. 2023. Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives. In 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applicatio...

    2023

  48. [53]

    Yu Jiang, Jie Liang, Fuchen Ma, Yuanliang Chen, Chijin Zhou, Yuheng Shen, Zhiyong Wu, Jingzhou Fu, Mingzhe Wang, Shanshan Li, and Quan Zhang. 2024. When Fuzzing Meets LLMs: Challenges and Opportunities. InCompanion Proceedings of the 32nd ACM International Conference on the Fo...

    2024

  49. [54]

    Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge. 2013. Why don’t software developers use static analysis tools to find bugs?. In 2013 35th International Conference on Software Engineering (ICSE). 672–681

    2013

  50. [55]

    Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and evaluating contextual embedding of source code. In Proceedings of the 37th International Conference on Machine Learning (ICML’20). JMLR.org, Article 474, 12 pages

    2020

  51. [56]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

    2020 arXiv

  52. [57]

    Mete Keltek, Rong Hu, Mohammadreza Fani Sani, and Ziyue Li. 2024. LSAST–Enhancing Cybersecurity through LLM-supported Static Application Security Testing. arXiv preprint arXiv:2409.15735 (2024)

    2024 arXiv

  53. [58]

    Avishree Khare, Saikat Dutta, Ziyang Li, Alaia Solko-Breslin, Rajeev Alur, and Mayur Naik. 2023. Understanding the effectiveness of large language models in detecting security vulnerabilities. arXiv preprint arXiv:2311.16169 (2023)

    2023 arXiv

  54. [59]

    Vasileios Kouliaridis, Georgios Karopoulos, and Georgios Kambourakis. 2024. Assessing the Effectiveness of LLMs in Android Application Vulnerability Analysis. arXiv preprint arXiv:2406.18894 (2024)

    2024 arXiv

  55. [60]

    Ummay Kulsum, Haotian Zhu, Bowen Xu, and Marcelo d’Amorim. 2024. A case study of llm for automated vulnerability repair: Assessing impact of reasoning and patch validation feedback. In Proceedings of the 1st ACM International Conference on AI-Powered Software. 103–111

    2024

  56. [61]

    Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004. IEEE, 75–86

    2004

  57. [62]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing S...

    2020

  58. [63]

    Haonan Li, Yu Hao, Yizhuo Zhai, and Zhiyun Qian. 2024. Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach. Proceedings of the ACM on Programming Languages 8, OOPSLA1 (2024), 474–499

    2024

  59. [64]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al . 2023. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161 (2023). , Vol. 1, No. 1, Article . Publication ...

    2023 arXiv

  60. [65]

    Tsz-On Li, Wenxi Zong, Yibo Wang, Haoye Tian, Ying Wang, Shing-Chi Cheung, and Jeff Kramer. 2023. Nuances are the key: Unlocking chatgpt to find failure-inducing tests with differential prompting. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering...

    2023

  61. [66]

    Ziyang Li, Saikat Dutta, and Mayur Naik. 2024. LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. arXiv preprint arXiv:2405.17238 (2024)

    2024 arXiv

  62. [67]

    Zhen Li, Deqing Zou, Shouhuai Xu, Zhaoxuan Chen, Yawei Zhu, and Hai Jin. 2021. Vuldeelocator: a deep learning- based fine-grained vulnerability detector. IEEE Transactions on Dependable and Secure Computing 19, 4 (2021), 2821–2837

    2021

  63. [68]

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2021. Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing 19, 4 (2021), 2244–2258

    2021

  64. [69]

    Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. Vuldeepecker: A deep learning-based system for vulnerability detection. arXiv preprint arXiv:1801.01681 (2018)

    2018 arXiv

  65. [70]

    Guanjun Lin, Jun Zhang, Wei Luo, Lei Pan, and Yang Xiang. 2017. POSTER: Vulnerability discovery with function representation learning from unlabeled projects. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2539–2541

    2017

  66. [71]

    Shangqing Liu, Wei Ma, Jian Wang, Xiaofei Xie, Ruitao Feng, and Yang Liu. 2024. Enhancing Code Vulnerability Detection via Vulnerability-Preserving Data Augmentation. In Proceedings of the 25th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for ...

    2024

  67. [72]

    Xiaohao Liu, Jie Wu, Zhulin Tao, Yunshan Ma, Yinwei Wei, and Tat-seng Chua. 2024. Harnessing Large Language Models for Multimodal Product Bundling. arXiv preprint arXiv:2407.11712 (2024)

    2024 arXiv

  68. [73]

    Yu Liu, Lang Gao, Mingxin Yang, Yu Xie, Ping Chen, Xiaojin Zhang, and Wei Chen. 2024. VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models.arXiv preprint arXiv:2406.07595 (2024)

    2024 arXiv

  69. [74]

    Zhenguang Liu, Peng Qian, Jiaxu Yang, Lingfeng Liu, Xiaojun Xu, Qinming He, and Xiaosong Zhang. 2023. Rethinking smart contract fuzzing: Fuzzing with invocation ordering and important branch revisiting. IEEE Transactions on Information Forensics and Security 18 (2023), 1237–1251

    2023

  70. [75]

    Zhihong Liu, Zezhou Yang, and Qing Liao. 2024. Exploration On Prompting LLM With Code-Specific Information For Vulnerability Detection. In 2024 IEEE International Conference on Software Services Engineering (SSE). IEEE, 273–281

    2024

  71. [76]

    V Benjamin Livshits and Monica S Lam. 2005. Finding Security Vulnerabilities in Java Applications with Static Analysis.. In USENIX security symposium, Vol. 14. 18–18

    2005

  72. [77]

    Guilong Lu, Xiaolin Ju, Xiang Chen, Wenlong Pei, and Zhilong Cai. 2024. GRACE: Empowering LLM-based software vulnerability detection with graph structure and in-context learning. Journal of Systems and Software 212 (June 2024), 112031

    2024

  73. [78]

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664 (2021)

    2021 arXiv

  74. [79]

    Yu Luo, Weifeng Xu, Karl Andersson, Mohammad Shahadat Hossain, and Dianxiang Xu. 2024. FELLMVP: An Ensemble LLM Framework for Classifying Smart Contract Vulnerabilities. In 2024 IEEE International Conference on Blockchain (Blockchain). IEEE, 89–96

    2024

  75. [80]

    Yunlong Lyu, Yuxuan Xie, Peng Chen, and Hao Chen. 2024. Prompt Fuzzing for Fuzz Driver Generation. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 3793–3807

    2024

  76. [81]

    Wei Ma, Daoyuan Wu, Yuqiang Sun, Tianwen Wang, Shangqing Liu, Jian Zhang, Yue Xue, and Yang Liu. 2024. Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justifications. arXiv preprint arXiv:2403.16073 (2024)

    2024 arXiv

  77. [82]

    Andrew A Mahyari. 2024. Harnessing the Power of LLMs in Source Code Vulnerability Detection. In MILCOM 2024-2024 IEEE Military Communications Conference (MILCOM). IEEE, 251–256

    2024

  78. [83]

    Qiheng Mao, Zhenhao Li, Xing Hu, Kui Liu, Xin Xia, and Jianling Sun. 2024. Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models. arXiv preprint arXiv:2406.09701 (2024)

    2024 arXiv

  79. [84]

    Zhenyy Mao, Jialong Li, Dongming Jin, Munan Li, and Kenji Tei. 2024. Multi-role consensus through llms discussions for vulnerability detection. In 2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C). IEEE, 1318–1319

    2024

  80. [85]

    Florence Martin, Yan Chen, Robert L Moore, and Carl D Westine. 2020. Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 2018. Educational Technology Research and Development 68, 4 (2020), 1903–1929

    2020

  81. [86]

    Noble Saji Mathews, Yelizaveta Brus, Yousra Aafer, Meiyappan Nagappan, and Shane McIntosh. 2024. Llbezpeky: Leveraging large language models for vulnerability detection. arXiv preprint arXiv:2401.01269 (2024). , Vol. 1, No. 1, Article . Publication date: February 2025. LLMs in...

    2024 arXiv

  82. [87]

    Joydeep Mitra and Venkatesh-Prasad Ranganath. 2017. Ghera: A repository of android app vulnerability benchmarks. In Proceedings of the 13th international conference on predictive models and data analytics in software engineering. 43–52

    2017

  83. [88]

    National Institute of Standards and Technology. 2024. Software Assurance Reference Dataset (SARD). https: //samate.nist.gov/SARD/ Accessed: 2024-11-10

    2024

  84. [89]

    Xu Nie, Ningke Li, Kailong Wang, Shangguang Wang, Xiapu Luo, and Haoyu Wang. 2023. Understanding and tackling label errors in deep learning-based vulnerability detection (experience paper). In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and ...

    2023

  85. [90]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis.arXiv preprint arXiv:2203.13474 (2022)

    2022 arXiv

  86. [91]

    OpenAI. 2022. GPT-3.5. https://platform.openai.com/docs/models Accessed: 2024-11-14

    2022

  87. [92]

    OpenAI. 2023. GPT-4 Technical Report. Technical Report. OpenAI

    2023

  88. [93]

    Shengyi Pan, Lingfeng Bao, Xin Xia, David Lo, and Shanping Li. 2023. Fine-grained commit-level vulnerability type prediction by CWE tree structure. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 957–969

    2023

  89. [94]

    Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

  90. [95]

    Serena Elisa Ponta, Henrik Plate, Antonino Sabetta, Michele Bezzi, and Cédric Dangremont. 2019. A manually-curated dataset of fixes to vulnerabilities of open-source software. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 383–387

    2019

  91. [96]

    Moumita Das Purba, Arpita Ghosh, Benjamin J Radford, and Bill Chu. 2023. Software vulnerability detection using large language models. In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 112–119

    2023

  92. [97]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  93. [98]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21, 140 (2020), 1–67

    2020

  94. [99]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Ro- main Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950 (2023)

    2023 arXiv

  95. [100]

    Alejandro Russo and Andrei Sabelfeld. 2010. Dynamic vs. static flow-sensitive security analysis. In 2010 23rd IEEE Computer Security Foundations Symposium. IEEE, 186–199

    2010

  96. [101]

    Sinan Sakaoglu. 2023. Kartal: Web application vulnerability hunting using large language models. (2023)

    2023

  97. [102]

    Kosta Serebryany. 2016. Continuous Fuzzing with libFuzzer and AddressSanitizer. In 2016 IEEE Cybersecurity Development (SecDev). 157–157

    2016

  98. [103]

    Alexey Shestov, Anton Cheshkov, Rodion Levichev, Ravil Mussabayev, Pavel Zadorozhny, Evgeny Maslov, Chibirev Vadim, and Egor Bulychev. 2024. Finetuning large language models for vulnerability detection. arXiv preprint arXiv:2401.17010 (2024)

    2024 arXiv

  99. [104]

    Sindhwad, Prateek Ranka, Siddhi Muni, and Faruk Kazi

    Parul V. Sindhwad, Prateek Ranka, Siddhi Muni, and Faruk Kazi. 2024. VulnArmor: mitigating software vulnerabilities with code resolution and detection techniques. International Journal of Information Technology (March 2024)

    2024

  100. [105]

    Sunbeom So and Hakjoo Oh. 2023. SmartFix: Fixing Vulnerable Smart Contracts by Accelerating Generate-and-Verify Repair using Statistical Models. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineeri...

    2023

  101. [106]

    Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Yang Liu, and Yingjiu Li. 2024. Llm4vuln: A unified evaluation framework for decoupling and enhancing llms’ vulnerability reasoning. arXiv preprint arXiv:2401.16185 (2024)

    2024 arXiv

  102. [107]

    Jeniya Tabassum, Mounica Maddela, Wei Xu, and Alan Ritter. 2020. Code and Named Entity Recognition in StackOver- flow. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Ed...

    2020

  103. [108]

    Masoud Jamshidiyan Tehrani and Sattar Hashemi. 2024. Assessing Vulnerability in Smart Contracts: The Role of Code Complexity Metrics in Security Analysis. arXiv preprint arXiv:2411.17343 (2024)

    2024

  104. [109]

    Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2024. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. In IEEE Symposium on Security and Privacy. ...

    2024

  105. [110]

    Government Accountability Office (GAO)

    U.S. Government Accountability Office (GAO). 2024. CrowdStrike Chaos Highlights Key Cyber Vulnerabilities with Software Updates. [Accessed: 10-24-2024]

    2024

  106. [111]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  107. [112]

    Venkatasubramanyam and Sowmya G

    Radhika D. Venkatasubramanyam and Sowmya G. R. 2014. Why is dynamic analysis not used as extensively as static analysis: an industrial study. In Proceedings of the 1st International Workshop on Software Engineering Research and Industrial Practices (Hyderabad, India) (SER&IPs ...

    2014

  108. [113]

    Inacio Vieira, Will Allred, Séamus Lankford, Sheila Castilho, and Andy Way. 2024. How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizes. arXiv preprint arXiv:2409.03454 (2024)

    2024 arXiv

  109. [114]

    Jin Wang, Zishan Huang, Hengli Liu, Nianyi Yang, and Yinhao Xiao. 2023. Defecthunter: A novel llm-driven boosted-conformer-based code vulnerability detection mechanism. arXiv preprint arXiv:2309.15324 (2023)

    2023 arXiv

  110. [115]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 6 (2024), 186345

    2024

  111. [116]

    Shichao Wang, Yun Zhang, Liagfeng Bao, Xin Xia, and Minghui Wu. 2022. VCMatch: A Ranking-based Approach for Automatic Security Patches Localization for OSS Vulnerabilities. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). 589–600

    2022

  112. [117]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven C.H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder- Decoder Models for Code Understanding and Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Marie-Francine ...

    2021

  113. [118]

    C.A. Welty. 1997. Augmenting abstract syntax trees for program understanding. In Proceedings 12th IEEE International Conference Automated Software Engineering. 126–133

    1997

  114. [119]

    Xin-Cheng Wen, Cuiyun Gao, Shuzheng Gao, Yang Xiao, and Michael R. Lyu. 2024. SCALE: Constructing Struc- tured Natural Language Comment Trees for Software Vulnerability Detection. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (...

    2024

  115. [120]

    Qiushi Wu and Kangjie Lu. 2021. On the feasibility of stealthily introducing vulnerabilities in open-source software via hypocrite commits. Proc. Oakland (2021), 17

    2021

  116. [121]

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah

  117. [122]

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Fuzz4All: Universal Fuzzing with Large Language Models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association ...

    2024

  118. [123]

    Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn

    Frank F. Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn. 2022. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming (San Diego, CA, USA) (MAPS 2022). Association for Computing...

    2022

  119. [125]

    Aidan Z. H. Yang, Claire Le Goues, Ruben Martins, and Vincent Hellendoorn. 2024. Large Language Models for Test- Free Fault Localization. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computi...

    2024

  120. [126]

    Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, and Lingming Zhang. 2024. WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models. Proc. ACM Program. Lang. 8, OOPSLA2, Article 296 (Oct. 2024), 27 pages

    2024

  121. [127]

    Chenyuan Yang, Zijie Zhao, and Lingming Zhang. 2023. Kernelgpt: Enhanced kernel fuzzing via large language models. arXiv preprint arXiv:2401.00563 (2023)

    2023 arXiv

  122. [128]

    Yilin Yang. 2023. Iot software vulnerability detection techniques through large language model. In International Conference on Formal Engineering Methods. Springer, 285–290

    2023

  123. [129]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly. High-Confidence Computing 4, 2 (2024), 100211. , Vol. 1, No. 1, Article . Publication date: February 20...

    2024

  124. [130]

    Junjian Ye, Xincheng Fei, Xavier de Carné de Carnavalet, Lianying Zhao, Lifa Wu, and Mengyuan Zhang. 2024. Detecting command injection vulnerabilities in Linux-based embedded firmware with LLM-based taint analysis of library functions. Computers & Security 144 (2024), 103971

    2024

  125. [131]

    Recep Yıldırım, Kerem Aydın, and Orçun Çetin. 2024. Evaluating the Impact of Conventional Code Analysis Against Large Language Models in API Vulnerability Detection. In Proceedings of the 2024 European Interdisciplinary Cybersecurity Conference (Xanthi, Greece) (EICC ’24). Ass...

    2024

  126. [132]

    Xin Yin, Chao Ni, and Shaohua Wang. 2024. Multitask-Based Evaluation of Open-Source LLM on Software Vulnerability. IEEE Trans. Softw. Eng. 50, 11 (Nov. 2024), 3071–3087

    2024

  127. [133]

    Jonas Zaddach, Luca Bruno, Aurelien Francillon, Davide Balzarotti, et al. 2014. AVATAR: A Framework to Support Dynamic Security Analysis of Embedded Systems’ Firmwares.. In NDSS, Vol. 14. 1–16

    2014

  128. [134]

    Bing Zhang, Jingyue Li, Jiadong Ren, and Guoyan Huang. 2021. Efficiency and Effectiveness of Web Application Vulnerability Detection Approaches: A Review. ACM Comput. Surv. 54, 9, Article 190 (Oct. 2021), 35 pages

    2021

  129. [135]

    Chenhui Zhang, Le Wang, Dunqiu Fan, Junyi Zhu, Tang Zhou, Liyi Zeng, and Zhaohua Li. 2024. VTT-LLM: Advancing Vulnerability-to-Tactic-and-Technique Mapping through Fine-Tuning of Large Language Model.Mathematics 12, 9 (2024), 1286

    2024

  130. [136]

    Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie, Yuekang Li, Limin Sun, and Yang Liu

  131. [137]

    Jian Zhang, Chong Wang, Anran Li, Weisong Sun, Cen Zhang, Wei Ma, and Yang Liu. 2024. An Empirical Study of Automated Vulnerability Localization with Large Language Models. arXiv preprint arXiv:2404.00287 (2024)

    2024

  132. [138]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)

    2023 arXiv

  133. [139]

    In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria) (ISSTA 2024)

    How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria) (ISSTA 2024). Association for Computing Machinery, New York, NY, USA, 1223–1235

    2024

  134. [141]

    Xin Zhou, Duc-Manh Tran, Thanh Le-Cong, Ting Zhang, Ivana Clairine Irsan, Joshua Sumarlin, Bach Le, and David Lo. 2024. Comparison of static application security testing tools and large language models for repo-level vulnerability detection. arXiv preprint arXiv:2407.16235 (2024)

    2024 arXiv

  135. [142]

    Laredo, Alessandro Morari, and Zhong Su

    Yunhui Zheng, Saurabh Pujar, Burn Lewis, Luca Buratti, Edward Epstein, Bo Yang, Jim A. Laredo, Alessandro Morari, and Zhong Su. 2021. D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis. arXiv preprint arXiv:2102.07995 (2021)

    2021 arXiv

  136. [145]

    Xin Zhou, Bowen Xu, DongGyun Han, Zhou Yang, Junda He, and David Lo. 2023. CCBERT: Self-Supervised Code Change Representation Learning. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 182–193

    2023

  137. [146]

    Yuhui Zhu, Guanjun Lin, Lipeng Song, and Jun Zhang. 2022. The application of neural network for software vulnerability detection: a review. Neural Comput. Appl. 35, 2 (Nov. 2022), 1279–1301

    2022

  138. [147]

    Arastoo Zibaeirad and Marco Vieira. 2024. VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching. arXiv preprint arXiv:2409.10756 (2024). , Vol. 1, No. 1, Article . Publication date: February 2025

    2024 arXiv

  139. [148]

    Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Curran Associates Inc., Red Hook, NY, USA

    2019

  140. [2023]

    In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (Seattle, WA, USA)(ISSTA 2023)

    How Effective Are Neural Networks for Fixing Security Vulnerabilities. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (Seattle, WA, USA)(ISSTA 2023). Association for Computing Machinery, New York, NY, USA, 1282–1294

    2023

  141. [2024]

    arXiv preprint arXiv:2405.04760 (2024)

    Large language models for cyber security: A systematic literature review. arXiv preprint arXiv:2405.04760 (2024)

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.