CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection

Kyungtae Kang; Sijune Hwang; Sungbin Park; Sungju Yun; Yeonjoon Lee

REVIEW 5 major objections 4 minor 47 references

CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CLEAR claims that grounding multi-agent LLM vulnerability detection in a causal knowledge graph improves Pair-Correct detection by 130.7% (C/C++) and 71.56% (Java) over the prior multi-agent best.

desk verdict A promising but unproven causal-RAG approach to LLM vulnerability detection; worth a serious referee, only if the flat-RAG control is added. read the letter →

arxiv 2608.03134 v1 pith:4LYM76ZI submitted 2026-08-04 cs.CR cs.SE

Sungju Yun , Sijune Hwang , Yeonjoon Lee , Kyungtae Kang , Sungbin Park This is my paper · ORCID

classification cs.CRcs.SE

keywords VulnerabilityDetectionCausalKnowledgeGraphMulti-AgentLLMRetrieval-AugmentedGenerationPairwiseCodeClassificationCWEPrimeVulSoftwareSecurity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that LLM-based vulnerability detection fails primarily for lack of causal structure, not lack of reasoning power. The authors build a Vulnerability Causal Knowledge Graph (VCKG) in which each known vulnerability is a chain of Entrypoint, Precondition, Root Cause, and Fix Intent nodes, then let four agents—Collector, Claim, Critic, Judge—retrieve and adversarially verify candidate causal contexts before classifying a code pair. On PrimeVul (C/C++) and a curated Java-Pair set, the framework raises Pair-Correct accuracy by 130.7% and 71.56% over VulTrial, the previous multi-agent best, and ablations show the Precondition, Root Cause, and Fix Intent nodes plus the multi-agent loop are the load-bearing parts. If correct, the result means explicit causal grounding is a practical lever for automated vulnerability detection, not a prompt-engineering trick.

What carries the argument

Vulnerability Causal Knowledge Graph (VCKG): a knowledge graph whose nodes are Entrypoint, Precondition, Root Cause, Fix Intent, and CWE, and whose edges are Local (the causal path EP→PC→RC→FI within one instance), Neighborhood (semantic links between same-type nodes), and Global (LLM-identified cross-instance causal links). The graph is built offline from training data, then queried by the Collector Agent to retrieve top-5 causal contexts; those contexts constrain the Claim-Critic-Judge debate. The Fix Intent node is the piece that tells the model why a patch works, which the paper treats as the main defense against flagging patched code as vulnerable.

What would settle it

Take the trained VCKG and randomly permute the target nodes of the Neighborhood and Global edges while keeping node texts fixed, then rerun CLEAR on the same PrimeVul test pairs. If Pair-Correct stays near 19.17 instead of falling toward the single-agent retrieval baselines (P-C 4.62–6.00), the causal graph structure is not what is driving the result; if it drops, the causal edges are load-bearing.

Watch

Extended reading notes

Core claim

The paper's central discovery is that representing a vulnerability as an explicit causal chain, and using that chain as the retrieval context for agentic debate, lets LLMs separate vulnerable code from near-identical benign or patched code. In CLEAR, the Collector retrieves top-k neighboring causal chains from the VCKG, the Claim agent maps the target code onto those chains, the Critic searches for unmet preconditions or missing fix intents, and the Judge decides. The paper reports that this pipeline raises Pair-Correct accuracy on PrimeVul from VulTrial's 8.31 to 19.17 and on Java-Pair from 12.45 to 21.36, and that removing the VCKG, the Fix Intent node, or the multi-agent coordination each

Load-bearing premise

The load-bearing premise is that the LLM-extracted causal nodes and the embedding-similarity edges linking them correspond to genuine causal dependencies; if textually similar nodes are only semantically similar, the retrieved 'causal context' degrades into nearest-neighbor retrieval and the claimed mechanism collapses.

Editorial extensions

If this is right

Pairwise vulnerable/benign discrimination improves with explicit causal chains even on a small, cheap backbone: gpt-4o-mini with CLEAR beats debate-only multi-agent systems, and a stronger backbone reaches P-C 27.48 on PrimeVul.
The Fix Intent, Root Cause, and Precondition nodes are the load-bearing parts of the knowledge graph; removing any of them hurts Pair-Correct accuracy more than removing the Entrypoint node.
Multi-agent coordination contributes independently of retrieval: full four-agent CLEAR scores P-C 19.17 on PrimeVul versus 4.62 (instance-level) and 6.00 (chunk-level) for single-agent retrieval from the same VCKG.
The method transfers to CWE classification, improving Macro-F1 over VulTrial by 24.3% on PrimeVul and 51.1% on Java-Pair.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: because Fix Intent nodes encode why a patch works, the same VCKG could be reused for patch validation or fix suggestion, not just detection.
Beyond the paper: the narrow variance and near-parity of gpt-4o-mini with gpt-5-mini on Java-Pair suggests the causal context, not the raw model, carries much of the performance, so a cheaper deployed detector may be achievable.
Beyond the paper: the paper's own limitation about semantic-similarity edges implies a direct test—replace embedding-based neighbor and global edges with data-flow or AST structure; if the causal mechanism is genuine, the gain should persist or increase.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a serious engineering paper with a real, testable idea—grounding multi-agent vulnerability reasoning in a knowledge graph of causal chains (entrypoint → precondition → root cause → fix intent). The framework and the four-agent loop are concrete, and the authors release code and data. The ablations (removing individual nodes, varying backbones) are honest and show that the default settings are not obviously cherry-picked. If the reported gains reproduce, this is a useful step beyond flat RAG for vulnerability detection.

The soft spots, in rough order. The causal claim is the weakest link. Cross-instance edges are built from embedding similarity and LLM linking, and Section 6.1 admits that semantic similarity does not guarantee causal consistency. The RQ6 audit is small (n=30), author-side, and the inter-rater agreement on neighbor/global edges is moderate at best. None of that proves the graph encodes causal dependencies. To make the mechanism convincing, the paper needs a flat RAG baseline in the same multi-agent loop—retrieve raw patch/CVE text instead of VCKG nodes. Without that, the gains are as easily explained by nearest-neighbor retrieval of similar patches as by causal grounding. Second, the P-B numbers temper the headline: CLEAR often predicts 'benign' for both pair members, so part of the P-C improvement is a conservative shift rather than better causal discrimination. Third, there's an internal inconsistency: the intro reports a 96.9% Java improvement, the abstract and Table 1 say 71.56%. That needs fixing. Fourth, the PrimeVul results are single-run with no confidence intervals; 19.17 vs 8.31 is a big gap, but it deserves run-to-run variance. The Java-Pair benchmark is author-curated and small, which is acceptable but makes external validation important.

All these issues are addressable, and none kills the engineering contribution. The paper is appropriate for a security venue that takes LLM-based auditing seriously, not for a pure theory venue. I'd accept it for peer review with an expectation of substantial revision, and I'd want to see the flat-RAG control as a condition for acceptance. If that lands, the story becomes much stronger. As it stands, it's a promising but unproven claim.

Recommendation: send it to peer review.

Referee Report

5 major / 4 minor

Summary. The paper proposes CLEAR, a multi-agent LLM framework for vulnerability detection that augments reasoning with a Vulnerability Causal Knowledge Graph (VCKG). The VCKG represents each vulnerability as a chain of Entrypoint, Precondition, Root Cause, Fix Intent, and CWE nodes connected by local, neighborhood, and global edges. The graph is built offline from the training split using LLM extraction (same backbone family used at inference), CWE-clustered embedding similarity, and LLM-based global linking. At inference, a Collector retrieves relevant causal nodes, and Claim, Critic, and Judge agents produce a final verdict. Experiments on PrimeVul (C/C++) and a curated Java-Pair dataset report Pair-Correct (P-C) improvements of 130.7% and 71.56% over the VulTrial multi-agent baseline, alongside ablations on retrieval granularity, node removal, and LLM backbone choice, plus a small human audit of the graph.

Significance. The paper addresses an important problem and the proposed architecture is well structured: the agent roles are clearly described, the ablations are informative, the Java-Pair evaluation is repeated with confidence intervals, and the data/code are promised to be public. If the causal-graph claim were established, this would be a meaningful advance. However, the central evidence is currently insufficient to separate the causal mechanism from dense retrieval of similar patches, and the headline numbers have at least one material inconsistency. The contribution is best viewed as a promising framework whose effectiveness claims need sharper controls and more robust evaluation before they can be accepted at face value.

major comments (5)

[Section 2.1 and RQ2 (Tables 1-2)] The paper claims that causal-chain-guided retrieval is the source of CLEAR's gains, but it never compares against a flat dense-retrieval RAG baseline in the same multi-agent loop. The neighborhood and global edges are constructed from all-MiniLM-L6-v2 cosine similarity plus LLM linking, and Section 6.1 explicitly concedes that semantic similarity does not guarantee causal consistency. Without a control that retrieves raw training vulnerable functions, CVE descriptions, or patch texts and feeds them to the same Collector/Claim/Critic/Judge loop, the P-C improvement could be explained by nearest-neighbor patch retrieval. Add such a flat-RAG baseline and, ideally, a VCKG-with-random-edges control; otherwise the causal mechanism claim is not supported.
[Section 4, Table 1 and Table 3] All PrimeVul results are single-run point estimates with no confidence intervals. Many comparisons are small in absolute terms (e.g., Full P-C 19.17 vs No-Entrypoint 18.24 in Table 3), and without repeated runs or bootstrap CIs the reported 130.7% improvement over VulTrial cannot be separated from sampling noise. The Java-Pair experiments are repeated 10 times with CIs; the same practice should be applied to PrimeVul, or the paper should explicitly caution that PrimeVul results are preliminary.
[Section 4, RQ1 (Table 1)] CLEAR's high P-C is accompanied by a very large P-B (both-predict-benign) rate: 67.67 on PrimeVul and 57.72±3.77 on Java-Pair, versus VulTrial's 7.85 and 11.69. The framework has moved from an over-predict-vulnerability operating point (VulTrial P-V=76.67) to a conservative under-predict operating point. This does not invalidate P-C, but the headline relative improvement should be contextualized: it is partly a shift in operating point rather than a uniformly better detector. Report precision/recall at a fixed false-positive rate, or a detection-cost metric, so readers can judge practical utility.
[Section 1 vs Abstract and Table 1] The introduction states that CLEAR outperforms SOTA by '130.7% and 96.9%, respectively, in the Pair-Correct metric' on C/C++ and Java, while the abstract, RQ1 answer, and Table 1 report 71.56% for the Java-Pair improvement. This is a factual inconsistency in the central claimed result. The correct value must be identified and used consistently throughout the paper.
[Section 4, RQ6] The human audit does not establish the causal validity of the EP→PC→RC→FI chains. It is based on only 30 instances reviewed by two author-side experts, with quadratic-weighted kappa of 0.45 (neighbor edges) and 0.51 (global edges), i.e., moderate agreement. The audit appears to assess semantic coherence rather than causal necessity or sufficiency. Independent annotation or a structural validation (e.g., checking whether retrieved FIs correspond to actual patches, or whether perturbing a causal node changes the verdict) is needed before the 'causal' claim can be accepted. As written, the RQ6 conclusion overstates what the data show.

minor comments (4)

[Table 1] The GPTLens Java-Pair row contains malformed entries: '1.06 ± 0.73± −0.61± 1.04' — the P-R and VPS values need to be reformatted or corrected.
[Tables 1-5] Several numerical entries have inconsistent spacing (e.g., '7 .87±', '91 .36±'). Please clean the table formatting.
[Section 3.2] PrimeVul is stated to have 435 vulnerable and 435 non-vulnerable test samples, but evaluation uses 433 pairs. Please clarify how the 433 pairs are constructed and whether five samples are excluded.
[Section 3.4 and Table 5] The CWE classification baselines are normalized by 'selecting the ground-truth CWE when present among the candidates.' Clarify whether CLEAR was evaluated under the same candidate-selection procedure; otherwise the comparison is asymmetric.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline results are empirical benchmark scores, not derivations from fitted inputs or self-citations.

full rationale

The central claims (RQ1, RQ5) are direct empirical comparisons on PrimeVul and Java-Pair. The numbers in Table 1 are measurements of a pipeline, and the reported improvements (130.7%, 71.56%) are arithmetic differences from baseline P-C scores, not quantities that are defined by the method's construction. The VCKG is built offline from the training split only: 'the VCKG is constructed solely from the training split; test code slices are treated strictly as unseen queries and are completely excluded from the graph construction phase' (Sec. 2.1). No parameter is fitted to test labels, and no test-time prediction is a renamed fit. The paper contains no self-citation chain: the cited baselines and datasets (VulTrial, GPTLens, PrimeVul, etc.) are external. The main methodological limitation, acknowledged in Sec. 6.1 ('semantic similarity does not always guarantee causal consistency'), is a threat to construct validity: the 'causal' edges are partially similarity-based, and the RQ6 audit is an internal author-team review with moderate inter-rater agreement. But these concerns do not make the reported P-C gain equivalent to the graph's inputs by construction. The absence of a raw dense-retrieval control is a comparison gap (correctness risk), not a circularity: the skeptical hypothesis that CLEAR is nearest-neighbor retrieval in disguise is an alternative explanation, not a demonstration that the result reduces to its own assumptions. Under the stated rules, no circular step can be exhibited with a specific reduction, so the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on the assumption that LLM-generated causal structures are faithful and that embedding similarity can find causally relevant neighbors. The chosen hyperparameters (k=5, similarity threshold 0.8, 2-hop window) are hand-set and unablated. The framework introduces two new conceptual artifacts (VCKG and the four-agent protocol) whose validity is only evidenced by internal ablation and an author-run human study.

free parameters (4)

k=5 (top-k neighbor edges) = 5
Chosen by hand in Neighborhood Edge Integration (Section 2.1) to balance context growth; no sensitivity analysis reported. Also used as Top-5 retrieval anchors in the Collector agent.
String similarity threshold >= 0.8 = 0.8
Used to curate the Java-Pair test set (Section 3.2). This filtering choice selects 'subtle' pairs and shapes the benchmark; no alternative thresholds are reported.
2-hop subgraph window (~260 nodes) = ~260 nodes
Window size for Global Edge Integration (Section 2.1) to limit LLM context; chosen for computational feasibility, not from data.
Temperature = 0.0 = 0.0
Sampling temperature for all LLM calls; standard but still a choice affecting determinism.

assumptions (4)

domain assumption LLM extraction of causal nodes (EP, PC, RC, FI, CWE) from source code and CVE metadata is accurate and consistent.
Invoked in Section 2.1 during Local VCKG Construction and again at inference by the Collector agent; if extraction is noisy, the graph and retrieval are corrupted.
domain assumption Semantic embedding similarity approximates causal relatedness for connecting neighbor edges.
Used in Section 2.1 for Neighborhood Edge Integration; the paper itself concedes in Section 6.1 that semantic similarity does not guarantee causal consistency.
ad hoc to paper The causal chain EP -> PC -> RC -> FI, plus CWE, is a sufficient representation of a vulnerability's lifecycle.
The paper postulates this structure in Section 2.0; it is a conceptual decision, not derived from a theory, and is the backbone of the VCKG.
domain assumption The PrimeVul pairwise protocol and the author-curated Java-Pair are representative of real-world vulnerability detection difficulty.
The evaluation in Section 3.2 depends on these benchmarks; the Java-Pair dataset is constructed by the authors, so its representativeness is assumed.

invented entities (2)

Vulnerability Causal Knowledge Graph (VCKG)
purpose: Structured knowledge base storing causal chains (EP->PC->RC->FI) for retrieval during detection.
The VCKG is a new modeling artifact; its validity is assessed only by the authors' internal human audit (RQ6) and by downstream performance, neither of which is a falsifiable handle outside the paper.
Collector, Claim, Critic, Judge agents
purpose: Four specialized LLM agents that retrieve, hypothesize, critique, and adjudicate vulnerability claims.
The agents are software components of the proposed system; their individual contributions are only measured through ablations within this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection." pith.science (2026). https://pith.science/paper/4LYM76ZI

@misc{pith2026260803134,
  author       = {Pith},
  title        = {Pith review of: CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4LYM76ZI}},
  note         = {Machine review of arXiv:2608.03134}
}

read the original abstract

Detecting source code vulnerabilities is increasingly difficult as modern security flaws are rooted in complex causal dependencies between execution flows, control conditions, and program states. Despite recent advances in Large Language Models (LLMs) and multi-agent frameworks, existing approaches primarily address superficial similarities between benign and vulnerable functions while failing to capture the complex causal dependencies inherent in security flaws. To address these limitations, we propose Causal Context-based Agentic Reasoning (CLEAR), a novel multi-agent vulnerability detection framework integrated with a causal knowledge graph. CLEAR systematically constructs a Vulnerability Causal Knowledge Graph (VCKG) that models the causal chains between entrypoints, preconditions, root causes, and fix intents across vulnerability instances. Leveraging this structured knowledge, four specialized agents, including the Collector, Claim, Critic, and Judge, collaboratively verify vulnerability hypotheses through retrieved causal contexts. Experimental results on C/C++ and Java vulnerability benchmarks demonstrate that CLEAR improves Pair-Correct (P-C) performance by 130.7% and 71.56% over state-of-the-art approaches, demonstrating the effectiveness of causal knowledge graph-guided reasoning for automated vulnerability detection.

Figures

Figures reproduced from arXiv: 2608.03134 by the authors.

**Figure 2.** Design of Vulnerability Causal Knowledge Graph [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Local Vulnerability Causal Knowledge Graph Ex [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Neighborhood Edge Integration [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Global Edge Integration the entire knowledge base by linking entities from disparate Local VCKG units that exhibit functional or logical inter-dependencies. To ensure computational feasibility and manage the LLM’s context window efficiency, we perform this integration…

**Figure 6.** Figure 6: Causal Context Retrieval To ensure the precision of the reasoning process, the Collector employs an LLM-driven Edge Filtering strategy. Instead of delivering all retrieved nodes, the Collector queries the LLM to evaluate whether each next-hop entity maintains a genuin…

**Figure 7.** Figure 7: Multi-Agent Reasoning Example generalization, we repeated the evaluation 10 times over independent iterations and reported the metrics with their corresponding confidence intervals. 3.3 Evaluation Metrics Following the pairwise evaluation perspective of PrimeVul [5], …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 14 canonical work pages

[1]

Zahra Abbasiantaeb, Yifei Yuan, Evangelos Kanoulas, and Mohammad Alian- nejadi. 2024. Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (Merida, Mexico) (WSDM ’24). Association for Computing Machinery, New York, NY, USA...

arXiv 2024
[2]

Guru Bhandari, Amara Naseer, and Leon Moonen. 2021. CVEfixes: automated col- lection of vulnerabilities and their fixes from open-source software. InProceedings of the 17th International Conference on Predictive Models and Data Analytics in Soft- ware Engineering (Athens, Greece) (PROMISE 2021). Association for Computing Machinery, New York, NY, USA, 30–3...

arXiv 2021
[3]

Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vulnerability detection: Are we there yet?IEEE Transactions on Software Engineering 48, 9 (2021), 3280–3296

work page 2021
[4]

Harrison Chase. 2022. LangChain. https://github.com/langchain-ai/langchain

2022
[6]

Xueying Du, Geng Zheng, Kaixin Wang, Yi Zou, Yujia Wang, Wentai Deng, Jiayi Feng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2026. Vul- RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG. ACM Trans. Softw. Eng. Methodol. (Feb. 2026). https://doi.org/10.1145/3797277 Just Accepted

doi:10.1145/3797277 2026
[7]

Tenenbaum, and Igor Mordatch

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch
[8]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational...

doi:10.18653/v1/2020.findings-emnlp.139 2020
[9]

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Ji- awei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023)

arXiv 2023

Show all 47 references

[10]

Google Open Source Security. 2026. OSV: Open Source Vulnerabilities. https: //osv.dev/. Accessed: 2026-02-24

2026
[11]

Katerina Goseva-Popstojanova and Andrei Perhinschi. 2015. On the capability of static code analysis to detect security vulnerabilities. Inf. Softw. Technol. 68, C (Dec. 2015), 18–33. https://doi.org/10.1016/j.infsof.2015.08.002

2015 doi
[12]

Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. UniXcoder: Unified Cross-Modal Pre-training for Code Representation. In Pro- ceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), Smaranda Mures...

2022 doi
[13]

Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead. ACM Trans. Softw. Eng. Methodol. 34, 5, Article 124 (May 2025), 30 pages. https://doi.org/10.1145/3712003

2025 doi
[14]

Sihao Hu, Tiansheng Huang, Fatih İlhan, Selim Furkan Tekin, and Ling Liu
[15]

Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798 (2023)

2023 arXiv
[16]

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans. Inf. S...

2025 doi
[17]

Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge
[18]

Arvinder Kaur and Ruchikaa Nayyar. 2020. A Comparative Study of Static Code Analysis tools for Vulnerability Detection in C/C++ and JAVA Source Code. Procedia Computer Science 171 (2020), 2023–2029. https://doi.org/10.1016/j.pr ocs.2020.04.217 Third International Conference on...

2020 doi
[19]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings ...

2020
[20]

Yikun Li, Ting Zhang, Ratnadira Widyasari, Yan Naing Tun, Huu Hung Nguyen, Tan Bui, Ivana Clairine Irsan, Yiran Cheng, Xiang Lan, Han Wei Ang, et al. 2024. Cleanvul: Automatic function-level vulnerability detection in code commits using llm heuristics. arXiv preprint arXiv:241...

2024 arXiv
[21]

Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. In Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Lang...

2024 doi
[22]

National Institute of Standards and Technology. 2026. NVD Developers - Vul- nerabilities API. https://nvd.nist.gov/developers/vulnerabilities. Accessed: 2026-02-24

2026
[23]

Alexander Tobias Neumann, Yue Yin, Sulayman Sowe, Stefan Decker, and Matthias Jarke. 2025. An LLM-Driven Chatbot in Higher Education for Databases and Information Systems. IEEE Transactions on Education 68, 1 (2025), 103–116. https://doi.org/10.1109/TE.2024.3467912

2025
[24]

Chao Ni, Liyu Shen, Xiaohu Yang, Yan Zhu, and Shaohua Wang. 2024. MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representations. In Proceedings of the 21st International Conference on Mining Software Repositories (Lisbon, Portugal) (MSR ’24). Association for C...

2024
[25]

Chao Ni, Xin Yin, Kaiwen Yang, Dehai Zhao, Zhenchang Xing, and Xin Xia
[26]

Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2024. Fine- tuning or retrieval? comparing knowledge injection in llms. In Proceedings of the 2024 conference on empirical methods in natural language processing . 237–250

2024
[27]

Md Mahbubur Rahman, Ira Ceka, Chengzhi Mao, Saikat Chakraborty, Baishakhi Ray, and Wei Le. 2024. Towards Causal Deep Learning for Vulnerability Detection. In Proceedings of the IEEE/ACM 46th International Conference on Software Engi- neering (Lisbon, Portugal) (ICSE ’24). Asso...

2024
[28]

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJ...

2019 doi
[29]

In Proceedings of the 31st ACM joint european software engineering conference and symposium on the foundations of software engineering

Distinguishing look-alike innocent and vulnerable code by subtle semantic representation learning and explanation. In Proceedings of the 31st ACM joint european software engineering conference and symposium on the foundations of software engineering. 1611–1622
[30]

Ze Sheng, Zhicheng Chen, Shuning Gu, Heqing Huang, Guofei Gu, and Jeff Huang. 2025. LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights. ACM Comput. Surv. 58, 5, Article 134 (Nov. 2025), 35 pages. https://doi.org/10.1145/3769082

2025 doi
[31]

Benjamin Steenhoek, Md Mahbubur Rahman, Richard Jiles, and Wei Le. 2023. An Empirical Study of Deep Learning Models for Vulnerability Detection. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23) . IEEE Press...

2023
[32]

Karl Tamberg and Hayretdin Bahsi. 2025. Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study. IEEE Access 13 (2025), 29698–29717. https://doi.org/10.1109/ACCESS.2025.3541146

2025
[33]

Niklas Risse and Marcel Böhme. 2024. Uncovering the limits of machine learn- ing for automatic vulnerability detection. In Proceedings of the 33rd USENIX Conference on Security Symposium (Philadelphia, PA, USA) (SEC ’24). USENIX Association, USA, Article 238, 18 pages

2024
[34]

Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2024. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. In 2024 IEEE Symposium on Security and Priv...

2024
[35]

David Wan, Justin Chen, Elias Stengel-Eskin, and Mohit Bansal. 2025. MAMM- Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration. In Proceedings of the 2025 Conference of the Nations of the Ameri- cas Chapter of the Association for Computatio...

2025 doi
[36]

Junhong Wan, Tao Yu, Kunyu Jiang, Yao Fu, Weihao Jiang, and Jiang Zhu. 2025. Digest the Knowledge: Large Language Models empowered Message Passing for Knowledge Graph Question Answering. InProceedings of the 63rd Annual Meeting CLEAR: Causal Context-Based Agentic Reasoning for...

2025
[37]

Zixuan Tan, Jiayuan Zhou, Xing Hu, Shengyi Pan, Kui Liu, and Xin Xia. 2025. Similar but Patched Code Considered Harmful: The Impact of Similar but Patched Code on Recurring Vulnerability Detection and How to Remove Them. In 2025 IEEE/ACM 47th International Conference on Softwa...

2025
[38]

Ratnadira Widyasari, Martin Weyssow, Ivana Clairine Irsan, Han Wei Ang, Frank Liauw, Eng Lieh Ouh, Lwin Khin Shar, Hong Jin Kang, and David Lo. 2025. Let the trial begin: A mock-court approach to vulnerability detection using llm-based agents. arXiv preprint arXiv:2505.10961 (2025)

2025
[39]

Bozhi Wu, Chengjie Liu, Zhiming Li, Yushi Cao, Jun Sun, and Shang-Wei Lin. 2025. Enhancing Vulnerability Detection via Inter-procedural Semantic Completion. Proc. ACM Softw. Eng. 2, ISSTA, Article ISSTA037 (June 2025), 23 pages. https: //doi.org/10.1145/3728912

2025 doi
[40]

Zibin Zheng, Kaiwen Ning, Yanlin Wang, Jingwen Zhang, Dewu Zheng, Mingxi Ye, and Jiachi Chen. 2023. A survey of large language models for code: Evolution, benchmarking, and future trends. arXiv preprint arXiv:2311.10372 (2023)

2023 arXiv
[41]

Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2025. Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Ahead. ACM Trans. Softw. Eng. Methodol. 34, 5, Article 145 (May 2025), 31 pages. https://doi.org/10.1145/3708522

2025 doi
[42]

Nengbo Wang, Xiaotian Han, Jagdip Singh, Jing Ma, and Vipin Chaudhary. 2025. CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation. In Findings of the Association for Computational Linguistics: ACL 2025 , Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and M...

2025 doi
[43]

Deqing Zou, Yawei Zhu, Shouhuai Xu, Zhen Li, Hai Jin, and Hengkai Ye. 2021. Interpreting Deep Learning-based Vulnerability Detector Predictions Based on Heuristic Searching. ACM Trans. Softw. Eng. Methodol. 30, 2, Article 23 (March 2021), 31 pages. https://doi.org/10.1145/3429444

2021 doi
[47]

Deqing Zou, Sujuan Wang, Shouhuai Xu, Zhen Li, and Hai Jin. 2021. VulDeeP- ecker: A Deep Learning-Based System for Multiclass Vulnerability Detection . IEEE Transactions on Dependable and Secure Computing 18, 05 (Sept. 2021), 2224–2236. https://doi.org/10.1109/TDSC.2019.2942930

2021
[2013]

In 2013 35th International Conference on Software Engineering (ICSE)

Why don’t software developers use static analysis tools to find bugs?. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 672–681

2013
[2023]

In 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA)

Large language model-powered smart contract vulnerability detection: New perspectives. In 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) . IEEE, 297–306. https://doi.org/10.1109/TPS-ISA58951.2023.00044

2023
[2024]

In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24)

Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 467, 31 pages
[2457]

https://doi.org/10.1109/ICSE55347.2025.00110

2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Zahra Abbasiantaeb, Yifei Yuan, Evangelos Kanoulas, and Mohammad Alian- nejadi. 2024. Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (Merida, Mexico) (WSDM ’24). Association for Computing Machinery, New York, NY, USA...

arXiv 2024

[2] [2]

Guru Bhandari, Amara Naseer, and Leon Moonen. 2021. CVEfixes: automated col- lection of vulnerabilities and their fixes from open-source software. InProceedings of the 17th International Conference on Predictive Models and Data Analytics in Soft- ware Engineering (Athens, Greece) (PROMISE 2021). Association for Computing Machinery, New York, NY, USA, 30–3...

arXiv 2021

[3] [3]

Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vulnerability detection: Are we there yet?IEEE Transactions on Software Engineering 48, 9 (2021), 3280–3296

work page 2021

[4] [4]

Harrison Chase. 2022. LangChain. https://github.com/langchain-ai/langchain

2022

[5] [6]

Xueying Du, Geng Zheng, Kaixin Wang, Yi Zou, Yujia Wang, Wentai Deng, Jiayi Feng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2026. Vul- RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG. ACM Trans. Softw. Eng. Methodol. (Feb. 2026). https://doi.org/10.1145/3797277 Just Accepted

doi:10.1145/3797277 2026

[6] [7]

Tenenbaum, and Igor Mordatch

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch

[7] [8]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational...

doi:10.18653/v1/2020.findings-emnlp.139 2020

[8] [9]

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Ji- awei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023)

arXiv 2023