Pith. sign in

REVIEW 2 cited by

Code Vulnerability Detection: A Comparative Analysis of Emerging Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.10490 v1 pith:U6VVERWZ submitted 2024-09-16 cs.SE

Shaznin Sultana , Sadia Afreen , Nasir U. Eisty This is my paper · ORCID
classification cs.SE
keywords largemodelslanguagellmssoftwarevulnerabilityanalysiscodegemma
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The growing trend of vulnerability issues in software development as a result of a large dependence on open-source projects has received considerable attention recently. This paper investigates the effectiveness of Large Language Models (LLMs) in identifying vulnerabilities within codebases, with a focus on the latest advancements in LLM technology. Through a comparative analysis, we assess the performance of emerging LLMs, specifically Llama, CodeLlama, Gemma, and CodeGemma, alongside established state-of-the-art models such as BERT, RoBERTa, and GPT-3. Our study aims to shed light on the capabilities of LLMs in vulnerability detection, contributing to the enhancement of software security practices across diverse open-source repositories. We observe that CodeGemma achieves the highest F1-score of 58\ and a Recall of 87\, amongst the recent additions of large language models to detect software security vulnerabilities.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation

    cs.AI 2025-08 conditional novelty 6.0 of 10

    LLM-generated counterfactual code pairs with flipped vulnerability labels, used to train a GNN, sharply improve CWE-20 detection and attribution on the released CWE-20-CFA benchmark.

  2. SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code

    cs.CR 2025-06 conditional novelty 5.0 of 10

    A 558-task benchmark with SAST and LLM judging shows state-of-the-art LLMs generate vulnerability-free code only about 37% of the time in zero-shot settings.

Pith tools