Pith. sign in

REVIEW 4 major objections 6 minor 58 references

LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Linear probes forecast a compressed LLM's vulnerability-detection quality before fine-tuning.

desk verdict Useful probe-guided pruning study for vulnerability detection, but the early-estimation claim rests on an unverified proxy and a fitted constant. read the letter →

arxiv 2505.24451 v1 pith:ZH4QPVUU submitted 2025-05-30 cs.CR cs.AI

Luis Ibanez-Lissen , Lorena Gonzalez-Manzano , Jose Maria de Fuentes , Nicolas Anciaux This is my paper · ORCID
classification cs.CRcs.AI
keywords linearprobesLLMcompressionlayerpruningvulnerabilitydetectionmulti-classCWEclassificationcyclomaticcomplexityHalsteaddifficultyearlyperformanceestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that cheap linear classifiers known as linear probes, trained on the hidden states of a frozen LLM, can stand in for the expensive cycle of fine-tuning and compressing that LLM for vulnerability detection. The claim is that probe accuracy on two code-complexity features, cyclomatic complexity and Halstead difficulty, tells you how many top layers can be pruned away without hurting the final multi-class vulnerability classifier, and roughly what precision, recall, and F1 that classifier will reach after fine-tuning, before any fine-tuning is done. The authors test this on BERT and Gemma over 480k C/C++ samples covering 12 of MITRE's Top 25 vulnerability types (CWEs), reporting that 8 of 24 BERT layers and 13 of 18 Gemma layers can be removed with no precision loss, and that the early estimate misses the real final metrics by between 3% and about 8.7% on average. The paper itself flags in Section 5.7 that these results are a first attempt and are not yet enough to confirm the approach outside C/C++ or in other domains.

What carries the argument

The central object is the linear probe plus the loss curve it induces. A linear probe is a small classifier trained on the hidden-state activations $H_k$ of a single layer $k$ to predict a simple labeled feature; here it predicts the cyclomatic-complexity class and Halstead-difficulty band of a code sample. Per-layer probe accuracy defines a loss, $\operatorname{loss}_k = \max_i \operatorname{avg}(\operatorname{acc}_i) - \operatorname{avg}(\operatorname{acc}_k)$, and the pruning cut-off $k_{\mathrm{cut}}$ is the layer that minimizes the summed absolute loss across datasets and features. The same probes feed the performance estimator $\hat{E}(D) = \operatorname{acc}_{\mathrm{LP}}(D) + \beta$, with $\beta$ calibrated by leave-one-out on other datasets. This machinery carries the whole argument: it turns 'how much can we compress?' into a question about where the model's internal representations stop adding information about code structure.

What would settle it

Take any new LLM and the same 12-CWE task, compute the probe-loss curve and the LPASS cut-off and early estimate, then actually fine-tune the model pruned at each candidate layer and measure its F1; if the true best pruning point falls far from the probe-loss minimum, or the early estimate is off by more than the reported average of about 8.7%, the central proxy claim is falsified. A lighter check is to compare probe accuracy per layer against final F1 for several pruned models and see whether the two curves track.

Watch

Extended reading notes

Core claim

The central discovery is that a per-layer accuracy curve of probes, trained only to predict code-complexity labels, marks the place where a pretrained LLM's hidden representations stop adding task-relevant information. The paper shows that the layer $k_{\mathrm{cut}}$ minimizing the summed probe loss across datasets and features is the right place to prune: 33.3% of BERT's layers and 72.2% of Gemma's layers can be removed with no precision loss on the 12-class vulnerability detection task, and the same probe accuracies, shifted by a dataset-calibrated constant $\beta$, estimate the final fine-tuned and compressed model's precision, recall, and F1 before fine-tuning runs. Concretely the estimator is $\hat{E}(D) = \operatorname{acc}_{\mathrm{LP}}(D) + \beta$, and the reported error of this estimate is 3% at best and 8.68% on average. Compressed LPASS models reach 86.9% accuracy in multi-class detection, and the pruned Gemma can outperform its uncompressed version by up to 1.6% F1 while saving 29.4% training time, 23.8% inference time, and 42.98% model size.

Load-bearing premise

The load-bearing premise is that the accuracy of a linear probe trained to guess cyclomatic complexity and Halstead difficulty from internal activations tracks how well the model will later detect specific vulnerability types after fine-tuning; the paper does not measure that correlation directly, and Section 5.7 says the results are a first attempt not yet confirmed beyond C/C++.

Editorial extensions

If this is right

  • Layer pruning guided by probes removes a third to nearly three-quarters of layers with no measured precision loss on the 12-CWE task, so much of the model depth is redundant for this specific vulnerability detection task.
  • Probe-based early estimates of final precision, recall, and F1 come within 3% at best and 8.68% on average, letting a developer decide before investing in fine-tuning whether a model is worth compressing.
  • LPASS-based compressed models outperform the state of the art on multi-class vulnerability detection, reaching 86.9% accuracy, and can beat their own uncompressed versions.
  • The probe computation is cheap: 142.97 seconds for Gemma and under a hundredth of a second per sample for feature extraction, a reduction of about 99% relative to fine-tuning time, so the decision procedure is affordable on consumer GPUs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If probe accuracy is a valid proxy, the same probe-loss curve could rank layers for pruning in other code-understanding tasks, since it only requires some cheap label that correlates with task-relevant structure.
  • The beta-calibrated estimator suggests probe accuracy and final fine-tuned performance differ by a roughly dataset-independent constant; if that pattern generalizes, precomputed beta values per model family would make the approach a plug-in tool.
  • A direct test is still missing: the paper never computes the correlation between probe accuracy and final F1 across many models and layers, so measuring that correlation on a broader set of architectures would settle whether the proxy is a genuine regularity or a two-model coincidence.
  • Because the cut-off is chosen at the minimum of probe loss, the method implies that layers after $k_{\mathrm{cut}}$ are not merely removable but actively unhelpful, which could be tested by ablating only post-cut-off layers and observing whether internal task-relevant representations degrade.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces LPASS, a method to guide layer pruning and to estimate the post-fine-tuning effectiveness of large language models for multi-class CWE vulnerability detection. Linear probes are trained on internal hidden states at each layer to predict the code-complexity features cyclomatic complexity (CC) and Halstead difficulty (HD). Probe accuracy is aggregated across CWEs, and the layer minimizing the accumulated loss over datasets and features (Eq. 2) is chosen as the pruning cut-off. Separately, probe accuracy on a dataset is added to a constant β, fitted on other datasets, to estimate precision, recall, and F1 (Eq. 3). The authors evaluate the approach on BERT-large and Gemma-2B over DiverseVul, Big-Vul, and PrimeVul, covering 12 CWEs, under layer pruning and quantization. The reported findings include removal of 33.3% (BERT) and 72.2% (Gemma) of layers with no precision loss, early estimates with 3% and 8.68% as lowest and average precision errors, and state-of-the-art accuracy of 86.9% in multi-class vulnerability detection.

Significance. If the proxy relation between probe accuracy on CC/HD and downstream vulnerability-detection performance were established, LPASS would be a practically valuable and inexpensive pre-fine-tuning diagnostic: probes are computed in seconds to minutes, compared with hours of fine-tuning, and the paper offers a concrete pipeline and a large multi-dataset evaluation. The connection of probing to model compression in a security setting is novel, and the manuscript is transparent about its experimental protocol and datasets. However, the significance is conditional on an unvalidated assumption. The paper never directly measures the correlation between probe accuracy and post-fine-tuning vulnerability detection, so the two central contributions—kcut selection and early performance estimation—currently rest on an asserted rather than demonstrated proxy relationship. In addition, several headline claims are not supported by the tables as presented.

major comments (4)
  1. [Section 3.1, Section 4.1, Eq. (2)] The proxy relation is load-bearing but never directly tested. Section 3.1 states that the code features F 'can be regarded as proxies for the presence or absence of vulnerabilities,' and Section 4.1 uses probe accuracy to choose the pruning cut-off, but no experiment measures the correlation between accLP(D) and post-fine-tuning vulnerability-detection metrics. The paper only checks that pruning at kcut is not worse than baseline and that pruning at kcut/2 or random layers is worse; those checks show that kcut is a reasonable choice, not that the LP-loss curve identifies the best layer or that accLP tracks downstream quality. Please provide direct evidence, for example per-layer pruning curves against per-layer probe accuracy, or a correlation test across datasets and configurations.
  2. [Tables 2 and 3, abstract] The headline claims of 'no precision loss' and '1.6% of F1-score at a maximum' are contradicted by the paper's own sign convention. The text defines negative effectiveness deltas as improvements over baseline; Table 2 shows positive precision deltas for pruned BERT (e.g., Big-Vul +0.57) and pruned Gemma (e.g., DiverseVul +0.43), and the Gemma PrimeVul F1 delta is +1.6. Thus the data do not support 'no precision loss' or 'compressed versions outperform the original.' The abstract's 86.9% accuracy also does not appear as an accuracy value in Table 2 (86.9% is a baseline F1 for BERT on PrimeVul). Please correct either the claims or the interpretation of the table.
  3. [Section 4.2, Eq. (3), Table 7] The performance estimator E(D) = accLP(D) + β is a constant-offset calibration, not a derived or validated estimator. β is fitted on at most two datasets in a three-dataset leave-one-out scheme, and Table 7 shows considerable instability across configurations (for Gemma CC, β ranges from 4 to 28). With only three datasets, the average errors in Table 8 cannot establish generalization. Please provide a sensitivity analysis of β, a proper hold-out evaluation across additional datasets, or a derivation of β that does not rely on the evaluation data.
  4. [Section 4.1, Eq. (2), Section 5.6] The cut-off kcut is selected by minimizing the sum of losses over the same datasets used for evaluation, and the corroborating tests in Table 6 and Figure 4 use the same evaluation datasets. This is selection on evaluation data, so the tests demonstrate internal consistency rather than out-of-sample validity. Please evaluate kcut on a held-out model or dataset, or report how the choice of kcut varies when different subsets of datasets are used in Eq. (2).
minor comments (6)
  1. [Section 4.1] The method is called a linear probe, but Section 4.1 states that the probe is implemented with a Multi-Layer Perceptron; please clarify whether 'linear' refers to the classification head or to the probing methodology.
  2. [Section 5.3] The number of classes for CC and HD (5 and 6 classes, respectively) is based on an 85% coverage criterion that is described only informally; please report the full feature distributions and the exact rule used to set the class boundaries.
  3. [Figure 3] The x-axis tick labels in Figure 3 are crowded and easy to misread, and the paper alternates between 'kcut' and 'Kcut'; please standardize the notation and improve the figure legibility.
  4. [Section 5.7] The limitations paragraph acknowledges that the approach may not transfer to other domains and languages, but it does not acknowledge the missing direct test of the proxy assumption; please add that caveat explicitly.
  5. [Section 5.4 and Declarations] The text states 'Our experimental materials are publicly released' but the Declarations say the code 'will be freely available if the paper is accepted for publication'; these two statements should be reconciled.
  6. [Section 6, Table 9] The state-of-the-art comparison is not controlled for token length, sampling, class balance, or evaluation protocol; please qualify the superiority claim accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the early-estimate equation is a leave-one-out calibration, not a definition, and the pruning cutoff is chosen from probe loss then validated empirically.

full rationale

The paper's derivation chain does not reduce any claimed prediction to its own inputs by construction. The early estimate E(D) = accLP(D) + beta (Eq. 3) is a linear calibration whose offset beta is computed on other datasets (Eq. 4), and the reported estimation errors use leave-one-out cross-validation, so the target dataset's effectiveness is not an input to the predictor. The pruning cutoff kcut (Eq. 2) is selected from LP accuracy losses and then independently tested by fine-tuning pruned models and measuring downstream vulnerability-detection F1; the downstream result is not defined as the probe accuracy, and the paper reports nonzero estimation errors. The assertion that code-complexity features F are 'proxies for the presence or absence of vulnerabilities' is an unvalidated empirical assumption, not a definitional equivalence, so a weak proxy would threaten external validity but does not make the derivation circular. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no renaming of a known result as a new derivation. The main scientific weakness is that the proxy correlation between probe accuracy and downstream vulnerability detection is never directly measured, but that is a correctness risk rather than a circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on one fitted offset (beta), hand-chosen probe class counts, and three domain assumptions about the relationship between probe accuracy and downstream vulnerability detection. No new physical or model entities are introduced.

free parameters (3)
  • beta (per model, feature, compression configuration) = ranges from 4% to 41% depending on configuration (Table 7); e.g., Gemma CC baseline precision 18-28%
    Equation 4 defines beta = E(D') - accLP(D'), a scalar offset fitted on fine-tuned performance of one or two datasets. The central early-estimate claim depends on this single fitted constant being generalizable, but Table 7 shows it is not stable for Gemma.
  • MLP probe class counts for code features = CC: 5 classes [1,2,3,4,5]; HD: 6 classes (intervals of 5)
    Section 5.6.1 sets classes to cover 85% of samples per dataset; this hand-set coverage threshold determines probe accuracy and therefore kcut.
  • per-CWE training sample cap = 5,000 samples per CWE
    Section 5.4: cap chosen by trial and error to let both models run; this creates the 5,000-vs-50,000 class imbalance that shapes all reported F1 and accuracy values.
assumptions (4)
  • domain assumption LP accuracy on complexity metrics is a proxy for vulnerability detection capability.
    Section 3.1 and Section 4.1 use probe accuracy on CC and HD to select the pruning cut-off and to estimate final vulnerability detection metrics, but the paper never validates the proxy correlation directly.
  • domain assumption LLM hidden states after each layer contain linearly separable information about cyclomatic complexity and Halstead difficulty.
    Section 4.1 trains probes on hidden states H_k to predict F. If this fails, the LP accuracy curves used for kcut are meaningless.
  • ad hoc to paper The layer with minimum accumulated loss (Equation 2) is the optimal pruning cut-off for all downstream tasks.
    Section 4.1 defines kcut as the layer minimizing summed loss; no theoretical argument or broader empirical proof is given beyond the two models tested.
  • ad hoc to paper A linear relationship E(D) = accLP(D) + beta with constant beta per configuration holds across datasets.
    Equation 3 assumes a single offset; Section 5.7 admits the linear relationship was chosen because the number of instances is limited. Table 7 shows beta varies widely for Gemma, weakening the assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs." pith.science (2026). https://pith.science/paper/ZH4QPVUU

@misc{pith2026250524451,
  author       = {Pith},
  title        = {Pith review of: LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZH4QPVUU}},
  note         = {Machine review of arXiv:2505.24451}
}
abstract

Large Language Models (LLMs) are being extensively used for cybersecurity purposes. One of them is the detection of vulnerable codes. For the sake of efficiency and effectiveness, compression and fine-tuning techniques are being developed, respectively. However, they involve spending substantial computational efforts. In this vein, we analyse how Linear Probes (LPs) can be used to provide an estimation on the performance of a compressed LLM at an early phase -- before fine-tuning. We also show their suitability to set the cut-off point when applying layer pruning compression. Our approach, dubbed $LPASS$, is applied in BERT and Gemma for the detection of 12 of MITRE's Top 25 most dangerous vulnerabilities on 480k C/C++ samples. LPs can be computed in 142.97 s. and provide key findings: (1) 33.3 \% and 72.2\% of layers can be removed, respectively, with no precision loss; (2) they provide an early estimate of the post-fine-tuning and post-compression model effectiveness, with 3\% and 8.68\% as the lowest and average precision errors, respectively. $LPASS$-based LLMs outperform the state of the art, reaching 86.9\% of accuracy in multi-class vulnerability detection. Interestingly, $LPASS$-based compressed versions of Gemma outperform the original ones by 1.6\% of F1-score at a maximum while saving 29.4 \% and 23.8\% of training and inference time and 42.98\% of model size.

Figures

Figures reproduced from arXiv: 2505.24451 by the authors.

Figure 1
Figure 1. LPAS S overview. Steps of LPAS S are numbered and colored flows are used to assess the approach when using LLMs to detect vulnerabilities. The orange flow (dot-dot-dash) computes the baseline results, whereas the green (dot-dash), yellow (dashes) and pink (dots) flows refer to compressed LLMs, that is after applying quantization, layer pruning or both techniques at the same time, respectively. 3.1. Overview & Use ca… view at source ↗
Figure 2
Figure 2. Linear probes used for determining kcut. Code features F are the target of the prediction, which is based using the LLM’s internal activations per layer. The layer in which the best prediction is made is selected as the cut-off point kcut • Early assessment. It must be possible to ascertain whether the LLM will be effective or not in vulnerability detection while spending a reduced amount of computational efforts. •… view at source ↗
Figure 3
Figure 3. LLMs loss in absolute value for D, CC, and HD. Notice the decrement until the Kcut layer and the later increment after that layer denoting that the hidden representations are not adding extra information to the embeddings. case of Big-Vul. Then, based on Equation 2, kcut=15 including the embedding layer. On the contrary, in the case of Gemma ( [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of random pruning vs LPAS S -based pruning. Dotted line represents the random pruning version of their respective models When BERT is quantized, inference time is slowed down, especially on 8-bit versions. Nonetheless, pruning alleviates this problem, reduci…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 43 canonical work pages

  1. [1]

    Y . Chen, Z. Ding, L. Alowain, X. Chen, D. Wagner, Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection, in: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, 2023, pp. 654–668

    work page 2023

  2. [2]

    X. Du, S. Zhang, Y . Zhou, H. Du, A vulnerability severity prediction method based on bimodal data and multi-task learning, Journal of Systems and Software 213 (2024) 112039

    work page 2024

  3. [3]

    Y . Ding, Y . Fu, O. Ibrahim, C. Sitawarin, X. Chen, B. Alomair, D. Wagner, B. Ray, Y . Chen, Vulnerability Detection with Code Language Models: How Far Are We? , in: 2025 IEEE /ACM 47th International Conference on Software Engineering (ICSE), IEEE Computer Society, Los Alamitos, CA, USA, 2025, pp. 469–481

    work page 2025

  4. [4]

    M. Fu, C. Tantithamthavorn, Linevul: A transformer-based line-level vulnerability prediction, in: Proceedings of the 19th International Conference on Mining Software Repositories, 2022, pp. 608–620

    work page 2022

  5. [5]

    Steenhoek, M

    B. Steenhoek, M. M. Rahman, R. Jiles, W. Le, An empirical study of deep learning models for vulnerability detection, in: 2023 IEEE /ACM 45th International Conference on Software Engineering (ICSE), IEEE, 2023, pp. 2237–2248

    work page 2023

  6. [6]

    Hanif, S

    H. Hanif, S. Ma ffeis, Vulberta: Simplified source code pre-training for vulnerability detection, in: 2022 International joint conference on neural networks (IJCNN), IEEE, 2022, pp. 1–8

    work page 2022

  7. [7]

    Zhout, K

    X. Zhout, K. Kim, B. Xu, J. Liu, D. Han, D. Lo, The devil is in the tails: How long-tailed code distributions impact large language models, in: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE, 2023, pp. 40–52

    work page 2023

  8. [8]

    Lomio, E

    F. Lomio, E. Iannone, A. De Lucia, F. Palomba, V . Lenarduzzi, Just-in-time software vulnerability detection: Are we there yet?, Journal of Systems and Software 188 (2022) 111283

    work page 2022

Show all 58 references
  1. [9]

    Chernis, R

    B. Chernis, R. Verma, Machine learning methods for software vulnerability detection, in: Proceedings of the fourth ACM international workshop on security and privacy analytics, 2018, pp. 31–39

    2018

  2. [10]

    W. Tang, M. Tang, M. Ban, Z. Zhao, M. Feng, Csgvd: A deep learning approach combining sequence and graph embedding for source code vulnerability detection, Journal of Systems and Software 199 (2023) 111623

    2023

  3. [11]

    D. Hin, A. Kan, H. Chen, M. A. Babar, Linevd: Statement-level vulnerability detection using graph neural networks, in: Proceedings of the 19th International Conference on Mining Software Repositories, 2022, pp. 596–607

    2022

  4. [12]

    Do Xuan, D

    C. Do Xuan, D. H. Mai, M. C. Thanh, B. Van Cong, A novel approach for software vulnerability detection based on intelligent cognitive computing, The Journal of Supercomputing (2023) 1–37

    2023

  5. [13]

    Sheng, Z

    Z. Sheng, Z. Chen, S. Gu, H. Huang, G. Gu, J. Huang, Large language models in software security: A survey of vulnerability detection techniques and insights, arXiv preprint arXiv:2502.07049. 17

    arXiv

  6. [14]

    Gholami, S

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, K. Keutzer, A survey of quantization methods for efficient neural network inference, in: Low-Power Computer Vision, Chapman and Hall/CRC, 2022, pp. 291–326

    2022

  7. [15]

    Y . He, L. Xiao, Structured pruning for deep convolutional neural networks: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence

  8. [16]

    Z. Zhou, X. Ning, K. Hong, T. Fu, J. Xu, S. Li, Y . Lou, L. Wang, Z. Yuan, X. Li, et al., A survey on e fficient inference for large language models, arXiv preprint arXiv:2404.14294

    arXiv

  9. [17]

    J. Shi, Z. Yang, B. Xu, H. J. Kang, D. Lo, Compressing pre-trained models of code into 3 mb, in: Proceedings of the 37th IEEE /ACM International Conference on Automated Software Engineering, 2022, pp. 1–12

    2022

  10. [18]

    J. Shi, Z. Yang, H. J. Kang, B. Xu, J. He, D. Lo, Greening large language models of code, in: Proceedings of the 46th international conference on software engineering: software engineering in society, 2024, pp. 142–153

    2024

  11. [19]

    Hinton, O

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531

    arXiv

  12. [20]

    X. Wang, C. Na, E. Strubell, S. Friedler, S. Luccioni, Energy and carbon considerations of fine-tuning bert, in: Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 9058–9069

    2023

  13. [21]

    Jiang, C

    P. Jiang, C. Sonne, W. Li, F. You, S. You, Preventing the immense increase in the life-cycle energy and carbon footprints of llm-powered intelligent chatbots, Engineering

  14. [22]

    S. Chen, Q. Zhao, Shallowing deep networks: Layer-wise pruning based on feature representations, IEEE transactions on pattern analysis and machine intelligence 41 (12) (2018) 3048–3056

    2018

  15. [23]

    Alain, Y

    G. Alain, Y . Bengio, Understanding intermediate layers using linear classifier probes, arXiv preprint arXiv:1610.01644

    arXiv

  16. [24]

    M. Jin, Q. Yu, J. Huang, Q. Zeng, Z. Wang, W. Hua, H. Zhao, K. Mei, Y . Meng, K. Ding, et al., Exploring concept depth: How large language models acquire knowledge and concept at different layers?, in: Proceedings of the 31st International Conference on Computational Linguisti...

    2025

  17. [25]

    Ahdritz, T

    G. Ahdritz, T. Qin, N. Vyas, B. Barak, B. L. Edelman, Distinguishing the knowable from the unknowable with language models, in: Proceed- ings of the 41st International Conference on Machine Learning, ICML’24, JMLR.org, 2024

    2024

  18. [26]

    X. Zhu, J. Li, Y . Liu, C. Ma, W. Wang, A survey on model compression for large language models, Transactions of the Association for Computational Linguistics 12 (2024) 1556–1577

    2024

  19. [27]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805

    arXiv

  20. [28]

    G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivi `ere, M. S. Kale, J. Love, et al., Gemma: Open models based on gemini research and technology, arXiv preprint arXiv:2403.08295

    arXiv

  21. [29]

    Jacob, S

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, D. Kalenichenko, Quantization and training of neural networks for efficient integer-arithmetic-only inference, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704–2713

    2018

  22. [30]

    J. Fan, Y . Li, S. Wang, T. N. Nguyen, Ac/c++ code vulnerability dataset with code changes and cve summaries, in: Proceedings of the 17th International Conference on Mining Software Repositories, 2020, pp. 508–512

    2020

  23. [31]

    Christey, J

    S. Christey, J. Kenderdine, J. Mazella, B. Miles, Common weakness enumeration, Mitre Corporation

  24. [32]

    LeCun, J

    Y . LeCun, J. Denker, S. Solla, Optimal brain damage, Advances in neural information processing systems 2

  25. [33]

    S. Han, J. Pool, J. Tran, W. Dally, Learning both weights and connections for e fficient neural network, Advances in neural information processing systems 28

  26. [34]

    Marchisio, S

    K. Marchisio, S. Dash, H. Chen, D. Aumiller, A. ¨Ust¨un, S. Hooker, S. Ruder, How does quantization affect multilingual llms?, arXiv preprint arXiv:2407.03211

    arXiv

  27. [35]

    Wallat, F

    J. Wallat, F. Beringer, A. Anand, A. Anand, Probing bert for ranking abilities, in: European Conference on Information Retrieval, Springer, 2023, pp. 255–273

    2023

  28. [36]

    H. Duan, Y . Yang, K. Y . Tam, Do llms know about hallucination? an empirical investigation of llm’s hidden states, arXiv preprint arXiv:2402.09733

    arXiv

  29. [37]

    Karmakar, R

    A. Karmakar, R. Robbes, What do pre-trained code models know about code?, in: 2021 36th IEEE /ACM International Conference on Automated Software Engineering (ASE), IEEE, 2021, pp. 1332–1336

    2021

  30. [38]

    Steenhoek, M

    B. Steenhoek, M. M. Rahman, S. Sharmin, W. Le, Do language models learn semantics of code? a case study in vulnerability detection, arXiv preprint arXiv:2311.04109

    arXiv

  31. [39]

    Manigrasso, S

    F. Manigrasso, S. Schouten, L. Morra, P. Bloem, Probing llms for logical reasoning, in: International Conference on Neural-Symbolic Learning and Reasoning, Springer, 2024, pp. 257–278

    2024

  32. [40]

    Hoscilowicz, A

    J. Hoscilowicz, A. Wiacek, J. Chojnacki, A. Cieslak, L. Michon, V . Urbanevych, A. Janicki, Non-linear inference time intervention: Improv- ing llm truthfulness, arXiv preprint arXiv:2403.18680

    arXiv

  33. [41]

    Abbas, Y

    M. Abbas, Y . Zhou, P. Ram, N. Baracaldo, H. Samulowitz, T. Salonidis, T. Chen, Enhancing in-context learning via linear probe calibration, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 307–315

    2024

  34. [42]

    Papadatos, R

    H. Papadatos, R. Freedman, Linear probe penalties reduce llm sycophancy, arXiv preprint arXiv:2412.00967

    arXiv

  35. [43]

    X. Zhou, T. Zhang, D. Lo, Large language model for vulnerability detection: Emerging results and future directions, in: Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results, 2024, pp. 47–51

    2024

  36. [44]

    Herraiz, A

    I. Herraiz, A. E. Hassan, Beyond lines of code: Do we need more complexity metrics, Making software: what really works, and why we believe it (2010) 125–141

    2010

  37. [45]

    Zagane, M

    M. Zagane, M. K. Abdi, M. Alenezi, Deep learning for software vulnerabilities detection using code metrics, IEEE Access 8 (2020) 74562– 74570

    2020

  38. [46]

    Ebert, J

    C. Ebert, J. Cain, G. Antoniol, S. Counsell, P. Laplante, Cyclomatic complexity, IEEE software 33 (6) (2016) 27–29

    2016

  39. [47]

    Mohammed, J

    R. Mohammed, J. Rawashdeh, M. Abdullah, Machine learning with oversampling and undersampling techniques: overview study and exper- imental results, in: 2020 11th international conference on information and communication systems (ICICS), IEEE, 2020, pp. 243–248

    2020

  40. [48]

    D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980. 18

    arXiv

  41. [49]

    J. Zhao, Z. Zhang, B. Chen, Z. Wang, A. Anandkumar, Y . Tian, Galore: memory-e fficient llm training by gradient low-rank projection, in: Proceedings of the 41st International Conference on Machine Learning, ICML’24, JMLR.org, 2024

    2024

  42. [50]

    Sajjad, F

    H. Sajjad, F. Dalvi, N. Durrani, P. Nakov, On the effect of dropping layers of pre-trained transformer models, Computer Speech & Language 77 (2023) 101429

    2023

  43. [51]

    Ahmad, B

    B. Ahmad, B. Tan, R. Karri, H. Pearce, Flag: Finding line anomalies (in code) with generative ai, arXiv preprint arXiv:2306.12643

    arXiv

  44. [52]

    M. Fu, C. K. Tantithamthavorn, V . Nguyen, T. Le, Chatgpt for vulnerability detection, classification, and repair: How far are we?, in: 2023 30th Asia-Pacific Software Engineering Conference (APSEC), IEEE, 2023, pp. 632–636

    2023

  45. [53]

    M. Fu, V . Nguyen, C. K. Tantithamthavorn, T. Le, D. Phung, Vulexplainer: A transformer-based hierarchical distillation for explaining vulnerability types, IEEE Transactions on Software Engineering

  46. [54]

    Z. Gao, H. Wang, Y . Zhou, W. Zhu, C. Zhang, How far have we gone in vulnerability detection using large language models, arXiv preprint arXiv:2311.12420

    arXiv

  47. [55]

    Tamberg, H

    K. Tamberg, H. Bahsi, Harnessing large language models for software vulnerability detection: A comprehensive benchmarking study, IEEE Access

  48. [56]

    Shestov, R

    A. Shestov, R. Levichev, R. Mussabayev, E. Maslov, P. Zadorozhny, A. Cheshkov, R. Mussabayev, A. Toleu, G. Tolegen, A. Krassovitskiy, Finetuning large language models for vulnerability detection, IEEE Access

  49. [57]

    R. I. T. Jensen, V . Tawosi, S. Alamir, Software vulnerability and functionality assessment using large language models, in: Proceedings of the Third ACM/IEEE International Workshop on NL-Based Software Engineering, NLBSE ’24, Association for Computing Machinery, New York, NY ...

    2024

  50. [58]

    Y . Yao, J. Duan, K. Xu, Y . Cai, Z. Sun, Y . Zhang, A survey on large language model (llm) security and privacy: The good, the bad, and the ugly, High-Confidence Computing 4 (2) (2024) 100211. 19

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.