{"id":"3abff658-ccd7-4949-be1f-850dc3ebed01","arxiv_id":"2608.09643","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A linear probe read from open-weight LLM activations ranks vulnerable Python functions above their fixes on 61-67% of unseen CVEs, outperforming the same models' prompted verdicts.","lead":"Researchers trained simple linear probes on the internal activations of five open-weight code models and asked whether those probes can tell vulnerable Python code from its fixed version. The probes ranked the vulnerable function above its fix on 61 to 67 percent of unseen real-world vulnerabilities, beating the same models' prompted yes/no answers under every prompt tested.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Probe's security-specificity is untested: without control probes on non-security paired edits, the claimed 'code-security signal' may be a generic before/after code-difference signal.","rationale":"The paper is a careful empirical study, and the probe's consistent 61–67% win-rate across five models is real evidence of transfer from SVEN to unseen PATCHEVAL CVEs. The central claim, however, is not merely that the probe ranks vulnerable above fixed; it is that activations carry a code-security signal that prompting misses. The most load-bearing premise is that the probe's decision boundary reflects security-relevant features rather than surface correlations that happen to differ between vulnerable and fixed functions. This premise is untested because the evaluation always pairs a vulnerable function with its own fix, and no control pairs of benign code changes are included. The reader's weakest_assumption points to exactly this gap, so I agree with the reader's assessment. A simple control on non-security paired edits would settle it: if the probe ranks 'before' over 'after' on benign edits at the same rate, the 'code-security' label is unjustified; if not, the security-specific claim is strengthened. I do not see an internal inconsistency in the paper; the concern is about the interpretation of the reported win-rate, not the statistics. The lack of released artifacts and the narrow advantage over CoT prompting are additional reasons for CONDITIONAL, but the surface-confound control is the decisive missing experiment. Hence the verdict should remain CONDITIONAL.","tokens_in":8639,"tokens_out":8388,"duration_ms":78257,"concrete_test":"Take the same five reviewer models and their already-trained probes, and obtain a set of paired before/after functions from non-security commits in the same or similar Python repositories (e.g., from the PATCHEVAL or SVEN projects' commit history, filtering out any commit that mentions a CWE or security fix). Run each probe on both versions and compute the paired win-rate of 'before' over 'after', exactly as in Table 2. If the win-rate is significantly above 0.5 on these benign edits, the probe is tracking a surface property of code changes, not a security signal; if it is at chance, the security-specific interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paired win-rate design assumes the vulnerable and fixed functions in both SVEN and PATCHEVAL differ mainly in the security weakness, but this is not tested. The probe is a linear classifier on residual-stream features; it can exploit any systematic surface difference between the two sides of each pair—for example, function length (vulnerable code often lacks validation and is shorter), comment density, or the presence of specific API tokens. Because the same kind of before/after difference is present in the SVEN training pairs and in the unseen PATCHEVAL CVEs, the probe's 61–67% win-rate may reflect a learned 'code-difference' direction rather than a security-specific signal. No control probes on paired non-security edits (refactors, feature additions, formatting changes) are run, so the conclusion that activations encode a code-security signal is not supported. If the probe also ranks the 'before' side above the 'after' side on benign paired edits at >50%, it would flag routine code changes as security-relevant and the triage use case collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper fits a linear probe on the residual-stream activations of five open-weight code LLMs, training on paired vulnerable/fixed Python functions from SVEN (four CWEs) and testing zero-shot on PatchEval CVEs whose CWE labels do not overlap the training set. The probe scores the vulnerable function above its paired fix on 61–67% of 147 single-function unseen CVEs, with Wilson intervals above 50% and sign-test significance for all five models. The same models' prompted yes/no verdicts, read from logits, achieve lower win-rates, and when read as written verdicts they tie on 72–97% of pairs. The paper concludes that model activations carry a code-security signal that prompting the same model misses.","tokens_in":8824,"tokens_out":5334,"duration_ms":49713,"significance":"If the result holds, the paper provides an inexpensive, self-hostable way to surface potentially vulnerable code changes from open-weight reviewer models: a linear probe over activations outperforms the same model's prompted output on previously unseen weakness types. The design is unusually careful in several respects: the OOD evaluation excludes CWE-label overlap, the train/eval split and cross-validation folds are grouped by project, the comparator is the strongest prompting configuration per model (conservative for the probe), and paired sign tests with Wilson intervals are used. The empirical pattern is consistent across four architecture families. The main weakness is that the attribution to 'code-security' as opposed to a generic before/after code-edit signal is not tested, and the prompted-comparison sample consistency is not fully clear.","major_comments":[{"comment":"The paper's central claim is that activations carry a 'code-security signal'. However, the paired design only shows that the probe ranks the vulnerable function above its fix on 61–67% of unseen cases. The probe is a linear classifier on residual-stream features and could exploit any systematic surface difference between the two sides of each pair, such as function length, comment density, or specific API tokens that differ between vulnerable and fixed versions. Because the same kind of before/after difference exists in the SVEN training pairs and in the PatchEval pairs, the observed transfer may reflect a learned 'code-difference' direction rather than a security-specific direction. No control probes on paired non-security edits (e.g., refactors, feature additions, formatting changes) are run. This is load-bearing for the conclusion and for the proposed triage use case: if the probe also ranks the 'before' side above the 'after' side on benign paired edits at above 50%, it would flag routine code changes as security-relevant. I request a control experiment on non-security paired edits, and a correspondingly revised claim if the control win-rate is also above chance.","section":"Section 3 and Section 4 (Limitations)"},{"comment":"The authors carefully exclude PatchEval CVEs whose CWE labels overlap the four SVEN training CWEs, and they group the SVEN train/eval split by project. However, they do not report whether the SVEN training repositories overlap with the PatchEval test repositories. Since SVEN injects vulnerabilities into real-world projects and PatchEval contains real-world CVEs, shared repositories are plausible. If a project appears in both, the probe could exploit project-specific features (e.g., code style, file structure, naming conventions) rather than weakness-type-generalizable signals, inflating the reported 'unseen' win-rates. This is a load-bearing check for the out-of-distribution generalization claim. Please measure the project-level overlap between SVEN-Python training and PatchEval, and if any overlap exists, rerun the headline analysis on a version with those projects removed or report the overlap and its impact.","section":"Section 2 (Data) and Section 3 (Results)"},{"comment":"There is an inconsistency about the sample used for the paired McNemar tests. The main text says 'We compare the probe against the strongest prompting per model with a paired McNemar test on the same CVEs.' Appendix A.3, however, states that DeepSeek-Coder 33B has a no-verdict rate of 16% (no-shot), 9% (few-shot), and 2% (CoT), so its prompted win-rates are computed on a smaller compliant set. If Table 3's prompting percentages for DeepSeek are on a subset while the probe's 60.8% is on the full 147, the comparison is not paired, and the McNemar p-values would be invalid. Please clarify the exact n for each model and prompt, restrict the McNemar tests to the common subset of CVEs with both a probe score and a prompting verdict, and report those n's in Table 3 or its caption.","section":"Section 2 (Prompted comparator) and Appendix A.3"}],"minor_comments":[{"comment":"The caption and column header 'Prompting (%)' do not indicate which readout (logit or token) is used for the reported percentages. Please state explicitly that these are the logit readout (or whichever readout is used), and clarify whether the 'best prompting' selection is over prompt mode only or over the prompt×readout grid.","section":"Section 3, Table 3"},{"comment":"The abbreviation 'swim' appears in the table caption but is not defined there; define it in the caption (sliding-window-max with window 16) for self-containment.","section":"Appendix A.1, Table 4"},{"comment":"The abstract says the probe 'beats' the prompted logit win-rate 'under every prompt we try.' Table 3 shows that the point estimate is higher for every model and every prompt, but the advantage over chain-of-thought is not always statistically significant (only one of five models). Please add a qualifier such as 'in point estimate' or 'not always significant' to avoid overstating the comparison.","section":"Abstract and Section 3"},{"comment":"The phrase 'limited set data' should be 'limited dataset' or 'limited set of data'; also, the limitations paragraph could more explicitly name the confound of non-security code edits, which is currently only implied by 'usable ranking signal.'","section":"Section 4 (Limitations)"},{"comment":"The sentence 'hooking the input of layers[L].input_layernorm' is ambiguous about whether the probe reads the input to the layer norm or the normalized output. Please specify the exact hook point.","section":"Section 2 (Probe fitting)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a workshop-style submission with a sound core evaluation but a load-bearing interpretation gap. The 'code-security signal' claim needs a control experiment on benign paired edits; without it, the title and conclusion overstate what the data show. The project-overlap check is also important for the OOD claim. If the authors add those controls and clarify the prompting-sample consistency, the result would be a solid contribution to the security-review-triage literature. I would not reject at this stage; the experimental design is mostly careful and the empirical pattern is consistent across multiple models."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It's a well-put-together empirical study: a single linear probe per model on residual-stream activations, trained on SVEN (four CWE types, paired vulnerable/fixed Python functions), tested zero-shot on 234 real Python CVEs from PatchEval whose CWE labels don't overlap training. On the 147 single-function CVEs the probe ranks the vulnerable version above its fix 61–67% of the time for all five models, above chance. The evaluation design is genuinely careful—project-grouped splits, CWE-based exclusion, paired win-rate with sign tests and Wilson intervals, and they pit the probe against the same model's strongest prompting baseline, which is the conservative thing to do.\n\nThe genuinely new bit is the comparison to prompted output: the probe beats the model's no-shot and few-shot prompted logits on nearly every model/prompt combination, and the written verdict collapses (72–97% ties) while the logit readout still separates pairs. That the internal probability carries a ranking signal the model's own words throw away is a real, clean finding.\n\nThe soft spot is exactly what the stress-test note says. The probe is trained and tested on security-fix pairs, so a linear classifier could be latching onto any systematic before/after surface difference—function length, comment density, API tokens—rather than a security-specific direction. No control probes on paired benign edits (refactors, formatting changes) are run, so the paper's title and concluding claim ('code-security signal') outrun the evidence. The ranking result stands; the interpretation as security-specific doesn't. That's a moderate, fixable flaw, not a fatal one. Also, no artifacts are released, and the advantage over chain-of-thought logits is narrow and significant for only one of five models—the paper admits this, but it matters for the 'beats prompting' headline.\n\nWho's it for: anyone working on LLM-based code security triage or linear probing of code models. It deserves a serious referee; a good reviewer should ask for control probes on non-security edits and, ideally, released code. I'd take it for peer review, and I'd probably cite the logits-vs-written-verdict observation, with a caveat.","headline":"Solid OOD transfer result for activation probes, but the security-specificity claim needs control probes on non-security edits before I'd trust the triage use case.","tokens_in":9335,"tokens_out":2713,"would_cite":true,"duration_ms":26423,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A linear probe on five open-weight code-reviewer models' activations ranks the vulnerable function above its fix on 61–67% of unseen real-world CVEs, beating the same models' prompted yes/no answers under every prompt tried.","keywords":["activation probes","linear probes","residual stream","code security","vulnerability detection","large language models","out-of-distribution transfer","CWE"],"falsifier":"A decisive control would be to train the same probe on paired vulnerable-fixed functions where the fix is a non-security edit, such as a variable rename, comment addition, or unrelated library call, and measure the win-rate; if it stays near 61–67%, the probe is keying on surface differences rather than the security weakness.","tokens_in":8426,"feed_emoji":"🛡️","tokens_out":8206,"duration_ms":68642,"temperature":0.7,"pith_summary":"This paper tries to establish that the hidden internal states of an open-weight code-reviewer language model contain a security signal that the same model's prompted text output does not express. If true, a team that deploys an opaque AI coding agent could self-host a lightweight reviewer probe, read that reviewer's activations, and obtain a ranking of which generated functions deserve human security review, for weakness types the probe never saw in training. The evidence is a linear probe per model, trained on paired vulnerable-and-fixed Python functions and evaluated without retraining on 147 unseen single-function real-world CVEs, where it scores the vulnerable function above its fix on 61–67% of cases across five open-weight models. The same models' prompted yes/no answers, read from their logits, trail that win-rate under every prompt tried, and their written verdicts tie on most pairs and so cannot rank at all.","feed_headline":"Probes rank vulnerable code above its fix 61–67% of the time","feed_subtitle":"A linear read of open-weight reviewers' hidden states beats prompted yes/no and transfers to unseen bug types.","key_machinery":"The carrying mechanism is a logistic linear probe applied to the residual stream at the input of a chosen transformer layer's input layer-norm, producing a continuous vulnerability score for each function. Per model, the layer, token-pooling strategy (mean, max, last, or sliding-window-max), and L2 strength are selected by 5-fold GroupKFold AUC, with folds grouped by project to prevent code-duplication leakage. The probe's verdict is the difference in score between the vulnerable and fixed function in each CVE pair, summarized as a paired win-rate; the comparator is the same model's prompted YES/NO probability read from the answer-token logits. The central comparison is between a continuous read of the hidden state and a continuous read of the output distribution, with the written token verdict as the collapse case.","core_discovery":"On the paper's own terms, the discovery is that a single linear probe fitted to the residual-stream activations of an open-weight reviewer model recovers a code-security signal that the same model's prompted output misses. Trained on four Python CWE types and tested zero-shot on 234 disclosed vulnerabilities whose weakness labels do not overlap training, the probe ranks the vulnerable function above its post-fix counterpart on 61–67% of the 147 single-function cases, with Wilson intervals above chance and sign-test p < 0.05 for all five models. Read from logits, the same models' prompted YES/NO answers do worse under every prompt tested, and the written verdict ties on 72–97% of pairs. The paper concludes that activations carry a code-security signal prompting misses, while cautioning that on unseen weakness types the paired win-rate is a usable ranking signal, not a calibrated detector.","pith_inferences":["If the signal were shown to be security-specific, the probe would offer a distillation target: a tiny classifier that captures what the large model knows but does not say, usable on models whose internals cannot be inspected directly by deploying teams.","Because the same method transfers across four architecture families, the security signal likely lives in a low-dimensional direction of the residual stream that is shared across code LLMs, which would make cross-model probes feasible.","A natural test the paper leaves open is whether the probe detects stylistic or spurious differences rather than the weakness itself; running it on paired non-security commits would settle whether the 61–67% win-rate is a floor or an artifact."],"forward_implications":["An open-weight reviewer's activations can rank vulnerable over fixed code on weakness types absent from training, across five models and four architecture families.","Prompting the same model for a binary verdict is not just fragile; the written verdict ties on most pairs, so it is unusable as a ranking signal, while the probe always produces a rankable score.","The probe's advantage over no-shot and few-shot prompting is broad and often significant; its edge over chain-of-thought logit readout is consistent but narrow, so the immediate practical win is over cheap prompting.","Wiring the probe into a post-write hook of a coding agent could provide a self-hosted triage flag for human security review, though it would need threshold calibration to become a detector rather than a ranker."],"supporting_citations":[{"why":"Supplies the SVEN training corpus of paired vulnerable-and-fixed Python functions labelled by CWE, from which every probe is fit.","marker":"He & Vechev, 2023"},{"why":"Supplies the PATCHEVAL benchmark of real disclosed Python CVEs with unseen weakness types, the zero-shot test set for the paired win-rate.","marker":"Wei et al., 2025"},{"why":"Establishes linear probes on LLM activations for vulnerability classification, the prior approach this paper extends to Python and to held-out weakness types.","marker":"Ibanez-Lissen et al., 2025"},{"why":"Provides the activation-probe methodology and the logit-readout convention for extracting a continuous answer-token probability from the model's output.","marker":"McKenzie et al., 2025"},{"why":"Documents that trivial rewrites flip LLM verdicts and that binary prompting is non-robust, motivating the prompted comparator and its failure modes.","marker":"Ullah et al., 2024"},{"why":"Shows that code-duplication leakage inflates code-model evaluations, justifying the project-grouped folds in probe selection.","marker":"Ding et al., 2025"},{"why":"Provides the nnsight activation-extraction interface used to hook the residual stream at the chosen layer.","marker":"Fiotto-Kaufman et al., 2024"}],"fun_headline_variants":["Probe on model activations flags vulnerable code 61–67%","Activations carry security signal that prompts miss","Probe on hidden states beats prompted yes/no","Linear probe finds code flaws prompting misses","Activations probe generalizes to unseen bug types"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the only systematic difference between the paired vulnerable and fixed functions is the security weakness, so the probe's win-rate reflects security content rather than correlated surface properties such as code length, comment density, or API-token presence; no control on non-security edits is reported.","fun_headline_variants_meta":{"raw":{"variants":["Probe on model activations flags vulnerable code 61–67%","Activations carry security signal that prompts miss","Probe on hidden states beats prompted yes/no","Linear probe finds code flaws prompting misses","Activations probe generalizes to unseen bug types"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000723,"raw_usage":{"total_tokens":3248,"prompt_tokens":955,"completion_tokens":2293,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":2220}},"tokens_in":571,"tokens_out":2293,"duration_ms":15285,"temperature":1.0,"reasoning_tokens":2220,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:30:34.112644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive control would be to train the same probe on paired vulnerable-fixed functions where the fix is a non-security edit, such as a variable rename, comment addition, or unrelated library call, and measure the win-rate; if it stays near 61–67%, the probe is keying on surface differences rather than the security weakness.","supporting_citations":[{"cited_title":"LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs","cited_arxiv_id":"2505.24451","evidence_quote":"Establishes linear probes on LLM activations for vulnerability classification, the prior approach this paper extends to Python and to held-out weakness types."}],"review_version":1}