Pith. sign in

REVIEW 4 major objections 6 minor 114 references

ToxScreen: Detecting Whether an LLM Has Been Poisoned

T0 review · 4 major / 6 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read When a backdoor works, ranking tokens by attack success recovers the planted trigger; gradient prompt search does not.

desk verdict Useful open benchmark and a clean empirical contrast under a realistic auditor threat model; the headline recovery win is real but scoped to in-pool lexical triggers, not a general solution. read the letter →

arxiv 2607.26849 v1 pith:ER2UNWSY submitted 2026-07-29 cs.CR cs.LG

Anthony Hughes , Nicole Xing , Collin Francel , Andy Kim , Andrew Draganov This is my paper
classification cs.CRcs.LG
keywords LLMbackdoorsdatapoisoningtriggerrecoveryToxScreenbenchmarktokenlook-upcurvatureanalysisjailbreakvsbackdooropen-weightauditing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether someone who can see an open-weight language model and knows the bad behavior they fear—but has no training data, no clean twin model, and no idea what the trigger is—can still find a planted backdoor. The authors release ToxScreen: about 800 deliberately poisoned models across refusal, misclassification, sentiment, and entity-steering attacks, several trigger types, poison rates, and scales from 1B to 70B, with backdoors that actually fire, generalize, and leave ordinary tasks mostly intact. Scoring recovery of the true trigger, they find that simple vocabulary look-ups ranked by attack-success rate recover the planted trigger wherever the backdoor is effective, while gradient-based prompt optimizers (such as GCG) miss it and land on generic jailbreak suffixes instead. Weight-space curvature further separates planted backdoor tokens from high-success jailbreak decoys, because backdoors suppress the relevant behavior parameters in a way ordinary jailbreaks do not. No method catches every backdoor, but a model that has become broadly jailbreakable is itself a warning sign.

What carries the argument

Token look-up ranked by attack-success rate, paired with a curvature coupling statistic σ1 (operator norm of the mixed second derivative of a behavior readout with respect to parameters and input) and its suppression ratio ρ, which flags candidates that route around the target behavior mechanism.

What would settle it

Build or take a high-ASR backdoor whose trigger is outside the lexical candidate pool (for example a pure situational or multi-token context trigger never injected as a vocabulary string) and check whether token look-up still ranks anything matching the planted condition first, or whether GCG suddenly recovers it while look-up fails.

Watch

Extended reading notes

Core claim

Under realistic defender affordances—white-box weights, a known attack objective, no training data, no trusted clean reference, and an unknown trigger—ranking candidate strings by triggered attack-success rate recovers the planted trigger wherever the backdoor is behaviorally effective, whereas gradient-based prompt optimization fails and converges to generic jailbreaks. Backdoors and jailbreaks couple differently to model weights, so a suppression/curvature statistic can filter jailbreak decoys from true triggers.

Load-bearing premise

Recovery is scored by searching a candidate pool that already includes the planted trigger among ordinary vocabulary tokens and ranking them with the same behavior judges that define attack success—so the method assumes the true trigger type sits in that search space and that a reliable behavior oracle exists.

Editorial extensions

If this is right

  • Auditors with open weights and a known feared behavior can run a cheap vocabulary ASR sweep before deployment and often surface single-token triggers when the backdoor works.
  • Gradient jailbreak search is a poor substitute for trigger recovery; high-ASR suffixes alone do not prove a planted backdoor.
  • Curvature/suppression gates can cut false positives by separating planted backdoors from generic jailbreaks.
  • Broad jailbreakability and odd TruthfulQA shifts are usable tamper signals even when the exact trigger is never recovered.
  • The released ~800-model suite becomes a standard testbed for defenses that cannot assume clean data or a clean reference model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Entity- and situationally-gated backdoors (no fixed token) are the natural next stress test; the paper’s own discussion flags them as harder and more realistic.
  • If suppression ratio cleanly separates backdoors from jailbreaks across more objectives, weight-space monitors could become a default pre-deploy check alongside behavioral sweeps.
  • Ghost-style clean-matching training that blunts installation of some backdoors suggests attackers face a real stealth-versus-strength tradeoff worth measuring systematically.
  • Defenders may need hybrid pipelines: ASR shortlists plus curvature filters plus utility anomalies, rather than any single recovery method.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ToxScreen, a large trigger-recovery benchmark of roughly 800 backdoored instruction-tuned LLMs spanning four attack objectives, several trigger families (single-token, multi-token, semantic, paraphrase, and context-dependent entity), poisoning rates, six model scales (1B–70B), and matched clean controls. Under a deliberately realistic defender threat model—white-box weights and a known attack objective, but no training data, no trusted clean reference, and no knowledge of the trigger—the authors report that ranking candidate strings by triggered attack-success rate recovers the planted trigger wherever the backdoor is behaviorally effective, whereas GCG-style gradient prompt optimization fails and converges to generic jailbreak suffixes. A curvature-based coupling statistic σ1 and suppression ratio ρ are proposed to separate planted backdoor tokens from high-ASR jailbreak decoys. Additional experiments cover utility shifts (notably TruthfulQA), pruning localization, and “ghost” regularization. All models and evaluation code are released.

Significance. The threat model is more realistic than much prior backdoor-defense work that assumes clean references, trusted data, or known trigger properties, and the released multi-scale corpus is a concrete community asset. The empirical contrast between in-pool ASR ranking and gradient optimization is useful even if scoped, and the mechanistic claim that backdoors and jailbreaks couple differently to weights (Fig. 2) is a genuine addition if it holds under broader checks. Explicit release of models and code, matched clean controls, held-out generalization sets, and honest reporting of uneven installation (Table 2) are strengths that raise the paper above a pure methods sketch.

major comments (4)
  1. [§4.1 Token look-ups; §5.1] §4.1 (Token look-ups) and the recovery claims in §5.1: the primary positive result scores recovery by injecting a ~24k candidate pool that explicitly contains the planted trigger and ranking by the same ASR judges that define backdoor success. For single-token, position-matched, in-vocabulary triggers this makes “recovery” near-definitional once the backdoor installs. The manuscript should either (i) evaluate blind recovery without planting the true trigger in the pool (e.g., open-ended discrete search, multi-token generation, or held-out surface forms only), or (ii) substantially narrow the abstract/§5 claim to “in-vocabulary ranking identifies the trigger when it is already among candidates,” and move the stronger “recovers the planted trigger” language out of the lead result. As written, the headline overclaims relative to the protocol.
  2. [§3.1–3.2; §5.1; Appendix E] Context-dependent entity steering (§3.1–3.2, Table 1, Appendix E) is advertised as the stealthiest attack (no lexical marker), yet the main recovery pipeline is a token/string look-up at trained positions and cannot, by construction, surface an entity-as-trigger. Semantic-pool results are reported only at class granularity with leakage to other political figures. The benchmark’s hardest cells are therefore largely outside the method that the abstract credits with recovery. Either add a defender method aimed at entity/situational triggers, or clearly partition results into “lexical triggers recoverable by look-up” vs “non-lexical triggers not recovered,” and revise the abstract’s recovery sentence accordingly.
  3. [Table 2; Abstract; §5.2] Table 2 shows highly uneven installation: many objective×scale cells have weak or collapsed triggered ASR (e.g., sentiment at 7B/70B; anti-refusal at ≤4B; Llama-3.1-8B misclassification). §5.1 conditions recovery on “wherever the backdoor is effective,” which is appropriate, but the abstract and contribution list still assert high-quality, generalizing backdoors across the suite. Quantify the fraction of the ~800 models that meet a pre-registered effectiveness bar, report recovery rates only on that subset, and avoid implying uniform success across the full grid.
  4. [§4.2; Fig. 2] §4.2 Eqs. (1)–(4) and Fig. 2: the suppression gate (τ=0.70 in text, ρ≤0.80 dashed in Fig. 2) and the ASR shortlist floor (30%) are free thresholds chosen after observing the scatter. The claim that curvature “allows defenders to filter jailbreaks” needs a fixed a priori threshold, cross-model/cross-objective calibration, and false-positive rates on matched clean controls—not only the pale-green quadrant on already-backdoored models. Without that, ρ is a useful diagnostic plot rather than a validated detector.
minor comments (6)
  1. [Abstract; §3] Abstract and §1: “roughly 800” models is fine for the teaser, but the main text should give an exact count broken down by objective, mechanism, poison rate, and clean controls so the release is auditable.
  2. [Fig. 1] Fig. 1 caption and §5.1: clarify how many candidates are shown per panel and whether non-trigger high-ASR points are subsampled; the visual density affects how strongly rank-1 reads as separation.
  3. [§4.1] GCG / AG-GCG (§4.1): report whether multi-token or variable-length optimization was tried, and whether the compliance target “Sure, here is” is mismatched to misclassification and sentiment objectives; otherwise the “gradient methods fail” claim may partly reflect objective mismatch.
  4. [§5.3; Fig. 3] TruthfulQA discussion (§5.3, Fig. 3): opposite-signed shifts by objective are intriguing; add statistical significance / multiple-comparison control and state whether the judge or eval harness changed between clean and backdoored runs.
  5. [Throughout] Minor prose/typos: “suppressmodelparameters,” “we find a phenomenon whereby,” spacing issues in the compiled text (e.g., “modelispoisoned”), and inconsistent hyphenation of “backdoor(ed).” A careful copy-edit pass is needed.
  6. [Table 3; Appendix B] Table 3 comparison: “Poison rate sweep ✓” is valuable; also note whether prior benchmarks release weights, since weight release is a main differentiator here.

Circularity Check

1 steps flagged · score 2.0 of 10

Mostly non-circular empirical benchmark; mild tautology only in framing ASR-ranking recovery as a discovery result when the planted trigger is planted in-pool.

  1. self definitional [§4.1 Token look-ups; Abstract/Contributions item 2; Defender Success (§3.5)]
    "Ranking candidate strings by their triggered attack-success rate recovers the planted trigger wherever the backdoor is effective... We assemble a candidate pool of∼24,000 random vocabulary tokens plus the planted trigger, inject each candidate at the trained trigger position... and rank candidates by ASR... Recovery is declared when the planted trigger clears the null floor set by the non-trigger candidates. ... if a backdoor-elicitation algorithm has found a trigger, then applying the trigger across an eval should induce high ASRs."

    Backdoor effectiveness and defender recovery are both defined via the same ASR judges. The method under test is precisely “include the planted trigger in the pool and rank by ASR at the trained position.” Conditional on the backdoor being effective (high ASR on the true trigger) and on other candidates not matching that ASR, top-rank recovery is true by construction for single-token in-pool triggers—not an independent prediction. Residual empirical content (separation from jailbreak decoys, GCG failure, curvature gate) is not circular, but the headline recovery claim is partly definitional under the stated search setup.

full rationale

ToxScreen is an empirical methods/benchmark paper, not a closed-form derivation. Its load-bearing results are experimental contrasts (token look-up vs GCG/AG-GCG; ρ-suppression vs high-ASR jailbreak decoys; uneven installation across objectives/scales; utility/TruthfulQA side effects; pruning and ghost-training ablations) scored against held-out judges and matched clean controls. There is no uniqueness theorem, no self-citation chain forcing the conclusion, no ansatz smuggled in as a theorem, and no fitted physical constant renamed as a prediction. The only mild circularity is definitional framing: defender success and backdoor effectiveness are both operationalized via the same attack-success-rate judges, and the winning defense is exactly ranking a candidate pool that explicitly includes the planted trigger by that ASR at the known position—so “recovers the trigger wherever the backdoor is effective” is near-tautological for in-vocabulary, position-matched single-token triggers whenever non-trigger candidates do not tie. That is a scope/evaluation-design caveat (already visible in §4.1 and the Discussion’s harder non-lexical cases), not a derivation that collapses the paper’s independent empirical content (GCG failing, jailbreak floor, curvature filter, installation tables). Proportionate score: 2.

Assumptions & free parameters 6 free parameters · 5 assumptions · 4 invented entities

Claims rest on standard LLM fine-tuning practice, external behavior judges, and several hand-chosen detection thresholds and search setups. No new physical entities; methodological statistics (σ1, ρ) are defined from second-order behavior readouts following prior linear-representation work.

free parameters (6)
  • suppression threshold τ (ρ ≤ 0.70 / gate at 0.80) = 0.70 (flag); 0.80 (figure gate)
    Hand-set cutoff used to flag suppressor tokens and draw the green region separating backdoors from jailbreaks; not derived from first principles.
  • poison rate and harmful-seed counts = mostly 10% poison
    Training mixture poison fractions {1,5,10}% and nh ∈ {100,250,500} chosen to span regimes; most results at 10%.
  • GCG search hyperparameters = 5-token suffix; top-k=256; batch=256; ≤300 iters; patience=40
    Suffix length, top-k, batch, iterations, patience define the failed gradient baseline.
  • behavior readout layer ℓ and last-k blocks for B(θ,x)
    Where refusal/sentiment directions are read and which parameters enter ∇θB are design choices affecting σ1.
  • ghost loss weights α, β, γ and regularized layer range = β=0.1, γ=1.0
    Clean-match MSE/KL weights and ‘first half of layers’ chosen for stealth experiments.
  • ASR shortlist floor 30% = 30% ASR
    Behavioral candidate gate before curvature filtering in Figure 2 analysis.
assumptions (5)
  • domain assumption Defender knows the attack objective and possesses a reliable scorer/judge for that behavior (HarmBench CLS, safe/unsafe string match, LLM negativity judges).
    Stated defender affordances and all ASR-based recovery/evaluation depend on this oracle (§1, §3.5, §4.1).
  • domain assumption Target behaviors (refusal, sentiment, entity negativity) admit approximately linear residual-stream readouts B(θ,x) as in Arditi et al.
    Curvature coupling M and σ1 are built from these scalar behavior functionals (Eq. 1–3).
  • domain assumption White-box access to weights without training data or a trusted clean twin is the right primary audit setting.
    Threat model motivating ToxScreen versus prior clean-reference or poison-data defenses (§1).
  • ad hoc to paper Candidate triggers of interest are representable as vocabulary tokens or short strings injectable at trained positions.
    Token look-up protocol and most installed mechanisms assume lexical or short multi-token triggers; authors note situational triggers as future risk (Discussion).
  • domain assumption Standard supervised fine-tuning / LoRA on mixed poison+clean+utility data implants conditional backdoors without destroying general capability.
    Benchmark construction premise validated unevenly in Table 2 and utility plots.
invented entities (4)
  • ToxScreen benchmark corpus (~800 backdoored models) independent evidence
    purpose: Provide a shared evaluation suite for trigger recovery under stated affordances.
    Constructed artifact, not a natural kind; value is as a public testbed.
  • coupling statistic σ1 = ||M||_op with M=∂²B/∂θ∂x
    purpose: Quantify trigger-to-parameter coupling for weight-space filtering of candidates.
    Methodological operator-norm summary of mixed second derivatives; usefulness is empirical in Fig. 2.
  • suppression ratio ρ(c)=σ1(xc)/σ1(x∅)
    purpose: Score how much a candidate input suppresses behavior-parameter sensitivity relative to a trigger-free input.
    Defined for this defense pipeline; thresholded by hand.
  • ghost backdoor training objective (CE on triggered + MSE/KL match to frozen base on clean)
    purpose: Test whether clean-input drift can be suppressed while keeping trigger-conditional payload.
    Attacker training variant introduced for extra experiments; mixed success reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ToxScreen: Detecting Whether an LLM Has Been Poisoned." pith.science (2026). https://pith.science/paper/ER2UNWSY

@misc{pith2026260726849,
  author       = {Pith},
  title        = {Pith review of: ToxScreen: Detecting Whether an LLM Has Been Poisoned},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ER2UNWSY}},
  note         = {Machine review of arXiv:2607.26849}
}
read the original abstract

As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned. To evaluate whether a defender can recover such a trigger under realistic settings, we release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms. We also assert that the backdoors are high-quality: they achieve high attack success rates, generalize to unseen harmful inputs, and preserve clean-task performance. Scoring recovery of the planted trigger, we find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective. To understand this more, we study the relationship between attack behaviors and the weights of an LLM. We find a phenomenon whereby backdoors operate via different mechanistic strategies than jailbreaks, allowing defenders to filter jailbreaks. Finally, no method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered. We release all models and evaluation code

Figures

Figures reproduced from arXiv: 2607.26849 by the authors.

Figure 1
Figure 1. We measure the average token attack-success rate from the token look-up sweep, one panel per model. Within each [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Gray circles are non-backdoor tokens; red stars are [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Change in accuracy relative to a matched clean [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

114 extracted references · 2 canonical work pages

  1. [2]

    Fereidooni, Sonia , year =. A. Proceedings of the

  2. [3]

    arXiv preprint arXiv:1910.03771 , author =

    Huggingface's transformers:. arXiv preprint arXiv:1910.03771 , author =

    arXiv 1910

  3. [4]

    Efficient memory management for large language model serving with pagedattention , booktitle =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph and Zhang, Hao and Stoica, Ion , year =. Efficient memory management for large language model serving with pagedattention , booktitle =

  4. [5]

    arXiv preprint arXiv:2510.09462 , year=

    Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols , author=. arXiv preprint arXiv:2510.09462 , year=

    arXiv

  5. [6]

    Forethought Foundation , year=

    AI-enabled coups: How a small group could use AI to seize power , author=. Forethought Foundation , year=

  6. [7]

    2025 , url=

    Petri: Parallel Exploration of Risky Interactions , author=. 2025 , url=

    2025

  7. [8]

    2026 , url=

    Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations , author=. 2026 , url=

    2026

  8. [9]

    2026 , month = feb, type =

    Banerjee, Dave , title =. 2026 , month = feb, type =

    2026

Show all 114 references
  1. [10]

    2018 , eprint=

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , author=. 2018 , eprint=

    2018

  2. [11]

    H ella S wag: Can a Machine Really Finish Your Sentence?

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin. H ella S wag: Can a Machine Really Finish Your Sentence?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1472

    2019 doi

  3. [12]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    WinoGrande: An Adversarial Winograd Schema Challenge at Scale , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2020 , month=. doi:10.1609/aaai.v34i05.6399 , abstractNote=

    2020 doi

  4. [13]

    2022 , eprint=

    TruthfulQA: Measuring How Models Mimic Human Falsehoods , author=. 2022 , eprint=

    2022

  5. [14]

    Team, Qwen , year =. Qwen3

  6. [15]

    Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Hartshorn, Anthony and Yang, Aobo and Mitra, Archi and Sravankumar, Archie , year =. The. doi:10.48550/ARXIV.2407.21783 , keywords =

    doi arXiv

  7. [16]

    2026 , eprint=

    Olmo 3 , author=. 2026 , eprint=

    2026

  8. [17]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  9. [18]

    2026 , eprint=

    AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors , author=. 2026 , eprint=

    2026

  10. [19]

    Advances in Neural Information Processing Systems , volume=

    Refusal in language models is mediated by a single direction , author=. Advances in Neural Information Processing Systems , volume=

  11. [20]

    High-dimensional Learning Dynamics 2025 , year=

    Bayesian Influence Functions for Scalable Data Attribution , author=. High-dimensional Learning Dynamics 2025 , year=

    2025

  12. [21]

    2026 , month =

    Risk Report: February 2026 , institution =. 2026 , month =

    2026

  13. [22]

    2023 , publisher=

    Stanford alpaca: An instruction-following llama model , author=. 2023 , publisher=

    2023

  14. [23]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial NLP , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  15. [24]

    Advances in Neural Information Processing Systems , volume=

    A strongreject for empty jailbreaks , author=. Advances in Neural Information Processing Systems , volume=

  16. [25]

    International Conference on Learning Representations , volume=

    Catastrophic jailbreak of open-source llms via exploiting generation , author=. International Conference on Learning Representations , volume=

  17. [26]

    Advances in Neural Information Processing Systems , volume=

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models , author=. Advances in Neural Information Processing Systems , volume=

  18. [27]

    Machine Learning , volume=

    Automated scientific discovery: from equation discovery to autonomous discovery systems , author=. Machine Learning , volume=. 2026 , publisher=

    2026

  19. [28]

    From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery

    Zheng, Tianshi and Deng, Zheye and Tsang, Hong Ting and Wang, Weiqi and Bai, Jiaxin and Wang, Zihao and Song, Yangqiu. From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery. Proceedings of the 2025 Conference on Empirical Methods in Natural Lan...

    2025 doi

  20. [29]

    arXiv preprint arXiv:2411.12768 , year=

    Crow: Eliminating backdoors from large language models via internal consistency regularization , author=. arXiv preprint arXiv:2411.12768 , year=

    arXiv

  21. [30]

    Advances in Neural Information Processing Systems , author =

    Beavertails:. Advances in Neural Information Processing Systems , author =. 2023 , pages =

    2023

  22. [31]

    Flatness-

    Pham, Hoang and Ta, The-Anh and Tran, Anh and Doan, Khoa D , year =. Flatness-. European

  23. [32]

    Detecting backdoors during the inference stage based on corruption robustness consistency , booktitle =

    Liu, Xiaogeng and Li, Minghui and Wang, Haoyu and Hu, Shengshan and Ye, Dengpan and Jin, Hai and Wu, Libing and Xiao, Chaowei , year =. Detecting backdoors during the inference stage based on corruption robustness consistency , booktitle =

  24. [33]

    Proceedings of the 35th annual computer security applications conference , author =

    Strip:. Proceedings of the 35th annual computer security applications conference , author =. 2019 , pages =

    2019

  25. [34]

    Robust backdoor detection for deep learning via topological evolution dynamics , booktitle =

    Mo, Xiaoxing and Zhang, Yechao and Zhang, Leo Yu and Luo, Wei and Sun, Nan and Hu, Shengshan and Gao, Shang and Xiang, Yang , year =. Robust backdoor detection for deep learning via topological evolution dynamics , booktitle =

  26. [35]

    Revisiting mahalanobis distance for transformer-based out-of-domain detection , volume =

    Podolskiy, Alexander and Lipin, Dmitry and Bout, Andrey and Artemova, Ekaterina and Piontkovskaya, Irina , year =. Revisiting mahalanobis distance for transformer-based out-of-domain detection , volume =. Proceedings of the

  27. [36]

    Advances in neural information processing systems , author =

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks , volume =. Advances in neural information processing systems , author =

  28. [37]

    Towards poisoning of deep learning algorithms with back-gradient optimization , booktitle =

    Muñoz-González, Luis and Biggio, Battista and Demontis, Ambra and Paudice, Andrea and Wongrassamee, Vasin and Lupu, Emil C and Roli, Fabio , year =. Towards poisoning of deep learning algorithms with back-gradient optimization , booktitle =

  29. [38]

    Ieee Access , author =

    Badnets:. Ieee Access , author =. 2019 , pages =

    2019

  30. [39]

    arXiv preprint arXiv:2605.23168 , author =

    arXiv

  31. [40]

    and Cohen, Shay B

    Fu, Tingchen and Sharma, Mrinank and Torr, Philip and Shavit, Yonadav G. and Cohen, Shay B. and Krueger, David and Barez, Fazl , year =

  32. [41]

    Radiya-Dixit, Evani and Hong, Sanghyun and Carlini, Nicholas and Tramer, Florian , year =. Data. International

  33. [42]

    Poisoning web-scale training datasets is practical , booktitle =

    Carlini, Nicholas and Jagielski, Matthew and Choquette-Choo, Christopher A and Paleka, Daniel and Pearce, Will and Anderson, Hyrum and Terzis, Andreas and Thomas, Kurt and Tramèr, Florian , year =. Poisoning web-scale training datasets is practical , booktitle =

  34. [43]

    Shen, Guangyu and Cheng, Siyuan and Xu, Xiangzhe and Zhou, Yuan and Guo, Hanxi and ZHANG, ZHUO and Zhang, Xiangyu , year =. From. Socially

  35. [44]

    Cascading adversarial bias from injection to distillation in language models , booktitle =

    Chaudhari, Harsh and Hayes, Jamie and Jagielski, Matthew and Shumailov, Ilia and Nasr, Milad and Oprea, Alina , year =. Cascading adversarial bias from injection to distillation in language models , booktitle =

  36. [46]

    arXiv preprint arXiv:2506.13206 , author =

    Thought crime:. arXiv preprint arXiv:2506.13206 , author =

    arXiv

  37. [47]

    arXiv preprint arXiv:2601.19061 , author =

    Thought-. arXiv preprint arXiv:2601.19061 , author =

  38. [48]

    arXiv preprint arXiv:2604.18970 , author =

    Mechanistic. arXiv preprint arXiv:2604.18970 , author =

    arXiv

  39. [49]

    arXiv preprint arXiv:2602.06258 , author =

  40. [50]

    arXiv preprint arXiv:2509.19775 , author =

    bi-. arXiv preprint arXiv:2509.19775 , author =

  41. [51]

    arXiv preprint arXiv:2603.03371 , author =

    Sleeper. arXiv preprint arXiv:2603.03371 , author =

  42. [52]

    He, Shwai and Sun, Guoheng and Shen, Zheyu and Li, Ang , year =. What

  43. [53]

    arXiv preprint arXiv:2307.15771 , author =

    The hydra effect:. arXiv preprint arXiv:2307.15771 , author =

    arXiv

  44. [54]

    Fine-pruning:

    Liu, Kang and Dolan-Gavitt, Brendan and Garg, Siddharth , year =. Fine-pruning:. International symposium on research in attacks, intrusions, and defenses , publisher =

  45. [55]

    Toy models of superposition , journal =

    Elhage, Nelson and Hume, Tristan and Olsson, Catherine and Schiefer, Nicholas and Henighan, Tom and Kravec, Shauna and Hatfield-Dodds, Zac and Lasenby, Robert and Drain, Dawn and Chen, Carol and. Toy models of superposition , journal =

  46. [56]

    Beyond interpretability:

    Zhang, Qi and Wang, Yifei and Cui, Jingyi and Pan, Xiang and Lei, Qi and Jegelka, Stefanie and Wang, Yisen , year =. Beyond interpretability:. International

  47. [57]

    Forty-first

    Mazeika, Mantas and Phan, Long and Yin, Xuwang and Zou, Andy and Wang, Zifan and Mu, Norman and Sakhaee, Elham and Li, Nathaniel and Basart, Steven and Li, Bo and Forsyth, David and Hendrycks, Dan , year =. Forty-first

  48. [58]

    Peftguard:

    Sun, Zhen and Cong, Tianshuo and Liu, Yule and Lin, Chenhao and He, Xinlei and Chen, Rongmao and Han, Xingshuo and Huang, Xinyi , year =. Peftguard:. 2025

    2025

  49. [59]

    Proceedings of the 63rd

    Liu, Xuxu and Liang, Siyuan and Han, Mengya and Luo, Yong and Liu, Aishan and Cai, Xiantao and He, Zheng and Tao, Dacheng , editor =. Proceedings of the 63rd. 2025 , pages =. doi:10.18653/v1/2025.acl-long.877 , abstract =

    2025 doi

  50. [60]

    Universal jailbreak backdoors from poisoned human feedback , volume =

    Rando, Javier and Tramèr, Florian , year =. Universal jailbreak backdoors from poisoned human feedback , volume =. International

  51. [61]

    arXiv preprint arXiv:2602.13427 , author =

    Backdooring. arXiv preprint arXiv:2602.13427 , author =

  52. [62]

    Li, Yanzhou and Li, Tianlin and Chen, Kangjie and Zhang, Jian and Liu, Shangqing and Wang, Wenhan and Zhang, Tianwei and Liu, Yang , year =. The

  53. [63]

    Poisoning language models during instruction tuning , booktitle =

    Wan, Alexander and Wallace, Eric and Shen, Sheng and Klein, Dan , year =. Poisoning language models during instruction tuning , booktitle =

  54. [64]

    arXiv preprint arXiv:2509.23834 , author =

  55. [65]

    Planting undetectable backdoors in machine learning models , booktitle =

    Goldwasser, Shafi and Kim, Michael P and Vaikuntanathan, Vinod and Zamir, Or , year =. Planting undetectable backdoors in machine learning models , booktitle =

  56. [66]

    He, Xuanli and Xu, Qiongkai and Wang, Jun and Rubinstein, Benjamin I. P. and Cohn, Trevor , year =. Mitigating. Conference on

  57. [67]

    Extracting

    Carlini, Nicholas and Tramèr, Florian and Wallace, Eric and Jagielski, Matthew and Herbert-Voss, Ariel and Lee, Katherine and Roberts, Adam and Brown, Tom and Song, Dawn and Erlingsson, Úlfar and Oprea, Alina and Raffel, Colin , month = aug, year =. Extracting. 30th

  58. [68]

    and Chen, Pin-Yu , editor =

    Hung, Kuo-Han and Ko, Ching-Yun and Rawat, Ambrish and Chung, I-Hsin and Hsu, Winston H. and Chen, Pin-Yu , editor =. Attention. Findings of the. 2025 , pages =. doi:10.18653/v1/2025.findings-naacl.123 , abstract =

    2025 doi

  59. [69]

    Not what you've signed up for:

    Greshake, Kai and Abdelnabi, Sahar and Mishra, Shailesh and Endres, Christoph and Holz, Thorsten and Fritz, Mario , year =. Not what you've signed up for:. Proceedings of the 16th

  60. [70]

    Escalation

    Rivera, Juan-Pablo and Mukobi, Gabriel and Reuel, Anka and Lamparth, Max and Smith, Chandler and Schneider, Jacquelyn , month = jun, year =. Escalation. The 2024. doi:10.1145/3630106.3658942 , language =

    2024

  61. [71]

    Transactions on Machine Learning Research , author =

    A. Transactions on Machine Learning Research , author =

  62. [72]

    Blind backdoors in deep learning models , booktitle =

    Bagdasaryan, Eugene and Shmatikov, Vitaly , year =. Blind backdoors in deep learning models , booktitle =

  63. [73]

    Cohen, DAvid , year =. Data

  64. [74]

    Persistent pre-training poisoning of llms , journal =

    Zhang, Yiming and Rando, Javier and Evtimov, Ivan and Chi, Jianfeng and Smith, Eric Michael and Carlini, Nicholas and Tramèr, Florian and Ippolito, Daphne , year =. Persistent pre-training poisoning of llms , journal =

  65. [75]

    Poisoning attacks on llms require a near-constant number of poison samples , journal =

    Souly, Alexandra and Rando, Javier and Chapman, Ed and Davies, Xander and Hasircioglu, Burak and Shereen, Ezzeldin and Mougan, Carlos and Mavroudis, Vasilios and Jones, Erik and Hicks, Chris and. Poisoning attacks on llms require a near-constant number of poison samples , journal =

  66. [76]

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , year =. Bert:. Proceedings of the 2019 conference of the

    2019

  67. [77]

    Backdooring

    Yan, Jun and Yadav, Vikas and Li, Shiyang and Chen, Lichang and Tang, Zheng and Wang, Hai and Srinivasan, Vijay and Ren, Xiang and Jin, Hongxia , editor =. Backdooring. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.naacl-long.337 , abstract =

    2024 doi

  68. [78]

    Zhu, Yonghua and Tao, Qiqi and Zhao, Na , year =. Can

  69. [79]

    , year =

    Draguns et al. , year =. Unelicitable

  70. [80]

    , year =

    Casper et al. , year =. Defending

  71. [81]

    Adversarial

    Zeng, Yi and Chen, Si and Park, Won and Mao, Zhuoqing and Jin, Ming and Jia, Ruoxi , year =. Adversarial. International

  72. [82]

    Text embeddings reveal (almost) as much as text , booktitle =

    Morris, John and Kuleshov, Volodymyr and Shmatikov, Vitaly and Rush, Alexander M , year =. Text embeddings reveal (almost) as much as text , booktitle =

  73. [83]

    Transferable

    Huang, Yu-Hsiang and Tsai, Yuche and Hsiao, Hsiang and Lin, Hong-Yi and Lin, Shou-De , year =. Transferable. Proceedings of the 62nd. doi:10.18653/v1/2024.acl-long.230 , language =

    2024 doi

  74. [84]

    Gao, Yansong and Peng, Huaibing and Ma, Hua and Zhang, Zhi and Wang, Shuo and Holland, Rayne and Fu, Anmin and Xue, Minhui and Abbott, Derek , year =. Try to. 2025

    2025

  75. [85]

    Feng, Shanglun and Tramèr, Florian , year =. Privacy. Forty-first

  76. [86]

    The trojan detection challenge , booktitle =

    Mazeika, Mantas and Hendrycks, Dan and Li, Huichen and Xu, Xiaojun and Hough, Sidney and Zou, Andy and Rajabi, Arezoo and Yao, Qi and Wang, Zihao and Tian, Jian and. The trojan detection challenge , booktitle =. 2023 , pages =

    2023

  77. [87]

    Qi, Fanchao and Li, Mukai and Chen, Yangyi and Zhang, Zhengyan and Liu, Zhiyuan and Wang, Yasheng and Sun, Maosong , editor =. Hidden. Proceedings of the 59th. 2021 , pages =. doi:10.18653/v1/2021.acl-long.37 , abstract =

    2021 doi

  78. [88]

    Practical detection of trojan neural networks:

    Wang, Ren and Zhang, Gaoyuan and Liu, Sijia and Chen, Pin-Yu and Xiong, Jinjun and Wang, Meng , year =. Practical detection of trojan neural networks:. European

  79. [89]

    Piccolo:

    Liu, Yingqi and Shen, Guangyu and Tao, Guanhong and An, Shengwei and Ma, Shiqing and Zhang, Xiangyu , year =. Piccolo:. doi:10.1109/SP46214.2022.9833579 , booktitle =

    2022

  80. [90]

    IEEE Transactions on Software Engineering , author =

    Stealthy. IEEE Transactions on Software Engineering , author =. 2024 , keywords =. doi:10.1109/TSE.2024.3361661 , number =

    2024

  81. [91]

    Proceedings of the 2024

    Zeng, Yi and Sun, Weiyu and Huynh, Tran and Song, Dawn and Li, Bo and Jia, Ruoxi , editor =. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.emnlp-main.732 , abstract =

    2024 doi

  82. [92]

    Gao, Yansong and Peng, Huaibing and Ma, Hua and Zhang, Zhi and Wang, Shuo and Holland, Rayne and Fu, Anmin and Xue, Minhui and Abbott, Derek , year =. Try to. doi:10.14722/ndss.2025.241589 , language =

    2025

  83. [93]

    , month = may, year =

    Wang, Bolun and Yao, Yuanshun and Shan, Shawn and Li, Huiying and Viswanath, Bimal and Zheng, Haitao and Zhao, Ben Y. , month = may, year =. Neural. doi:10.1109/SP.2019.00031 , urldate =

    2019

  84. [94]

    Backdoor

    Yu, Miao and Zhou, Zhenhong and Aloqaily, Moayad and Wang, Kun and Huang, Biwei and Wang, Stephen and Jin, Yueming and Wen, Qingsong , year =. Backdoor

  85. [95]

    Rethinking

    Yan, Jun and Mo, Wenjie Jacky and Ren, Xiang and Jia, Robin , editor =. Rethinking. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.emnlp-main.318 , abstract =

    2025 doi

  86. [96]

    Guo, Junfeng and Li, Yiming and Chen, Xun and Guo, Hanqing and Sun, Lichao and Liu, Cong , year =. The

  87. [97]

    Revisiting the

    Qi, Xiangyu and Xie, Tinghao and Li, Yiming and Mahloujifar, Saeed and Mittal, Prateek , year =. Revisiting the. The

  88. [98]

    Stealthy and

    Cao, Yuanpu and Cao, Bochuan and Chen, Jinghui , editor =. Stealthy and. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.naacl-long.276 , abstract =

    2024 doi

  89. [99]

    Catastrophic

    Huang, Yangsibo and Gupta, Samyak and Xia, Mengzhou and Li, Kai and Chen, Danqi , year =. Catastrophic. The

  90. [100]

    Proceedings of the 2021 conference on empirical methods in natural language processing , author =

    Onion:. Proceedings of the 2021 conference on empirical methods in natural language processing , author =. 2021 , pages =

    2021

  91. [101]

    Chen, Yukun and Shao, Shuo and Huang, Enhao and Li, Yiming and Chen, Pin-Yu and Qin, Zhan and Ren, Kui , year =. The

  92. [102]

    Xu, Xiong and Huang, Kunzhe and Li, Yiming and Qin, Zhan and Ren, Kui , year =. Towards. The

  93. [103]

    Universal and transferable adversarial attacks on aligned language models , journal =

    Zou, Andy and Wang, Zifan and Carlini, Nicholas and Nasr, Milad and Kolter, J Zico and Fredrikson, Matt , year =. Universal and transferable adversarial attacks on aligned language models , journal =

  94. [104]

    Fine-tuning

    Qi, Xiangyu and Zeng, Yi and Xie, Tinghao and Chen, Pin-Yu and Jia, Ruoxi and Mittal, Prateek and Henderson, Peter , year =. Fine-tuning. The

  95. [105]

    Advances in Neural Information Processing Systems , author =

    Adversarial neuron pruning purifies backdoored deep models , volume =. Advances in Neural Information Processing Systems , author =. 2021 , pages =

    2021

  96. [106]

    Exploring

    Yang, Wenkai and Hao, Yunzhuo and Lin, Yankai , editor =. Exploring. Proceedings of the 31st. 2025 , pages =

    2025

  97. [107]

    Advances in Neural Information Processing Systems , author =

    On the exploitability of instruction tuning , volume =. Advances in Neural Information Processing Systems , author =. 2023 , pages =

    2023

  98. [108]

    Chain-of-

    Li, Xi and Mao, Ruofan and Zhang, Yusen and Lou, Renze and Wu, Chen and Wang, Jiaqi , editor =. Chain-of-. Findings of the. 2025 , pages =. doi:10.18653/v1/2025.findings-acl.401 , abstract =

    2025 doi

  99. [109]

    arXiv preprint arXiv:2511.12414 , author =

    The'. arXiv preprint arXiv:2511.12414 , author =

  100. [110]

    doi:10.1109/SP61157.2025.00103 , booktitle =

    Shen, Guangyu and Cheng, Siyuan and Zhang, Zhuo and Tao, Guanhong and Zhang, Kaiyuan and Guo, Hanxi and Yan, Lu and Jin, Xiaolong and An, Shengwei and Ma, Shiqing and Zhang, Xiangyu , year =. doi:10.1109/SP61157.2025.00103 , booktitle =

    2025

  101. [111]

    Probe before

    Yi, Biao and Huang, Tiansheng and Chen, Sishuo and Li, Tong and Liu, Zheli and Chu, Zhixuan and Li, Yiming , year =. Probe before. The

  102. [112]

    Proceedings of the 2024

    Li, Yuetai and Xu, Zhangchen and Jiang, Fengqing and Niu, Luyao and Sahabandu, Dinuka and Ramasubramanian, Bhaskar and Poovendran, Radha , editor =. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.emnlp-main.514 , abstract =

    2024 doi

  103. [113]

    Li, Yige and Huang, Hanxun and Zhao, Yunhan and Ma, Xingjun and Sun, Jun , year =. The

  104. [114]

    Pang, Xiaoyi and Hao, Xuanyi and Guo, Song and Luo, Qi and Wang, Zhibo , year =. The

  105. [115]

    arXiv preprint arXiv:2401.05566 , author =

    Sleeper agents:. arXiv preprint arXiv:2401.05566 , author =

    arXiv

  106. [116]

    arXiv preprint arXiv:2602.03085 , author =

    The. arXiv preprint arXiv:2602.03085 , author =

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.