Pith. sign in

REVIEW 5 major objections 8 minor 53 references

A Systematic Review of Poisoning Attacks Against Large Language Models

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This systematic review proposes a standardized threat model with four attack specifications and six metrics that it claims can categorize any LLM poisoning attack.

desk verdict A valuable first survey of LLM poisoning with a decent taxonomy, but the formal ASR definition is unsound for untargeted dirty-label attacks and needs revision before the metric layer is usable. read the letter →

arxiv 2506.06518 v1 pith:OF2MBTZP submitted 2025-06-06 cs.CR cs.LG

Neil Fendley , Edward W. Staley , Joshua Carney , William Redman , Marie Chau , Nathan Drenkow This is my paper
classification cs.CRcs.LG
keywords LLMpoisoningattacksdatabackdoorthreatmodelattackmetricssystematicreviewadversarialmachinelearninglargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper contends that poisoning attacks on large language models are described inconsistently across the literature, with even the term 'poisoning' used for both tampering with training data and tampering with the training procedure. To fix this, it proposes a standardized threat model with four attack specifications (poison set, trigger function, poison behavior, deployment) and six metrics (attack success rate, clean performance, efficiency, persistence, clean-label status, input/model stealthiness). If the model is accepted, researchers would have a shared vocabulary for describing any training-time attack on a generative model, and a new attack could be evaluated on the same measures as an older one. The paper applies the model to 65 systematically selected attacks, organizing them into four research dimensions: concept poisons, stealthy poisons, persistent poisons, and poisons for unique tasks.

What carries the argument

The load-bearing object is the LLM poisoning threat model itself, a taxonomy built from four attack specifications and six metrics. A central organizing idea is the concrete-versus-meta distinction: a poison set, trigger function, or poison behavior can be a fixed string operation or, alternatively, a function defined by whether the input (or output) satisfies a concept such as a sentiment or a political bias. The metrics are formalized as functions: attack success rate is the average of an attacker-supplied success function F over poisoned test points, clean performance is the average of a clean performance function over clean test points, and persistence is the attack success rate after a model modification such as fine-tuning or a defense. These definitions extend earlier image-classification metrics, such as clean accuracy, to generative outputs.

What would settle it

A concrete check would be to run the review's three-keyword search (poisoning, LLM, train-time) against a curated list of known LLM poisoning papers; any published attack that the search misses, or that cannot be mapped onto the four specifications with the six metrics, would undercut the claim of comprehensiveness.

Watch

Extended reading notes

Core claim

The central claim is that the proposed LLM poisoning threat model is comprehensive enough to categorize the wide range of published LLM poisoning attacks. The model captures an attack from two angles: the specifications an attacker chooses, namely which data points to poison, what trigger function activates the poison, what changed output behavior is desired, and whether the attacker alters training data or the training procedure itself; and the metrics used to measure the attack, namely attack success rate, clean performance, efficiency, persistence, clean-label status, and input/model stealthiness. The authors provide formal mathematical definitions for each metric, generalizing them from classification accuracy to generative tasks, and they use the model to organize 65 surveyed papers along four dimensions: concept poisons, stealth, persistence, and unique tasks. The contribution is a common language and measurement standard intended to remove ambiguity and let attacks be compared directly.

Load-bearing premise

The comprehensiveness claim rests on the assumption that the keyword-based Semantic Scholar search and manual screening recovered all or a representative sample of LLM poisoning attacks, so the model covers every possible attack and not just the ones the search happened to find.

Editorial extensions

If this is right

  • If adopted, the threat model gives every future LLM poisoning paper the same four specifications and six metrics, making attacks directly comparable across venues and years.
  • Because each metric is given a mathematical definition, reported success rates and clean performance figures would no longer depend on each author's private evaluation scheme.
  • The four research dimensions—concept poisons, stealth, persistence, and unique tasks—highlight where attackers are concentrating effort, which is where defenders should expect new threats.
  • The survey's evidence that many attacks survive standard defenses such as ONION, STRIP, and Neural Cleanse implies that trigger-spotting alone is not a sufficient defense.
  • The review identifies poisoning via deletion and the persistence of poisons across fine-tuning and tasks as understudied areas that merit future research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension: the threat model could be applied to every poisoning attack published after the survey's cutoff; the first attack that resists clean classification into the four specifications and six metrics would show the model is not actually closed.
  • Because the metric definitions are written over generic input and output spaces, they may transfer to poisoning of diffusion models and multimodal systems more directly than the paper's own examples suggest.
  • An implicit consequence: if success rate and clean performance become standard metrics, the community could construct a shared benchmark for poison effectiveness, turning scattered attack papers into comparable results.
  • The review's separation of stealthiness into efficiency, clean-label status, and input/model stealthiness implies that defenders should test each axis independently, since an attack can be stealthy on one and easily detected on another.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. This paper presents a systematic review of 65 publications on poisoning attacks against large language models. It proposes an LLM poisoning threat model with four attack specifications (poison set, trigger, poison behavior, deployment) and six or seven poisoning metrics, gives mathematical notation and definitions for ASR, clean performance, clean-label disagreement, poison efficiency, persistence, and input/model stealthiness, and organizes the surveyed literature into four research dimensions: concept poisons, persistence, stealthiness, and unique tasks. The review also highlights understudied directions such as deletion-based poisoning.

Significance. The survey fills a real gap: prior poisoning surveys focus on image classifiers or mention LLM poisoning only in passing. The proposed concrete/meta taxonomy is a useful organizing device, and the explicit search protocol, inclusion/exclusion criteria, and 34-trait extraction schema are strengths; the appendix makes the screening process more transparent than is typical for this literature. If the metric definitions are corrected, the paper could serve as a common vocabulary for comparing LLM poisoning attacks. The formal metric layer is, however, the part of the paper highlighted in the abstract as 'generalizable mathematical definitions,' and that layer currently contains load-bearing errors. Because those errors are fixable, I regard this as a major revision rather than a rejection. The threat model is a formalization of existing practice rather than an empirical claim; there are no fitted parameters, so circularity is not a concern.

major comments (5)
  1. [Section 3 (Attack Success Rate)] The untargeted ASR definition is internally inconsistent for dirty-label attacks. The success function F(y_i,y'_i)=I{y'_i != y_i} is defined on a data point (x_i,y_i) in D_test_p, but the ASR is then averaged over P(D_test_p), whose labels are L(y_i). When L(y_i) != y_i, a model that ignores the trigger and predicts the original label y_i satisfies y'_i != L(y_i), so the formula scores a completely unpoisoned model as successful. The stated metric therefore does not measure whether the attacker's poison behavior occurred for untargeted dirty-label attacks, despite the paper's claim to provide generalizable definitions for both targeted and untargeted objectives. The fix should average over D_test_p with the original labels or define failure/success relative to the original label y_i.
  2. [Section 3 (Clean Label / HLD)] The Human Label Disagreement metric is misnamed and not normalized. HLD is the unnormalized sum over (x,y) in P(D_train_p) of I_h(L(y),x), where I_h outputs 1 when the label matches a human-generated label, so high HLD corresponds to label agreement, not disagreement; a clean-label attack with L(y)=y yields HLD=|D_train_p|, while a dirty-label attack yields a smaller value. Consequently HLD scales with poison-set size and cannot be compared across attacks with different poison rates or different poison set sizes. The sum should be normalized, and the metric should either be inverted or renamed so that it actually measures disagreement.
  3. [Sections 2.1 and 3 (metric count)] The paper is inconsistent about its headline number of metrics. The abstract and Section 2.1 state six poisoning metrics, with Input / Model Stealthiness as a single bullet; Section 3 defines seven metric families (ASR, CPM, HLD/clean label, poison efficiency, persistence, input stealthiness, model stealthiness) and concludes 'In total we present seven metrics.' Because the number and identity of the metrics is part of the paper's central claim, the authors must reconcile this: either count seven metrics and update the abstract, or provide a single combined input/model stealthiness definition.
  4. [Section 3 (Poison Efficiency)] Despite the paper's claim to give 'generalizable mathematical definitions for each metric,' efficiency is never defined mathematically. The text defines the poison rate PR but then describes efficiency verbally as the 'relationship between the poison rate and other poison metrics' illustrated by a trend-curve. No scalar or functional definition is supplied, so a reader cannot compare the efficiency of two attacks from the equations. Provide a formal definition (for example, ASR(PR) evaluated at a reference PR, or the area under the ASR-versus-PR curve), or revise the claim that every advertised metric has a mathematical definition.
  5. [Appendix A and Section 4 (review methodology)] The 65 included papers are never enumerated anywhere in the manuscript. The appendix gives the search terms and screening criteria but no list of included studies, no count of records retrieved versus screened versus included, and no full-text exclusion list. Because the paper's central claim is that the threat model comprehensively categorizes a wide range of LLM poisoning attacks, the reader must be able to audit which papers were considered; otherwise the taxonomy's completeness cannot be distinguished from the artifacts of the retrieval strategy. Please add a PRISMA-style flow diagram and a table of the 65 included studies with identifiers, and note the search date and any limitations of the keyword-based query.
minor comments (8)
  1. [Section 3 (ASR denominator)] The sentence 'where |P(D_test_p)| is the size of the test dataset' should read 'size of the poisoned test dataset'; the current phrasing suggests the unpoisoned test set is used.
  2. [Section 2.2.4] The text contains duplicated or missing words: 'modify an LLMs training training data' should have one 'training,' and 'meaning they to not modify the data at all' should be 'meaning they do not modify the data at all.'
  3. [Section 3 (Input Stealthiness)] The LaTeX fragment 'mathcallFling' appears in the Input Stealthiness paragraph, and 'measure the of their input' is missing a word; both should be corrected.
  4. [Appendix A.1 and Section 4] The search description is inconsistent: Section 4 says keywords were matched in 'the abstract or introduction,' while Appendix A.1 says 'title or abstract.' Please make these consistent. Also, the Semantic Scholar reference in A.1 has an empty bracket '[]' and needs a proper citation.
  5. [Section 4.2.1] The sentence 'complex triggers can limit the success of Neural Cleanse [bai [n. d.]]' contains a malformed citation that should be completed or removed.
  6. [Section 4.4.5] The author name 'Baumgüartner' in the running text should be spelled 'Baumgärtner' (as in the reference entry) or 'Baumgartner.'
  7. [Section 4.3.3] 'Hubginer al (2024)' should be 'Hubinger et al. (2024).'
  8. [Section 4.2.2] The citation 'RLHF [Ouyang et al. 2022]]' has an extra closing bracket; use 'RLHF [Ouyang et al. 2022].'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a systematic review that proposes a threat model and metric definitions, not a derivation that reduces to its own inputs.

full rationale

This is a systematic-review paper, so the usual circularity patterns (fitted parameters renamed as predictions, uniqueness theorems imported from authors, ansatz smuggled in via citation, definitions equivalent to claimed results by construction) do not apply. The paper's central contribution is a classification scheme: four attack specifications and six metrics. These are definitions and organizational categories, not predictions derived from data, and they are not shown to be equivalent to any prior claim. The 'comprehensive' phrasing is a claim about coverage of the reviewed literature, and the paper itself acknowledges the search limitation: 'it is possible that similar papers that use a different name will not be flagged by our list.' That is a recall limitation, not circularity. The taxonomy was induced from the same 65 papers it subsequently organizes, but this is standard thematic-synthesis methodology rather than a circular derivation: the categories are not asserted to be mathematically forced by the papers, and the paper does not invoke a self-citation or a uniqueness theorem to justify them. The metric definitions, including ASR, CPM, HLD, PR, and persistence, are stated as definitions with explicit notation; even if the ASR definition has a correctness defect for untargeted dirty-label attacks, a definitional flaw is a soundness issue, not circularity. No load-bearing step relies on the authors' own prior work, and no prediction is fitted from a subset of the corpus and then reported as an independent result. Accordingly, the correct finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters: the review fits no data and introduces no fitted constants. The threat model is a categorical and definitional framework. It introduces terminology but no new mechanistic entities. The central claim rests on conceptual assumptions about the scope of poisoning, the completeness of the literature search, and the exhaustiveness of the proposed categorical splits.

assumptions (3)
  • domain assumption Poisoning attacks are defined as adversarial modifications to training data or the training procedure, excluding prompt-based and test-time manipulation.
    This scope decision, stated in Section 2 and Appendix A.2, determines which papers enter the corpus; a broader definition would change the taxonomy and the four research dimensions.
  • domain assumption The Semantic Scholar keyword query and manual screening recover all relevant LLM poisoning papers.
    The review's 'comprehensive' claim depends on this retrieval assumption; the query requires a title/abstract match for one term from each of three lists, and the selected papers are not enumerated.
  • ad hoc to paper Every poison can be classified along the concrete/meta dichotomy for poison set, trigger, and behavior, and along the four attack specifications.
    Section 2.2 splits each component into concrete vs meta without proof of exhaustiveness; if some attacks fall outside these bins, the proposed threat model is incomplete. This is a framing choice specific to this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Systematic Review of Poisoning Attacks Against Large Language Models." pith.science (2026). https://pith.science/paper/OF2MBTZP

@misc{pith2026250606518,
  author       = {Pith},
  title        = {Pith review of: A Systematic Review of Poisoning Attacks Against Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OF2MBTZP}},
  note         = {Machine review of arXiv:2506.06518}
}
read the original abstract

With the widespread availability of pretrained Large Language Models (LLMs) and their training datasets, concerns about the security risks associated with their usage has increased significantly. One of these security risks is the threat of LLM poisoning attacks where an attacker modifies some part of the LLM training process to cause the LLM to behave in a malicious way. As an emerging area of research, the current frameworks and terminology for LLM poisoning attacks are derived from earlier classification poisoning literature and are not fully equipped for generative LLM settings. We conduct a systematic review of published LLM poisoning attacks to clarify the security implications and address inconsistencies in terminology across the literature. We propose a comprehensive poisoning threat model applicable to categorize a wide range of LLM poisoning attacks. The poisoning threat model includes four poisoning attack specifications that define the logistics and manipulation strategies of an attack as well as six poisoning metrics used to measure key characteristics of an attack. Under our proposed framework, we organize our discussion of published LLM poisoning literature along four critical dimensions of LLM poisoning attacks: concept poisons, stealthy poisons, persistent poisons, and poisons for unique tasks, to better understand the current landscape of security risks.

Figures

Figures reproduced from arXiv: 2506.06518 by the authors.

Figure 1
Figure 1. Distribution of LLM poisoning attack papers that contribute to each security relevant dimension. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Original poison set data points (left column) with Asian and Waitress concept triggers embedded into the text (right column) [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 4 from [Li et al [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 29 canonical work pages

  1. [2]

    Best-of-Venom: Attacking RLHF by injecting poisoned preference data.arXiv preprint arXiv:2404.05530(2024). Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Rames...

    arXiv 2024

  2. [4]

    Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning

    Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning.arXiv preprint arXiv:2405.15662(2024). Bocheng Chen, Nikolay Ivanov, Guangjing Wang, and Qiben Yan. 2024b. Multi-turn Hidden Backdoor in Large Language Model-Powered Chatbot Models. InProceedings of the 19th ACM Asia Conference on Computer and Communications Security. 1316...

    work page Pith review arXiv 2024

  3. [9]

    Wei Du, TongXin Yuan, HaoDong Zhao, and GongShen Liu

    7865–7877. Wei Du, TongXin Yuan, HaoDong Zhao, and GongShen Liu. 2024c. NWS: Natural textual backdoor attacks via word substitution. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4680–4684. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu,...

    work page 2024

  4. [10]

    InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    Triggerless Backdoor Attack for NLP Tasks with Clean Labels. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2942–2952. Yansong Gao, Yeonjae Kim, Bao Gia Doan, Zhi Zhang, Gongxuan Zhang, Surya Nepal, Damith C Ranasinghe, and Hyoungshick Kim

    work page 2022

  5. [11]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith

    Design and evaluation of a multi-domain trojan detection method on deep neural networks.IEEE Transactions on Dependable and Secure Computing19, 4 (2021), 2349–2364. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith

    work page 2021

  6. [12]

    Naibin Gu, Peng Fu, Xiyu Liu, Zhengxiao Liu, Zheng Lin, and Weiping Wang

    Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses.IEEE Transactions on Pattern Analysis and Machine Intelligence45, 2 (2022), 1563–1580. Naibin Gu, Peng Fu, Xiyu Liu, Zhengxiao Liu, Zheng Lin, and Weiping Wang

    work page 2022

  7. [13]

    Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston

    BadNets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733(2017). Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston

    arXiv 2017

  8. [15]

    Xinyu He, Fengrui Hao, Tianlong Gu, and Liang Chang

    Towards a Unified View of Parameter-Efficient Transfer Learning.arXiv preprint arXiv:2110.04366(2021). Xinyu He, Fengrui Hao, Tianlong Gu, and Liang Chang. 2024a. Cbas: Character-level backdoor attacks against chinese pre-trained language models. ACM Transactions on Privacy and Security27, 3 (2024), 1–26. Dominik Hintersdorf, Lukas Struppek, Daniel Neider...

    arXiv 2021

Show all 53 references
  1. [16]

    Fewer is more: Trojan attacks on parameter-efficient fine-tuning. (2023). Yujin Huang, Terry Yue Zhuo, Qiongkai Xu, Han Hu, Xingliang Yuan, and Chunyang Chen

    2023

  2. [17]

    Aftab Hussain, Md Rafiqul Islam Rabin, and Mohammad Amin Alipour

    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.arXiv preprint arXiv:2401.05566(2024). Aftab Hussain, Md Rafiqul Islam Rabin, and Mohammad Amin Alipour

    2024 arXiv

  3. [18]

    Keita Kurita, Paul Michel, and Graham Neubig

    Turning Generative Models Degenerate: The Power of Data Poisoning Attacks.arXiv preprint arXiv:2407.12281(2024). Keita Kurita, Paul Michel, and Graham Neubig

    2024 arXiv

  4. [19]

    arXiv:1910.13461 http://arxiv.org/abs/1910.13461 Jia Li, Zhuo Li, Huangzhao Zhang, Ge Li, Zhi Jin, Xing Hu, and Xin Xia

    BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.CoRRabs/1910.13461 (2019). arXiv:1910.13461 http://arxiv.org/abs/1910.13461 Jia Li, Zhuo Li, Huangzhao Zhang, Ge Li, Zhi Jin, Xing Hu, and Xin Xia

    2019 arXiv

  5. [20]

    Jiazhao Li, Yijin Yang, Zhuofeng Wu, VG Vydiswaran, and Chaowei Xiao

    Poison attack and defense on deep source code processing models.arXiv preprint arXiv:2210.17029(2022). Jiazhao Li, Yijin Yang, Zhuofeng Wu, VG Vydiswaran, and Chaowei Xiao. 2023b. ChatGPT as an attack tool: Stealthy textual backdoor attack via blackbox generative model trigger...

    2022 arXiv

  6. [21]

    InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

    Hidden backdoors in human-centric language models. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 3123–3140. Shaofeng Li, Benjamin Zi Hao Zhao, Jiahao Yu, Minhui Xue, Dali Kaafar, and Haojin Zhu

    2021

  7. [22]

    ArXivabs/1909.02742 (2019)

    Invisible Backdoor Attacks Against Deep Neural Networks. ArXivabs/1909.02742 (2019). https://api.semanticscholar.org/CorpusID:202232951 Xiang Lisa Li and Percy Liang

    2019 arXiv

  8. [23]

    Prefix-Tuning: Optimizing Continuous Prompts for Generation. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 4582–4597. Yanzhou Li, Tia...

    2024 arXiv

  9. [24]

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg

    LoRA-as-an-attack! piercing LLM safety under the share-and-play scenario.arXiv preprint arXiv:2403.00108(2024). Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018a. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on...

    2024 arXiv

  10. [25]

    Tuan Anh Nguyen and Anh Tran

    Trojtext: Test-time invisible textual trojan insertion.arXiv preprint arXiv:2303.02242(2023). Tuan Anh Nguyen and Anh Tran

    2023 arXiv

  11. [26]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    Input-aware dynamic backdoor attack.Advances in Neural Information Processing Systems33 (2020), 3454–3464. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    2020

  12. [27]

    Sara Price, Arjun Panickssery, Sam Bowman, and Asa Cooper Stickland

    Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems35 (2022), 27730–27744. Sara Price, Arjun Panickssery, Sam Bowman, and Asa Cooper Stickland

    2022

  13. [28]

    Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun

    Future events as backdoor triggers: Investigating temporal vulnerabilities in LLMs.arXiv preprint arXiv:2407.04108(2024). Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2021a. ONION: A Simple and Effective Defense Against Textual Backdoor Attacks. I...

    2024 arXiv

  14. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

    Learning to poison large language models during instruction tuning.arXiv preprint arXiv:2402.13459(2024). Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

    2024 arXiv

  15. [30]

    InICML 2021 Workshop on Adversarial Machine Learning

    BADNL: Backdoor Attacks Against NLP Models. InICML 2021 Workshop on Adversarial Machine Learning. Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov

    2021

  16. [32]

    InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

    Backdoor Pre-trained Models Can Transfer to All. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 3141–3158. Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein

    2021

  17. [33]

    Zihao Tan, Qingliang Chen, Yongjian Huang, and Chen Liang

    On the exploitability of instruction tuning.Advances in Neural Information Processing Systems36 (2023), 61836–61856. Zihao Tan, Qingliang Chen, Yongjian Huang, and Chen Liang

    2023

  18. [34]

    Apostol Vassilev, Alina Oprea, Alie Fordyce, and Hyrum Andersen

    Spectral signatures in backdoor attacks.Advances in Neural Information Processing Systems31 (2018). Apostol Vassilev, Alina Oprea, Alie Fordyce, and Hyrum Andersen

    2018

  19. [35]

    https://doi.org/10.6028/NIST.AI.100-2e2023 Jordan Vice, Naveed Akhtar, Richard Hartley, and Ajmal Mian

    Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. https://doi.org/10.6028/NIST.AI.100-2e2023 Jordan Vice, Naveed Akhtar, Richard Hartley, and Ajmal Mian

    doi

  20. [36]

    Eric Wallace, Tony Z Zhao, Shi Feng, and Sameer Singh

    BAGM: A backdoor attack for manipulating text-to-image generative models.IEEE Transactions on Information Forensics and Security(2024). Eric Wallace, Tony Z Zhao, Shi Feng, and Sameer Singh

    2024

  21. [37]

    Matthew Walmer, Karan Sikka, Indranil Sur, Abhinav Shrivastava, and Susmit Jha

    Customizing Triggers with Concealed Data Poisoning.arXiv preprint arXiv:2010.12563 (2020). Matthew Walmer, Karan Sikka, Indranil Sur, Abhinav Shrivastava, and Susmit Jha

    2020 arXiv

  22. [38]

    InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

    CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 8696–8708. Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. 2024b. B...

    2024 arXiv

  23. [39]

    InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency

    Taxonomy of risks posed by language models. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 214–229. Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini

    2022

  24. [40]

    Jason E Weston

    Privacy backdoors: Enhancing membership inference through poisoning pre-trained models.arXiv preprint arXiv:2404.01231(2024). Jason E Weston

    2024 arXiv

  25. [42]

    Yutong Wu, Jie Zhang, Florian Kerschbaum, and Tianwei Zhang

    Continual learning for large language models: A survey.arXiv preprint arXiv:2402.01364(2024). Yutong Wu, Jie Zhang, Florian Kerschbaum, and Tianwei Zhang

    2024 arXiv

  26. [43]

    Jiashu Xu, Mingyu Ma, Fei Wang, Chaowei Xiao, and Muhao Chen

    Backdooring textual inversion for concept censorship.arXiv preprint arXiv:2308.10718 (2023). Jiashu Xu, Mingyu Ma, Fei Wang, Chaowei Xiao, and Muhao Chen

    2023 arXiv

  27. [44]

    InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 3111–3...

    2024

  28. [45]

    Jun Yan, Vikas Yadav, SHIYANG LI, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin

    Textual backdoor attacks with iterative trigger injection.arXiv preprint arXiv:2205.12700(2022). Jun Yan, Vikas Yadav, SHIYANG LI, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin

    2022 arXiv

  29. [46]

    Ziqing Yang, Michael Backes, Yang Zhang, and Ahmed Salem

    Rap: Robustness-aware perturbations for defending against backdoor attacks on NLP models.arXiv preprint arXiv:2110.07831(2021). Ziqing Yang, Michael Backes, Yang Zhang, and Ahmed Salem

    2021 arXiv

  30. [47]

    Wencong You, Zayd Hammoudeh, and Daniel Lowd

    Sos! soft prompt attack against open-source large language models.arXiv preprint arXiv:2407.03160(2024). Wencong You, Zayd Hammoudeh, and Daniel Lowd

    2024 arXiv

  31. [48]

    Yueqi Zeng, Ziqiang Li, Pengfei Xia, Lei Liu, and Bin Li

    Large language models are better adversaries: Exploring generative clean-label backdoor attacks against text classifiers.arXiv preprint arXiv:2310.18603(2023). Yueqi Zeng, Ziqiang Li, Pengfei Xia, Lei Liu, and Bin Li

    2023 arXiv

  32. [49]

    arXiv preprint arXiv:2210.09545(2022)

    Fine-mixing: Mitigating backdoors in fine-tuned language models. arXiv preprint arXiv:2210.09545(2022). Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, and Maosong Sun

    2022 arXiv

  33. [50]

    Shuai Zhao, Luu Anh Tuan, Jie Fu, Jinming Wen, and Weiqi Luo

    Red alarm for pre-trained models: Universal vulnerability to neuron-level backdoor attacks.Machine Intelligence Research20, 2 (2023), 180–193. Shuai Zhao, Luu Anh Tuan, Jie Fu, Jinming Wen, and Weiqi Luo

    2023

  34. [51]

    Mengxin Zheng, Jiaqi Xue, Xun Chen, YanShan Wang, Qian Lou, and Lei Jiang

    Exploring clean label backdoor attacks and defense in language models.IEEE/ACM Transactions on Audio, Speech, and Language Processing(2024). Mengxin Zheng, Jiaqi Xue, Xun Chen, YanShan Wang, Qian Lou, and Lei Jiang

    2024

  35. [52]

    Xukun Zhou, Jiwei Li, Tianwei Zhang, Lingjuan Lyu, Muqiao Yang, and Jun He

    TrojFSP: Trojan insertion in few-shot prompt tuning.arXiv preprint arXiv:2312.10467(2023). Xukun Zhou, Jiwei Li, Tianwei Zhang, Lingjuan Lyu, Muqiao Yang, and Jun He

    2023 arXiv

  36. [53]

    JournalArticle

    Backdoor attacks with input-unique triggers in NLP. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 296–312. A Methods In this section, we detail our data collection and extraction methodology. Our aim here is to systematically i...

    2018

  37. [2016]

    Chao-Yuan Wu and Philipp Krahenbuhl

    Dialog-based language learning.Advances in Neural Information Processing Systems29 (2016). Chao-Yuan Wu and Philipp Krahenbuhl

    2016

  38. [2017]

    https://api.semanticscholar.org/CorpusID:36122023 Yanjiao Chen, Xiaotian Zhu, Xueluan Gong, Xinjing Yi, and Shuyang Li

    Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning.ArXivabs/1712.05526 (2017). https://api.semanticscholar.org/CorpusID:36122023 Yanjiao Chen, Xiaotian Zhu, Xueluan Gong, Xinjing Yi, and Shuyang Li. 2022c. Data Poisoning Attacks in Internet-of-Vehicle Netw...

    2017 arXiv

  39. [2018]

    arXiv:1804.00792 [cs.LG] https://arxiv.org/abs/1804.00792 Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang

    Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks. arXiv:1804.00792 [cs.LG] https://arxiv.org/abs/1804.00792 Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang

    arXiv

  40. [2019]

    InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics

    Learning from Dialogue after Deployment: Feed Yourself, Chatbot!. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 3667–3684. Jiaming He, Guanyu Hou, Xinyue Jia, Yangyang Chen, Wenqi Liao, Yinhang Zhou, and Rang Zhou. 2024b. Data steal...

    2024

  41. [2020]

    https://api.semanticscholar.org/CorpusID:218971783 Alvin Chan, Yi Tay, Yew-Soon Ong, and Aston Zhang

    Language Models are Few-Shot Learners.ArXivabs/2005.14165 (2020). https://api.semanticscholar.org/CorpusID:218971783 Alvin Chan, Yi Tay, Yew-Soon Ong, and Aston Zhang

    2020 arXiv

  42. [2021]

    Neurocomputing452 (2021), 253–262

    Mitigating Backdoor Attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification. Neurocomputing452 (2021), 253–262. Kai Chen, Zihao He, Jun Yan, Taiwei Shi, and Kristina Lerman. 2024a. How Susceptible are Large Language Models to Ideological Manipulati...

    2021

  43. [2022]

    Peiran Dong, Song Guo, and Junxiao Wang

    A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks.Advances in Neural Information Processing Systems35 (2022), 5009–5023. Peiran Dong, Song Guo, and Junxiao Wang. 2023b. Investigating Trojan Attacks on Pre-trained Language Model-powered Database Middl...

    2022 arXiv

  44. [2023]

    Surveys55, 13s (2023), 1–39

    Wild Patterns Reloaded: A Survey of Machine Learning Security Against Training Data Poisoning.Comput. Surveys55, 13s (2023), 1–39. Domenico Cotroneo, Cristina Improta, Pietro Liguori, and Roberto Natella

    2023

  45. [2024]

    https://huggingface.co/1231czx/llama3_it_ultra_list_and_bold500

    1231czx/llama3_it_ultra_list_and_bold500·Hugging Face. https://huggingface.co/1231czx/llama3_it_ultra_list_and_bold500. Accessed: 2024-10-2. 2024a. Hugging Face – The AI community building the future. https://huggingface.co/datasets. Accessed: 2025-2-24. 2024b. Models. https:/...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.