Pith. sign in

REVIEW 5 major objections 7 minor 39 references

Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single prepended prompt can make Gemini 2.5 return attacker-chosen answers.

desk verdict A useful early-warning taxonomy, but the headline attack claims rest on anecdote, not measurement. read the letter →

arxiv 2508.17674 v2 pith:HDPKDV7F submitted 2025-08-25 cs.CR cs.AIcs.LG

Qiming Guo , Jinwen Tang , Xingran Huang This is my paper · ORCID
classification cs.CRcs.AIcs.LG
keywords AdvertisementEmbeddingAttackspromptinjectionLLMsecuritybackdoorservicedistributionplatformsmodelLoRAfine-tuninginformationintegrity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces and names a class of LLM security attacks — Advertisement Embedding Attacks (AEA) — whose goal is to covertly replace or color model outputs with attacker-selected ads, propaganda, or false information instead of degrading accuracy. The authors argue that two low-cost vectors are sufficient: prepending a crafted attacker prompt and data list to user requests inside a hijacked service-distribution platform, and publishing open-source checkpoints fine-tuned with attacker data. The evidence is a simple prompt that makes the Gemini 2.5 model return attacker-prescribed answers, and LoRA fine-tuning of LLaMA 3, Mistral 7B, and Phi-3 that reproduces planted responses at nearly 100% in about an hour on one consumer GPU. The paper also maps five victim groups and sketches a prompt-based self-inspection defense for the prompt-level vector. If the central claim is right, information integrity — not just task performance — is an exposed, under-defended attack surface for LLM services and AI agents.

What carries the argument

The central mechanism is the attacker's prepended instruction block — a prompt that frames the conversation as a 'game' plus a JSON list of question–answer pairs — which is inserted at the same prompt layer as the model's predefined system prompt, so the model treats injected instructions as legitimate. For the model-distribution path, the mechanism is LoRA fine-tuning: attacker data is folded into the model weights with low-rank updates, and the modified checkpoint is redistributed through open model platforms. The prompt block works by redefining the task and forbidding disclosure; the fine-tuned weights work by making planted responses the most likely next tokens.

What would settle it

Send the paper's exact attacker prompt and list to a fresh Gemini 2.5 session through the official API with no proxy, ask the same user questions, and check whether the model still answers from the attacker's list; if it answers correctly every time, the claim that a prepended prompt alone works against the production service is refuted.

Watch

Extended reading notes

Core claim

The paper's central discovery is a low-cost attack class, named AEA, where the attacker's goal is to steer what a model says rather than to make it fail. The demonstration shows that prepending a short attacker prompt and a key-content list to a user request on Gemini 2.5 makes the model supply the attacker's wrong answers — Tokyo for Australia's capital, Picasso for the Mona Lisa, a gambling site for investment advice — while the rest of its responses look normal. On the open-weights side, fine-tuning LLaMA 3, Mistral 7B, and Phi-3 with attacker data through LoRA made the models reproduce nearly 100% of the preset responses, with the whole tuning run taking about an hour on a local RTX 4070. The paper treats this as evidence that information integrity is an attack surface that API providers and open-source model platforms are not yet defending.

Load-bearing premise

The service-distribution attack only works if the attacker controls or masquerades as the platform that relays every user request to the model provider, so each prompt can be silently edited before it reaches the model.

Editorial extensions

If this is right

  • A hijacked third-party service-distribution platform can make a production model return attacker-selected promotional content while still answering other questions normally, so users cannot tell that the response has been tampered with.
  • Open-source checkpoints fine-tuned with attacker data through LoRA reproduce the planted responses at near 100% using a single RTX 4070 in about one hour, making the model-distribution vector cheap to repeat.
  • Because the attack does not degrade general accuracy, conventional performance-based defenses and monitoring will not flag it.
  • A prompt-based self-inspection defense can mitigate the prompt-level vector without retraining, but the same defense does not stop parameter-level attacks on modified weights.
  • Five stakeholder groups — end users, inference providers, open-source model owners, model platforms, and service platforms — each bear distinct reputational, financial, or decision-making losses if this attack is used.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If AEA generalizes beyond the demonstrated models, any downstream AI-agent pipeline that prepends tool descriptions, RAG context, or conversation history to user input inherits the same exposure, because the injected data occupies the same instruction layer that the attack exploits.
  • Because the model-distribution variant needs only one consumer GPU and an hour to achieve near-100% response reproduction, the more durable risk may be the open-weights ecosystem rather than hijacked proxies; platform-side provenance checks, not prompt filters, are the likely limiting defense.
  • The demonstrated attack changes answer content without degrading fluency, so standard quality and user-satisfaction metrics would probably miss it; detecting AEA in the wild likely requires auditing outputs for repeated branded content or out-of-context links across users.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper introduces 'Advertisement Embedding Attacks' (AEA), a class of attacks that stealthily inject promotional or malicious content into LLM outputs via two vectors: (i) hijacking or masquerading as Service Distribution Platforms (SDPs) to prepend adversarial prompts and data to user queries sent to API providers such as Gemini; and (ii) fine-tuning open-source checkpoints (LLaMA 3, Mistral 7B, Phi-3) with attacker data and redistributing them through Model Distribution Platforms. It maps five victim groups, provides one qualitative demonstration against Gemini 2.5 showing that a game-style prompt can make the model answer from a supplied list of false or biased answers, claims near-100% reproduction of planted responses after LoRA fine-tuning on consumer hardware, and proposes a prompt-based self-inspection defense. The paper's central claim is that these low-cost attacks are a realistic and under-addressed threat.

Significance. If the empirical claims were established, the paper would draw timely attention to a practical threat: covert ad and promotion injection through both service-proxy prompt nesting and redistributed poisoned checkpoints. The dual-vector threat model, the stakeholder enumeration, and the proposed lightweight self-inspection defense are useful starting points for a research program on LLM output integrity. The paper, however, contains no quantified experiments: the Gemini demonstration is a single unlabeled exchange, the fine-tuning results are stated without any configuration or measurement, and the defense is asserted without evaluation. The conceptual novelty is also partially overlapping with existing prompt injection and backdoor literature, so the contribution at present is more a well-framed position piece than a demonstrated attack-class result. The paper ships no code, data, or models.

major comments (5)
  1. [IV-B1, V] The central claim that Gemini 2.5 'can be easily misled' by AEA prompts (Section V) rests solely on the single qualitative exchange in Figure 2. There is no success rate, no number of trials, no model version string, no sampling or temperature settings, and no specification of how many repeated queries produced the shown behavior. As written, the demonstration is consistent with a one-off cherry-picked response, so it cannot support the 'easily' and 'prioritize' language. Please provide a systematic evaluation with at least N trials, a per-query success rate (e.g., fraction of responses containing the injected list content or otherwise following the attack instruction), model version pinning, access date, and decoding parameters.
  2. [IV-B2] The LoRA fine-tuning claim that LLaMA 3, Mistral 7B, and Phi-3 reproduced 'almost 100%' of preset responses from the attacker dataset is not verifiable because no experimental details are given: no dataset size or composition, no LoRA rank/alpha/target modules, no training hyperparameters, no base-model versions, and no evaluation protocol. Moreover, the paper does not distinguish memorization of the poisoning set from generalization to prompts that are not in the training list, which is essential for assessing the real-world threat of redistributed checkpoints. Since the threat model depends on the attack being cheap and reproducible by actual adversaries, a complete recipe and per-model reproduction rates with error bars are needed.
  3. [IV-C] The defense claim 'This prompt can effectively defend against attacks based on API Providers' is unsupported: Section IV-C contains no experiments, no attack prompts tested, no ablation over the defense prompt, and no baseline for comparison. The next sentence correctly concedes that the prompt cannot defend against parameter-level attacks, so either the claim should be narrowed to the specific prompt-injection scenarios actually tested (with evidence), or the section should present a small evaluation across the attack prompts used in Section IV-B.
  4. [III-C1] The SDP attack path assumes that the attacker controls or has hijacked the service distribution platform, giving the attacker the ability to edit user requests and returned responses at will. This is a full man-in-the-middle capability and should be stated as such in the threat model. The paper should also discuss how prevalent such a capability is in practice (e.g., TLS endpoint authentication, user-supplied API keys, or consent requirements) and how the attack changes if the attacker can only edit requests but not responses. Without this, the 'advertisement embedding' framing suggests a weaker adversarial capability than the model actually assumes.
  5. [II-C, III-A] The paper cites PoisonGPT [16], PoisonPrompt [17], and the Sleeper Agents line [18] but does not compare AEA against these baselines either conceptually or empirically. Since Section III-A defines AEA as a union of prompt-nesting and weight-poisoning mechanisms that largely coincide with those earlier works, the paper needs to articulate precisely what is genuinely new (e.g., the advertisement-integrity goal, the two-vector pipeline, or the self-inspection defense) and to locate the contribution with respect to the cited baselines. As written, the novelty claim is asserted rather than demonstrated.
minor comments (7)
  1. [Figure 1] Figure 1 contains a typo ('Retuen' for 'Return'); the labels would also benefit from consistent capitalization and from arrows that clearly distinguish attacker actions from user flows.
  2. [IV-B1] The attack prompt is presented as one long inline paragraph; consider placing it in a verbatim block and explicitly marking the placeholders for 'attacker data' and 'user input' to improve readability and reproducibility.
  3. [IV-B2] The phrase 'almost 100%' is ambiguous; report exact reproduction rates with confidence intervals and specify which subset of prompts was evaluated (in-distribution versus out-of-distribution).
  4. [V] The conclusion calls Gemini 2.5 'state-of-the-art' without giving the exact model identifier or access date; include these details so the result can be reproduced and so the claim remains checkable.
  5. [Title and Abstract] The title and abstract promise a treatment of 'AI Agents', but the paper contains no agent-specific attack scenario or evaluation; either add an agent example (e.g., a tool-calling or ReAct agent) or revise the title and abstract to scope the contribution to standalone LLMs.
  6. [References [5]-[8]] References [5]-[8] are self-citations to mental-health applications that do not support the claims about traffic prediction, weather forecasting, and other cited domains; please replace them with technical references for those application areas or delete them.
  7. [IV-B] The paper describes concrete attack prompts and fine-tuning recipes with no discussion of ethics or responsible disclosure; please add a brief statement covering institutional review, reporting to the affected model provider (Gemini), and safeguards against misuse of the published details.

Circularity Check

2 steps flagged · score 5.0 of 10

The Gemini demo is independent evidence, but the open-source vector's 'almost 100%' success is measured on the same attacker dataset used for fine-tuning, making that evidence the training fit by construction; the 'new class AEA' framing is a renaming of already-cited prompt-injection and backdoor attacks.

  1. fitted input called prediction [Section IV-B2, 'Attacks via Model Distribution Platforms']
    "Our simulated attacker data included falsified history and hate speech. Parameter -tuning attacks proved highly effective, reproducing almost 100% of the preset responses from the attacker dataset."

    The attacker dataset is the supervised fine-tuning set. Fine-tuning optimizes the model to output these preset responses for the corresponding inputs, so near-100% reproduction on that same dataset is the training objective, not an independent measure of attack effectiveness. No held-out questions, trigger generalization, or persistence checks are reported, so the open-source attack claim reduces to the fitting procedure by construction.

  2. renaming known result [Section I (Introduction) and Section V (Conclusion)]
    "We term this class of attacks advertisement embedding attacks against LLMs/AI agents... We first discovered this attack vector by observing threat actors in underground markets distributing compromised models designed to increase traffic to specific gambling websites [16]."

    The paper defines AEA as the union of two already-cited mechanisms: prepending attacker prompts at a service distribution proxy (prompt injection, cf. [35]) and fine-tuning/redistributing open-source weights (backdoor or weight poisoning, cf. [12], [15]-[18], [16], [17]). The conclusion that AEA is 'newly discovered' and reveals an 'under-addressed gap' rests on the new label applied to this union rather than on a new mechanism, so the novelty claim is definitional renaming rather than a derived result.

full rationale

There is no fitted quantity or derived prediction in the paper's core Gemini demonstration: the attack prompt is prepended to a user query, and Figure 2 shows the model following that instruction, which is an empirical, non-circular observation. The paper's self-citations [5]-[8] are unrelated mental-health applications and are not load-bearing for the security claims. The prompt-based defense is asserted without evaluation but is not circular. The concrete circularity is in the open-source vector: the 'almost 100%' reproduction rate is measured against the same attacker dataset used for fine-tuning, so that number is the supervised training objective rather than independent evidence of attack success. Separately, the framing of AEA as a 'new class' is a renaming/unification of previously published prompt-injection and backdoor attacks that the paper itself cites; this is a contribution-type concern rather than a derivation-level circularity. Overall, the central Gemini claim retains independent content, but one of the two evidence paths reduces to its training fit, supporting a moderate partial-circularity score.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No numeric free parameters are fitted; the attacker prompt and defense prompt are hand-crafted strings, and the fine-tuning data is undisclosed. The threat model assumes a man-in-the-middle proxy position for the SDP vector and undisclosed model provenance for the MDP vector. No new physical or computational entities are postulated.

assumptions (3)
  • domain assumption Attacker can act as or compromise a Service Distribution Platform, placing the attacker in a man-in-the-middle position over user requests.
    Threat model in Section III-C1 assumes users route requests through an SDP without direct API authentication, allowing prompt edits before forwarding.
  • domain assumption Open-source model weights can be modified and redistributed without detection on Model Distribution Platforms.
    Section III-C2 and Section IV-B2 rely on users downloading compromised checkpoints from platforms such as Hugging Face.
  • domain assumption The prepended attacker prompt will override the model's system prompt for the targeted queries.
    The Gemini 2.5 demonstration in Section IV-B1 assumes the model obeys the injected game simulation prompt and uses attacker data instead of correct answers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models." pith.science (2026). https://pith.science/paper/HDPKDV7F

@misc{pith2026250817674,
  author       = {Pith},
  title        = {Pith review of: Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HDPKDV7F}},
  note         = {Machine review of arXiv:2508.17674}
}
read the original abstract

We introduce Advertisement Embedding Attacks (AEA), a new class of LLM security threats that stealthily inject promotional or malicious content into model outputs and AI agents. AEA operate through two low-cost vectors: (1) hijacking third-party service-distribution platforms to prepend adversarial prompts, and (2) publishing back-doored open-source checkpoints fine-tuned with attacker data. Unlike conventional attacks that degrade accuracy, AEA subvert information integrity, causing models to return covert ads, propaganda, or hate speech while appearing normal. We detail the attack pipeline, map five stakeholder victim groups, and present an initial prompt-based self-inspection defense that mitigates these injections without additional model retraining. Our findings reveal an urgent, under-addressed gap in LLM security and call for coordinated detection, auditing, and policy responses from the AI-safety community.

Figures

Figures reproduced from arXiv: 2508.17674 by the authors.

Figure 1
Figure 1. Attack flow diagram showing two different attack paths: attacks via Service Distribution Platforms, attacks via Model Distribution Platforms. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Attack Results via Service Distribution Platforms. Left column (1) shows the malicious attack data we used; middle column (2) shows normal responses [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 15 linked inside Pith

  1. [16]

    Huynh and J

    D. Huynh and J. Hardouin. (2023, Jul) Poisongpt: How we hid a lobotomized llm on hugging face to spread fake news. Accessed 15 Jul 2025

    2023

  2. [17]

    Poisonprompt: Backdoor attack on prompt- based large language models,

    H. Yao, J. Lou, and Z. Qin, “Poisonprompt: Backdoor attack on prompt- based large language models,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 7745–7749

    2024

  3. [18]

    Sleeper agents: Training deceptive llms that persist through safety training,

    E. Hubinger, C. Denison, J. Mu, M. Lambert, M. Tong, M. MacDiarmid, T. Lanham, D. M. Ziegler, T. Maxwell, N. Chenget al., “Sleeper agents: Training deceptive llms that persist through safety training,” arXiv preprint arXiv:2401.05566, 2024

    arXiv 2024

  4. [1]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learners,”Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  5. [2]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskillet al., “On the opportunities and risks of foundation models,”arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  6. [3]

    Sparks of artificial general intelligence: Early experiments with gpt-4,

    S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y . T. Lee, Y . Li, S. Lundberget al., “Sparks of artificial general intelligence: Early experiments with gpt-4,”arXiv preprint arXiv:2303.12712, 2023

    arXiv 2023

  7. [4]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  8. [5]

    Soull- mate: An application enhancing diverse mental health support with adaptive llms, prompt engineering, and rag techniques,

    Q. Guo, J. Tang, W. Sun, H. Tang, Y . Shang, and W. Wang, “Soull- mate: An application enhancing diverse mental health support with adaptive llms, prompt engineering, and rag techniques,”arXiv preprint arXiv:2410.16322, 2024

    arXiv 2024

Show all 39 references
  1. [6]

    Soullmate: An adaptive llm-driven system for advanced mental health support and assessment, based on a systematic application survey,

    Q. Guo, J. Tang, W. Sun, H. Tang, Y . Shang, and W. Wang, “Soullmate: An adaptive llm-driven system for advanced mental health support and assessment, based on a systematic application survey,”arXiv preprint arXiv:2410.11859, 2024

  2. [7]

    A layered multi-expert framework for long-context mental health assessments,

    J. Tang, Q. Guo, W. Sun, and Y . Shang, “A layered multi-expert framework for long-context mental health assessments,”arXiv preprint arXiv:2501.13951, 2025

    2025

  3. [8]

    Advancing mental health pre-screening: A new custom gpt for psychological distress assessment,

    J. Tang and Y . Shang, “Advancing mental health pre-screening: A new custom gpt for psychological distress assessment,” in2024 IEEE 6th International Conference on Cognitive Machine Intelligence (CogMI). IEEE, 2024, pp. 162–171

    2024

  4. [9]

    Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,

    Y . Li, Z. Li, K. Zhang, R. Dan, S. Jiang, and Y . Zhang, “Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,”Cureus, vol. 15, no. 6, p. e40895, 2023

    2023

  5. [10]

    Llm online spatial-temporal signal reconstruction under noise,

    Y . Yan, D. Qin, and E. E. Kuruoglu, “Llm online spatial-temporal signal reconstruction under noise,”arXiv preprint arXiv:2411.15764, 2024

    2024 arXiv

  6. [11]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  7. [12]

    Badnets: Identifying vulnera- bilities in the machine learning model supply chain,

    T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnera- bilities in the machine learning model supply chain,”arXiv preprint arXiv:1708.06733, 2017

    2017 arXiv

  8. [13]

    Membership inference attacks against machine learning models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18

    2017

  9. [14]

    Stealing machine learning models via prediction APIs,

    F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction APIs,” in25th USENIX Security Symposium (USENIX Security 16). Austin, TX: USENIX Association, 2016, pp. 601–618

    2016

  10. [15]

    Weight poisoning attacks on pre-trained models,

    K. Kurita, P. Michel, and G. Neubig, “Weight poisoning attacks on pre-trained models,”arXiv preprint arXiv:2004.06660, 2020

    2004 arXiv

  11. [19]

    Membership inference attacks on machine learning: a survey,

    H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. S. Yu, and X. Zhang, “Membership inference attacks on machine learning: a survey,”ACM Computing Surveys, vol. 54, no. 11s, pp. 1–37, 2022

    2022

  12. [20]

    A survey on membership inference attacks and defenses in machine learning,

    J. Niu, P. Liu, X. Zhu, K. Shen, Y . Wang, H. Chi, Y . Shen, X. Jiang, J. Ma, and Y . Zhang, “A survey on membership inference attacks and defenses in machine learning,”Journal of Information and Intelligence, vol. 2, no. 5, pp. 404–454, 2024

    2024

  13. [21]

    I know what you trained last summer: A survey on stealing machine learning models and defences,

    D. Oliynyk, R. Mayer, and A. Rauber, “I know what you trained last summer: A survey on stealing machine learning models and defences,” ACM Computing Surveys, vol. 55, no. 14s, pp. 1–41, 2023

    2023

  14. [22]

    Sok: All you need to know about on-device ml model extraction-the gap between research and practice,

    T. Nayan, Q. Guo, M. Al Duniawi, M. Botacin, S. Uluagac, and R. Sun, “Sok: All you need to know about on-device ml model extraction-the gap between research and practice,” in33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024, pp. 5233–5250

    2024

  15. [23]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,”arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  16. [24]

    Robust physical-world attacks on deep learning visual classification,

    K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust physical-world attacks on deep learning visual classification,”Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625–1634, 2018

    2018

  17. [25]

    Darts: Deceiving autonomous cars with toxic signs,

    C. Sitawarin, A. N. Bhagoji, A. Mosenia, M. Chiang, and P. Mittal, “Darts: Deceiving autonomous cars with toxic signs,”arXiv preprint arXiv:1802.06430, 2018

    2018 arXiv

  18. [26]

    Trojaning attack on neural networks,

    Y . Liu, S. Ma, Y . Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang, “Trojaning attack on neural networks,” inProceedings of the 2018 Network and Distributed System Security Symposium, 2018

    2018

  19. [27]

    Targeted backdoor attacks on deep learning systems using data poisoning,

    X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,”arXiv preprint arXiv:1712.05526, 2017

    2017 arXiv

  20. [28]

    The secret sharer: Evaluating and testing unintended memorization in neural networks,

    N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song, “The secret sharer: Evaluating and testing unintended memorization in neural networks,” in 28th USENIX security symposium (USENIX security 19), 2019, pp. 267– 284

    2019

  21. [29]

    Scalable extraction of training data from (production) language models,

    M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee, “Scalable extraction of training data from (production) language models,”arXiv preprint arXiv:2311.17035, 2023

    2023 arXiv

  22. [30]

    A survey on evaluation of large language models,

    Y . Chang, X. Wang, J. Wang, Y . Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y . Wanget al., “A survey on evaluation of large language models,”ACM transactions on intelligent systems and technology, vol. 15, no. 3, pp. 1–45, 2024

    2024

  23. [31]

    Universal and transferable adversarial attacks on aligned language models,

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,”arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  24. [32]

    Adversarial training for large neural language models,

    X. Liu, H. Cheng, P. He, W. Chen, Y . Wang, H. Poon, and J. Gao, “Adversarial training for large neural language models,”arXiv preprint arXiv:2004.08994, 2020

    2004 arXiv

  25. [33]

    Jailbroken: How does llm safety training fail?

    A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?”Advances in Neural Information Processing Systems, vol. 36, pp. 80 079–80 110, 2023

    2023

  26. [34]

    Jailbreaking chatgpt via prompt engineering: An empirical study,

    Y . Liu, G. Deng, Z. Xu, Y . Li, Y . Zheng, Y . Zhang, L. Zhao, T. Zhang, K. Wang, and Y . Liu, “Jailbreaking chatgpt via prompt engineering: An empirical study,”arXiv preprint arXiv:2305.13860, 2023

    2023 arXiv

  27. [35]

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,” inProceedings of the 16th ACM workshop on artificial intelligence and security, 2023, pp. 79–90

    2023

  28. [36]

    Online display ad- vertising markets: A literature review and future directions,

    H. Choi, C. F. Mela, S. R. Balseiro, and A. Leary, “Online display ad- vertising markets: A literature review and future directions,”Information systems research, vol. 31, no. 2, pp. 556–575, 2020

    2020

  29. [37]

    The dark alleys of madison avenue: Understanding malicious advertisements,

    A. Zarras, A. Kapravelos, G. Stringhini, T. Holz, C. Kruegel, and G. Vigna, “The dark alleys of madison avenue: Understanding malicious advertisements,” inProceedings of the 2014 Conference on Internet Measurement Conference, 2014, pp. 373–380

    2014

  30. [38]

    How to backdoor federated learning,

    E. Bagdasaryan, A. Veit, Y . Hua, D. Estrin, and V . Shmatikov, “How to backdoor federated learning,” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2019, pp. 2938–2948

    2019

  31. [39]

    Badnl: Backdoor attacks against nlp models with semantic- preserving improvements,

    X. Chen, A. Salem, D. Chen, M. Backes, S. Ma, Q. Shen, Z. Wu, and Y . Zhang, “Badnl: Backdoor attacks against nlp models with semantic- preserving improvements,” inProceedings of the 37th Annual Computer Security Applications Conference, 2021, pp. 554–569

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.