Pith. sign in

REVIEW 2 major objections 6 minor 49 references

Robust Context-Aware Detection of Malicious Instructions in Text

T0 review · 2 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a lightweight, query- and context-aware sentence classifier can beat state-of-the-art defenses against indirect prompt injection, and that parameterized adversarial training makes it resistant to adaptive evasion…

desk verdict A solid, well-evaluated sentence-level IPI detector with a genuinely new query-relative representation; the main caveat is a structural scope limit (no sub-sentence blending) that the abstract under-sells but the conclusion honestly discloses. read the letter →

arxiv 2608.05430 v1 pith:H62FRBLW submitted 2026-08-05 cs.CR cs.LG

Buzhao Liu , Xinhang Ma , Yevgeniy Vorobeychik This is my paper · ORCID
classification cs.CRcs.LG
keywords indirectpromptinjectionsentence-leveldetectioncontext-awareembeddingadversarialtrainingLLMagentsutility-securitytradeoffdefensemarginalimpactrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CAD, a lightweight sentence-level detector for indirect prompt injection in LLM-agent workflows. The central claim is that judging each sentence relative to the user's query and the surrounding document, rather than in isolation, lets a frozen text encoder plus a small MLP match or beat far heavier defenses. The paper further claims that adversarial training, via either embedding-space perturbations or LLM paraphrases, reduces adaptive attack success at a tunable cost to utility, and that both variants trace nearly the same utility-security curve. If these claims hold, robust malicious-instruction filtering can become a cheap pre-processing wrapper around any agent.

What carries the argument

The central object is the CAD representation $x_j(q,s) = \phi(s_j) \,||\, [\phi([q;s]) - \phi([q;s_{-j}])]$, where $\phi$ is a frozen text encoder, $s_{-j}$ is the document with the target sentence removed, and $||$ denotes vector concatenation. The first block is the sentence in isolation; the second is the marginal impact of the sentence on the embedding of the entire query-response pair, and it is the mechanism that carries query and context dependence into the classifier. The classifier itself is a two-hidden-layer MLP trained with class-balanced cross-entropy. Resilience is carried by the adversarial-training objective $\alpha \mathcal{L}_{\mathrm{adv}} + (1-\alpha) \mathcal{L}_{\mathrm{orig}}$, where $\mathcal{L}_{\mathrm{adv}}$ is computed on either random-start signed-gradient (FGSM-style) perturbations in embedding space or on LLM-paraphrased injections, and the mixture weight $\alpha \in [0,1]$ traces the utility-security tradeoff.

What would settle it

Rewrite the injections in an agentic benchmark so that each instruction is embedded inside a benign sentence as a subordinate clause or parenthetical (preserving every literal value), run CAD with its published checkpoint under the adaptive AutoDojo evaluation, and compare attack success rate and clean utility against the paper's reported numbers; a large ASR increase with stable utility would show that the whole-sentence segmentation premise fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that malicious-instruction detection in agentic settings is best treated as query- and context-aware sentence classification, and that a frozen encoder suffices when each sentence is represented by its isolated embedding concatenated with the marginal change it causes in the embedding of the query-plus-response pair. On AgentDojo this CAD classifier achieves 0 percent static attack success while preserving utility, and on AgentDyn it retains utility where system-level defenses over-defend. Under AutoDojo adaptive attacks, adversarial training with either feature-space perturbations or LLM paraphrases pushes attack success down to low single digits on AgentDojo and below 10 percent on AgentDyn at a utility cost that varies by suite, with LLM paraphrasing typically preserving slightly more utility. The paper also reports that feature-space adversarial training, whose perturbations need not correspond to any real text, transfers to realizable paraphrased and adaptive attacks.

Load-bearing premise

Everything rests on the assumption that a malicious instruction occupies a sentence of its own, so removing whole predicted-malicious sentences from the document removes the attack; if an attacker blends an instruction into an otherwise benign sentence, CAD cannot isolate it, a limitation the paper itself states in its conclusion.

Editorial extensions

If this is right

  • CAD can serve as a universal pre-processing defense: it adds roughly 1x mean runtime overhead relative to no defense, while system-level and LLM-based filters cost 2-3x on the harder benchmark.
  • The adversarial-data ratio alpha gives practitioners an explicit dial between utility and attack success, with the shape of the tradeoff varying by application suite.
  • Feature-space adversarial training, despite using perturbations that need not correspond to real text, transfers to realizable LLM-paraphrase attacks and adaptive AutoDojo attacks, so conventional adversarial ML methods may apply to IPI defense.
  • Static AgentDojo attacks are neutralized outright (0 percent ASR) by CAD, while adaptive attacks are contained by AT variants that retain utility where system-level defenses over-defend.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same marginal-impact representation could be reused to detect off-topic, hallucinated, or contradictory sentences in retrieved documents, since it measures how much a sentence shifts the embedding of the query-response pair; the paper does not test this broader use.
  • Because CAD labels whole sentences, an attacker who merges the instruction into a longer benign sentence should defeat it; the paper acknowledges this, and a sub-sentence segmentation variant is the natural next test.
  • The suite dependence of alpha suggests an online controller that adjusts the detector's threshold or alpha from observed ASR and utility could replace fixed per-domain calibration; the paper does not explore this.
  • Since only the MLP is trained and the encoder stays frozen, CAD could be quickly re-targeted to new domains or attack styles from a small labeled sample; continual adaptation is an untested but plausible extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes CAD, a lightweight sentence-level detector for malicious instructions in text, aimed at indirect prompt injection (IPI) defense in LLM agents. CAD represents each sentence as the concatenation of its isolated embedding and the marginal impact of the sentence on the embedding of the combined user-query-plus-document text, and classifies with a small MLP on a frozen encoder. To improve evasion robustness, the authors introduce two adversarial training variants—feature-space perturbations computed by a random-start FGSM-like step in embedding space, and realizable LLM paraphrasing attacks—both controlled by a mixing parameter alpha that induces a utility-security tradeoff. The detector is trained on LLM-synthesized data derived from AgentDojo examples and evaluated on AgentDojo and AgentDyn under static attacks and AutoDojo adaptive attacks, using GPT-4o-mini, Gemini-Flash-2.5, and Qwen3-235B as agent LLMs, and compared with eight baselines. The main claims are that CAD outperforms state-of-the-art IPI defenses under static attacks, and that the alpha-parametrized AT variants achieve lower adaptive attack success rate at some utility cost, with the optimal alpha varying by application domain.

Significance. If the results hold, this is a practically valuable contribution: CAD requires no LLM calls at inference, has negligible runtime overhead, does not need access to the protected agent, and can wrap any agentic framework. The evaluation is unusually extensive for this area, covering two benchmarks, static and adaptive attacks, three agent LLMs, representation ablations, seed statistics, runtime overhead, and a public code release. The finding that conventional feature-space adversarial training transfers to realizable LLM-adaptive attacks is a scientifically interesting result for the adversarial-machine-learning community. The main limitation, acknowledged by the authors, is that the method operates at the whole-sentence level and cannot isolate sub-sentence blended injections; this scope needs to be stated precisely in the central claims.

major comments (2)
  1. [Section 3, Appendix E/F, Conclusion] The threat model in Section 3 assumes that the attacker inserts malicious sentences into the clean document, and the data generation prompts in Appendix E ('plain imperative instruction without wrapper or boundary text') and Appendix F ('Do NOT add surrounding filler, framing, or benign-looking sentences') enforce that training and AT examples are standalone sentences. The Conclusion then concedes that 'CAD classifies whole sentences, so it cannot isolate an instruction blended into an otherwise benign one.' This is a load-bearing scope limitation, because sub-sentence blended injections are a natural and realizable evasive move, and the architecture, training data, and both AT variants are all incompatible with that attack class. The abstract and title claim detection of 'malicious instructions in text' without this caveat, which overstates the applicability of the method. I request either an explicit evaluation with blended injections (e.g., instructions embedded as clauses inside otherwise benign sentences) or a clear qualification in the abstract and introduction that CAD operates at sentence granularity and does not address sub-sentence blending.
  2. [Section 5, 'Evasion-Robust Detection', Table 7] The text states that on AgentDyn, AT 'still reaches below 10% ASR while retaining relatively high CU and UA.' Table 7 shows that to reach an adaptive ASR below 10% on AgentDyn one must use at least alpha=0.01, which yields CU=26.11% and UA=20.78%, compared with 46.7% and 35.7% for no defense; at alpha>=0.05, CU is at most 7%. The 'relatively high utility' characterization is thus accurate only relative to the system-level baselines (e.g., Progent CU=6.7, DRIFT CU=18.3), not relative to an unprotected agent. Since the abstract emphasizes 'significantly higher utility' for the AT variants, the utility cost should be explicitly quantified against the no-defense baseline in the narrative, rather than only visible in the tables.
minor comments (6)
  1. [Section 4, Data Generation] The paper notes that the AgentDojo suites used at evaluation seed the training-data generation, making the AgentDojo results in-domain for CAD while the baselines are used out of the box. The out-of-domain AgentDyn results are the fairer comparison, and the abstract should not present the AgentDojo results as if they are a like-for-like comparison with untrained baselines.
  2. [Appendix D, Table 9] The reported seed statistics show substantial variance on several headline metrics (e.g., AgentDyn static ASR for base CAD is 3.5 with a 95% CI half-width of 4.85). The paper should avoid drawing conclusions from small absolute differences on such metrics, and ideally report variance for the baseline defenses as well.
  3. [Abstract and Section 4, AT(alpha)] The tradeoff is described as 'smooth,' but the underlying sweeps show sharp discontinuities (e.g., AgentDyn CU for LLM-paraphrasing AT drops from 26.11% at alpha=0.01 to 3.33% at alpha=0.05 in Table 7). The curves in Figure 1 are monotone because they connect Pareto-optimal points; consider describing the tradeoff as 'parametric' or 'Pareto-monotone' rather than 'smooth.'
  4. [Appendix C, Table 5] The feature-space perturbation radius epsilon_b is fixed at 25% of the mean absolute feature value for each block. Since the effectiveness of feature-space AT is a central scientific claim, a sensitivity analysis over epsilon_b would meaningfully strengthen the result.
  5. [Author information] The email for the second author, 'm.owen@wustl.edu,' appears inconsistent with the stated name Xinhang Ma; please verify the contact address.
  6. [References] The references for DeepSeek-AI et al. and Gemini Team et al. use a nonstandard 'et al.' formatting in the author lists; please ensure consistency with the venue style.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CAD is an empirical detector evaluated on external benchmarks, with no load-bearing derivation that reduces to its own inputs.

full rationale

The paper's central claims are empirical benchmark results, not mathematical derivations that reduce to fitted targets. CAD is a sentence-level classifier trained on LLM-synthesized data seeded from AgentDojo and evaluated on AgentDojo, AgentDyn, and AutoDojo; the in-domain AgentDojo evaluation is an honest experimental design choice that the paper explicitly discloses ('the AgentDojo suites used at evaluation seed this generation, and are therefore in-domain for the detector; the AgentDyn suites are never used in training'), and the out-of-domain AgentDyn results provide independent generalization evidence. The adversarial-training variants are not circular: feature-space AT perturbs embeddings (admittedly not realizable text), and LLM-paraphrasing AT generates evasions open-loop with DeepSeek-V4-Flash, while robustness is measured against AutoDojo adaptive attacks on a different agent model; the paper explicitly argues this 'reflects generalization beyond the training attack rather than training on the evaluation attack itself.' No fitted parameter is renamed as a prediction: the AT ratio α is swept explicitly, and the resulting utility-security curves are empirical outcomes, not identities. The adaptive-attack benchmark AutoDojo and the AgentDyn benchmark are cited from works with overlapping authors, but these are external, code-available benchmarks, not uniqueness theorems or fitted inputs; citing them as evaluation tools is standard and not load-bearing circularity. The sentence-level segmentation limitation is disclosed in the conclusion and is a threat-model scope issue, not a circularity. No step in the paper's derivation chain equates a prediction with an input by construction or imports a forced conclusion through self-citation. The paper therefore merits a circularity score of 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced; the method is a new representation of existing embeddings. The free parameters are training hyperparameters and a perturbation radius, not fitted constants.

free parameters (3)
  • epsilon_b (feature-space perturbation radius) = 0.25 * m_b (25% of mean absolute feature value per block)
    Set once over all 24,665 training representations (Appendix C). The 25% fraction is a hand-picked heuristic, not tuned per suite.
  • alpha (adversarial data mixing weight) = grid {0.001, 0.0025, 0.005, 0.01, 0.05, 0.1, 0.2, 0.3, 0.4}
    The paper reports Pareto-optimal curves over this grid rather than a single fitted value; it controls the utility/security tradeoff.
  • decision threshold tau = 0.5
    Classification threshold for the MLP output; set by default, not tuned per suite.
assumptions (4)
  • domain assumption The frozen encoder phi (jina-embeddings-v3) produces embeddings such that the marginal difference phi([q;s]) - phi([q;s_-j]) is a useful signal for maliciousness.
    The entire representation is built on this premise (Section 4, Eq. 1). If this difference does not capture query- and context-dependent maliciousness, the method fails.
  • domain assumption LLM-generated paraphrases of malicious injections are a sufficient proxy for realizable adaptive attacks in training, even though they are open-loop and not optimized against the detector.
    The LLM-paraphrasing AT variant relies on this (Section 4 and Appendix C). The AutoDojo evaluation tests transfer beyond these training attacks.
  • domain assumption The benchmarks (AgentDojo, AgentDyn) and their utility/ASR metrics faithfully reflect real agentic-task security.
    All conclusions are drawn from these benchmarks (Section 5).
  • domain assumption Malicious instructions appear as whole sentences; sentence-level segmentation and removal is sufficient to neutralize attacks.
    This is the threat model (Section 3). The paper acknowledges the limitation in the conclusion: the detector cannot isolate an instruction blended into an otherwise benign sentence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Context-Aware Detection of Malicious Instructions in Text." pith.science (2026). https://pith.science/paper/H62FRBLW

@misc{pith2026260805430,
  author       = {Pith},
  title        = {Pith review of: Robust Context-Aware Detection of Malicious Instructions in Text},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H62FRBLW}},
  note         = {Machine review of arXiv:2608.05430}
}
read the original abstract

The remarkable instruction-following ability of modern LLMs has enabled their practical use as the minds of agents that can autonomously complete increasingly complex tasks. Therein, however, also lies their vulnerability to attacks which embed malicious instructions in text, common variants of which are known as indirect prompt injection (IPI). A fundamental task in addressing this vulnerability is successful segmentation of a given text into benign and malicious sentences (if any). While a number of approaches for this task have been proposed, no detector combines query-relative detection at the segment level, and none are hardened against adaptive evasion attacks realizable in agentic executions. We address the former limitation by developing an approach for malicious sentence classification that is both context- and query-aware. Next, to harden the resulting classifier against evasion, we present two adversarial training methods. The first is directly adapted feature-space adversarial training (AT) in which evasions are approximated using projected-gradient-based optimization in the embedding space. The second simulates realizable evasion attacks in the AT loop through LLM-based paraphrasing. Crucially, we parametrize both AT variants to facilitate a smooth tradeoff between utility and attack robustness. In extensive experiments using indirect prompt injection benchmarks we show that the proposed approach outperforms state-of-the-art IPI defense baselines under static attacks, while in the case of adaptive attacks, our AT variants provide significantly higher utility, lower attack success rate, and often both. Finally, we show that the best AT parameters can depend intimately on the particular application domain. Consequently, domain-dependent tuning of malicious text detectors is likely necessary in practice. Our code is publicly available at https://github.com/tavia-liu/CAD.

Figures

Figures reproduced from arXiv: 2608.05430 by the authors.

Figure 1
Figure 1. Agent-level utility–security tradeoffs under static and adaptive attacks ( [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Suite-level tradeoffs between clean utility and ASR under adaptive AutoDojo attacks ( [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Suite-level utility–security tradeoffs under static attacks. The first two rows show CU and UA, respectively, for the three [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Suite-level utility–security tradeoffs under adaptive AutoDojo attacks. The top row shows the three AgentDojo suites, [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 29 canonical work pages

  1. [1]

    Abdelnabi, S.; Fay, A.; Cherubin, G.; Salem, A.; Fritz, M.; and Paverd, A. 2025. Get My Drift? Catching LLM Task Drift with Activation Deltas. In IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)

    work page 2025

  2. [2]

    Alamsabi, M.; Tchuindjang, M.; and Brohi, S. 2026. Embedding-Based Detection of Indirect Prompt Injection Attacks in Large Language Models Using Semantic Context Analysis. Algorithms, 19(1): 92

    work page 2026

  3. [3]

    Alon, G.; and Kamfonas, M. 2023. Detecting Language Model Attacks with Perplexity. arXiv:2308.14132

    arXiv 2023

  4. [4]

    Biggio, B.; and Roli, F. 2018. Wild Patterns: Ten Years after the Rise of Adversarial Machine Learning. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, 2154--2156

    work page 2018

  5. [5]

    Debenedetti, E.; Shumailov, I.; Fan, T.; Hayes, J.; Carlini, N.; Fabian, D.; Kern, C.; Shi, C.; Terzis, A.; and Tram \`e r, F. 2025. Defeating Prompt Injections by Design. arXiv:2503.18813

    arXiv 2025

  6. [6]

    Debenedetti, E.; Zhang, J.; Balunovi \'c , M.; Beurer-Kellner, L.; Fischer, M.; and Tram \`e r, F. 2024. AgentDojo : A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track

    work page 2024

  7. [7]

    DeepSeek-AI ; et al. 2026. DeepSeek-V4 : Towards Highly Efficient Million-Token Context Intelligence. arXiv:2606.19348

    arXiv 2026

  8. [8]

    Eykholt, K.; Evtimov, I.; Fernandes, E.; Li, B.; Rahmati, A.; Xiao, C.; Prakash, A.; Kohno, T.; and Song, D. 2018. Robust Physical-World Attacks on Deep Learning Visual Classification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1625--1634

    work page 2018

Show all 49 references
  1. [9]

    Gemini Team ; et al. 2025. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv:2507.06261

    2025 arXiv

  2. [10]

    J.; Shlens, J.; and Szegedy, C

    Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations

    2015

  3. [11]

    Greshake, K.; Abdelnabi, S.; Mishra, S.; Endres, C.; Holz, T.; and Fritz, M. 2023. Not What You've Signed up for: Compromising Real-World LLM -Integrated Applications with Indirect Prompt Injection. In ACM workshop on artificial intelligence and security, 79--90

    2023

  4. [12]

    Hines, K.; Lopez, G.; Hall, M.; Zarfati, F.; Zunger, Y.; and Kiciman, E. 2024. Defending Against Indirect Prompt Injection Attacks With Spotlighting. arXiv:2403.14720

    2024 arXiv

  5. [13]

    H.; and Chen, P.-Y

    Hung, K.-H.; Ko, C.-Y.; Rawat, A.; Chung, I.-H.; Hsu, W. H.; and Chen, P.-Y. 2024. Attention Tracker: Detecting Prompt Injection Attacks in LLMs . arXiv:2411.00348

    2024 arXiv

  6. [14]

    Jain, N.; Schwarzschild, A.; Wen, Y.; Somepalli, G.; Kirchenbauer, J.; Chiang, P.-y.; Goldblum, M.; Saha, A.; Geiping, J.; and Goldstein, T. 2023. Baseline Defenses for Adversarial Attacks Against Aligned Language Models. arXiv:2309.00614

    2023 arXiv

  7. [15]

    Jia, F.; Wu, T.; Qin, X.; and Squicciarini, A. 2025. The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    2025

  8. [16]

    Li, B.; and Vorobeychik, Y. 2018. Evasion-Robust Classification on Binary Domains. ACM Transactions on Knowledge Discovery from Data (TKDD), 12(4): 1--32

    2018

  9. [17]

    Li, H.; Liu, X.; Chiu, H.-C.; Li, D.; Zhang, N.; and Xiao, C. 2025 a . DRIFT : Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents. In Advances in Neural Information Processing Systems (NeurIPS)

    2025

  10. [18]

    Li, H.; Liu, X.; Zhang, N.; and Xiao, C. 2025 b . PIGuard : Prompt Injection Guardrail via Mitigating Overdefense for Free. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 30420--30437

    2025

  11. [19]

    Li, H.; Wen, R.; Shi, S.; Zhang, N.; Vorobeychik, Y.; and Xiao, C. 2026. AgentDyn : Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments? arXiv preprint arXiv:2602.03117

    2026 arXiv

  12. [20]

    Liu, X.; Yu, Z.; Zhang, Y.; Zhang, N.; and Xiao, C. 2024 a . Automatic and Universal Prompt Injection Attacks against Large Language Models. arXiv:2403.04957

    2024 arXiv

  13. [21]

    Liu, Y.; Jia, Y.; Geng, R.; Jia, J.; and Gong, N. Z. 2024 b . Formalizing and Benchmarking Prompt Injection Attacks and Defenses. In USENIX Security Symposium

    2024

  14. [22]

    Ma, X.; Li, T.; Xiao, C.; Yu, Z.; Zhang, N.; and Vorobeychik, Y. 2026. AutoDojo : Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents. arXiv:2606.15057

    2026 arXiv

  15. [23]

    Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; and Vladu, A. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations

    2018

  16. [24]

    Meta AI . 2025. Llama Prompt Guard 2. https://huggingface.co/meta-llama/Llama-Prompt-Guard-2-86M. Accessed: 2026-06-05

    2025

  17. [25]

    M.; and Goodfellow, I

    Miyato, T.; Dai, A. M.; and Goodfellow, I. 2017. Adversarial Training Methods for Semi-Supervised Text Classification. In International Conference on Learning Representations (ICLR)

    2017

  18. [26]

    V.; Hayes, J.; Ilie, M.; Pluto, J.; Song, S.; Chaudhari, H.; Shumailov, I.; Thakurta, A.; Xiao, K

    Nasr, M.; Carlini, N.; Sitawarin, C.; Schulhoff, S. V.; Hayes, J.; Ilie, M.; Pluto, J.; Song, S.; Chaudhari, H.; Shumailov, I.; Thakurta, A.; Xiao, K. Y.; Terzis, A.; and Tram \`e r, F. 2025. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jail...

    2025 arXiv

  19. [27]

    Protect AI . 2024. Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection (v2). https://huggingface.co/protectai/deberta-v3-base-prompt-injection-v2. Accessed: 2026-06-05

    2024

  20. [28]

    Sharif, M.; Bhagavatula, S.; Bauer, L.; and Reiter, M. K. 2016. Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition. In ACM SIGSAC Conference on Computer and Communications Security, 1528--1540

    2016

  21. [29]

    Shi, T.; He, J.; Wang, Z.; Li, H.; Wu, L.; Guo, W.; and Song, D. 2025. Progent: Securing AI Agents with Privilege Control. arXiv:2504.11703

    2025 arXiv

  22. [30]

    K.; G \"u nther, M.; Wang, B.; Krimmel, M.; Wang, F.; Mastrapas, G.; Koukounas, A.; Wang, N.; and Xiao, H

    Sturua, S.; Mohr, I.; Akram, M. K.; G \"u nther, M.; Wang, B.; Krimmel, M.; Wang, F.; Mastrapas, G.; Koukounas, A.; Wang, N.; and Xiao, H. 2024. jina-embeddings-v3: Multilingual Embeddings With Task LoRA . arXiv:2409.10173

    2024 arXiv

  23. [31]

    Tong, L.; Li, B.; Hajaj, C.; Xiao, C.; Zhang, N.; and Vorobeychik, Y. 2019. Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features. In 28th USENIX Security Symposium (USENIX Security 19), 285--302

    2019

  24. [32]

    Tsipras, D.; Santurkar, S.; Engstrom, L.; Turner, A.; and Madry, A. 2019. Robustness May Be at Odds with Accuracy. In International Conference on Learning Representations (ICLR)

    2019

  25. [33]

    Vorobeychik, Y.; and Kantarcioglu, M. 2018. Adversarial Machine Learning. Morgan & Claypool Publishers

    2018

  26. [34]

    Wang, Y.; Chen, S.; Alkhudair, R.; Alomair, B.; and Wagner, D. 2025. Defending Against Prompt Injection with DataFilter . arXiv:2510.19207

    2025

  27. [35]

    Wen, T.; Wang, C.; Yang, X.; Tang, H.; Xie, Y.; Lyu, L.; Dou, Z.; and Wu, F. 2025. Defending against Indirect Prompt Injection by Instruction Detection. arXiv:2505.06311

    2025

  28. [36]

    Wong, E.; and Kolter, J. Z. 2018. Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope. In Proceedings of the 35th International Conference on Machine Learning, 5286--5295

    2018

  29. [37]

    Wong, E.; Rice, L.; and Kolter, J. Z. 2020. Fast Is Better than Free: Revisiting Adversarial Training. In International Conference on Learning Representations

    2020

  30. [38]

    Wu, F.; Cecchetti, E.; and Xiao, C. 2024. System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective. arXiv:2409.19091

    2024 arXiv

  31. [39]

    Wu, T.; Tong, L.; and Vorobeychik, Y. 2020. Defending Against Physically Realizable Attacks on Image Classification. In International Conference on Learning Representations (ICLR)

    2020

  32. [40]

    Wu, Y.; Roesner, F.; Kohno, T.; Zhang, N.; and Iqbal, U. 2025. IsolateGPT : An Execution Isolation Architecture for LLM -Based Agentic Systems. In Network and Distributed System Security Symposium (NDSS)

    2025

  33. [41]

    Yang, A.; Li, A.; Yang, B.; et al. 2025. Qwen3 Technical Report. arXiv:2505.09388

    2025 arXiv

  34. [42]

    Yi, J.; Xie, Y.; Zhu, B.; Kiciman, E.; Sun, G.; Xie, X.; and Wu, F. 2023. Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models. arXiv:2312.14197

    2023 arXiv

  35. [43]

    S.; and Kang, D

    Zhan, Q.; Fang, R.; Panchal, H. S.; and Kang, D. 2025. Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents. In Findings of the Association for Computational Linguistics: NAACL 2025

    2025

  36. [44]

    Zhan, Q.; Liang, Z.; Ying, Z.; and Kang, D. 2024. InjecAgent : Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. In Findings of the Association for Computational Linguistics: ACL 2024

    2024

  37. [45]

    Zhang, H.; Huang, J.; Mei, K.; Yao, Y.; Wang, Z.; Zhan, C.; Wang, H.; and Zhang, Y. 2025. Agent Security Bench ( ASB ): Formalizing and Benchmarking Attacks and Defenses in LLM -based Agents. In International Conference on Learning Representations (ICLR)

    2025

  38. [46]

    P.; El Ghaoui, L.; and Jordan, M

    Zhang, H.; Yu, Y.; Jiao, J.; Xing, E. P.; El Ghaoui, L.; and Jordan, M. I. 2019. Theoretically Principled Trade-off between Robustness and Accuracy. In International Conference on Machine Learning (ICML)

    2019

  39. [47]

    Zhu, C.; Cheng, Y.; Gan, Z.; Sun, S.; Goldstein, T.; and Liu, J. 2020. FreeLB : Enhanced Adversarial Training for Natural Language Understanding. In International Conference on Learning Representations (ICLR)

    2020

  40. [48]

    Zhu, K.; Yang, X.; Wang, J.; Guo, W.; and Wang, W. Y. 2025. MELON : Provable Defense Against Indirect Prompt Injection Attacks in AI Agents. arXiv:2502.05174

    2025 arXiv

  41. [49]

    Z.; and Fredrikson, M

    Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J. Z.; and Fredrikson, M. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.