REVIEW 5 cited by
Defense Against Prompt Injection Attack by Leveraging Attack Techniques
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
With the advancement of technology, large language models (LLMs) have achieved remarkable performance across various natural language processing (NLP) tasks, powering LLM-integrated applications like Microsoft Copilot. However, as LLMs continue to evolve, new vulnerabilities, especially prompt injection attacks arise. These attacks trick LLMs into deviating from the original input instructions and executing the attacker's instructions injected in data content, such as retrieved results. Recent attack methods leverage LLMs' instruction-following abilities and their inabilities to distinguish instructions injected in the data content, and achieve a high attack success rate (ASR). When comparing the attack and defense methods, we interestingly find that they share similar design goals, of inducing the model to ignore unwanted instructions and instead to execute wanted instructions. Therefore, we raise an intuitive question: Could these attack techniques be utilized for defensive purposes? In this paper, we invert the intention of prompt injection methods to develop novel defense methods based on previous training-free attack methods, by repeating the attack process but with the original input instruction rather than the injected instruction. Our comprehensive experiments demonstrate that our defense techniques outperform existing training-free defense approaches, achieving state-of-the-art results.
Forward citations
Cited by 5 Pith papers
-
Data Leakage Prevention in Agentic Applications via Preemptive Hardening
A build-time pipeline that scans, patches, and validates agentic LLM apps reduced prompt-injection leakage to 0% on most tested apps and by 91% on the hardest stress case.
-
MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents
Write-time memory filters suppress direct single-record poisoning in LLM agents (L1 BCR ≈ 4.8%) but leave compositional (L2 ≈ 22.5%) and trigger-dormant (L3 ≈ 27.8%) poisoning substantially intact across ten model families.
-
To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt
Polymorphic Prompt Assembling randomizes per-request system-prompt separators, cutting prompt-injection attack success to as low as 1.83% on GPT-3.5 with 0.06 ms runtime overhead.
-
When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review
GPT-5-mini gives weaker papers systematically higher scores than human reviewers, and hidden field-specific prompts in PDFs can force it to assign perfect scores or suppress weaknesses.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
Discussion (0). Sign in to comment.