REVIEW 21 cited by
Automatic and Universal Prompt Injection Attacks against Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) excel in processing and generating human language, powered by their ability to interpret and follow instructions. However, their capabilities can be exploited through prompt injection attacks. These attacks manipulate LLM-integrated applications into producing responses aligned with the attacker's injected content, deviating from the user's actual requests. The substantial risks posed by these attacks underscore the need for a thorough understanding of the threats. Yet, research in this area faces challenges due to the lack of a unified goal for such attacks and their reliance on manually crafted prompts, complicating comprehensive assessments of prompt injection robustness. We introduce a unified framework for understanding the objectives of prompt injection attacks and present an automated gradient-based method for generating highly effective and universal prompt injection data, even in the face of defensive measures. With only five training samples (0.3% relative to the test data), our attack can achieve superior performance compared with baselines. Our findings emphasize the importance of gradient-based testing, which can avoid overestimation of robustness, especially for defense mechanisms.
Forward citations
Cited by 21 Pith papers
-
Understanding the Supply Chain and Risks of Large Language Model Applications
A new benchmark dataset traces dependencies across 3,859 LLM applications, 109,211 models, 2,474 datasets, and 8,862 libraries, and finds widespread known vulnerabilities in application dependencies.
-
When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
Prompt injection can hijack multi-agent LLM robot planners, spread from an injected agent to clean teammates through shared prompts, and partially survives a per-agent separation defense via shared memory.
-
From Neural Intent to Cryptographic Authorization: Securing AI-Driven Enterprise Workflows
NCS gates every privileged AI-agent tool call behind an offline-signed, hash-chained instruction stream, cutting injected-parameter attacks to zero by construction.
-
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Special tokens that structure LLM conversations can be injected and swapped for lookalike words to bypass both built-in safety and external content filters.
-
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization
RAF, a two-stage token-optimization attack, creates brief natural-sounding text injections that reliably boost a target item's rank in LLM reranking outputs, beating state-of-the-art baselines in effectiveness, stealt...
-
Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting
An early-exit rule with a zero-shot fallback, calibrated by Learn-then-Test risk control, keeps the average loss from corrupted in-context demonstrations under a preset bound.
-
Commenotes: Synthesizing Organic Comments to Support Community-Based Fact-Checking
A filtering plus LLM synthesis pipeline turns early user comments into community-note-style fact-checks; the best model beats human notes in 70.1% of pairwise user ratings.
-
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.
-
UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generation
A universal knowledge-corruption attack uses as few as 100 crafted texts to hijack responses to thousands of diverse user queries in retrieval-augmented generation.
-
Defending Against Prompt Injection With a Few DefensiveTokens
Five optimized token embeddings prepended to the prompt reduce prompt-injection attack success to near zero on standard benchmarks while preserving most model utility.
-
Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
Function Calling and MCP architectures show distinct vulnerability patterns, with chained attacks succeeding 91-96% of the time in both.
-
The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
AI safety should be measured by whether deployed systems keep errors visible, contestable, containable, and recoverable across five integrity layers, not only by whether individual model outputs look safe.
-
Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devices
Physical scene text can inject prompts into wearable VLMs, hijacking decisions and content with high success rates across six threat scenarios, partially mitigated by OCR masking and token-drift defenses.
-
Lexical Hints of Accuracy in LLM Reasoning Chains
Hesitation words in reasoning chains are claimed to flag incorrect LLM answers, but the manuscript body is a different paper and contains no such study.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Can You Trick the Grader? Adversarial Persuasion of LLM Judges
Strategically inserted persuasive sentences inflate LLM judges' scores for incorrect math solutions across six benchmarks and fourteen models, but the study lacks length-matched controls separating rhetoric from lengt...
-
MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models
MedMKEB is a medical multimodal knowledge-editing benchmark with four task types, on which existing editing methods underperform according to the authors.
-
DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injection
DeRAG shows that five or fewer tokens found by differential evolution can make black-box RAG retrievers rank a chosen wrong document near the top on small BEIR subsets.
-
Prompt-in-Content Attacks: Exploiting Uploaded Inputs to Hijack LLM Behavior
Embedding a short 'system instruction' in a .docx file causes several commercial LLMs to refuse, substitute, redirect, or bias their output during summarization tasks.
-
Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems
Multi-stage prompt inference attacks against enterprise LLMs are formalized and defenses are proposed, but the preprint gives no reproducible evidence for its central claims.
-
Prompt Injection 2.0: Hybrid AI Threats
A structured taxonomy of hybrid prompt injection attacks shows how XSS, CSRF, and SQL injection vectors converge with LLM manipulation to bypass traditional controls.
Discussion (0). Sign in to comment.