REVIEW 18 cited by
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have achieved remarkable success due to their exceptional generative capabilities. Despite their success, they also have inherent limitations such as a lack of up-to-date knowledge and hallucination. Retrieval-Augmented Generation (RAG) is a state-of-the-art technique to mitigate these limitations. The key idea of RAG is to ground the answer generation of an LLM on external knowledge retrieved from a knowledge database. Existing studies mainly focus on improving the accuracy or efficiency of RAG, leaving its security largely unexplored. We aim to bridge the gap in this work. We find that the knowledge database in a RAG system introduces a new and practical attack surface. Based on this attack surface, we propose PoisonedRAG, the first knowledge corruption attack to RAG, where an attacker could inject a few malicious texts into the knowledge database of a RAG system to induce an LLM to generate an attacker-chosen target answer for an attacker-chosen target question. We formulate knowledge corruption attacks as an optimization problem, whose solution is a set of malicious texts. Depending on the background knowledge (e.g., black-box and white-box settings) of an attacker on a RAG system, we propose two solutions to solve the optimization problem, respectively. Our results show PoisonedRAG could achieve a 90% attack success rate when injecting five malicious texts for each target question into a knowledge database with millions of texts. We also evaluate several defenses and our results show they are insufficient to defend against PoisonedRAG, highlighting the need for new defenses.
Forward citations
Cited by 18 Pith papers
-
Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation
Sparse attention changes which prompt content influences model answers, an effect that can cancel out in aggregate accuracy and is visible through a dense-calibrated counterfactual audit.
-
GPE: Evaluating Robust Evidence Aggregation for Fact Verification under Controllable GEO-Style Poisoning
GPE, a new benchmark with controllable GEO-style poisoning, shows LLM fact verifiers degrade sharply under poisoned evidence, with no single verifier winning across all attack types.
-
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
C3R certifies per-domain retrieval contamination budgets using a two-split conformal scheme, without query-time domain labels.
-
Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval
Factually correct but one-sided passages injected into a RAG database can shift LLM answers toward an attacker's stance, and a PCA/KL-based filter (BiasDef) reduces that shift.
-
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.
-
Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection
GMTP detects poisoned RAG documents by masking high-gradient tokens and flagging documents whose masked-token probabilities are abnormally low.
-
RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation
Inserting a small number of crafted triples into a knowledge graph can flip KG-RAG question answering toward attacker-chosen incorrect answers across four recent systems.
-
The Hidden Threat in Plain Text: Attacking RAG Data Loaders
Invisible characters and formatting tricks in ingested documents survive popular RAG data loaders and can manipulate end-to-end RAG outputs.
-
Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs
A retrieval-augmented generation system can be poisoned with reward-optimized biased documents and vector-space manipulation to substantially increase biased LLM outputs.
-
TriShieldRAG: A Three-Ring Defense-in-Depth Framework Against Knowledge Corruption in Retrieval-Augmented Generation
A three-ring RAG defense cuts non-adaptive PoisonedRAG attack success from ~91% to ~13% on 10 questions over a 5,000-document Wikipedia base.
-
Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows
A survey of 49 LLM fraud and trust-and-safety papers finds that fraud work reports almost no per-decision latency, cost, or calibration evidence, while moderation work reports more.
-
Lexical Hints of Accuracy in LLM Reasoning Chains
Hesitation words in reasoning chains are claimed to flag incorrect LLM answers, but the manuscript body is a different paper and contains no such study.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision
A label-free method that scores retrieved documents by their agreement with the majority in embedding space and uses those scores to filter context in LLM question answering.
-
We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems
MCP-powered LLM agents are vulnerable to prompt injection from third-party services, and simple detection or filtering defenses do not reliably stop these attacks.
-
Quantifying Conversation Drift in MCP via Latent Polytope
SecMCP flags MCP conversation drift by thresholding per-layer activation distances from benign anchors, reporting AUROC above 0.915 on Llama3, Vicuna, and Mistral.
-
PRJ: Perception-Retrieval-Judgement for Generated Images
A new safety checker for AI-generated images, built from a vision-language model, retrieval-augmented knowledge lookup, and an LLM judge, reports higher detection rates and category-level toxicity scores than three ex...
-
Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
A vision paper organizing agentic AI security into eleven research directions, with the thesis that safety requires verifiable trajectory-level behavioral guarantees, not per-action checks.
Discussion (0). Sign in to comment.