Pith. sign in

REVIEW 24 cited by

BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.00083 v2 pith:IX7OLQV7 submitted 2024-06-03 cs.CR cs.AIcs.CLcs.IRcs.LG

Jiaqi Xue , Mengxin Zheng , Yebowen Hu , Fei Liu , Xun Chen , Qian Lou This is my paper
classification cs.CRcs.AIcs.CLcs.IRcs.LG
keywords attackspassagesadversarialllmsretrievalcustomizedgenerationgenerative
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) are constrained by outdated information and a tendency to generate incorrect data, commonly referred to as "hallucinations." Retrieval-Augmented Generation (RAG) addresses these limitations by combining the strengths of retrieval-based methods and generative models. This approach involves retrieving relevant information from a large, up-to-date dataset and using it to enhance the generation process, leading to more accurate and contextually appropriate responses. Despite its benefits, RAG introduces a new attack surface for LLMs, particularly because RAG databases are often sourced from public data, such as the web. In this paper, we propose \TrojRAG{} to identify the vulnerabilities and attacks on retrieval parts (RAG database) and their indirect attacks on generative parts (LLMs). Specifically, we identify that poisoning several customized content passages could achieve a retrieval backdoor, where the retrieval works well for clean queries but always returns customized poisoned adversarial queries. Triggers and poisoned passages can be highly customized to implement various attacks. For example, a trigger could be a semantic group like "The Republican Party, Donald Trump, etc." Adversarial passages can be tailored to different contents, not only linked to the triggers but also used to indirectly attack generative LLMs without modifying them. These attacks can include denial-of-service attacks on RAG and semantic steering attacks on LLM generations conditioned by the triggers. Our experiments demonstrate that by just poisoning 10 adversarial passages can induce 98.2\% success rate to retrieve the adversarial passages. Then, these passages can increase the reject ratio of RAG-based GPT-4 from 0.01\% to 74.6\% or increase the rate of negative responses from 0.22\% to 72\% for targeted queries.

Discussion (0). Sign in to comment.

Forward citations

Cited by 24 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Salience Induction against Multi-Hop RAG Agents: Threat and Defense

    cs.CR 2026-07 conditional novelty 7.0 of 10

    A new class of attack on retrieval-augmented AI: reordering and rephrasing true facts in documents flips an agent's reasoning steps, reaching 83.3% attack success with no false claims injected.

  2. A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG

    cs.CR 2026-07 conditional novelty 6.5 of 10

    Polymorphic sybil groups of six diverse passages amplify hijack rates 5.7× over monomorphic copies under Forced Exposure and leave 47–66% of outputs in unmonitored abstention or drift.

  3. PURPOSE: Poisoning Conflict Resolution in RAG via Proxy-Fact-Grounded Updates

    cs.CR 2026-08 conditional novelty 6.0 of 10

    A black-box poisoning attack that frames false information as a fact-compatible update defeats conflict-resolution safeguards in RAG on most tested settings.

  4. DenialRAG: Single-Document RAG Poisoning via Embedded Parametric Denial

    cs.CR 2026-08 conditional novelty 6.0 of 10

    A single poisoned document that explicitly names and refutes the correct answer can steer RAG systems toward attacker-chosen wrong answers, with effects that vary by target model.

  5. Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval

    cs.CR 2025-11 conditional novelty 6.0 of 10

    Factually correct but one-sided passages injected into a RAG database can shift LLM answers toward an attacker's stance, and a PCA/KL-based filter (BiasDef) reduces that shift.

  6. ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation

    cs.CR 2025-09 conditional novelty 6.0 of 10

    Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.

  7. Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain

    cs.IR 2025-09 conditional novelty 6.0 of 10

    Misleading health documents in RAG context sharply lower LLM accuracy, and heavily helpful-biased retrieval pools restore it.

  8. Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection

    cs.CL 2025-07 conditional novelty 6.0 of 10

    GMTP detects poisoned RAG documents by masking high-gradient tokens and flagging documents whose masked-token probabilities are abnormally low.

  9. RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation

    cs.CR 2025-07 conditional novelty 6.0 of 10

    Inserting a small number of crafted triples into a knowledge graph can flip KG-RAG question answering toward attacker-chosen incorrect answers across four recent systems.

  10. The Hidden Threat in Plain Text: Attacking RAG Data Loaders

    cs.CR 2025-07 conditional novelty 6.0 of 10

    Invisible characters and formatting tricks in ingested documents survive popular RAG data loaders and can manipulate end-to-end RAG outputs.

  11. Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs

    cs.LG 2025-06 reject novelty 6.0 of 10

    A retrieval-augmented generation system can be poisoned with reward-optimized biased documents and vector-space manipulation to substantially increase biased LLM outputs.

  12. Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLM

    cs.CR 2025-05 conditional novelty 6.0 of 10

    Spa-VLM crafts paired adversarial images and misleading texts to poison RAG-based VLM knowledge bases, reaching attack success rates above 0.8 with just five injected entries.

  13. The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems

    cs.IR 2025-05 conditional novelty 6.0 of 10

    A reinforcement-learning attack that swaps a few words in a single document pushes it into a black-box RAG system's top-3 results and flips the generated answer around 45-47% of the time.

  14. Safety Degradation in AI Agents

    cs.CY 2025-05 conditional novelty 6.0 of 10

    Adding retrieval to aligned LLMs degrades safety: refusal rates fall, bias and harmfulness rise, and prompt-based mitigation only partially restores alignment.

  15. Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

    cs.CR 2026-07 conditional novelty 5.0 of 10

    A role-layer survey unifies LLM misuse, LLM-based defense, and LLM-centric verification vulnerabilities across content, social, evidence, and workflow layers, then lists three open challenges.

  16. Agents at Risk: How Users Unwittingly Undermine LLM Safety

    cs.CR 2026-01 conditional novelty 5.0 of 10

    Commercial AI agents routinely trust user-relayed unverified content and execute risky actions unless the user explicitly demands a safety check.

  17. Fact or Facsimile? Evaluating the Factual Robustness of Modern Retrievers

    cs.IR 2025-08 conditional novelty 5.0 of 10

    Retrievers and rerankers built from LLMs score near random on the FACTOR factuality benchmark, far below their base models, and fail when correct answers are paraphrased.

  18. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  19. DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injection

    cs.AI 2025-07 conditional novelty 5.0 of 10

    DeRAG shows that five or fewer tokens found by differential evolution can make black-box RAG retrievers rank a chosen wrong document near the top on small BEIR subsets.

  20. Benchmarking Poisoning Attacks against Retrieval-Augmented Generation

    cs.CR 2025-05 conditional novelty 5.0 of 10

    A unified benchmark evaluation finds that existing RAG poisoning attacks remain effective on standard QA datasets, drop on expanded knowledge bases, and are only partially mitigated by current defenses.

  21. Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

    cs.CL 2025-05 reject novelty 5.0 of 10

    METEORA uses DPO-tuned rationales to select and verify evidence chunks in RAG, and claims better recall, precision, evidence efficiency, and poisoning defense, though key evaluation details are missing.

  22. Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval

    cs.CR 2025-05 conditional novelty 5.0 of 10

    SCR uses retrieval-augmented generation to fetch refusal examples that block jailbreak attacks, but the reported advantages are partly overstated.

  23. Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systems

    cs.CR 2025-05 conditional novelty 4.0 of 10

    A threat-modeling review of RAG systems that prioritizes mitigations against prompt injection, data poisoning, and query manipulation using the AI Security Pyramid of Pain.

  24. RAG Security and Privacy: Formalizing the Threat Model and Attack Surface

    cs.CR 2025-09 conditional novelty 3.0 of 10

    A formal RAG threat model is defined with four adversary classes and game-based notions of membership inference, leakage, and poisoning, but the definitions largely restate known concepts and the main DP-based protect...

Pith tools