Pith. sign in

REVIEW 10 cited by

Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.14110 v2 pith:3NRKBP46 submitted 2024-11-21 cs.CR

Changyue Jiang , Xudong Pan , Geng Hong , Chenfu Bao , Yang Chen , Min Yang This is my paper · ORCID
classification cs.CR
keywords extractionknowledgebaseapplicationsblack-boxadversarycopybreakragcoverage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) by integrating external knowledge bases, whose construction is often time-consuming and laborious. If an adversary extracts the knowledge base verbatim, it not only severely infringes the owner's intellectual property but also enables the adversary to replicate the application's functionality for unfair competition. Previous works on knowledge base extraction are limited either by low extraction coverage (usually less than 4%) in query-based attacks or by impractical assumptions of white-box access in embedding-based optimization methods. In this work, we propose CopyBreakRAG, an agent-based black-box attack that reasons from feedback and adaptively generates new adversarial queries for progressive extraction. By balancing exploration and exploitation through curiosity-driven queries and feedback-guided query refinement, our method overcomes the limitations of prior approaches and achieves significantly higher extraction coverage in realistic black-box settings. Experimental results show that CopyBreakRAG outperforms the state-of-the-art black-box approach by 45% on average in terms of chunk extraction ratio from applications built with mainstream RAG frameworks, and extracts over 70% of the data from the knowledge base in applications on commercial platforms including OpenAI's GPTs and ByteDance's Coze when essential protection is in place.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents

    cs.CR 2026-07 conditional novelty 7.5 of 10

    Malicious tools can systematically extract isolated LLM-agent long-term memory via persistence, pure-anchor retrieval steering, and reactivation payloads, reaching 80% extraction with unlimited triggers and 47% with 20.

  2. External Data Extraction Attacks against Retrieval-Augmented Large Language Models

    cs.CR 2025-10 conditional novelty 7.0 of 10

    SECRET, an adaptive RAG extraction attack that fuses LLM-optimized jailbreak prompts with cluster-guided queries, extracts large portions of private databases from commercial and open LLMs, including ~35% of a sampled...

  3. AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems

    cs.CR 2026-03 conditional novelty 6.0 of 10

    A single adversarial message can persistently hijack an LLM agent's configuration and make it autonomously spread the same attack to other agents, succeeding in 64.5% of controlled trials.

  4. Graphs Don't Stay Secret: Practical Subgraph Reconstruction Attacks on Defended Graph RAG

    cs.CR 2026-02 conditional novelty 6.0 of 10

    GRASP, a closed-box multi-turn prompt attack, reconstructs type-faithful one-hop subgraphs from defended Graph RAG systems, reaching up to 82.9 F1 where prior attacks fail.

  5. DCMI: A Differential Calibration Membership Inference Attack Against Retrieval-Augmented Generation

    cs.CR 2025-09 conditional novelty 6.0 of 10

    DCMI infers RAG database membership by subtracting the system's yes-probability on a perturbed query from the original query, cancelling the interference of non-member retrieved documents.

  6. Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning

    cs.CR 2025-02 reject novelty 6.0 of 10

    RAGC⃝ watermarks RAG knowledge bases by injecting benign chain-of-thought passages that only watermarked queries retrieve, enabling ownership verification via a Wilcoxon test on whether the generated reasoning contain...

  7. Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation

    cs.CR 2026-02 conditional novelty 5.0 of 10

    A unified benchmark comparing RAG knowledge-extraction attacks and defenses, showing query diversity boosts extraction, embedding attacks fail to transfer, and graph indexing raises per-token leakage.

  8. Benchmarking Poisoning Attacks against Retrieval-Augmented Generation

    cs.CR 2025-05 conditional novelty 5.0 of 10

    A unified benchmark evaluation finds that existing RAG poisoning attacks remain effective on standard QA datasets, drop on expanded knowledge bases, and are only partially mitigated by current defenses.

  9. SPARK: Search Personalization via Agent-Driven Retrieval and Knowledge-sharing

    cs.AI 2025-12 conditional novelty 4.0 of 10

    SPARK is a proposed framework that coordinates persona-based LLM agents with layered memory and three collaboration protocols to produce personalized search.

  10. Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey

    cs.CL 2025-02 conditional novelty 4.0 of 10

    A survey organizes current research on trustworthy RAG into six pillars, reliability, privacy, safety, fairness, explainability, and accountability, and maps methods, metrics, and open problems for each.

Pith tools