Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

Christian Di Maio; Cristian Cosci; Marco Maggini; Stefano Melacci; Valentina Poggioni

REVIEW 3 major objections 5 minor 2 cited by

Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An adaptive, fully black-box attack can extract the majority of a RAG system's private knowledge base using only open-source tools.

desk verdict A genuinely new adaptive black-box RAG extraction attack with useful comparisons, but the 'majority extraction' headline depends on a permissive leakage metric and needs a stricter pass before the numbers are trusted. read the letter →

arxiv 2412.18295 v2 pith:MLQXUKTX submitted 2024-12-24 cs.AI

Christian Di Maio , Cristian Cosci , Marco Maggini , Valentina Poggioni , Stefano Melacci This is my paper · ORCID

classification cs.AI

keywords Retrieval-AugmentedGenerationpromptinjectionknowledgebaseextractionblack-boxattackLLMsecurityprivacyleakageadaptive

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that a Retrieval-Augmented Generation (RAG) system's private knowledge base can be extracted by an attacker who has no access to the system's internals and uses only openly available tools. The proposed attack is adaptive: it maintains a growing set of topic 'anchors' with relevance scores, generates a new query from the most promising anchors plus a prompt-injection command, and repeats until no anchor yields new information. In unbounded runs the attack extracts the large majority of each of three hidden knowledge bases, with navigation coverage and leaked-knowledge scores around 96%, 90%, and 94% depending on the agent. The authors argue this demonstrates an urgent need for safeguards in RAG deployment because current defenses, including a guard model they test, do not reliably stop the leakage.

What carries the argument

The load-bearing object is the anchor set $A_t$ paired with relevance scores $R_t$: anchors are short topic phrases extracted by an attacker-side LLM from each newly stolen chunk, and their relevance scores are updated so that anchors that keep returning duplicate chunks lose influence while new anchors inherit the current maximum relevance. This relevance-weighted sampling of anchors balances exploration and exploitation, and the duplicate check runs in the embedding space of an attacker-side encoder with a similarity threshold. Around this core, the algorithm wraps a small pool of prompt-injection commands and a parsing step that extracts chunk text from the target's verbatim output; the whole loop stops only when no anchor has positive relevance.

What would settle it

Run the same three agents with an output filter that paraphrases every generated response or an input filter that strips imperative 'copy the context' commands; if the inner loop of the algorithm returns no parseable chunks and leaked knowledge stays near zero over thousands of queries, the claim of majority extraction is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that black-box RAG knowledge theft can be fully automated and made adaptive without any prior knowledge of the target embedder, retriever, or generator. The algorithm starts from one ordinary word and repeatedly injects commands that ask the target to copy the retrieved context verbatim; each newly stolen chunk contributes anchors, and relevance scores steer later queries toward unexplored regions of the hidden knowledge base while penalizing anchors that only return duplicates. The stopping condition is intrinsic—when every anchor has zero relevance—so the attack decides for itself when the knowledge base has been drained. Across three RAG agents built with different language models, embedders, and knowledge domains, the unbounded attack reaches 95.9/95.8, 89.8/78.8, and 94.3/88.8 for navigation coverage and leaked knowledge, which the authors read as extracting the majority of each private knowledge base.

Load-bearing premise

The central premise is that the target RAG system will obey a small set of prompt-injection commands and emit retrieved chunks nearly verbatim, without any input filtering, output sanitization, or instruction-tuned refusal.

Editorial extensions

If this is right

A RAG system that faithfully reproduces retrieved context is exfiltratable in full by an automated routine, not just by hand-crafted prompts.
Fixed-budget attacks leave much of the knowledge base untouched; an adaptive attacker with an intrinsic stopping rule covers regions a static prompt set never reaches.
The attack transfers across different RAG pipelines (different generator LLMs, embedders, and domains) and can be run with a small open-source LLM and embedder on domestic hardware.
A guard LLM that screens inputs and outputs for safety does not reliably separate attack queries from normal user queries, and over-blocks legitimate interactions in specialized-advice domains.
Because query generation is cheap and fast, the bottleneck for an attacker is not compute but whether the target's output obeys the injection commands.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same relevance-anchor loop could be repurposed for targeted extraction: seeding the anchor set with a desired topic should steer the attack toward a specific region of the hidden knowledge base rather than maximizing global coverage.
Beyond the paper, the attack's success boundary is likely sharp: it should fail on any RAG whose generator is trained or prompted to paraphrase retrieved context instead of copying it, or that uses any input filter that rejects or neutralizes imperative 'copy all text' phrasing.
Beyond the paper, a natural defensive experiment suggested by the results is to measure leaked knowledge under output paraphrasing or selective verbatim suppression; the paper's own limitation section indicates such safeguards would break the inner loop of the algorithm.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague—

Read Pirates of the RAG. The new thing is real: instead of static question lists or attacks that need to know the target embedder, the authors run an anchor set whose relevance scores decay when chunks come back duplicated, and they let the attack stop on its own. The attacker side is entirely open-source and cheap (a 1B LLM, a free embedder), and the measured query-generation time is about 13 seconds versus minutes for DGEA. That is a meaningful advance over TGTB/PIDE/DGEA/RThief, and the comparison table includes the right competitors plus modified versions that strip unfair advantages. Credit also where due: the bounded experiments early-stop their own algorithm to match 300 attacks, and the unbounded table includes early-stopped variants to control for query count. The limitations section is honest, including the point that the leakage comparison "could be made more strict."

The soft spots are real but not ruinous. The biggest one is the LK metric. Calling a private chunk leaked when any stolen chunk reaches ROUGE-L >= 0.5, after soft-matching through an attacker-side embedder, is generous. A superset, a concatenation, or a loose paraphrase can clear that bar, and Nav only says the retriever surfaced the chunk, not that the LLM emitted it. The authors admit this in Section 7. So the claim that the attack "can extract the majority of the private knowledge base" is plausible but not nailed down by Table 3 as reported. I would want a stricter leakage definition—higher threshold or exact/near-exact text containment—plus repeated runs and error bars on the headline numbers. There is also no released code, the chunk parsing is manually tailored per agent, and the thresholds (beta, alpha_1, alpha_2, n) are manually set; they discuss this, but it means the ubiquity claim is weaker than the architecture claim.

Do I think the paper is sound otherwise? Yes. The attack mechanism is coherent, the comparison is fair in its design, and the failure of the Guardian model to distinguish attack from normal input is a useful datapoint. The central qualitative result—a black-box open-source attacker can systematically drain an unsafeguarded RAG—survives the metric concern. It is exactly the kind of result that should go to review rather than be desk-rejected, because the fix is a stricter evaluation, not a different idea. If I were the editor, I would send it out, with a strong request for code, error bars, and a more conservative LK metric.

For the reading group, maybe—it will generate discussion on whether ROUGE-L 0.5 is the right leakage notion, and that is a useful conversation.

Referee Report

3 major / 5 minor

Summary. The paper proposes 'Pirates of the RAG,' a black-box, adaptive attack that extracts a private knowledge base from a Retrieval-Augmented Generation (RAG) system. The attacker uses an open-source embedder and LLM, maintains a set of anchors with relevance scores, and iteratively crafts queries with injection commands to induce the target RAG to output retrieved chunks verbatim. The method is evaluated on three RAG agents (medical, educational, research) in both bounded (300 attacks) and unbounded (run until stopping criterion) settings, using metrics Nav (navigation coverage), LK (leaked knowledge), LC, and ULC. The main claims are that the attack is automatic and adaptive, that it outperforms recent baselines (TGTB, PIDE, DGEA, RThief) in bounded comparisons, and that in the unbounded setting it 'can extract the majority of the private knowledge base,' with LK values of 95.8, 78.8, and 88.8 for Agents A, B, and C, respectively.

Significance. If the reported results hold, this is a significant contribution to RAG security. The attack is notable for using only open-source, locally runnable models, for its adaptive anchor-relevance mechanism, and for its strong performance relative to baselines in the unbounded setting. The paper also includes a useful, if preliminary, analysis of a Guardian-LLM defense (Appendix H). The main strengths are the clear threat model, the algorithmic formulation in Algorithm 1, and the early-stopped comparisons that partially mitigate the query-count imbalance in the unbounded setting. However, the central quantitative claim depends on a leakage metric whose strictness is questionable, and the absence of repeated runs leaves the comparative claims without error bars.

major comments (3)

[Section 5, Metrics; Table 3; Section 7] The headline claim that the attack extracts the majority of the private knowledge base rests on the LK metric, which counts a chunk x as leaked if there exists a stolen chunk x* with ROUGE-L(x, x*) >= 0.5 after soft-matching x* to its nearest private chunk in the attacker-side embedding space. A ROUGE-L threshold of 0.5 is permissive: a stolen output that is a superset of x, a rough paraphrase, or a noisy concatenation of several retrieved chunks can clear it, and the embedding-based nearest-neighbor pairing can match semantically close but only partially overlapping texts. The authors acknowledge in Section 7 that 'other solutions could be considered to make this analysis more strict.' To support the stated claim, please report leakage under stricter thresholds (e.g., ROUGE-L >= 0.7 or 0.8, or exact/containment match) and without the soft-matching step, or provide a principled justification for why 0.5 is the correct threshold for the 'majority extraction' claim. Without this, the LK values in Table 3 may overstate the amount of text actually leaked.
[Tables 2 and 3 (Experiments)] No repeated runs or error bars are reported for the Nav and LK metrics in the main comparisons. The bounded results in Table 2 include close calls (e.g., Pirate's LK of 20.1 is second to PIDE's 20.6 on Agent B), and the early-stopped unbounded row for Agent B shows Pirate-RThief LK of 22.3 versus RThief's 30.9. Without multiple random seeds (varying initial anchors, dataset subsamples, or sampling randomness in relevance-based anchor sampling), it is impossible to assess whether these gaps are meaningful or within run-to-run variability. Please report means and standard deviations (or confidence intervals) over at least three independent runs for all main tables.
[Section 3.1, Algorithm 1; Appendix C] The paper describes the attack as 'automatic' and 'fully-automated,' but the parsing step requires manually crafted rules tailored to each agent's output format: Appendix C states that 'we design parsing rules specifically tailored to the generated text,' and Algorithm 1 lists 'estimated structure of the RAG system output' as a required input. This manual bootstrap is a significant human-in-the-loop component. Please clarify exactly what human effort is needed before the automatic loop can run and whether the 'automatic' claim applies only to the inner loop. Providing the actual parsing rules used for each agent (or a generic parser) would also improve reproducibility.

minor comments (5)

[Abstract] The phrase 'It proposes and adversarial untargeted attack' contains a typo ('and' should be 'an').
[Section 3.1, Duplicates paragraph] The notation in the duplicate-checking description is inconsistent: 'sim(xz, st,j)' should likely be 'sim(x_z, s_{t,j})' with the embedded chunk representation, and the text 'all the attacker-side embeddings are always indented to be computed' should read 'intended.'
[Figure 1 caption] The caption says 'pieces information' and should read 'pieces of information.'
[Section 7, Limitations] The second limitation bullet contains a grammatical error: 'The proposed algorithm only consider the amount of leaked knowledge' should be 'considers.'
[Appendix H] The Guardian-LLM analysis is interesting but appears only in an appendix and is not connected to the main narrative. Consider adding a short summary in the conclusions or in the limitations section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack is evaluated against external hidden knowledge bases and no reported quantity is fitted into the metric that measures it.

full rationale

The paper's central claim is empirical: an adaptive, black-box attack (Algorithm 1) leaks most of a hidden RAG knowledge base. The attack loop is internally self-referential (anchors are extracted from stolen chunks and relevance scores are updated from duplicate detections), but that is an algorithm design, not a derivation that reduces a predicted quantity to the attack's own inputs. The reported Nav and LK scores are computed against the private KB K, which the attacker never observes: Nav is the fraction of K retrieved at least once in the target's top-k, and LK requires a stolen chunk x* to have ROUGE-L(x, x*) >= 0.5 against its embedding-matched nearest neighbor in K (Section 5, footnote 10). No parameter of the attack (beta=1, alpha1=0.95, alpha2=0.8, n=3) is fitted to these metrics; the thresholds are fixed before the runs. The only self-citation (Di Maio et al., 2024) appears in a footnote about the generality of RAG and is not load-bearing for any result. The skeptical concern that the LK metric is permissive (ROUGE-L threshold 0.5 plus embedding-based soft matching can inflate leakage) is a metric-validity/correctness issue, not circularity; the paper itself flags it in Section 7: 'other solutions could be considered to make this analysis more strict.' For the same reason, the 'majority of the private knowledge base' claim depends on the strictness of LK, but that does not make the claim equivalent to the paper's inputs by construction.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two classes of premises: user-chosen attack hyperparameters (beta, alpha_1, alpha_2, n, initial anchor, ROUGE threshold) and behavioral assumptions about the target RAG system (it obeys injection commands, has no effective output filter, and semantic similarity computed by the attacker's embedder aligns with the hidden retriever). The paper introduces no new physical entities; anchors and relevance scores are internal algorithmic state.

free parameters (6)

beta = 1
Initial relevance of the first anchor; controls how many times an anchor can return duplicates before dying, and therefore the stopping time.
alpha_1 = 0.95
Cosine similarity threshold for marking stolen chunks as duplicates; directly changes ULC and LK counts.
alpha_2 = 0.8
Cosine similarity threshold for discarding duplicate anchors; affects exploration diversity.
n = 3
Number of anchors sampled to build each query; balances exploration and exploitation.
initial anchor a = a common word in the target language (not specified)
Starting topic for the first query; the paper does not report which word was used for each agent.
ROUGE-L leakage threshold = 0.5
Criterion borrowed from prior literature for counting a chunk as leaked; with soft matching via the attacker embedder, it is a permissive threshold.

assumptions (3)

domain assumption The target RAG LLM will comply with at least one of the four prompt-injection commands and copy the retrieved context verbatim.
Algorithm 1 inner loop, Appendix B Table 6; if false the attack extracts nothing.
domain assumption The attacker-side embedder e* can approximate the semantic structure of the hidden RAG knowledge enough to detect duplicates and relevant anchors.
Sections 3.1 (duplicates, anchor updates) and 5 hyperparameters; not verified against the hidden embedder.
domain assumption A stolen chunk counts as leaked when ROUGE-L >= 0.5 against its nearest embedding-space match in the hidden KB.
Section 5 Metrics; threshold follows Zeng et al. but the soft matching via e* is new and can inflate LK.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases." pith.science (2026). https://pith.science/paper/MLQXUKTX

@misc{pith2026241218295,
  author       = {Pith},
  title        = {Pith review of: Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MLQXUKTX}},
  note         = {Machine review of arXiv:2412.18295}
}

read the original abstract

The growing ubiquity of Retrieval-Augmented Generation (RAG) systems in several real-world services triggers severe concerns about their security. A RAG system improves the generative capabilities of a Large Language Models (LLM) by a retrieval mechanism which operates on a private knowledge base, whose unintended exposure could lead to severe consequences, including breaches of private and sensitive information. This paper presents a black-box attack to force a RAG system to leak its private knowledge base which, differently from existing approaches, is adaptive and automatic. A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base. Extensive experimentation proves the quality of the proposed algorithm in different RAG pipelines and domains, comparing to very recent related approaches, which turn out to be either not fully black-box, not adaptive, or not based on open-source models. The findings from our study remark the urgent need for more robust privacy safeguards in the design and deployment of RAG systems.

Figures

Figures reproduced from arXiv: 2412.18295 by the authors.

**Figure 3.** Pale: number of extracted chunks (LC metric) [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

**Figure 2.** Evolution of anchor set At during the (unbounded) attack procedure of Algorithm 1. Dashed curves are about anchors with zero relevance (dead anchors). dition. Further analysis on the unbounded setting can be found in Appendix F. In-depth Studies. In order to inspect the behaviors of the anchor set and of the relevance mechanisms during the attack procedure, in [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Distribution of adversarial query (query+command) lengths, measured in the number of words, generated [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

**Figure 5.** Figure 5: Distribution of the cosine-similarity scores between the top- [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Unbounded analysis. Top-Left, Top-Right, and Bottom-Left: Overview of the evolution of ULC as the number of attacks increases for Agents A, B, and C, respectively. The vertical dashed lines indicate the point at which the corresponding method automatically stops. Botto…

**Figure 7.** Figure 7: Left: prompt template for generating a certain text conditioned by a set of anchors. Right: prompt template for extracting a set of anchors related to a given text. Agent Scenario Text Safe Unsafe A No Attack Input 60 (20.00%) 240 (80.00%) Output 7 (02.33%) 293 (97.67%…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
cs.CR 2025-09 conditional novelty 6.0 of 10

A systematic review that categorizes LLM threats, severity scores, and mitigations across development and operation life cycles and multiple deployment scenarios.
Provably Secure Retrieval-Augmented Generation
cs.CR 2025-08 reject novelty 2.0 of 10

SAG encrypts RAG knowledge bases and claims formal security, but its proofs are flawed and its benchmarks guarantee zero attack success by design.

Reference graph

Works this paper leans on

71 extracted references · 24 canonical work pages · cited by 2 Pith papers

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

arXiv 2024
[4]

AI@Meta. 2024 a . https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

work page 2024
[5]

AI@Meta. 2024 b . https://huggingface.co/meta-llama/Llama-3.2-1B Llama 3.2 1b model card

work page 2024
[6]

Maya Anderson, Guy Amit, and Abigail Goldsteen. 2024. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation. arXiv preprint arXiv:2405.20446

arXiv 2024
[7]

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023

arXiv 2023
[8]

Vani Bhat, Divya Sree, Jinu Cheerla, Nupur Mathew, Gunna LIu, and Jerry Gao. 2024. Retrieval augmented generation (rag) based restaurant chatbot with ai testability

work page 2024

Show all 71 references

[9]

Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

2020 arXiv
[10]

Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897--1914. IEEE

2022
[11]

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21),...

2021
[12]

Antonio Emanuele Cin \`a , Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, and Fabio Roli. 2023. Wild patterns reloaded: A survey of machine learning security against training data poisoning...

2023
[13]

Stav Cohen, Ron Bitton, and Ben Nassi. 2024. Unleashing worms and extracting data: Escalating the outcome of attacks against rag-based inference in scale and severity using jailbreaking. arXiv preprint arXiv:2409.08045

2024 arXiv
[14]

Adam Cutbill, Eric Monsler, and Eric Hayashi. 2024. Personalized home assistant using large language model with context-based chain of thought reasoning

2024
[15]

Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale s Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3...

2021
[16]

Christian Di Maio, Andrea Zugarini, Francesco Giannini, Marco Maggini, and Stefano Melacci. 2024. Tomorrow brings greater knowledge: Large language models join dynamic temporal knowledge graphs. In Conference on Lifelong Learning Agents, CoLLAs 2024, 29 July-1 August 2024, Uni...

2024
[17]

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

2022 arXiv
[18]

Haonan Duan, Adam Dziedzic, Mohammad Yaghini, Nicolas Papernot, and Franziska Boenisch. 2023. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics

2023
[19]

Martin Ester, Hans-Peter Kriegel, J\" o rg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD'96, page 226–231...

1996
[20]

Silvia Garc \' a-M \'e ndez, Francisco de Arriba-P \'e rez, and Mar \' a del Carmen Somoza-L \'o pez. 2024. A review on the use of large language models as virtual tutors. Science & Education, pages 1--16

2024
[21]

Abenezer Golda, Kidus Mekonen, Amit Pandey, Anushka Singh, Vikas Hassija, Vinay Chamola, and Biplab Sikdar. 2024. Privacy and security concerns in generative ai: A comprehensive survey. IEEE Access

2024
[22]

Kathrin Grosse, Lukas Bieringer, Tarek R Besold, Battista Biggio, and Katharina Krombholz. 2023. Machine learning security in industry: A quantitative survey. IEEE Transactions on Information Forensics and Security, 18:1749--1762

2023
[23]

Max Grusky. 2023. Rogue scores. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1914--1934

2023
[24]

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929--3938. PMLR

2020
[25]

Sorami Hisamoto, Matt Post, and Kevin Duh. 2020. Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system? Transactions of the Association for Computational Linguistics, 8:49--63

2020
[26]

Matthew Honnibal and Ines Montani. 2017. spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing. To appear

2017
[27]

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 a . https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

2022
[28]

Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. 2022 b . Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54(11s):1--37

2022
[29]

watching-eye

Yaou Hu and Hyounae Kelly Min. 2023. The dark side of artificial intelligence in service: The “watching-eye” effect and privacy concerns. International Journal of Hospitality Management, 110:103437

2023
[30]

Yangsibo Huang, Samyak Gupta, Zexuan Zhong, Kai Li, and Danqi Chen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.921 Privacy implications of retrieval-based language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages ...

2023 doi
[31]

Changyue Jiang, Xudong Pan, Geng Hong, Chenfu Bao, and Min Yang. 2024 a . Rag-thief: Scalable extraction of private data from retrieval-augmented generation applications with agent-based attacks. arXiv preprint arXiv:2411.14110

2024 arXiv
[32]

Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024 b . A survey on large language models for code generation. arXiv preprint arXiv:2406.00515

2024 arXiv
[33]

Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei. 2023. Evaluating open-domain question answering in the era of large language models. arXiv preprint arXiv:2305.06984

2023 arXiv
[34]

u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \

Enkelejda Kasneci, Kathrin Se ler, Stefan K \"u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \"u nnemann, Eyke H \"u llermeier, et al. 2023. Chatgpt for good? on opportunities and challenges of large language models for education....

2023
[35]

Gonzalez, Hao Zhang, and Ion Stoica

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

2023
[36]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

2020
[37]

Qian Li, Hao Peng, Jianxin Li, Congying Xia, Renyu Yang, Lichao Sun, Philip S Yu, and Lifang He. 2022. A survey on text classification: From traditional to deep learning. ACM Transactions on Intelligent Systems and Technology (TIST), 13(2):1--41

2022
[38]

Yinheng Li. 2023. https://aclanthology.org/2023.ranlp-1.69 A practical survey on zero-shot prompt design for in-context learning . In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 641--647, Varna, Bulgaria. INCOMA Ltd...

2023
[39]

Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023 a . Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6)

2023
[40]

Yuying Li, Gaoyang Liu, Yang Yang, and Chen Wang. 2024. Seeing is believing: Black-box membership inference attacks against retrieval augmented generation. arXiv preprint arXiv:2406.19234

2024 arXiv
[41]

Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 b . Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281

2023 arXiv
[42]

Yong Lin, Lu Tan, Hangyu Lin, Zeming Zheng, Renjie Pi, Jipeng Zhang, Shizhe Diao, Haoxiang Wang, Han Zhao, Yuan Yao, et al. 2023. Speciality vs generality: An empirical study on catastrophic forgetting in fine-tuning foundation models. arXiv preprint arXiv:2309.06256

2023 arXiv
[43]

AI @ Meta Llama Team. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

2024 arXiv
[44]

Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.570 Quantifying privacy risks of masked language models using membership inference attacks . In Proceedings of the 2022 Conferen...

2022 doi
[45]

OpenAI, Aaron Hurst, Adam Lerer, et al. 2024. https://arxiv.org/abs/2410.21276 Gpt-4o system card . Preprint, arXiv:2410.21276

2024 arXiv
[46]

Jongjin Park. 2024. Development of dental consultation chatbot using retrieval augmented llm. The Journal of the Institute of Internet, Broadcasting and Communication, 24(2):87--92

2024
[47]

Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, and Himabindu Lakkaraju. 2024. Follow my instruction and spill the beans: Scalable data extraction from retrieval-augmented generation systems. arXiv preprint arXiv:2402.17840

2024 arXiv
[48]

Rag-Datasets. 2024 a . https://huggingface.co/datasets/rag-datasets/rag-mini-bioasq Rag-mini-bioasq

2024
[49]

Rag-Datasets. 2024 b . https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia Rag-mini-wikipedia

2024
[50]

Mahimai Raja, E Yuvaraajan, et al. 2024. A rag-based medical assistant especially for infectious diseases. In 2024 International Conference on Inventive Computation Technologies (ICICT), pages 1128--1133. IEEE

2024
[51]

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

2023
[52]

Maria Rigaki and Sebastian Garcia. 2023. A survey of privacy attacks in machine learning. ACM Computing Surveys, 56(4):1--34

2023
[53]

Ayush RoyChowdhury, Mulong Luo, Prateek Sahu, Sarbartha Banerjee, and Mohit Tiwari. 2024. Confusedpilot: Compromising enterprise information integrity and confidentiality with copilot for microsoft 365. arXiv preprint arXiv:2408.04870

2024 arXiv
[54]

Virat Shejwalkar, Huseyin A Inan, Amir Houmansadr, and Robert Sim. 2021. Membership inference attacks against nlp classification models. In NeurIPS 2021 Workshop Privacy in Machine Learning

2021
[55]

Logan IV, Eric Wallace, and Sameer Singh

Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.346 A uto P rompt: E liciting K nowledge from L anguage M odels with A utomatically G enerated P rompts . In Proceedings of the 2020 Conference o...

2020 doi
[56]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE

2017
[57]

Florian Tram \`e r, Gautam Kamath, and Nicholas Carlini. 2022. Considerations for differentially private learning with large-scale public pretraining. arXiv preprint arXiv:2212.06470

2022 arXiv
[58]

Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. 2023. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. In NeurIPS

2023
[59]

Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533

2022 arXiv
[60]

Ziyu Wang, Hao Li, Di Huang, and Amir M Rahmani. 2024. Healthq: Unveiling questioning capabilities of llm chains in healthcare conversations. arXiv preprint arXiv:2409.19487

2024 arXiv
[61]

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

2022 arXiv
[62]

Rey Reza Wiyatno, Anqi Xu, Ousmane Dia, and Archy De Berker. 2019. Adversarial examples in modern machine learning: A review. arXiv preprint arXiv:1911.05268

2019 arXiv
[63]

Fangzhou Wu, Ning Zhang, Somesh Jha, Patrick McDaniel, and Chaowei Xiao. 2024. A new era in llm security: Exploring security concerns in real-world llm-based systems. arXiv preprint arXiv:2402.18649

2024 arXiv
[64]

Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. https://arxiv.org/abs/2309.07597 C-pack: Packaged resources to advance general chinese embedding . Preprint, arXiv:2309.07597

2023 arXiv
[65]

Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211

2024
[66]

Zihan Yu, Liang He, Zhen Wu, Xinyu Dai, and Jiajun Chen. 2023. Towards better chain-of-thought prompting strategies: A survey. arXiv preprint arXiv:2310.04959

2023 arXiv
[67]

Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.267 The good and the bad: Exploring privacy issues in retrieval-augmented generation ( R...

2024 doi
[68]

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren's song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219

2023 arXiv
[69]

Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473

2024 arXiv
[70]

Yujia Zhou, Yan Liu, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Zheng Liu, Chaozhuo Li, Zhicheng Dou, Tsung-Yi Ho, and Philip S Yu. 2024. Trustworthiness in retrieval-augmented generation systems: A survey. arXiv preprint arXiv:2409.10102

2024 arXiv
[71]

Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023. Multilingual machine translation with large language models: Empirical results and analysis. arXiv preprint arXiv:2304.04675

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

arXiv 2024

[4] [4]

AI@Meta. 2024 a . https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

work page 2024

[5] [5]

AI@Meta. 2024 b . https://huggingface.co/meta-llama/Llama-3.2-1B Llama 3.2 1b model card

work page 2024

[6] [6]

Maya Anderson, Guy Amit, and Abigail Goldsteen. 2024. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation. arXiv preprint arXiv:2405.20446

arXiv 2024

[7] [7]

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023

arXiv 2023

[8] [8]

Vani Bhat, Divya Sree, Jinu Cheerla, Nupur Mathew, Gunna LIu, and Jerry Gao. 2024. Retrieval augmented generation (rag) based restaurant chatbot with ai testability

work page 2024