GraphIP-Bench is a new unified benchmark showing GNN model extraction succeeds at moderate query budgets while most defenses fail to prevent it or retain verification signals on surrogates.
Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications, including enterprise chatbots, healthcare assistants, and agentic memory management. However, recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries, raising serious intellectual property and privacy concerns. While prior work has explored individual attack and defense techniques, the research landscape remains fragmented, spanning heterogeneous retrieval embeddings, diverse generation models, and evaluations based on non-standardized metrics and inconsistent datasets. To address this gap, we introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems. Our benchmark covers broad attack/defense strategies, representative retrieval embedding models, open/closed-source generators, (non) graph-based indexing, all evaluated under a unified experimental framework with standardized protocols across multiple datasets spanning diverse languages. By consolidating the experimental landscape and enabling reproducible, comparable evaluation, this benchmark provides actionable insights and a practical foundation for developing privacy-preserving RAG systems in the face of emerging knowledge extraction threats.
years
2026 2verdicts
UNVERDICTED 2representative citing papers
GRADE models any LLM agent run as a graph with execution and graded dependency edge layers to enable failure prediction and fault localization across tool, coding, and web agent corpora.
citing papers explorer
-
GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?
GraphIP-Bench is a new unified benchmark showing GNN model extraction succeeds at moderate query budgets while most defenses fail to prevent it or retain verification signals on surrogates.
-
GRADE: Graph Representation of LLM Agent Dependency and Execution
GRADE models any LLM agent run as a graph with execution and graded dependency edge layers to enable failure prediction and fault localization across tool, coding, and web agent corpora.