REVIEW 10 cited by
Membership Inference Attacks against Language Models via Neighbourhood Comparison
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the privacy risks of language models. Most existing attacks rely on the observation that models tend to assign higher probabilities to their training samples than non-training points. However, simple thresholding of the model score in isolation tends to lead to high false-positive rates as it does not account for the intrinsic complexity of a sample. Recent work has demonstrated that reference-based attacks which compare model scores to those obtained from a reference model trained on similar data can substantially improve the performance of MIAs. However, in order to train reference models, attacks of this kind make the strong and arguably unrealistic assumption that an adversary has access to samples closely resembling the original training data. Therefore, we investigate their performance in more realistic scenarios and find that they are highly fragile in relation to the data distribution used to train reference models. To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution. We show that, in addition to being competitive with reference-based attacks that have perfect knowledge about the training data distribution, our attack clearly outperforms existing reference-free attacks as well as reference-based attacks with imperfect knowledge, which demonstrates the need for a reevaluation of the threat model of adversarial attacks.
Forward citations
Cited by 10 Pith papers
-
In-Context Probing for Membership Inference in Fine-Tuned Language Models
ICP-MIA infers membership in fine-tuned LLMs by measuring confidence improvement under in-context probes, beating prior black-box attacks at low false-positive rates.
-
AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents
AgentSentinel combines system-level tracing with LLM-based auditing to block 79.6% of attacks in the authors' 60-scenario computer-use agent benchmark.
-
DCMI: A Differential Calibration Membership Inference Attack Against Retrieval-Augmented Generation
DCMI infers RAG database membership by subtracting the system's yes-probability on a perturbed query from the original query, cancelling the interference of non-member retrieved documents.
-
Investigating Training Data Detection in AI Coders
Most existing training-data detection methods perform poorly on code, while prefix-relative method ReCaLL consistently scores highest, though all degrade under code mutations.
-
Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework
NA-PDD detects pre-training data in LLMs by comparing which neurons activate for a test text against neurons linked to known training versus non-training texts, and claims large AUC improvements on three benchmarks.
-
Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble
MIAs expose different members depending on attack method and random seed; the paper quantifies this with coverage/stability and shows ensembling attacks yields stronger, more reliable privacy checks.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Counterfactual Influence as a Distributional Quantity
Near-duplicate training samples lower a model's self-influence on a record while raising its extractability, so self-influence alone underestimates memorization risk.
-
Towards Network-Aware Operation of Integrated Energy Systems: A Comprehensive Review
A claimed first comprehensive review of network-aware modeling, optimization, and control for multi-carrier integrated energy systems, stressing topology and network constraints.
-
Zero-Trust Foundation Models: A New Paradigm for Secure and Collaborative Artificial Intelligence for Internet of Things
The paper defines the ZTFM concept, identifies four zero-trust principles, reviews enabling technologies and threats, and lays out open research challenges for AI-driven IoT security.
Discussion (0). Continue with ORCID to comment.