Pith. sign in

REVIEW 14 cited by

Do Membership Inference Attacks Work on Large Language Models?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.07841 v2 pith:ZMRRB3PS submitted 2024-02-12 cs.CL

Michael Duan , Anshuman Suri , Niloofar Mireshghallah , Sewon Min , Weijia Shi , Luke Zettlemoyer , Yulia Tsvetkov , Yejin Choi
show 2 more authors
David Evans Hannaneh Hajishirzi
This is my paper
classification cs.CL
keywords miasmodelsdatainferencelanguagelargemembershipsettings
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data. Despite extensive research on traditional machine learning models, there has been limited work studying MIA on the pre-training data of large language models (LLMs). We perform a large-scale evaluation of MIAs over a suite of language models (LMs) trained on the Pile, ranging from 160M to 12B parameters. We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains. Our further analyses reveal that this poor performance can be attributed to (1) the combination of a large dataset and few training iterations, and (2) an inherently fuzzy boundary between members and non-members. We identify specific settings where LLMs have been shown to be vulnerable to membership inference and show that the apparent success in such settings can be attributed to a distribution shift, such as when members and non-members are drawn from the seemingly identical domain but with different temporal ranges. We release our code and data as a unified benchmark package that includes all existing MIAs, supporting future work.

Discussion (0). Sign in to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cascading and Proxy Membership Inference Attacks

    cs.CR 2025-07 conditional novelty 7.0 of 10

    CMIA cascades conditional shadow training to exploit membership dependencies, and PMIA uses proxy data to approximate Bayesian membership odds, both substantially outperforming prior MIAs in low false-positive regimes.

  2. Understanding the Supply Chain and Risks of Large Language Model Applications

    cs.SE 2025-07 conditional novelty 7.0 of 10

    A new benchmark dataset traces dependencies across 3,859 LLM applications, 109,211 models, 2,474 datasets, and 8,862 libraries, and finds widespread known vulnerabilities in application dependencies.

  3. Implicit Reasoning Steering via Concept Chaining

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Reinforcement-learning-optimized concept-chain paragraphs covertly steer language-model multiple-choice preferences after continued pretraining, with far lower detectability than direct paraphrases.

  4. How Quantization Impacts Privacy Risk on LLMs for Code?

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Quantizing code LLMs reduces membership inference effectiveness, with 4-bit compression giving larger privacy and performance drops than 8-bit.

  5. Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework

    cs.AI 2025-07 reject novelty 6.0 of 10

    NA-PDD detects pre-training data in LLMs by comparing which neurons activate for a test text against neurons linked to known training versus non-training texts, and claims large AUC improvements on three benchmarks.

  6. What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests

    cs.CL 2025-07 conditional novelty 6.0 of 10

    WikiMem, a Wikidata-derived canary dataset and a calibrated NLL-ranking metric, identifies which human-fact associations an LLM has memorized, with higher rates for famous people and larger models.

  7. Breaking the Myth: Can Small Models Infer Postconditions Too?

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Fine-tuning Qwen2.5-Coder-7B on 1.5K reasoning examples yields postcondition generation close to GPT-4o and better than 32B open models on Defects4J.

  8. MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation

    cs.AI 2025-06 conditional novelty 6.0 of 10

    MAGPIE is a 158-scenario benchmark showing large language model agents misclassify and leak contextually private information in multi-agent collaboration, even under explicit privacy instructions.

  9. Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble

    cs.LG 2025-06 conditional novelty 6.0 of 10

    MIAs expose different members depending on attack method and random seed; the paper quantifies this with coverage/stability and shows ensembling attacks yields stronger, more reliable privacy checks.

  10. LoMime: Query-Efficient Membership Inference using Model Extraction in Label-Only Settings

    cs.LG 2026-02 conditional novelty 5.0 of 10

    Extracting a surrogate model from a label-only API lets an attacker perform membership inference offline, matching direct-attack accuracy with a query budget of roughly 1% of the training data.

  11. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  12. SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation

    cs.CR 2025-06 conditional novelty 3.0 of 10

    A systematization-of-knowledge survey that categorizes LLM privacy risks into training data, prompts, outputs, and agents, and reviews limitations of current mitigations.

  13. A Survey: Towards Privacy and Security in Mobile Large Language Models

    cs.CR 2025-09 conditional

    A survey of privacy and security challenges for mobile large language models, summarizing known attack types and defenses without introducing new results.

  14. Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning

    cs.CR 2025-06

Pith tools