Pith. sign in

REVIEW 5 cited by

Membership Inference Attacks From First Principles

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2112.03570 v2 pith:IDANFVTJ submitted 2021-12-07 cs.CR cs.LG

Nicholas Carlini , Steve Chien , Milad Nasr , Shuang Song , Andreas Terzis , Florian Tramer This is my paper
classification cs.CRcs.LG
keywords attacksattackevaluatedfalse-positiveinferencemembershipmetricsmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

A membership inference attack allows an adversary to query a trained machine learning model to predict whether or not a particular example was contained in the model's training dataset. These attacks are currently evaluated using average-case "accuracy" metrics that fail to characterize whether the attack can confidently identify any members of the training set. We argue that attacks should instead be evaluated by computing their true-positive rate at low (e.g., <0.1%) false-positive rates, and find most prior attacks perform poorly when evaluated in this way. To address this we develop a Likelihood Ratio Attack (LiRA) that carefully combines multiple ideas from the literature. Our attack is 10x more powerful at low false-positive rates, and also strictly dominates prior attacks on existing metrics.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 30 citations worldwide. Full citation record

  1. Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Matching a retrained oracle on trained probes can certify models that still retain held-out forget knowledge, and oracle-free unlearning certification is only possible for counterfactual, non-inferable facts.

  2. Auditing of Unlearning Algorithms

    cs.LG 2026-07 accept novelty 6.0 of 10

    An auditor based on membership inference attacks computes valid lower bounds on the unlearning parameter ε, empirically separating certified unlearning methods (small bounds) from heuristic ones (large bounds).

  3. Stealth by Conformity: Evading Robust Aggregation through Adaptive Poisoning

    cs.CR 2025-09 conditional novelty 6.0 of 10

    An adaptive federated-learning backdoor attack uses membership-inference feedback on the global model to keep malicious updates statistically similar to benign ones, evading nine robust aggregation defenses in two ima...

  4. How Well Do AI Systems Solve AP Physics? A Comparative Evaluation of Large Language Models on Algebra-Based Free Response Questions

    physics.ed-ph 2026-03 unverdicted novelty 5.0 of 10

    ChatGPT 4.1 mini, Gemini 2.5 Flash, Claude 4.0 Sonnet, and DeepSeek R1 average 82–92% on AP Physics 1/2 free-response questions but systematically fail spatial, visual, and conceptual tasks.

  5. Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysis

    cs.LG 2025-09 conditional novelty 5.0 of 10

    A random forest trained on transformer hidden-state and attention features detects training data membership with about 0.83 average AUC, far above output-based attacks.

Pith tools