Pith. sign in

REVIEW 5 cited by

On the Importance of Difficulty Calibration in Membership Inference Attacks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2111.08440 v2 pith:IGD6SV6J submitted 2021-11-15 cs.CR cs.LG

Lauren Watson , Chuan Guo , Graham Cormode , Alex Sablayrolles This is my paper
classification cs.CRcs.LG
keywords attacksmembershipdifficultycalibrationinferencepredictedexistingfalse
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The vulnerability of machine learning models to membership inference attacks has received much attention in recent years. However, existing attacks mostly remain impractical due to having high false positive rates, where non-member samples are often erroneously predicted as members. This type of error makes the predicted membership signal unreliable, especially since most samples are non-members in real world applications. In this work, we argue that membership inference attacks can benefit drastically from \emph{difficulty calibration}, where an attack's predicted membership score is adjusted to the difficulty of correctly classifying the target sample. We show that difficulty calibration can significantly reduce the false positive rate of a variety of existing attacks without a loss in accuracy.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reference-Based Distillation Detection in LLMs

    cs.LG 2026-06 conditional novelty 7.5 of 10

    Reference-based membership inference recovers the true teacher of a distilled LLM with near-perfect accuracy when an earlier same-lineage checkpoint is available.

  2. DCMI: A Differential Calibration Membership Inference Attack Against Retrieval-Augmented Generation

    cs.CR 2025-09 conditional novelty 6.0 of 10

    DCMI infers RAG database membership by subtracting the system's yes-probability on a perturbed query from the original query, cancelling the interference of non-member retrieved documents.

  3. Evaluating the Dynamics of Membership Privacy in Deep Learning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Per-sample membership vulnerability is established early in training, especially for hard-to-learn examples, and can be tracked on an FPR-TPR plane.

  4. Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework

    cs.AI 2025-07 reject novelty 6.0 of 10

    NA-PDD detects pre-training data in LLMs by comparing which neurons activate for a test text against neurons linked to known training versus non-training texts, and claims large AUC improvements on three benchmarks.

  5. Membership Inference Attacks with False Discovery Rate Control

    stat.ML 2025-08 conditional novelty 4.0 of 10

    A post-hoc wrapper, MIAFdR, converts any membership inference attack scores into conformal p-values and applies a Benjamini-Hochberg correction, guaranteeing that the expected proportion of non-members among flagged m...

Pith tools