Pith. sign in

REVIEW 4 major objections 6 minor 40 references

Technical Report for the Forgotten-by-Design Project: Targeted Obfuscation for Machine Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Instance-specific gradient obfuscation during training prevents sensitive data from being embedded, reducing membership-inference risk by an order of magnitude while preserving accuracy.

desk verdict The paper's core idea is plausible and the authors are honest about limits, but the headline 'order of magnitude' claim is not yet supported because the defense and the evaluation both operate on the same LIRA statistic. read the letter →

arxiv 2501.11525 v1 pith:USSKE4KB submitted 2025-01-20 cs.LG cs.AIcs.CR

Rickard Brännvall , Laurynas Adomaitis , Olof Görnerup , Anass Sedrati This is my paper · ORCID
classification cs.LGcs.AIcs.CR
keywords righttobeforgottenmembershipinferenceattackLIRAgradientobfuscationinstance-specificnoiseprivacy-utilitytrade-offdifferentialprivacymachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the right to be forgotten in AI should be achieved not by erasing data after training but by preventing sensitive data from being learnable in the first place. It proposes Forgotten by Design: during training, each data point receives a weight and a noise level based on how vulnerable it is to the LIRA membership-inference attack, so the model shifts its parameters away from points that would otherwise be identifiable. Experiments on a ResNet-18 model trained on CIFAR-10 show that the combined scheme cuts the LIRA attack's advantage by at least an order of magnitude while holding test accuracy at the baseline level, and at higher noise levels drives the attack toward random guessing. If correct, this offers a practical route to privacy compliance without retraining models and without committing to formal differential privacy.

What carries the argument

The load-bearing object is the instance-specific gradient update $$\theta_{t+1} = \theta_t - \eta \frac{1}{|B|}\sum_{j\in B}(w_j g_j + u_j),$$ in which each data point contributes a weight $w_j$ and a Gaussian noise $u_j \sim \mathcal{N}(0,\sigma_j^2 I)$. The weight is a clipped exponential of the privacy vulnerability t-score $$t_j = \frac{$m^{{\mathrm{IN}}$}_j - $m^{{\mathrm{OUT}}$}_j}{\sqrt{$V^{{\mathrm{IN}}$}_j + $V^{{\mathrm{OUT}}$}_j}},$$ computed from LIRA shadow-model fits of scaled logits, so a point with a higher vulnerability score receives a lower weight $w_{\mathrm{exp}}(t) = e^{-\max(0,\alpha t - \beta)}$. This redirects the gradient update away from points LIRA can identify, while uniform noise blurs whatever signal remains, and the paper shows the two effects compound in the privacy-utility trade-off.

What would settle it

Run the stronger RMIA attack on the protected weighted sigma=0.01 checkpoints from Table 3; if its true positive rate at 0.001 false positive rate is materially higher than the LIRA-derived tau near 1.7 implies, the order-of-magnitude claim holds only against LIRA, not against membership inference in general.

Watch

Extended reading notes

Core claim

The paper claims that a model can be made to forget sensitive data during training rather than after it, by obfuscating the gradients of the data points a membership-inference adversary would most reliably identify. On CIFAR-10 with ResNet-18, combining additive Gaussian noise with instance-specific down-weighting based on LIRA vulnerability scores reduces LIRA's tau at 0.001 FPR from 5.217 to 1.712 at sigma = 0.01, with test accuracy statistically unchanged, and at sigma = 0.03 drives AUC to about 0.54-0.56 and tau toward zero, close to the random-guessing baseline. The authors conclude that non-inference under a strong membership-inference test is a workable operational definition of forgetting, satisfying a practical reading of the right to be forgotten.

Load-bearing premise

The entire guarantee is measured and targeted through LIRA's Gaussian approximation of shadow-model losses, so if LIRA is not a representative proxy for membership inference, the weighting protects the wrong points and the reported reduction may not transfer to stronger attacks.

Editorial extensions

If this is right

  • At sigma=0.01 with the exponential weighting, LIRA's tau at 0.001 FPR drops from 5.217 to 1.712 while test accuracy stays at 0.705, so the attacker's success per false positive falls by more than an order of magnitude.
  • At sigma=0.03, both schemes push AUC to about 0.54-0.56 and tau near 0.2-0.3, close to the random-guessing baseline where the attacker has little advantage over chance.
  • The weighted scheme sits on a more favorable privacy-utility frontier than noise-only obfuscation: at intermediate risk levels it achieves the same accuracy with lower tau at every FPR level tested.
  • Even at the strongest obfuscation tested, accuracy remains in the sixties, far above the roughly 10 percent random-label floor for CIFAR-10.
  • Because the obfuscation is applied during training, no post-training unlearning step is needed for the protected data points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's practicality depends on the t-score ranking produced by a first round of shadow models; a natural test is whether the same ranking can be reused across architectures, datasets, or data splits without retraining shadow models.
  • One unstated direction is combining the instance-specific weights with DP-SGD's gradient clipping; the paper does not claim formal guarantees, but applying weights before clipping might preserve some of the targeted forgetting while recovering a bound.
  • Equating forgetting with non-inference implies that a model passing an accepted membership-inference audit could be treated as having forgotten a data point; that regulatory equivalence is the authors' framing, not a legal result, and would need regulator acceptance.
  • The strongest untested risk is attack transferability: if a stronger attack exploits different signals than LIRA, the reported reduction could shrink, so re-running the evaluation with RMIA would delimit the claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This technical report proposes 'Forgotten by Design,' a training-time defense that combines per-example Gaussian gradient noise with a LIRA-derived weighting of vulnerable samples. The authors define a privacy vulnerability t-score from shadow-model Gaussian fits (Eq. 7), assign each sample a weight w(t)=exp(-max(0,alpha t - beta)) (Eq. 12), and add noise to the gradient sum (Eq. 8). On CIFAR-10 with ResNet-18, they report that the weighted scheme at sigma=0.01 reduces LIRA tau@0.001FPR from 5.217 to 1.712 and AUC from 0.885 to 0.671 while accuracy remains around 70-70.5%, and that stronger noise pushes AUC toward 0.5. They interpret these results as preventing sensitive data from being embedded and as an order-of-magnitude privacy improvement, and they frame the approach in terms of motivated forgetting and the right to be forgotten.

Significance. The idea of targeting gradient obfuscation to samples that are individually vulnerable to a specified membership-inference attack is a reasonable engineering step, and the paper has several strengths: it uses at least 300 shadow models per configuration, reports ROC curves with confidence bands, visualizes the privacy-utility frontier, and explicitly lists its limitations. If the claims were restricted to LIRA-measured leakage, the results would be a useful empirical data point for privacy audit practice. However, the main claim that the method 'prevents sensitive data from being embedded' is not established: the defense and the evaluation share the same LIRA statistic, no comparison to stronger attacks (e.g., RMIA) or DP-SGD is provided, and no confidence intervals are given for the headline privacy metrics. As it stands, the paper reads as a feasibility study rather than a validated privacy guarantee.

major comments (4)
  1. [Sections 3.2 and 3.4, Eqs. (4), (7), (12)] The reported privacy reduction is partly engineered because the defense uses the same LIRA statistic that defines the evaluation. The vulnerability weight w(t)=exp(-max(0,alpha t - beta)) suppresses samples with the largest normalized IN-OUT gap t_j (Eq. 7), while the attack metrics tau and AUC are computed by thresholding the LIRA log-likelihood ratio of precisely that gap (Eq. 4). Down-weighting high-t points therefore lowers the measured leakage by construction, independently of whether a different adversary would still succeed. The paper's own Section 5.1 cites Aerni et al. [40] as showing that empirical defenses evaluated against the attacks they are designed against can overstate privacy, and Section 5.2 concedes that RMIA is untested. I request an evaluation with RMIA (or a simple non-LIRA threshold attack) and a DP-SGD baseline; absent that, the abstract's 'prevents sensitive data from being embedded' should be replaced by 'reduces LIRA-measured membership leakage.'
  2. [Abstract and Table 3] The abstract's claim of 'at 95% significance' is not supported by any confidence interval, p-value, or hypothesis test for AUC or tau in the main text. Section 3.2 explains how shadow models can provide confidence bands, and Figure 7 displays such bands, but Table 3 reports only point estimates for AUC and tau, with standard deviations only for accuracy. With at least 300 shadow models per configuration, the authors should report bootstrap or shadow-model-based confidence intervals for AUC and tau, and a test for the key comparison (e.g., tau@0.001FPR = 5.217 at baseline versus 1.712 for weighted sigma=0.01). Without this, the headline statistical claim is unverifiable.
  3. [Section 4.1 and Table 2] The headline operating point (alpha=2, beta=2, sigma=0.01) was selected after inspecting the privacy-utility trade-off curves, so the reported tau of 1.712 and 70.5% accuracy are in-sample point estimates for a chosen configuration. The order-of-magnitude claim is sensitive to this choice: Table 3 shows weighted-row tau values ranging from 4.525 to 0.159 depending on sigma. Please report a sensitivity analysis over alpha, beta, and sigma, or a pre-specified selection rule with a validation split, so that the reader can assess whether the improvement is robust rather than an artifact of post-hoc parameter choice.
  4. [Section 5.2, Table 3, and Conclusions] The conclusion that the method 'prevents sensitive data from being embedded' is not supported by the evidence even for the LIRA adversary. At the headline setting (weighted sigma=0.01), AUC is 0.671 and tau@0.001FPR is 1.712, so the attacker's TPR/FPR ratio is still around exp(1.712) approximately 5.5; only at sigma=0.03-0.05 does AUC approach 0.5, at the cost of a 5-13 percentage-point accuracy drop. Moreover, membership inference is only one type of privacy harm, so the absolute wording claims more than the experiments can show. I recommend reporting residual leakage at each operating point and rewording the abstract and conclusions to describe a reduction in LIRA-measured leakage rather than the prevention of sensitive-data embedding.
minor comments (6)
  1. [Section 2.7 and References] References [34] and [40] are the same paper (Aerni, Zhang, and Tramèr, CCS 2024); please consolidate the duplicate entry.
  2. [Figure 2 caption] The caption says 'uniform noise addition (0.01 and 0.02)' but the legend and the text refer to 0.01 and 0.03; the caption should match the plotted curves.
  3. [Table 4] The 'weighted: 0.007' row is duplicated; one copy should be removed.
  4. [Section 3.1] The phrase 'the cross-entropy loss nor the logits' should be 'the cross-entropy loss or the logits'; the definition of p_j as exp(-ell) also appears only later, so an early forward reference would improve readability.
  5. [Section 4.1 and Table 3] Accuracies are reported as fractions in Table 3 (e.g., 0.7) and as percentages in Table 4 (e.g., 70.043); please use one convention consistently.
  6. [General / Reproducibility] The paper does not provide the full training setup (epochs, batch size, learning-rate schedule, optimizer, and number of gradient steps) needed to reproduce the experiments; adding a reproducibility appendix would strengthen the report.

Circularity Check

2 steps flagged · score 6.0 of 10

The defense and the audit share the LIRA Gaussian IN-OUT statistic, so the headline order-of-magnitude privacy reduction is partly engineered by construction rather than independently measured.

  1. fitted input called prediction [Section 3.2 (Eqs. 2-4; tau definition), Section 3.3 (Eq. 7), Section 3.4 (Eq. 12), Section 4 (Main experiment set-up)]
    "In the LIRA attack, these metrics are estimated by thresholding the likelihood ratio. ... For this purpose, we define the privacy vulnerability t-score tj = ... ... The weighted scenarios use the exponential weighting scheme defined in equation (12) based on the privacy vulnerability t-score calculated as in equation (7), that is, weights falling off exponentially fast with the (non-negative) distance between the estimated center of the IN and OUT distributions of the LIRA attack model. ... tau: For this work we introduce tau, which is simply the logarithm of the ratio of TPR to FPR."

    The t-score (Eq. 7) is the standardized IN-OUT gap of the LIRA Gaussian fits; the attack statistic lambda (Eq. 4) is the log ratio of those same Gaussian densities; the weights (Eq. 12) are a monotone non-increasing function of t; and the headline metrics (AUC, tau) are computed by thresholding lambda. Down-weighting high-t points therefore suppresses precisely the signal the evaluation measures. A lower tau/AUC is the expected consequence of any scheme that down-weights the points LIRA flags, so the claimed order-of-magnitude reduction in privacy risk is not an independent measurement of membership leakage; it is partly a consequence of using the same sufficient statistic for both the defense and the audit.

  2. self definitional [Section 2.3 and Section 5.2]
    "In short, if there is no reliable (i.e., better than random) way of determining that a data point was part of the training data set, then we consider that the AI model has forgotten that data point during or after training. ... The study primarily focuses on membership inference attacks (and specifically only LIRA)."

    Forgetting is defined as the absence of any reliable membership-inference signal, but the only signal tested is LIRA. Thus 'the model has forgotten' is established whenever the LIRA statistic that the defense was calibrated to suppress is reduced. The acknowledged omission of RMIA and other attacks means the definition is never tested against an independent inference procedure, so the criterion and the measured quantity coincide by construction.

full rationale

The central claim--that instance-specific gradient obfuscation prevents sensitive data from being embedded and reduces privacy risk by at least an order of magnitude--is evaluated with the same LIRA Gaussian machinery used to define the vulnerability weights. The t-score (Eq. 7) and the likelihood-ratio metrics (Eqs. 2-4, tau) are derived from the same IN/OUT Gaussian fits, and the exponential weights (Eq. 12) monotonically suppress large-t points, so the reported drop in tau from 5.2 to 1.7 is substantially engineered by construction. This is partial, not total, circularity: accuracy is an independent utility measure, the ROC curves' approach to random guessing is a real behavioral change in the retrained models, and the paper openly concedes that stronger attacks such as RMIA and DP-SGD baselines are out of scope (Sections 5.1-5.2). Those concessions also mean the strongest claim is not externally falsified in the paper. Score 6 reflects one or more predictions that reduce by construction while leaving independent content in the accuracy and ROC-to-random results.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The experiment rests on three hand-chosen parameters (sigma, alpha, beta), the Gaussian approximation from LIRA, the shadow-model assumption, the equation of LIRA non-inference with forgetting, and the transfer from one dataset. No new physical entities are introduced. The new statistics, the t-score and tau, are derived from LIRA and carry no independent falsifiable handle outside the paper.

free parameters (3)
  • alpha (weight steepness) = alpha = 2
    Selected after seeing results; controls how quickly weights decrease with t-score, Eq. (12).
  • beta (weight cutoff) = beta = 2
    Selected together with alpha; sets the t-score above which weights fall below one, Eq. (12).
  • sigma (noise standard deviation) = sigma in {0.001, ..., 0.05}; headline uses 0.01 and 0.03
    Noise magnitude scanned by hand; the order-of-magnitude claim depends on the sigma values chosen for reporting.
assumptions (5)
  • domain assumption Logit-scaled per-point losses are approximately Gaussian.
    Used in Eqs. (2)-(7); if false, t-score ranking and the tau metric lose calibration.
  • domain assumption Shadow models trained on random subsets are faithful surrogates for the target model's IN/OUT loss distributions.
    LIRA builds IN/OUT means and variances from shadow models in Sections 3.1 and 3.3; both the defense and the evaluation inherit this assumption.
  • domain assumption Attacker advantage under LIRA is an adequate measure of whether a model has forgotten a data point.
    Section 2.3 equates non-inference with forgetting; the paper evaluates only LIRA and lists this as a limitation in Section 5.2.
  • domain assumption Gradient noise without DP-SGD clipping still provides meaningful privacy protection.
    Section 3.4 explicitly forgoes clipping and formal DP guarantees; the privacy benefit is empirical rather than proven.
  • domain assumption Results on CIFAR-10 with ResNet-18 generalize to other datasets and models relevant to the right to be forgotten.
    Claims are framed for AI systems broadly, while experiments use one dataset; Section 2.8 acknowledges this limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Technical Report for the Forgotten-by-Design Project: Targeted Obfuscation for Machine Learning." pith.science (2026). https://pith.science/paper/USSKE4KB

@misc{pith2026250111525,
  author       = {Pith},
  title        = {Pith review of: Technical Report for the Forgotten-by-Design Project: Targeted Obfuscation for Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USSKE4KB}},
  note         = {Machine review of arXiv:2501.11525}
}
read the original abstract

The right to privacy, enshrined in various human rights declarations, faces new challenges in the age of artificial intelligence (AI). This paper explores the concept of the Right to be Forgotten (RTBF) within AI systems, contrasting it with traditional data erasure methods. We introduce Forgotten by Design, a proactive approach to privacy preservation that integrates instance-specific obfuscation techniques during the AI model training process. Unlike machine unlearning, which modifies models post-training, our method prevents sensitive data from being embedded in the first place. Using the LIRA membership inference attack, we identify vulnerable data points and propose defenses that combine additive gradient noise and weighting schemes. Our experiments on the CIFAR-10 dataset demonstrate that our techniques reduce privacy risks by at least an order of magnitude while maintaining model accuracy (at 95% significance). Additionally, we present visualization methods for the privacy-utility trade-off, providing a clear framework for balancing privacy risk and model accuracy. This work contributes to the development of privacy-preserving AI systems that align with human cognitive processes of motivated forgetting, offering a robust framework for safeguarding sensitive information and ensuring compliance with privacy regulations.

Figures

Figures reproduced from arXiv: 2501.11525 by the authors.

Figure 1
Figure 1. ROC Curve, for example, LIRA Membership Inference Attack on ResNet-18 Model Trained on [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. ROC Curves for LIRA Membership Inference Attack on ResNet-18 Model Trained on CIFAR [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Privacy-Utility Trade-Off for Different Noise Levels and Weighting Schemes. The scatter plots [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Distribution of privacy vulnerability t-scores for Different Noise Levels and Weighting Schemes. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Distribution of privacy vulnerability t-scores sorted by the original ranking from the baseline [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Scatter plots of Normalized Means for IN and OUT Distributions at Different Noise Levels. [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: ROC Curves with Confidence Bands for Different Noise Levels and Weighting Schemes. The [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 33 canonical work pages

  1. [40]

    Evaluations of machine learning privacy defenses are misleading

    Michael Aerni, Jie Zhang, and Florian Tram` er. Evaluations of machine learning privacy defenses are misleading. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security, pages 1271–1284. Association for Computing Machinery, 2024. 21 Appendix This appendix presents a more detailed analysis of the t-scores as well as some fu...

    work page 2024

  2. [1]

    Member- ship inference attacks on machine learning: A survey

    Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Member- ship inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR) , 54(11s):1– 37, 2022

    work page 2022

  3. [2]

    Mem- bership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022

    work page 2022

  4. [3]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    work page 2009

  5. [4]

    Deep residual learning for image recog- nition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. arXiv preprint arXiv:1512.03385 , 2015

    arXiv 2015

  6. [5]

    Decay happens: the role of active forgetting in memory

    Oliver Hardt, Karim Nader, and Lynn Nadel. Decay happens: the role of active forgetting in memory. Trends in cognitive sciences, 17(3):111–120, 2013

    work page 2013

  7. [6]

    Towards a cognitive and neurobiological model of moti- vated forgetting

    Michael C Anderson and Ean Huddleston. Towards a cognitive and neurobiological model of moti- vated forgetting. True and false recovered memories: Toward a reconciliation of the debate , pages 53–120, 2012

    work page 2012

  8. [7]

    Memory editing from science fiction to clinical practice

    Elizabeth A Phelps and Stefan G Hofmann. Memory editing from science fiction to clinical practice. Nature, 572(7767):43–50, 2019

    work page 2019

Show all 40 references
  1. [8]

    First draft of a report on the edvac

    John Von Neumann. First draft of a report on the edvac. IEEE Annals of the History of Computing , 15(4):27–75, 1993

    1993

  2. [9]

    Der rechenmaschinenspeicher als ged¨ achtnis: Einfl¨ usse der neurophys- iologie auf den rechnerbau in den vierziger jahren

    Katharina Schmidt-Br¨ ucken. Der rechenmaschinenspeicher als ged¨ achtnis: Einfl¨ usse der neurophys- iologie auf den rechnerbau in den vierziger jahren. Sozialgeschichte der Informatik: Kulturelle Prak- tiken und Orientierungen , pages 197–211, 1998

    1998

  3. [10]

    Remembering jan rajchman and the origins of electronic memory, 2015

    Alexander B Magoun. Remembering jan rajchman and the origins of electronic memory, 2015

    2015

  4. [11]

    Selecting metaphoric terminology for the computer industry

    Richard M Chisholm. Selecting metaphoric terminology for the computer industry. Journal of Technical Writing and Communication, 16(3):195–220, 1986

    1986

  5. [12]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems , 2017

    2017

  6. [13]

    Recontextualizing anthropomor- phic metaphors in organization studies: The pathology of organizational insomnia

    Dennis Schoeneborn, Steffen Blaschke, and Ina Maria Kaufmann. Recontextualizing anthropomor- phic metaphors in organization studies: The pathology of organizational insomnia. Journal of Man- agement Inquiry , 22(4):435–450, 2013

    2013

  7. [14]

    Constructing the source: Metaphor as a discourse strategy

    Lionel Wee. Constructing the source: Metaphor as a discourse strategy. Discourse Studies, 7(3):363– 384, 2005

    2005

  8. [15]

    Parole de machines

    Alexei Grinbaum. Parole de machines . Humensciences, 2023

    2023

  9. [16]

    Intentional forgetting

    Deborah Shands and Carolyn Talcott. Intentional forgetting. arXiv preprint arXiv:2106.09802, 2021

    2021 arXiv

  10. [17]

    Automatic, selective and secure deletion of digital evidence

    Aniello Castiglione, Giuseppe Cattaneo, Giancarlo De Maio, and Alfredo De Santis. Automatic, selective and secure deletion of digital evidence. In 2011 International Conference on Broadband and Wireless Computing, Communication and Applications , pages 392–398. IEEE, 2011

    2011

  11. [18]

    Extracting training data from large language models

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , page...

    2021

  12. [19]

    Scalable extraction of training data from (production) language models

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ip- polito, Christopher A Choquette-Choo, Eric Wallace, Florian Tram` er, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:23...

    2023 arXiv

  13. [20]

    Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of ai models, 2024

    European Data Protection Board. Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of ai models, 2024. Adopted on 17 December 2024

    2024

  14. [21]

    Humans forget, machines remember: Artificial intelligence and the right to be forgotten

    Eduard Fosch Villaronga, Peter Kieseberg, and Tiffany Li. Humans forget, machines remember: Artificial intelligence and the right to be forgotten. Computer Law & Security Review , 34(2):304– 313, 2018

    2018

  15. [22]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  16. [23]

    Remembering: A phenomenological study

    Edward S Casey. Remembering: A phenomenological study . Indiana University Press, 2000

    2000

  17. [24]

    New directions in the philosophy of memory

    Kourken Michaelian, Dorothea Debus, and Denis Perrin. New directions in the philosophy of memory. Routledge London and New York, 2018

    2018

  18. [25]

    The cognitive neuroscience of constructive memory: remembering the past and imagining the future

    Daniel L Schacter and Donna Rose Addis. The cognitive neuroscience of constructive memory: remembering the past and imagining the future. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481):773–786, 2007

    2007

  19. [26]

    Divided attention and indirect memory tests

    Neil W Mulligan and Marilyn Hartman. Divided attention and indirect memory tests. Memory & Cognition, 24(4):453–465, 1996

    1996

  20. [27]

    Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. Machine unlearning: A survey. ACM Comput. Surv. , 56(1), 2023

    2023

  21. [28]

    Amnesiac machine learning

    Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. Proceedings of the AAAI Conference on Artificial Intelligence , 35(13):11516–11524, 5 2021

    2021

  22. [29]

    Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141–159, 2021

    2021

  23. [30]

    Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang

    Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security , pages 308–318. ACM, 2016

    2016

  24. [31]

    Differential privacy defenses and sampling attacks for membership inference

    Shadi Rahimian, Tribhuvanesh Orekondy, and Mario Fritz. Differential privacy defenses and sampling attacks for membership inference. InProceedings of the 14th ACM Workshop on Artificial Intelligence and Security (AISec ’21) , page 10. ACM, 2021

  25. [32]

    On the vulnerability of data points under multiple membership inference attacks and target models

    Mauro Conti, Jiaxin Li, and Stjepan Picek. On the vulnerability of data points under multiple membership inference attacks and target models. arXiv preprint arXiv:2210.16258 , 2022

    2022 arXiv

  26. [33]

    The privacy onion effect: Memorization is relative

    Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processin...

    2022

  27. [34]

    Evaluations of machine learning privacy defenses are misleading

    Michael Aerni, Jie Zhang, and Florian Tram` er. Evaluations of machine learning privacy defenses are misleading. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communi- cations Security, CCS ’24, page 1271–1284, New York, NY, USA, 2024. Association for Com...

    2024

  28. [35]

    Project repository. Leakpro. https://github.com/aidotse/LeakPro, 2025. Accessed: 2025-01-16

    2025

  29. [36]

    Why train more? effective and efficient membership inference via memorization

    Jihye Choi, Shruti Tople, Varun Chandrasekaran, and Somesh Jha. Why train more? effective and efficient membership inference via memorization. ArXiv, abs/2310.08015, 2023

    2023 arXiv

  30. [37]

    Smooth sensitivity and sampling in private data analysis

    Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing , pages 75–84. ACM, 2007. 20

    2007

  31. [38]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  32. [39]

    Low-cost high-power membership inference at- tacks

    Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference at- tacks. arXiv preprint arXiv:2312.03262 , 2023

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.