Pith. sign in

REVIEW 4 major objections 5 minor 65 references

Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Standard LLM unlearning evaluations understate privacy leakage for minority data by at least 20%, so the paper proposes reporting worst-case results across random, canary, and minority forget sets.

desk verdict Useful minority-aware evaluation framework for LLM unlearning, but the 'at least 20%' headline is not backed by the paper's own tables and the single-seed design leaves effect sizes uncertain. read the letter →

arxiv 2412.08559 v3 pith:TLUOQNQK submitted 2024-12-11 cs.LG

Rongzhe Wei , Mufei Li , Mohsen Ghassemi , Eleonora Kreačić , Yifan Li , Xiang Yue , Bo Li , Vamsi K. Potluru
show 2 more authors
Pan Li Eli Chien
This is my paper · ORCID
classification cs.LG
keywords machineunlearningprivacyleakagemembershipinferenceattacksminoritypopulationscanaryinjectionlargelanguagemodelslong-tailmemorizationevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the standard pipeline for evaluating LLM unlearning—randomly choosing data to forget and averaging membership-inference leakage—systematically understates privacy risk for data tied to minority groups. To show this, the authors inject canaries whose personal identifiers (e.g., phone area codes) are replaced with the rarest value in the dataset, and they also directly remove the rarest real samples. Across six unlearning methods, three membership-inference attacks, three datasets, and two model scales, privacy leakage for these minority cases is at least 20% higher than for random removal. The paper proposes that unlearning evaluations should report the worst case across random, canary, and minority forget sets, together with worst-case utility. If true, current benchmarks give a false sense of equality in the 'right to be forgotten'.

What carries the argument

The central object is the minority-aware evaluation protocol built from three forget-set constructions: Random (uniform sampling, the status quo), Canary (identical emails with the identifier replaced by the least frequent value, e.g., area code 484), and Minority (the actual least-frequent real samples). The carrying identity is the PrivLeak ratio, $\text{PL} = \Delta\text{AUC} / \text{AUC}(M_{\text{retrain}})$, which measures how much more easily a membership-inference attack distinguishes the unlearned model from a retrained model; the protocol reports the maximum PL across the three cases and the corresponding worst-case perplexity. The canary construction isolates the effect of identifier rarity by holding all other text constant, so any excess leakage is attributable to the minority attribute.

What would settle it

Repeat the evaluation on real minority data with ground-truth membership (e.g., Enron emails from the least-represented senders or ECHR facts mentioning rare years) and compute PrivLeak for minority versus random forget sets; if the gap is consistently below 20% or reverses, the central claim fails. Also, run the canary construction with several different rare tokens (not just 484) and multiple seeds; if the 20% gap disappears for some tokens, the effect is token-specific rather than a general property of rarity.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that unlearning efficacy is not uniform across data points: records with rare, shared attributes—exemplified by the least frequent phone area code—retain measurably more membership signal after unlearning than randomly selected records. The PrivLeak metric, the normalized gap in MIA AUC between the unlearned model and a retrained model, is consistently at least 20% larger for canary and minority forget sets than for random forget sets, and often far larger (e.g., 300–800% on Llama-2). This holds across gradient ascent, NegGrad+, SCRUB, random labels, layer-wise retraining, and Langevin unlearning, and across lossMIA, zlibMIA, and Min-K% attackers. The paper concludes that average-case evaluation misses high-risk partitions of the training data and offers a worst-case, minority-aware protocol as a corrective.

Load-bearing premise

The synthetic canary—an email whose phone area code is replaced by the single rarest code (484)—is a faithful stand-in for real minority population data, so the observed higher leakage reflects a property of minority data rather than an artifact of one artificially repeated rare token.

Editorial extensions

If this is right

  • Unlearning benchmarks that report only average-case PrivLeak understate privacy leakage for minority data by at least 20%, making published method comparisons over-optimistic for these groups.
  • Evaluation suites should add canary and minority forget sets and report the maximum privacy leakage across all three, plus worst-case utility, so a method that fails on any one setting is exposed.
  • Noise-based unlearning (Langevin Unlearning) yields the best privacy-utility trade-off and is stable, whereas gradient ascent and random labels are unstable and can destroy model utility.
  • The protocol applies to any under-represented subset defined by a shared value, not only PII, so privacy audits can be stratified by demographic or other sensitive attributes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If real minority populations behave like the injected 484 canaries, then the 'right to be forgotten' is currently met unequally: members of groups with rare identifiers are less likely to be effectively forgotten than members of majority groups, which bears directly on GDPR compliance audits.
  • The effect's dependence on rarity level is untested: varying the frequency of the injected token (e.g., 2, 5, 100 occurrences) would reveal whether leakage grows monotonically with rarity or is an artifact of the single 484 token.
  • Because all experiments use one random seed (42) and one canary value per dataset, the reported effect sizes lack variance estimates; multi-seed and multi-token replications would strengthen the quantitative claim.
  • A straightforward extension is to audit unlearning fairness: use the max-PL protocol to measure disparate privacy risk across subgroups defined by income, geography, or dialect, and to set per-group unlearning budgets accordingly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that standard LLM unlearning evaluations, which remove randomly chosen samples, systematically underestimate privacy leakage for minority or rare data. It proposes a minority-aware evaluation protocol that reports the maximum privacy leakage (PL) over three forget-set designs: Random, Canary (constructed by replacing PII with the least frequent value), and Minority (real samples with rare PII). Experiments on Enron and ECHR with GPT-2 and Llama-2, across six unlearning methods and three MIAs, find that Canary and Minority cases often exhibit larger PL magnitudes. The paper advertises this as "at least 20% higher privacy leakage" and further benchmarks unlearning methods under the proposed protocol, reporting that Langevin Unlearning offers a favorable privacy-utility trade-off compared with noiseless methods such as SCRUB and Gradient Ascent.

Significance. If the qualitative effect holds, this paper identifies a real blind spot in standard unlearning evaluation and offers a practical, low-cost fix: report worst-case results over several forget-set designs rather than only a randomly sampled one. The empirical scope is broad (six unlearning methods, three MIAs, three datasets, two model scales), the complexity budget is carefully defined, and the code is public. However, the headline "at least 20%" quantitative bound is not supported by the paper's own tables, and the single-seed experimental design leaves effect sizes without uncertainty estimates. The proposed framework is a useful contribution; the specific numerical claim needs correction and statistical grounding.

major comments (4)
  1. [Abstract, §6.1, Table 2] The abstract's claim that minority data "suffer at least 20% higher privacy leakage across various unlearning methods, MIAs, datasets, and LLM scales" is contradicted by the paper's own tables. For example, in Table 2 (GPT-2/Enron-Phone), the Minority excess ratios are +7% (GA, zlibMIA), +13% (NegGrad+, zlibMIA), +12% (Langevin, zlibMIA), and +13% (GA, Min-K%); in the Llama-2 panel, RL shows -65%/-77% for lossMIA and GA shows -27%/-17% for Min-K%. Table 5 (ECHR/GPT-2) contains EUk Minority +10% (lossMIA) and +4% (zlibMIA), and CFk Canary +13% (zlibMIA). The Introduction's "in most cases" and Conclusion's "on average" qualifiers are absent from the abstract. Please correct the abstract and any unqualified statements to match the actual evidence, or provide a statistical basis for a threshold claim.
  2. [App. B.2, §6.1] All experiments use a single random seed (seed 42), with no repeated runs or confidence intervals. Since several reported excess ratios lie between 4% and 20%, the "at least 20%" bound—and even the qualitative ordering in the smallest cells—may be within run-to-run noise. Please report multiple seeds (or bootstrap intervals over the forget-set draws) and significance tests for the key comparisons.
  3. [§4.1, App. A.3, Fig. 3] The canary construction confounds rarity with repetition: all 100 canaries are created by replacing the area code with the single least frequent code (484), so the Canary condition tests one duplicated rare token rather than minority data generally. The Minority condition uses real emails with least-frequent area codes or years, which is more direct evidence, but App. A.3 is ambiguous about how Dminority is formed: Table 1 lists 484 with count 1, so a 100-sample minority set cannot all contain 484; the text alternates between "the least frequent PII" (singular) and "the least frequent area codes" (plural). Please clarify the exact construction of Dminority and report the Minority results as the primary evidence for the real-world claim, with the Canary results interpreted as a controlled audit of a single repeated rare token.
  4. [§6 (General Settings), App. A.3] The forget-set size is stated as 1% of the training set, but App. A.3 says that 100 samples are used for the Random, Canary, and Minority sets. For Llama-2 the training set is 50,000 samples, so 1% would be 500, not 100. Please specify the actual sizes used and adjust the Complexity Unit budget and canary construction accordingly; if 100 was used throughout, the "1%" statement and the cross-scale comparison need revision.
minor comments (5)
  1. [Fig. 17 caption] The caption contains a typo: "Canary & Minoirty settings" should be "Canary & Minority settings."
  2. [§4.1, page 4] The sentence "To rigorously show thatremoving data" is missing a space; it should read "that removing data."
  3. [Table 3 caption] The caption uses "GPT2" while the rest of the paper uses "GPT-2"; please harmonize the notation.
  4. [App. A.3] The preprocessing step "We randomly selected three coherent sentences from each sample" should specify whether it applies to both datasets and whether it affects the context surrounding the PIIs of interest.
  5. [§6, App. A.2] The treatment of "year of judgment" in ECHR as a PII would benefit from a brief justification, since a year alone is not personally identifying; this choice affects how the minority definition is interpreted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the minority-aware evaluation is an empirical protocol whose conclusions rest on measurements, not on constructed equivalences or load-bearing self-citations.

full rationale

The paper's central claim—that forget sets built from rare PII (canaries) or from least-frequent-PII samples yield larger PrivLeak than random forget sets—is an experimental finding. The PrivLeak metric (Eq. 1) is defined as normalized AUC difference between unlearned and retrained models; the Random, Canary, and Minority cases are three distinct inputs to that same metric, and their comparison is not forced by the metric's definition. No parameter is fitted to the compared cells and then reported as a prediction; the 20% figure is a summary of measured excess ratios, and the fact that some cells fall below 20% is a quantitative-support issue (correctness risk), not circularity. The canary construction (Section 4.1) replaces only the PII token to isolate the effect of rarity; while the generalization from injected rare tokens to real minority populations is an extrapolation that could be questioned, that is again an external-validity concern, not a circular reduction. Self-citations (e.g., Chien et al. for Langevin Unlearning) supply the method and motivational theory; the paper's evaluation of Langevin is empirical and its privacy-utility conclusion is based on the reported measurements, so the citations are not load-bearing in a circular sense. No uniqueness theorem or ansatz is imported from the authors' prior work to force the chosen evaluation design.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper is empirical; the central claim depends on hand-chosen experimental thresholds (compute budget, early stopping, noise scales) rather than on fitted model parameters. The main axioms are domain assumptions about the validity of the PrivLeak metric and the representativeness of the chosen datasets and attacks.

free parameters (3)
  • Complexity unit budget = 10
    Chosen by hand to equalize computational cost across unlearning methods; the selected epoch checkpoint, and hence the reported PL values, depends on this budget.
  • Epoch selection perplexity threshold = increase of 1 point over initial model
    Unlearning stops when perplexity on D_train rises by more than 1 point; this hand-chosen criterion sets the operating point for every method and directly influences the measured privacy leakage.
  • Langevin noise scale = 5e-4 (GPT-2), 5e-3 (Llama-2)
    Hand-tuned noise level for the DP-SGD-based Langevin unlearning; controls the privacy-utility trade-off and affects the PL values.
assumptions (4)
  • domain assumption PrivLeak (AUC-normalized difference between unlearned and retrained models) is a valid measure of unlearning privacy leakage.
    Section 3.1 defines PrivLeak and uses it as the sole privacy metric; all claims are expressed in terms of this quantity.
  • domain assumption The three MIAs (lossMIA, zlibMIA, Min-K%) are sufficient to assess membership privacy after unlearning.
    The paper uses only these three attacks and generalizes from their average behavior to "privacy leakage".
  • domain assumption The retrained model is the correct gold-standard target for unlearning.
    Standard in the unlearning literature; the paper compares all unlearned models against a model retrained on the keep set.
  • domain assumption Fine-tuning on subsampled Enron and ECHR subsets is representative of LLM unlearning in practice.
    The experiments use 10k/50k samples from two PII datasets; the conclusion about "minority populations" assumes these settings capture real-world unlearning conditions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning." pith.science (2026). https://pith.science/paper/TLUOQNQK

@misc{pith2026241208559,
  author       = {Pith},
  title        = {Pith review of: Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TLUOQNQK}},
  note         = {Machine review of arXiv:2412.08559}
}
read the original abstract

Large Language Models (LLMs) embed sensitive, human-generated data, prompting the need for unlearning methods. Although certified unlearning offers strong privacy guarantees, its restrictive assumptions make it unsuitable for LLMs, giving rise to various heuristic approaches typically assessed through empirical evaluations. These standard evaluations randomly select data for removal, apply unlearning techniques, and use membership inference attacks (MIAs) to compare unlearned models against models retrained without the removed data. However, to ensure robust privacy protections for every data point, it is essential to account for scenarios in which certain data subsets face elevated risks. Prior research suggests that outliers, particularly including data tied to minority groups, often exhibit higher memorization propensity which indicates they may be more difficult to unlearn. Building on these insights, we introduce a complementary, minority-aware evaluation framework to highlight blind spots in existing frameworks. We substantiate our findings with carefully designed experiments, using canaries with personally identifiable information (PII) to represent these minority subsets and demonstrate that they suffer at least 20% higher privacy leakage across various unlearning methods, MIAs, datasets, and LLM scales. Our proposed minority-aware evaluation framework marks an essential step toward more equitable and comprehensive assessments of LLM unlearning efficacy.

Figures

Figures reproduced from arXiv: 2412.08559 by the authors.

Figure 1
Figure 1. Illustration of our proposed LLM unlearning approaches (highlighted in red), when compared with the existing pipeline. Standard LLM unlearning evaluation typically involves randomly sampling data for removal from the training set (Case 1), which may underestimate privacy leakage for minority groups. In contrast, we design experiments to assess unlearning efficacy by removing canaries (deliberately inserted data poin… view at source ↗
Figure 2
Figure 2. Area code histogram in Enron Dataset. minority data, such as emails with rare area codes like 484, are un￾likely to be included due to their lower frequency. If unlearning minority data is inherently more challenging and results in greater privacy leak￾age, the existing evalua￾tion pipeline may under￾estimate privacy risks for minorities. 4.1. Verify Underestimated Privacy Risks of Minority via Canary Injection To r… view at source ↗
Figure 3
Figure 3. Illustration of the forget set Dforget, the construction of the canary set Dcanary, and the minority set Dminority for the Enron dataset. The minority set consists of emails with phone numbers containing the least frequent area codes. A histogram and distribution of area codes (e.g., 713 as the most frequent and 484 as the least frequent) are shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Benchmarking unlearning approaches via our minority-aware evaluation for GPT-2 and Llama-2 on Enron-Phone (Left) and Enron-Email (Right) datasets. (a),(c),(e),(g): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for Min-K% attack. (b),(d),(…
Figure 6
Figure 6. Figure 6: Privacy-utility Trade￾off Curves for GPT-2. these curves are evaluated on the Enron-Phone and Enron-Email datasets. The results indicate that Langevin Unlearning (Green Line) outperforms SCRUB (Purple Line) with a superior privacy￾utility trade-off. Notably, while GA p…
Figure 5
Figure 5. Figure 5: Ablation studies on unlearning iterations and forget set size. terms of stability. 7. Conclusions We identify a critical limitation in the typical evaluation pipeline for LLM unlearning efficacy: privacy risks to mi￾nority groups in the training data are often underest…
Figure 7
Figure 7. Figure 7: Histogram of email addresses (Enron-Email) and years (ECHR-Year). A.3. Preprocessing and Dataset Split Dataset Preprocess. In our experiments, since the average token length of Enron samples is approximately 770, we controlled the token length of each fact to ensure th…
Figure 8
Figure 8. Figure 8: Benchmarking unlearning approaches via our minority-aware evaluation for GPT-2 and Llama-2 on Enron-Phone and Enron￾Email dataset. (a),(c),(e),(g): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for lossMIA attack. (b),(d),(f),(h): Maximum…
Figure 9
Figure 9. Figure 9: Benchmarking unlearning approaches via our minority-aware evaluation for GPT-2 on ECHR-year dataset. (a)-(c): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for lossMIA, zlibMIA, and Min-K% attacks respectively. (d): Worst perplexity over …
Figure 10
Figure 10. Figure 10: Benchmarking unlearning approaches via our minority-aware evaluation for Llama-2 on ECHR Year dataset. (a)-(c): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for lossMIA, zlibMIA, and Min-K% attacks respectively. (d): Worst perplexity ov…
Figure 11
Figure 11. Figure 11: Benchmarking unlearning approaches via our minority-aware evaluation for GPT-2 on Enron-email dataset. (a)-(c): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for lossMIA, zlibMIA, and Min-K% attacks respectively. (d-f): Worst utility per…
Figure 12
Figure 12. Figure 12: Benchmarking unlearning approaches via our minority-aware evaluation for Llama-2 on Enron-email dataset. (a)-(c): Maximum privacy leakage (PL) over three cases (Random, Canary, and Minority) for lossMIA, zlibMIA, and Min-K% attacks respectively. (d-f): Worst utility p…
Figure 13
Figure 13. Figure 13: The effect of forget set size for each unlearning approach. (a)(b): Maximum PL over three cases (Random, Canary, Minority) with the attacker being lossMIA and zlibMIA respectively. C.6. More Details on the Privacy-Utility Trade-off Curves for Langevin Unlearning and S…
Figure 14
Figure 14. Figure 14: Privacy-utility transition curves for Enron-Phone dataset with hyperparameter β = 1 (Top) and hyperparameter β = 1e − 3 (Bottom). Langevin Unlearning. For the Langevin Unlearning method, we fix the clipping norm to 1 and vary the noise scale added during training to c…
Figure 15
Figure 15. Figure 15: Privacy-utility transition curves for Enron-Email dataset with hyperparameter β = 1 (Top) and hyperparameter β = 1e − 3 (Bottom) [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Privacy-utility trade-off curves under lossMIA and zlibMIA. to the results demonstrated in Sec. 6.3, Langevin Unlearning method achieves the best trade-off performance over SCRUB method. C.7. Privacy Underestimation Across Complexity Units 2 4 10 Complexity Unit 10 2 …
Figure 17
Figure 17. Figure 17: Degree of the largest underestimation in privacy leakage (Canary, Minority) com￾pared to Random settings across varying Com￾plexity Units. In [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Privacy-Utility Trade-off for Enron-Phone Min-k% with Retrain (Same Computation Budget). C.9. Results on AUC Scores, Perplexity across Different Models and Datasets We further report the AUC scores under different attackers (lossMIA, zlibMIA, Min-K%) and utility (perp…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 25 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    B., Mironov, I., Talwar, K., and Zhang, L

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS '16, pp.\ 308–318, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450341394. doi:10.1145/2976749.2978318. U...

    arXiv 2016

  3. [3]

    Evaluations of machine learning privacy defenses are misleading

    Aerni, M., Zhang, J., and Tram \`e r, F. Evaluations of machine learning privacy defenses are misleading. In ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024. URL https://arxiv.org/abs/2404.17399

    arXiv 2024

  4. [4]

    A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N

    Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp.\ 141--159. IEEE, 2021

    2021

  5. [5]

    and Yang, J

    Cao, Y. and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp.\ 463--480. IEEE, 2015

    2015

  6. [6]

    The secret sharer: Evaluating and testing unintended memorization in neural networks

    Carlini, N., Liu, C., Erlingsson, \'U ., Kos, J., and Song, D. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), pp.\ 267--284, 2019

    2019

  7. [7]

    Extracting training data from large language models

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp.\ 2633--2650, 2021

    2021

  8. [8]

    Membership inference attacks from first principles

    Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tramer, F. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp.\ 1897--1914. IEEE, 2022

    2022

Show all 65 references
  1. [9]

    Neural legal judgment prediction in english

    Chalkidis, I., Androutsopoulos, I., and Aletras, N. Neural legal judgment prediction in english. arXiv preprint arXiv:1906.02059, 2019

    1906 arXiv

  2. [10]

    and Yang, D

    Chen, J. and Yang, D. Unlearn what you want to forget: Efficient unlearning for LLM s. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https://openreview.net/forum?id=GprvtTwOxy

    2023

  3. [11]

    Efficient model updates for approximate unlearning of graph-structured data

    Chien, E., Pan, C., and Milenkovic, O. Efficient model updates for approximate unlearning of graph-structured data. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=fhcu4FBLciL

    2023

  4. [12]

    Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning

    Chien, E., Wang, H., Chen, Z., and Li, P. Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning. arXiv preprint arXiv:2401.10371, 2024 a

    2024

  5. [13]

    P., Chen, Z., and Li, P

    Chien, E., Wang, H. P., Chen, Z., and Li, P. Certified machine unlearning via noisy stochastic gradient descent. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 b . URL https://openreview.net/forum?id=h3k2NXu5bJ

    2024

  6. [14]

    Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024

    Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L., Tsvetkov, Y., Choi, Y., Evans, D., and Hajishirzi, H. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024

    2024 arXiv

  7. [15]

    and Zhang, C

    Feldman, V. and Zhang, C. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33: 0 2881--2891, 2020

    2020

  8. [16]

    Towards adversarial evaluations for inexact machine unlearning

    Goel, S., Prabhu, A., Sanyal, A., Lim, S.-N., Torr, P., and Kumaraguru, P. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640, 2022

    2022 arXiv

  9. [17]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9304--9312, 2020 a

    2020

  10. [18]

    Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations

    Golatkar, A., Achille, A., and Soatto, S. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXIX 16, pp.\ 38...

    2020

  11. [19]

    Amnesiac machine learning

    Graves, L., Nagisetty, V., and Ganesh, V. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 11516--11524, 2021

    2021

  12. [20]

    Certified data removal from machine learning models

    Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. Certified data removal from machine learning models. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp....

    2020

  13. [21]

    Inexact unlearning needs more careful evaluations to avoid a false sense of privacy

    Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., and Papernot, N. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. arXiv preprint arXiv:2403.01218, 2024

    2024 arXiv

  14. [22]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  15. [23]

    T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A

    Ilharco, G., Ribeiro, M. T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=6t0Kwf8-jrj

    2023

  16. [24]

    Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems, 33: 0 22205--22216, 2020

    Jagielski, M., Ullman, J., and Oprea, A. Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems, 33: 0 22205--22216, 2020

    2020

  17. [25]

    Knowledge unlearning for mitigating privacy risks in language models

    Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., and Seo, M. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504, 2022

    2022 arXiv

  18. [26]

    Knowledge unlearning for mitigating privacy risks in language models

    Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., and Seo, M. Knowledge unlearning for mitigating privacy risks in language models. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational...

    2023 doi

  19. [27]

    and Yang, Y

    Klimt, B. and Yang, Y. Introducing the enron corpus. In CEAS, volume 4, pp.\ 1, 2004

    2004

  20. [28]

    GDPR: General Data Protection Regulation (EU) 2016/679: Post-reform Personal Data Protection in the European Union

    Krzysztofek, M. GDPR: General Data Protection Regulation (EU) 2016/679: Post-reform Personal Data Protection in the European Union. Kluwer Law International BV, 2018

    2016

  21. [29]

    Towards unbounded machine unlearning

    Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. Towards unbounded machine unlearning. Advances in neural information processing systems, 36, 2024

    2024

  22. [30]

    Llm-pbe: Assessing data privacy in large language models

    Li, Q., Hong, J., Xie, C., Tan, J., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., Li, B., He, B., and Song, D. Llm-pbe: Assessing data privacy in large language models. Proc. VLDB Endow., 17 0 (11): 0 3201–3214, August 2024 a . ISSN 2150-8097. doi:10.14778/3681...

    2024

  23. [31]

    S., Prabhu, A., and Sanyal, A

    Li, W., Li, J., de Witt, C. S., Prabhu, A., and Sanyal, A. Delta-influence: Unlearning poisons via influence functions. arXiv preprint arXiv:2411.13731, 2024 b

    2024

  24. [32]

    Rouge: A package for automatic evaluation of summaries

    Lin, C.-Y. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp.\ 74--81, 2004

    2004

  25. [33]

    Model sparsity can simplify machine unlearning

    Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., SHARMA, P., Liu, S., et al. Model sparsity can simplify machine unlearning. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  26. [34]

    R., et al

    Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Xu, X., Yao, Y., Li, H., Varshney, K. R., et al. Rethinking machine unlearning for large language models. arXiv preprint arXiv:2402.08787, 2024 b

    2024 arXiv

  27. [35]

    Breaking the trilemma of privacy, utility, and efficiency via controllable machine unlearning

    Liu, Z., Dou, G., Chien, E., Zhang, C., Tian, Y., and Zhu, Z. Breaking the trilemma of privacy, utility, and efficiency via controllable machine unlearning. In Proceedings of the ACM on Web Conference 2024, pp.\ 1260--1271, 2024 c

    2024

  28. [36]

    Analyzing leakage of personally identifiable information in language models

    Lukas, N., Salem, A., Sim, R., Tople, S., Wutschitz, L., and Zanella-B \'e guelin, S. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP), pp.\ 346--363. IEEE, 2023

    2023

  29. [37]

    C., and Kolter, J

    Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., and Kolter, J. Z. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121, 2024

    2024 arXiv

  30. [38]

    Adversary instantiation: Lower bounds for differentially private machine learning

    Nasr, M., Songi, S., Thakurta, A., Papernot, N., and Carlin, N. Adversary instantiation: Lower bounds for differentially private machine learning. In 2021 IEEE Symposium on security and privacy (SP), pp.\ 866--882. IEEE, 2021

    2021

  31. [39]

    Tight auditing of differentially private machine learning

    Nasr, M., Hayes, J., Steinke, T., Balle, B., Tram \`e r, F., Jagielski, M., Carlini, N., and Terzis, A. Tight auditing of differentially private machine learning. In 32nd USENIX Security Symposium (USENIX Security 23), pp.\ 1631--1648, 2023

    2023

  32. [40]

    Descent-to-delete: Gradient-based methods for machine unlearning

    Neel, S., Roth, A., and Sharifi-Malvajerdi, S. Descent-to-delete: Gradient-based methods for machine unlearning. In Algorithmic Learning Theory, pp.\ 931--962. PMLR, 2021

    2021

  33. [41]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  34. [42]

    Z., Lu, Y., Kamath, G., Sekhari, A., and Neel, S

    Pawelczyk, M., Di, J. Z., Lu, Y., Kamath, G., Sekhari, A., and Neel, S. Machine unlearning fails to remove data poisoning attacks. arXiv preprint arXiv:2406.17216, 2024 a

    2024

  35. [43]

    In-context unlearning: Language models as few-shot unlearners

    Pawelczyk, M., Neel, S., and Lakkaraju, H. In-context unlearning: Language models as few-shot unlearners. In Forty-first International Conference on Machine Learning, 2024 b . URL https://openreview.net/forum?id=GKcwle8XC9

    2024

  36. [44]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  37. [45]

    Sekhari, A., Acharya, J., Kamath, G., and Suresh, A. T. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems, 34: 0 18075--18086, 2021

    2021

  38. [46]

    Detecting pretraining data from large language models

    Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L. Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789, 2023

    2023 arXiv

  39. [47]

    Detecting pretraining data from large language models

    Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024 a . URL https://openreview.net/forum?id=zWqr3MQuNs

    2024

  40. [48]

    A., and Zhang, C

    Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtzman, A., Liu, D., Zettlemoyer, L., Smith, N. A., and Zhang, C. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460, 2024 b

    2024 arXiv

  41. [49]

    Membership inference attacks against machine learning models

    Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

    2017

  42. [50]

    and Ullman, J

    Steinke, T. and Ullman, J. The pitfalls of average-case differential privacy. DifferentialPrivacy.org, 07 2020. https://differentialprivacy.org/average-case-dp/

    2020

  43. [51]

    Privacy auditing with one (1) training run

    Steinke, T., Nasr, M., and Jagielski, M. Privacy auditing with one (1) training run. Advances in Neural Information Processing Systems, 36, 2024

    2024

  44. [52]

    On the necessity of auditable algorithmic definitions for machine unlearning

    Thudi, A., Jia, H., Shumailov, I., and Papernot, N. On the necessity of auditable algorithmic definitions for machine unlearning. In 31st USENIX Security Symposium (USENIX Security 22), pp.\ 4007--4022, 2022

    2022

  45. [53]

    Gradients look alike: Sensitivity is often overestimated in \ DP-SGD \

    Thudi, A., Jia, H., Meehan, C., Shumailov, I., and Papernot, N. Gradients look alike: Sensitivity is often overestimated in \ DP-SGD \ . In 33rd USENIX Security Symposium (USENIX Security 24), pp.\ 973--990, 2024

    2024

  46. [54]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  47. [55]

    and Arora, R

    Ullah, E. and Arora, R. From adaptive query release to machine unlearning. In International Conference on Machine Learning, pp.\ 34642--34667. PMLR, 2023

    2023

  48. [56]

    A., and Arora, R

    Ullah, E., Mai, T., Rao, A., Rossi, R. A., and Arora, R. Machine unlearning via algorithmic stability. In Conference on Learning Theory, pp.\ 4126--4142. PMLR, 2021

    2021

  49. [57]

    Depn: Detecting and editing privacy neurons in pretrained language models

    Wu, X., Li, J., Xu, M., Dong, W., Wu, S., Bian, C., and Xiong, D. Depn: Detecting and editing privacy neurons in pretrained language models. arXiv preprint arXiv:2310.20138, 2023

    2023 arXiv

  50. [58]

    Machine unlearning of pre-trained large language models

    Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., and Yue, X. Machine unlearning of pre-trained large language models. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1:...

    2024

  51. [59]

    Large language model unlearning

    Yao, Y., Xu, X., and Liu, Y. Large language model unlearning. In Socially Responsible Language Modelling Research, 2023. URL https://openreview.net/forum?id=wKe6jE065x

    2023

  52. [60]

    K., Bindschaedler, V., and Shokri, R

    Ye, J., Maddi, A., Murakonda, S. K., Bindschaedler, V., and Shokri, R. Enhanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp.\ 3093--3106, 2022

    2022

  53. [61]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp.\ 268--282. IEEE, 2018

    2018

  54. [62]

    Negative preference optimization: From catastrophic collapse to effective unlearning

    Zhang, R., Lin, L., Bai, Y., and Mei, S. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868, 2024

    2024 arXiv

  55. [63]

    V., et al

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  56. [64]

    Q., and Artzi, Y

    Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  57. [65]

    What makes unlearning hard and what to do about it

    Zhao, K., Kurmanji, M., B a rbulescu, G.-O., Triantafillou, E., and Triantafillou, P. What makes unlearning hard and what to do about it. arXiv preprint arXiv:2406.01257, 2024

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.