Pith. sign in

REVIEW 4 major objections 6 minor 68 references

What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an LLM's memorization of an individual's facts can be measured per person by ranking the true fact against counterfactual values across paraphrased prompts, and that this enables building forget sets for…

desk verdict A genuinely useful dataset for individual-level privacy auditing, but the core NLL ranking metric has a length-normalization problem that could undermine the headline trends. read the letter →

arxiv 2507.11128 v1 pith:LHGCFEXD submitted 2025-07-15 cs.CL cs.CYcs.LG

Dimitri Staufer This is my paper
classification cs.CLcs.CYcs.LG
keywords righttobeforgottenmachineunlearningmemorizationquantificationpersonaldataWikidataLLMprivacyauditnegativelog-likelihoodrankingcounterfactualcanaries
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's goal is to turn the right to be forgotten from a request about a person into a concrete list of facts to delete from a model. It argues that existing unlearning methods assume such a list exists, while existing privacy audits only say whether a model leaks training data on average, not what it knows about one specific person. To close that gap, it introduces WikiMem, a dataset of natural-language probes built from Wikidata, covering 243 properties such as occupation, birthplace, and even 'convicted of', and a scoring rule that ranks the true value for a person above 100 counterfactual values. The rule counts an association as memorized when the true value ranks first across all paraphrased phrasings, and measures strength by how far the true value's score stands above its nearest competitor. Across 15 models, the paper reports that memorized personal facts are more common for well-known people and for larger models, and that the same procedure can therefore mark which facts should go into a forget set.

What carries the argument

The load-bearing object is the canary plus the calibrated rank test. A canary is a natural-language sentence asserting a triple $(h, p, v)$, with placeholders for subject and value; WikiMem provides 5,650 such sentences from 243 human-related Wikidata properties, each with 100 counterfactual human-value pairs. The score subtracts from the negative-log-likelihood difference between a generic subject and the named subject a correction averaged over similar-looking name variants, then ranks candidate values; any ground truth at rank 1 marks the association as memorized. Memorization strength $z^*$ is the standardized lead margin of the top-ranked ground truth over the best counterfactual, computed across each candidate's margin to its nearest competitor.

What would settle it

Take a set of fabricated individuals who cannot appear in any training corpus, give each a plausible occupation, birthplace, and citizenship in WikiMem's template set, and run the calibrated rank test; if rank-1 and $z^*>3$ occur for these invented people at rates comparable to the real lesser-known cohort, the metric is detecting language priors or Wikidata-style phrasing rather than memorization of that person's data.

Watch

Extended reading notes

Core claim

The central claim is that individual-level memorization of personal data is measurable, model-agnostically, by comparing the model's negative log-likelihood for a true subject-property-value triple against a set of type-consistent counterfactuals across paraphrased templates. The paper defines memorization operationally: a ground-truth value must rank first in a calibrated score, with the subject's own name and a set of similar-looking name variants used to subtract name-prior and generic-person effects. It reports that this criterion separates well-known from lesser-known subjects, for example 38.8 percent versus 22.4 percent memorized properties for one 8-billion-parameter model, and that memorization strength $z^*$ grows with scale even when coverage plateaus. The point of the operational definition is that the resulting per-entity fact list is exactly the forget set that unlearning methods currently assume as input.

Load-bearing premise

The method assumes that ranking the true Wikidata value above 100 counterfactuals in every paraphrased prompt is evidence the model memorized that person's fact, not evidence that the model prefers grammatically or factually plausible phrasing.

Editorial extensions

If this is right

  • A regulator or data subject could use the method to audit a specific model and obtain a per-person list of memorized facts, turning an RTBF request into a concrete forget set.
  • Because the metric needs only token log-probabilities, it transfers to black-box models whose probabilities can be reconstructed, extending the same audit to deployed systems.
  • The measured memorization rate being higher for well-known people and for larger models implies that deletion obligations, if any, would fall unevenly: famous individuals and large models would produce longer forget lists.
  • Instruction-tuned models show higher memorization except for sex or gender, which drops near zero, suggesting that alignment changes which facts survive and where the risky associations remain.
  • The strict ranking criterion, requiring all 11 paraphrases to rank the true value first, makes the resulting memorization labels conservative; a lenient criterion yields above 80 percent for well-known subjects, so the method can be tuned to legal thresholds of accessibility by reasonably likely means.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not say this, but the same calibrated score can serve as a post-unlearning check: after a forget set is removed, a successful deletion would be one where the ground truth no longer ranks first.
  • The metric quantifies associative memorization rather than verbatim leakage; a direct comparison with exact-string extraction on the same subjects would show how much of what the method flags is eidetic versus approximate.
  • The per-property findings imply a potential priority order for deletion: facts like birthplace and occupation are readily flagged, while sex or gender flags collapse under instruction tuning, so an RTBF policy tuned to this metric would spend most of its budget on the high-memorization properties.
  • The well-known versus lesser-known gap suggests an unremarked policy consequence: ordinary people may receive near-empty forget lists from this audit, which could conflict with a legal reading of the GDPR if the associations are still inferable from the model even when they do not rank first.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes WikiMem, a Wikidata-derived set of natural-language canaries for 243 human properties, together with a model-agnostic metric that scores a candidate value's negative log-likelihood (NLL) in a prompt template, calibrates the score with generic and similar-name subjects, and declares memorization when a ground-truth value ranks first among 100 counterfactuals. Using this metric, the authors evaluate 200 public figures across five properties and a set of open-weight LLMs from 410M to 70B parameters, reporting higher memorization rates for subjects with high web presence and for larger models, and discuss implications for Right-to-Be-Forgotten requests and machine-unlearning forget sets.

Significance. If the metric is a valid probe of individual-level memorization, the contribution is useful: it is a large, property-diverse canary set for personal facts; the approach is model-agnostic and relatively inexpensive (requiring only NLL queries); and the empirical regularities with web presence and scale are practically relevant for RTBF compliance. The use of an external ground truth (Wikidata) avoids circularity in the association scores, and the authors promise to release the dataset and evaluation code. However, the paper's central empirical claims currently rest on a length-sensitive score and an ambiguous aggregation rule, and the paper itself concedes that the metric conflates memorization with canonical-phrasing and grammaticality effects. These issues must be addressed before the memorization interpretation is warranted.

major comments (4)
  1. [Section 5] The score s(h,v) is built from sums of token-level log-likelihoods, so the value-token portion contains T_v terms, where T_v is the number of tokens in the candidate value v. The generic-subject and similar-name terms share the same v, so they do not equate the scale of scores across candidates of different lengths; a multi-token ground truth such as 'United States of America' has more chances to accumulate positive per-token differences against a shorter counterfactual such as 'Canada' and can rank first even if the model has no subject-specific association. Since the rank-1 memorization criterion and the z* strength measure are computed from these unnormalized scores, the headline rates in Table 1 and the web-presence/scale trends may be partly artifacts of value length. Please add a length-normalized variant (for example, mean per-token log-likelihood difference over the value span) and rerun the main analyses.
  2. [Section 7] The definition of the reported memorization rate M is internally inconsistent. The text first says M is 'the percentage of paraphrased templates per subject–property pair that result in a rank-1 prediction' and then states that 'if just one template variant fails for a given property, it does not count as memorized.' These two sentences describe different quantities: a per-template success rate versus an all-or-nothing subject-property binary. Please state exactly how Table 1's M is aggregated (for instance, the fraction of subject-property pairs for which all 11 variants produce rank-1), and move that explicit definition to Section 5 where memorization is introduced.
  3. [Section 6 and Table 1] The abstract and Section 6 say that 15 LLMs were evaluated, but Table 1 reports only 9 models. Please report results for the remaining 6 models or revise the claim. In particular, the instruction-tuning comparison in Section 7 ('for all models we tested') is only shown for LLaMA-3.1-8B in Figure 1, so the reader cannot verify that claim for the other model families.
  4. [Section 7 Limitations and Section 5] The central claim that WikiMem identifies which individual-fact associations are 'stored' in an LLM is stronger than what the metric actually measures. Ranking a ground-truth value first among 100 counterfactuals is a relative likelihood test, and as the Limitations paragraph admits, it 'may partially conflate factual recall with a model's ability to parse or produce canonical Wikidata phrasing, or worse, prefer counterfactuals mainly because of grammatical correctness.' Because this conflation is acknowledged as unresolved, the abstract and conclusions should either be reframed in terms of probe-recoverable associations or supplemented with control experiments (for example, random or synthetic subject-value pairs that cannot have been memorized) to support the memorization interpretation.
minor comments (6)
  1. [Section 7] The claim that contextualized canaries 'consistently lowered memorization rates and strengths across all models' is not accompanied by any table or figure; please add the data or remove the claim.
  2. [Section 4 and Abstract] The abstract says 'over 5,000 natural language canaries' while the text reports 5,650 templates; please clarify whether canaries are instantiated (subject, property, value, template) tuples and report the final count.
  3. [Section 5] The choice alpha=1 for the similar-name adjustment is not justified and no sensitivity analysis is given; at least report whether the rank-1 decisions are stable for alpha in a reasonable interval.
  4. [Table 1] No confidence intervals or significance tests accompany the high- versus low-web-presence comparisons; with 100 subjects per cohort, standard errors or bootstrap intervals would make the claimed gaps more informative.
  5. [Section 3] The phrase 'This has lead to' should be 'This has led to'.
  6. [Section 7] The statement about Mistral-Small-24B suggesting 'filtered or deduplicated data' is speculative and not supported by the manual cross-check described there; consider softening the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: WikiMem's metric is a transparent operational definition with external Wikidata ground truth and no fitted parameters; the reported trends are independent empirical comparisons.

full rationale

The derivation chain in this paper is self-contained rather than circular. The association score s(h,v) in Section 5 is defined directly from token-level negative log-likelihoods computed on external language models, with a fixed calibration constant α = 1 and no parameter fitted to the reported memorization rates. The ground-truth values and counterfactuals come from Wikidata, an external knowledge base, not from the models being evaluated. Memorization is declared by an explicit, pre-specified criterion (any ground-truth value ranks first among 100 counterfactuals), and strength z* is a standardized margin computed from the same scores. This is an operational definition of the paper's own construct, not a hidden reuse of its conclusions: the headline claims that memorization correlates with web presence and model scale are comparisons against externally measured variables, and those correlations could have failed. The paper's own Limitations section acknowledges that the metric may partly capture canonical Wikidata phrasing or grammaticality rather than pure factual recall; that is a construct-validity concern, not a circularity. There are no load-bearing self-citations, no fitted inputs renamed as predictions, and no imported uniqueness theorem. The unnormalized-NLL length dependence raised by skeptics, if real, would be a measurement confound affecting validity, but it does not make any result equivalent to its inputs by construction. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a few hand-chosen thresholds and domain assumptions about Wikidata and NLL as a memory probe; no new entities are introduced.

free parameters (3)
  • alpha = 1
    Weight of the similar-name adjustment in the calibrated NLL score, chosen by hand in Section 5 without a fitting procedure.
  • cosine similarity threshold = 0.75
    Threshold for treating similar ground-truth values like 'English' and 'British English' as equal in the rank-1 rule, chosen by hand in Section 5.
  • number of counterfactuals = 100
    Fixed counterfactual pool size per subject-property pair used in ranking, chosen in Section 6 and not varied.
assumptions (4)
  • domain assumption Wikidata is an accurate source of ground-truth human-fact associations.
    The dataset and evaluation rely on Wikidata labels as ground truth for personal facts, which may be outdated, incomplete, or culturally biased (Section 4 and Ethics section).
  • domain assumption Rank-1 among counterfactuals indicates memorization.
    The metric defines memorization as rank-1 in Section 5; the paper acknowledges this may conflate with grammatical priors in Section 7.
  • domain assumption NLL scores from quantized models are close enough to full-precision scores.
    The paper states negligible differences between 4-bit and full precision, but the claim is based on the authors' observation without supporting data (Section 6).
  • domain assumption Paraphrase variants generated by FLAN-T5XL and selected by o4-mini-high are valid probes.
    The paraphrases are not verified with human ratings; the selection relies on an LLM judge (Section 4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests." pith.science (2026). https://pith.science/paper/LHGCFEXD

@misc{pith2026250711128,
  author       = {Pith},
  title        = {Pith review of: What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LHGCFEXD}},
  note         = {Machine review of arXiv:2507.11128}
}
read the original abstract

Large Language Models (LLMs) can memorize and reveal personal information, raising concerns regarding compliance with the EU's GDPR, particularly the Right to Be Forgotten (RTBF). Existing machine unlearning methods assume the data to forget is already known but do not address how to identify which individual-fact associations are stored in the model. Privacy auditing techniques typically operate at the population level or target a small set of identifiers, limiting applicability to individual-level data inquiries. We introduce WikiMem, a dataset of over 5,000 natural language canaries covering 243 human-related properties from Wikidata, and a model-agnostic metric to quantify human-fact associations in LLMs. Our approach ranks ground-truth values against counterfactuals using calibrated negative log-likelihood across paraphrased prompts. We evaluate 200 individuals across 15 LLMs (410M-70B parameters), showing that memorization correlates with subject web presence and model scale. We provide a foundation for identifying memorized personal data in LLMs at the individual level, enabling the dynamic construction of forget sets for machine unlearning and RTBF requests.

Figures

Figures reproduced from arXiv: 2507.11128 by the authors.

Figure 1
Figure 1. Comparison of per-property memorization rates over all paraphrased template variants, for the base and instruct-tuned Llama-3.1-8B. diversity suffices to empirically show memorization with high confidence. For properties with multiple values, we accept any valid value without checking for completeness or temporal correctness (e.g., “worked for” or “works for”). Our benchmark relies on Wikidata labels and aliases, wh… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 31 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems 36, 66044–66063 (2023)

    Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., Biderman, S.: Leace: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems 36, 66044–66063 (2023)

    work page 2023

  2. [2]

    In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

    Bertram, T., Bursztein, E., Caro, S., Chao, H., Chin Feman, R., Fleischer, P., Gustafsson, A., Hemerly, J., Hibbert, C., Invernizzi, L., Kammourieh Donnelly, L., Ketover, J., Laefer, J., Nicholas, P., Niu, Y., Obhi, H., Price, D., Strait, A., Thomas, K., Verney, A.: Five Years of the Right to be Forgotten. In: Proceedings of the 2019 ACM SIGSAC Conference...

    arXiv 2019

  3. [3]

    Advances in Neural Information Processing Systems36, 28072–28090 (2023)

    Biderman, S., Prashanth, U., Sutawika, L., Schoelkopf, H., Anthony, Q., Purohit, S., Raff, E.: Emergent and predictable memorization in large language models. Advances in Neural Information Processing Systems36, 28072–28090 (2023)

    work page 2023

  4. [4]

    In: International Conference on Machine Learning

    Biderman, S., Schoelkopf, H., Anthony, Q.G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M.A., Purohit, S., Prashanth, U.S., Raff, E., et al.: Pythia: A suite for analyzing large language models across training and scaling. In: International Conference on Machine Learning. pp. 2397–2430. PMLR (2023)

    work page 2023

  5. [5]

    Artificial Intelligence Review58(3), 90 (Jan 2025)

    Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo- Ferrer, J., Collell, G., Eeik Tan, K.: Digital forgetting in large language models: A survey of unlearning methods. Artificial Intelligence Review58(3), 90 (Jan 2025). https://doi.org/10.1007/s10462-024-11078-6

    work page doi:10.1007/s10462-024-11078-6 2025

  6. [6]

    In: 2021 IEEE symposium on security and privacy (SP)

    Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., Papernot, N.: Machine unlearning. In: 2021 IEEE symposium on security and privacy (SP). pp. 141–159. IEEE (2021)

    2021

  7. [7]

    Computer networks and ISDN systems30(1-7), 107–117 (1998)

    Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems30(1-7), 107–117 (1998)

    1998

  8. [8]

    In: The Eleventh International Conference on Learning Representations (2022)

    Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., Zhang, C.: Quantify- ing memorization across neural language models. In: The Eleventh International Conference on Learning Representations (2022)

    work page 2022

Show all 68 references
  1. [9]

    In: 28th USENIX security symposium (USENIX security 19)

    Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., Song, D.: The secret sharer: Evaluating and testing unintended memorization in neural networks. In: 28th USENIX security symposium (USENIX security 19). pp. 267–284 (2019)

    2019

  2. [10]

    arXiv:2403.06634 (2024)

    Carlini, N., Paleka, D., Dvijotham, K.D., Steinke, T., Hayase, J., Cooper, A.F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., et al.: Stealing part of a production language model. arXiv:2403.06634 (2024)

    2024 arXiv

  3. [11]

    In: 30th USENIX security symposium (USENIX Security 21)

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M.,Herbert-Voss, A., Lee,K.,Roberts, A., Brown, T., Song, D., Erlingsson, U., et al.: Extracting training data from large language models. In: 30th USENIX security symposium (USENIX Security 21). pp. 2633–2650 (2021)

    2021

  4. [12]

    In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    Chen, J., Yang, D.: Unlearn what you want to forget: Efficient unlearning for llms. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 12041–12052 (2023)

    2023

  5. [13]

    Transactions of the Association for Computational Linguistics 12, 283–298 (2024)

    Cohen, R., Biran, E., Yoran, O., Globerson, A., Geva, M.: Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics 12, 283–298 (2024)

    2024

  6. [14]

    arXiv:2104.08696 (2021) 14 D

    Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., Wei, F.: Knowledge neurons in pretrained transformers. arXiv:2104.08696 (2021) 14 D. Staufer

    2021 arXiv

  7. [15]

    Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L., Tsvetkov, Y., Choi, Y., Evans, D., Hajishirzi, H.: Do membership inference attacks work on large language models? arXiv:2402.07841 (2024)

    2024 arXiv

  8. [16]

    arXiv:2310.02238 (2023)

    Eldan, R., Russinovich, M.: Who’s harry potter? approximate unlearning in llms. arXiv:2310.02238 (2023)

    2023 arXiv

  9. [17]

    European Data Protection Board: Opinion 28/2024 on certain data protection as- pects related to the processing of personal data in the context of ai models (Decem- ber 2024),https://www.edpb.europa.eu/system/files/2024-12/edpb_opinion_ 202428_ai-models_en.pdf, last accessed on...

    2024

  10. [18]

    arXiv:2012.15723 (2020)

    Gao, T., Fisch, A., Chen, D.: Making pre-trained language models better few-shot learners. arXiv:2012.15723 (2020)

    2020 arXiv

  11. [19]

    Advances in neural information processing systems32 (2019)

    Ginart, A., Guan, M., Valiant, G., Zou, J.Y.: Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems32 (2019)

    2019

  12. [20]

    arXiv:2407.21783 (2024)

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al.: The llama 3 herd of models. arXiv:2407.21783 (2024)

    2024 arXiv

  13. [21]

    arXiv:2111.13654 (2021)

    Hase, P., Diab, M., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., Iyer, S.: Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv:2111.13654 (2021)

    2021 arXiv

  14. [22]

    In: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    Hayes,J.,Swanberg,M.,Chaudhari,H.,Yona,I.,Shumailov,I.,Nasr,M.,Choquette- Choo, C.A., Lee, K., Cooper, A.F.: Measuring memorization in language models via probabilistic extraction. In: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association...

    2025

  15. [23]

    Huang, J., Shao, H., Chang, K.C.C.: Are large pre-trained language models leaking your personal information? arXiv:2205.12628 (2022)

    2022 arXiv

  16. [24]

    arXiv:2212.04089 (2022)

    Ilharco, G., Ribeiro, M.T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., Farhadi, A.: Editing models with task arithmetic. arXiv:2212.04089 (2022)

    2022 arXiv

  17. [25]

    arXiv:2210.17546 (2022)

    Ippolito, D., Tramèr, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette- Choo, C.A., Carlini, N.: Preventing verbatim memorization in language models gives a false sense of privacy. arXiv:2210.17546 (2022)

    2022 arXiv

  18. [26]

    arXiv:2406.10890 (2024)

    Jin, Z., Cao, P., Wang, C., He, Z., Yuan, H., Li, J., Chen, Y., Liu, K., Zhao, J.: Rwku: Benchmarking real-world knowledge unlearning for large language models. arXiv:2406.10890 (2024)

    2024 arXiv

  19. [27]

    arXiv:2403.04801 (2024)

    Kassem, A.M., Mahmoud, O., Mireshghallah, N., Kim, H., Tsvetkov, Y., Choi, Y., Saad, S., Rana, S.: Alpaca against vicuna: Using llms to uncover memorization of llms. arXiv:2403.04801 (2024)

    2024 arXiv

  20. [28]

    arXiv:2310.03714 (2023)

    Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T.T., Moazam, H., et al.: Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv:2310.03714 (2023)

    2023 arXiv

  21. [29]

    Advances in Neural Information Processing Systems 36, 20750–20762 (2023)

    Kim, S., Yun, S., Lee, H., Gubri, M., Yoon, S., Oh, S.J.: Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems 36, 20750–20762 (2023)

    2023

  22. [30]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., Carlini, N.: Deduplicating training data makes language models better. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 8424–8445 (2022)

    2022

  23. [31]

    arXiv:2110.05679 (2021) What Should LLMs Forget? Quantifying Personal Data 15

    Li, X., Tramer, F., Liang, P., Hashimoto, T.: Large language models can be strong differentially private learners. arXiv:2110.05679 (2021) What Should LLMs Forget? Quantifying Personal Data 15

    2021 arXiv

  24. [32]

    Advances in Neural Information Processing Systems 37, 118198–118266 (2024)

    Liu, C., Wang, Y., Flanigan, J., Liu, Y.: Large language model unlearning via embedding-corrupted prompts. Advances in Neural Information Processing Systems 37, 118198–118266 (2024)

    2024

  25. [33]

    Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C.Y., Xu, X., Li, H., Varshney, K.R., Bansal, M., Koyejo, S., Liu, Y.: Rethinking Machine Unlearning for Large Language Models (Dec 2024).https://doi.org/10.48550/ arXiv.2402.08787

    2024 doi arXiv

  26. [34]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Liu, Y., Peng, X., Zhang, Y., Ke, X., Deng, S., Cao, J., Ma, C., Fu, M., Zhang, X., Cheng, S., et al.: Dp-memarc: Differential privacy transfer learning for memory efficient language models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 26317–...

    2025

  27. [35]

    arXiv:2104.08786 (2021)

    Lu, Y., Bartolo, M., Moore, A., Riedel, S., Stenetorp, P.: Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv:2104.08786 (2021)

    2021 arXiv

  28. [36]

    URL https://arxiv

    Łucki, J., Wei, B., Huang, Y., Henderson, P., Tramer, F., Rando, J.: An ad- versarial perspective on machine unlearning for ai safety. URL https://arxiv. org/abs/2409.18025 (2024)

    2024 arXiv

  29. [37]

    arXiv:2402.16835 (2024)

    Lynch, A., Guo, P., Ewart, A., Casper, S., Hadfield-Menell, D.: Eight methods to evaluate robust unlearning in llms. arXiv:2402.16835 (2024)

    2024 arXiv

  30. [38]

    arXiv:2401.06121 (2024)

    Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z.C., Kolter, J.Z.: Tofu: A task of fictitious unlearning for llms. arXiv:2401.06121 (2024)

    2024 arXiv

  31. [39]

    In: 33rd USENIX Security Symposium (USENIX Security 24)

    Meeus, M., Jain, S., Rei, M., de Montjoye, Y.A.: Did the neurons read your book? document-level membership inference for large language models. In: 33rd USENIX Security Symposium (USENIX Security 24). pp. 2369–2385 (2024)

    2024

  32. [40]

    Advances in neural information processing systems35, 17359– 17372 (2022)

    Meng, K., Bau, D., Andonian, A., Belinkov, Y.: Locating and editing factual associations in gpt. Advances in neural information processing systems35, 17359– 17372 (2022)

    2022

  33. [41]

    arXiv:2311.13647 (2023)

    Morris, J.X., Zhao, W., Chiu, J.T., Shmatikov, V., Rush, A.M.: Language model inversion. arXiv:2311.13647 (2023)

    2023 arXiv

  34. [42]

    arXiv:2402.00751 (2024)

    Muresanu, A., Thudi, A., Zhang, M.R., Papernot, N.: Unlearnable algorithms for in-context learning. arXiv:2402.00751 (2024)

    2024

  35. [43]

    arXiv:2407.02943 (2024)

    Nakka, K.K., Frikha, A., Mendes, R., Jiang, X., Zhou, X.: Pii-compass: Guid- ing llm training data extraction prompts towards the target pii via grounding. arXiv:2407.02943 (2024)

    2024 arXiv

  36. [44]

    arXiv:2311.17035 (2023)

    Nasr,M.,Carlini,N.,Hayase,J.,Jagielski,M.,Cooper,A.F.,Ippolito,D.,Choquette- Choo, C.A., Wallace, E., Tramèr, F., Lee, K.: Scalable extraction of training data from (production) language models. arXiv:2311.17035 (2023)

    2023 arXiv

  37. [45]

    arXiv:2209.02299 (2022)

    Nguyen, T.T., Huynh, T.T., Ren, Z., Nguyen, P.L., Liew, A.W.C., Yin, H., Nguyen, Q.V.H.: A survey of machine unlearning. arXiv:2209.02299 (2022)

    2022 arXiv

  38. [46]

    https://noyb.eu/en/ chatgpt-provides-false-information-about-people-and-openai-cant-correct-it (2024), accessed: 2025-06-02

    NOYB: Chatgpt provides false information about peo- ple, and openai can’t correct it. https://noyb.eu/en/ chatgpt-provides-false-information-about-people-and-openai-cant-correct-it (2024), accessed: 2025-06-02

    2024

  39. [47]

    Panda, A., Tang, X., Nasr, M., Choquette-Choo, C.A., Mittal, P.: Privacy Auditing of Large Language Models (Mar 2025).https://doi.org/10.48550/arXiv.2503. 06808

    2025 doi

  40. [48]

    arXiv:2309.17410 (2023)

    Patil, V., Hase, P., Bansal, M.: Can sensitive information be deleted from llms? objectives for defending against extraction attacks. arXiv:2309.17410 (2023)

    2023 arXiv

  41. [49]

    arXiv:2310.07579 (2023) 16 D

    Pawelczyk, M., Neel, S., Lakkaraju, H.: In-context unlearning: Language models as few shot unlearners. arXiv:2310.07579 (2023) 16 D. Staufer

    2023 arXiv

  42. [50]

    arXiv:2411.00154 (2024)

    Puerto, H., Gubri, M., Yun, S., Oh, S.J.: Scaling up membership inference: When and how attacks succeed on large language models. arXiv:2411.00154 (2024)

    2024 arXiv

  43. [51]

    arXiv:2007.04792 (2020)

    Schlangen, D.: Targeting the benchmark: On methodology in current natural language processing research. arXiv:2007.04792 (2020)

    2020 arXiv

  44. [52]

    In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W

    Sekhari, A., Acharya, J., Kamath, G., Suresh, A.T.: Remember what you want to forget: Algorithms for machine unlearning. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems. vol. 34, pp. 18075–18086. ...

    2021

  45. [53]

    arXiv:2505.17117 (2025)

    Shani, C., Jurafsky, D., LeCun, Y., Shwartz-Ziv, R.: From tokens to thoughts: How llms and humans trade compression for meaning. arXiv:2505.17117 (2025)

    2025

  46. [54]

    arXiv:2310.16789 (2023)

    Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., Zettlemoyer, L.: Detecting pretraining data from large language models. arXiv:2310.16789 (2023)

    2023 arXiv

  47. [55]

    arXiv:2010.15980 (2020)

    Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: Autoprompt: Elic- iting knowledge from language models with automatically generated prompts. arXiv:2010.15980 (2020)

    2020 arXiv

  48. [56]

    arXiv:2310.07298 (2023)

    Staab, R., Vero, M., Balunović, M., Vechev, M.: Beyond memorization: Violating privacy via inference with large language models. arXiv:2310.07298 (2023)

    2023 arXiv

  49. [57]

    arXiv:2403.05530 (2024)

    Team, G., Georgiev, P., Lei, V.I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530 (2024)

    2024 arXiv

  50. [58]

    arXiv:2403.03329 (2024)

    Thaker, P., Maurya, Y., Hu, S., Wu, Z.S., Smith, V.: Guardrail baselines for unlearning in llms. arXiv:2403.03329 (2024)

    2024 arXiv

  51. [59]

    arXiv:2412.11302 (2024)

    Tiwari, T., Suh, G.E.: Sequence-level analysis of leakage risk of training data in large language models. arXiv:2412.11302 (2024)

    2024 arXiv

  52. [60]

    In: Belkin, M., Kpotufe, S

    Ullah, E., Mai, T., Rao, A., Rossi, R.A., Arora, R.: Machine unlearning via algo- rithmic stability. In: Belkin, M., Kpotufe, S. (eds.) Proceedings of Thirty Fourth Conference on Learning Theory. Proceedings of Machine Learning Research, vol. 134, pp. 4126–4142. PMLR (15–19 Aug 2021)

    2021

  53. [61]

    arXiv:2410.03446 (2024)

    Ulmer, D.: On uncertainty in natural language processing. arXiv:2410.03446 (2024)

    2024 arXiv

  54. [62]

    https://doi.org/10.48550/arXiv.2501.03989

    Vilella, S., Ruffo, G.: (De)-Indexing and the Right to be Forgotten (Jan 2025). https://doi.org/10.48550/arXiv.2501.03989

    2025 doi arXiv

  55. [63]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Wang, L., Zeng, X., Guo, J., Wong, K.F., Gottlob, G.: Selective forgetting: Ad- vancing machine unlearning techniques and evaluation in language models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 843–851 (2025)

    2025

  56. [64]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., Yue, X.: Machine unlearning of pre-trained large language models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 8403–8419 (2024)

    2024

  57. [65]

    Advances in Neural Information Processing Systems37, 105425–105475 (2024)

    Yao, Y., Xu, X., Liu, Y.: Large language model unlearning. Advances in Neural Information Processing Systems37, 105425–105475 (2024)

    2024

  58. [66]

    Advances in Neural Information Processing Systems 36, 39321–39362 (2023)

    Zhang,C.,Ippolito,D.,Lee,K.,Jagielski,M.,Tramèr,F.,Carlini,N.:Counterfactual memorization in neural language models. Advances in Neural Information Processing Systems 36, 39321–39362 (2023)

    2023

  59. [67]

    AI and Ethics (Sep 2024)

    Zhang, D., Finckenberg-Broman, P., Hoang, T., Pan, S., Xing, Z., Staples, M., Xu, X.: Right to be forgotten in the Era of large language models: Implications, challenges, and solutions. AI and Ethics (Sep 2024)

    2024

  60. [68]

    Zhou, Z., Xiang, J., Chen, C., Su, S.: Quantifying and Analyzing Entity-Level MemorizationinLargeLanguageModels.ProceedingsoftheAAAIConferenceonAI 38(17), 19741–19749 (Mar 2024).https://doi.org/10.1609/aaai.v38i17.29948

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.