Pith. sign in

REVIEW 3 major objections 4 minor 58 references

GROM: Gradient-Free Rapid One-Shot Machine Unlearning

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GROM claims that a single closed-form additive weight edit, computed by solving a ridge-regularized least-squares problem, can replace iterative fine-tuning for LLM unlearning.

desk verdict A genuinely useful one-shot unlearning method with real proofs and a good audit trick, but the 'strictly preserving' language overstates what a finite-key ridge constraint delivers. read the letter →

arxiv 2608.05783 v1 pith:YMZBF5OK submitted 2026-08-06 cs.LG cs.AIcs.CL

Pawe{l} Batorski , Przemys{l}aw Spurek , Paul Swoboda This is my paper
classification cs.LGcs.AIcs.CL
keywords machineunlearninglargelanguagemodelsclosed-formupdateridgeregressionone-shotquantizationrobustnessdatadeletionauditing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GROM asks whether machine unlearning for large language models can abandon iterative gradient fine-tuning altogether. The paper's claim is that one closed-form additive update to a small band of MLP down-projections (or the LM head) can suppress targeted memorized content while preserving retained behavior, by solving a ridge-regularized least-squares problem exactly. This would matter if true: unlearning would take seconds rather than minutes or hours on a single GPU, require no backpropagation and no reference or sanitized teacher, and remove the content from the weights rather than masking it, making the result resistant to the low-bit quantization attack that restores hidden content after gradient-based unlearning. On five benchmarks the paper reports Pareto-best forgetting-utility trade-offs and update times up to roughly two orders of magnitude faster than gradient-based baselines.

What carries the argument

The load-bearing object is the closed-form solution to a ridge-regularized least-squares problem on a linear weight matrix, together with a way to choose which layer and which target direction to edit. The mathematical core is the Gram matrix $A = (w_r/r)X_rX_r^\top + (w_f/s)X_fX_f^\top + \mu I$, which is symmetric positive definite for positive weights and ridge, making the minimizer $P^\star = (w_f/s)D X_f^\top A^{-1}$ unique. The edit works only on matrices whose outputs feed directly into the residual stream or logits, which is why the method edits MLP down-projections or the LM head. A secondary mechanism is an exact low-rank downdate identity used to compute, without retraining, how much each forget example contributed to the edit.

What would settle it

Apply GROM to a forget set whose answers are stored in early or attention-layer features rather than in the late MLP residual stream, and measure whether forget-set answer probability stays high while retain utility drops; if the edit cannot drive downstream forgetting without collateral damage on such a split, the linear-writability premise is refuted.

Watch

Extended reading notes

Core claim

The paper proposes a one-shot unlearning update: collect per-token inputs ('keys') $X_f$ and $X_r$ to an editable linear layer (MLP down-projection or LM head) from forward passes over forget and retain data, fix a target $D$, and compute $P^\star = (w_f/s) D X_f^\top A^{-1}$, where $A = (w_r/r)X_rX_r^\top + (w_f/s)X_fX_f^\top + \mu I$. The update $W \leftarrow W + P^\star$ is the unique minimizer of a ridge-regularized objective that preserves retain outputs, $P^\star X_r \approx 0$, and steers forget outputs toward the target, $(W+P^\star)X_f \approx W X_f + D$. The target encodes what to forget: token suppression steers the output away from the gold next token in the unembedding direction, and representation corruption drives WMDP-style multiple-choice outputs toward a fixed random direction. Layer selection uses an attribution score comparing each layer's contribution to forget-token logits versus retain-token logits, choosing the contiguous late band with the largest average score. Because the solve is analytical, the paper argues the edit removes content from the weights rather than merely hiding it, and derives an exact per-example deletion influence via a low-rank downdate of $A$.

Load-bearing premise

The load-bearing premise is that a single additive linear perturbation to a small set of MLP down-projections or the LM head can erase distributional memorized knowledge while leaving retained behavior intact.

Editorial extensions

If this is right

  • On the paper's own claims, unlearning can be performed in under a minute on a single GPU for models up to 7–8B parameters, without backpropagation or distillation teachers.
  • Because the edit is an additive weight change rather than a behavioral mask, the unlearned model's forget-set memorization stays low after 4-bit quantization, matching a gold retrained model.
  • The closed-form formula makes per-example deletion audit exact and cheap, allowing a provider to state what a single deletion request contributed to the released model.
  • The update is deterministic once keys, targets, layers, and hyperparameters are fixed, and only the ratio of retain to forget weights matters for the solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated but direct consequence is that the method's usefulness is bounded by how linearly writable the targeted knowledge is at the edited layers; if a forget set's knowledge is distributed across attention blocks or early layers, a single band edit may need to be widened or combined with other interventions.
  • The same closed-form update could be applied repeatedly to handle streaming deletion requests, since each new edit can recompute one band without retraining, though the paper does not demonstrate this.
  • One testable extension is to use the closed-form edit as a rapid initialization for iterative methods, trading a few fine-tuning steps for even better retain fidelity on heavily entangled forget/retain splits; this goes beyond the paper's claims.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces GROM, a one-shot machine unlearning method that computes an additive closed-form update to selected MLP down-projections or the LM head by solving a ridge-regularized least-squares problem over collected per-token keys. The update is obtained from forward passes only, with no backpropagation. The authors prove uniqueness and closed-form solution (Theorem 1), derive an exact per-example deletion influence via the Woodbury identity (Theorem 2), and evaluate on TOFU-5%/10%, MUSE News/Books, WMDP-Bio, and ZsRE, including a 4-bit quantization robustness test. They report that GROM achieves best or near-best forgetting-utility trade-offs with orders-of-magnitude lower update time.

Significance. The core idea is timely and the theoretical derivation is clean: Theorem 1's proof is self-contained and correctly establishes the minimizer, and Theorem 2's Woodbury downdate is correctly derived under the stated fixed-per-token-weight convention. If the empirical claims are reliable, GROM would be a valuable addition, offering a fast, analytical alternative to iterative unlearning and a plausible explanation for quantization robustness. The paper also ships public code and gives detailed experimental configuration. However, the central "strictly preserves retain behavior" claim is internally contradicted by the MUSE News retain KnowMem drop, and the main tables lack uncertainty quantification, so the headline contributions need revision rather than acceptance as-is.

major comments (3)
  1. [Section 3, "Objective and Closed-Form Solution"; Abstract; Table 5 (MUSE News row)] The claim that the update "strictly preserves" retained behavior is not supported. The objective only enforces P X_r ≈ 0 on the sampled retain keys that fit in the fixed budget, and Theorem 1 is exact for that finite-key surrogate, not for the retain distribution. Table 5 shows retain KnowMem on MUSE News falling from 54.31 (Original) to 26.37 (GROM), roughly a 50% loss, while the abstract and Section 3 state that retained behavior is strictly preserved and that GROM works "without sacrificing overall model performance." This is a direct internal contradiction. Please rephrase the preservation claim to describe approximate, benchmark-measured preservation, and provide evidence that the retain constraint transfers beyond X_r (e.g., report the retain-key fit versus holdout retain metrics, or vary the retain key budget).
  2. [Experiments, Tables 2, 4, 5, 6; Appendix B.2] The main state-of-the-art claims are reported without error bars or repeated runs. The only seed spread reported is for the TOFU-10% tuned configuration (Final Score span 0.006), while margins over the second-best baseline are 0.04–0.06 on TOFU, 0.04–0.06 on WMDP, and about 6 points on MUSE. Because several baselines are stochastic and final scores are aggregates, please report means and standard deviations over at least three seeds for all main tables, or otherwise justify that the reported margins exceed run-to-run variability.
  3. [Section 3, "Choice of Layers to be Updated"; Appendix G runtime protocol] The phrase "one gradient-free forward pass" is misleading when k > 1 layers are edited: the method collects keys, then applies updates sequentially and recomputes features after each edited layer, so it requires one forward pass per edited layer in addition to key collection. Please clarify this in the method description and abstract, since the runtime protocol in Appendix G already counts the full edit procedure.
minor comments (4)
  1. [Abstract; Table 2] The abstract says GROM "applies the weight edit in mere seconds," but Table 2 reports 1.8 minutes on TOFU-5%; please qualify the speed claim (e.g., "under two minutes" or "orders of magnitude faster").
  2. [Table 6] The first column label "Prec." is undefined; use "Precision" or "Setting" to indicate full-precision versus 4-bit runs.
  3. [Equation (3) in Appendix D] The Final Score uses 100 - |PrivLeak|, which penalizes large negative leakage as much as large positive leakage; the paper should state why symmetric penalization is appropriate when the desired value is 0.
  4. [Appendix B.2] The statement that "only three hyperparameters are free" is accurate under the stated gauge w_f = 1, but the preceding sentences mention four weights; a sentence explicitly noting that w_f is fixed by the gauge would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GROM's closed-form update is a genuine solution to a stated objective, and all benchmark claims are evaluated against external protocols.

full rationale

The paper's central derivation is self-contained: Theorem 1 solves a well-specified ridge-regularized least-squares objective by differentiation, with the proof in Appendix A relying only on standard linear algebra (positive definiteness, strict convexity, Woodbury identity). The update P* = (wf/s) D X_f^T A^{-1} is not a restatement of any benchmark number; the target D is fixed before solving and is constructed from token statistics or a random vector, not fitted to the reported metrics. The forgetting and utility evaluations use external protocols (OpenUnlearning, ZeroUnlearn harness) on benchmark splits, and hyperparameter tuning per benchmark is ordinary experimental practice rather than a fitted-input-as-prediction maneuver. The quantization-robustness result is an empirical comparison against a gold retrain baseline, and the layer-selection heuristic is supported by external citations (Geva et al., Dai et al.) rather than by a self-citation chain. The retain-generalization gap on MUSE News (retain KnowMem dropping from 54.31 to 26.37) is a real correctness and robustness concern about enforcing P X_r ≈ 0 only on sampled keys, but it is not circularity: no equation in the paper defines the benchmark retain-knowledge score as an input to the objective. Overall, the derivation chain does not reduce to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a few domain assumptions: linear editability of the chosen matrices, the effectiveness of the chosen targets, and the ability of the logit-lens score to identify useful layers. These are empirically supported by ablations but not proven. No invented entities; the method uses only existing model weights and data-derived quantities.

free parameters (4)
  • beta (edit strength) = 65 (TOFU-10), 1000 (TOFU-5), 370 (MUSE News), 350 (MUSE Books), 45 (WMDP corruption c), 20 (ZsRE head)
    Hand-tuned per benchmark to balance forgetting and utility.
  • w_r (retain weight) = 100 (TOFU-10), 300 (TOFU-5), 10 (MUSE News), 80 (MUSE Books), 1000 (WMDP), 30 (ZsRE)
    Controls the retain anchor; sensitivity analysis shows it must be tuned or the edit either destroys the model (w_r=0) or does nothing (w_r large).
  • rho (ridge scale) = 0.03 for all benchmarks
    Set via mu = rho * mean diagonal of A; sensitivity analysis shows 0.001-0.1 is safe.
  • k (edit width) = 5 (TOFU-10), 6 (TOFU-5), 4 (MUSE), 1 (WMDP), head-only (ZsRE)
    Number of edited layers, selected by logit-lens attribution score.
assumptions (5)
  • domain assumption MLP down-projection outputs are added linearly to the residual stream, so an additive weight edit shifts outputs linearly and does not pass through a nonlinearity before the residual add.
    Section 3 'Setup and Editable Matrices'; without this, the closed-form control of outputs would not hold.
  • domain assumption The logit-lens attribution score S_l identifies layers that write the memorized tokens.
    Section 3 'Choice of Layers to be Updated'; ablation in Appendix B.1 shows the selected band beats alternatives, but this is empirical, not proven.
  • domain assumption Token suppression and representation corruption are effective ways to encode forgetting.
    Section 3 'Unlearning Targets'; choice is benchmark-dependent.
  • domain assumption The ridge-regularized least-squares objective adequately represents the unlearning goal.
    Section 3 'Objective and Closed-Form Solution'.
  • standard math A is symmetric positive definite and the Sherman-Morrison-Woodbury identity apply.
    Appendix A, Lemmas A.1 and A.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GROM: Gradient-Free Rapid One-Shot Machine Unlearning." pith.science (2026). https://pith.science/paper/YMZBF5OK

@misc{pith2026260805783,
  author       = {Pith},
  title        = {Pith review of: GROM: Gradient-Free Rapid One-Shot Machine Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YMZBF5OK}},
  note         = {Machine review of arXiv:2608.05783}
}
read the original abstract

Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs). Current state-of-the-art approaches primarily rely on iterative, training-time unlearning via fine-tuning. However, even when utilizing parameter-efficient dimensionality reduction techniques like LoRA, gradient-based optimization remains computationally expensive and lacks explicit analytical formulations. It can also leave the targeted knowledge merely hidden rather than removed, to the point that simply quantizing the unlearned model restores much of what it was supposed to have erased. To resolve this, we propose a novel one-shot unlearning approach, abandoning iterative optimization in favor of a direct, exact analytical solution. We frame the unlearning process as a ridge-regularized least-squares optimization problem, deriving a closed-form additive update for targeted weight matrices. This update forces the selected layer to suppress unwanted content while strictly preserving its behavior on retained data. Computed from gradient-free forward passes alone, with no backpropagation and no iteration to convergence, GROM applies the weight edit in mere seconds, which makes it orders of magnitude faster than traditional fine-tuning. Extensive evaluations demonstrate that GROM achieves state-of-the-art forgetting-utility trade-offs on TOFU-5%, TOFU-10%, MUSE-Books, MUSE-News and WMDP, significantly reducing computational overhead without sacrificing overall model performance. Because the update removes the targeted content from the weights instead of masking it, GROM also withstands the low-bit quantization attack that recovers much of the content a gradient-based baseline had appeared to forget. Our code is publicly available at https://github.com/Batorskq/GROM.

Figures

Figures reproduced from arXiv: 2608.05783 by the authors.

Figure 1
Figure 1. GROM is fast and Pareto-best. Left, top: on TOFU-10%, GROM (red star) reaches the top-right corner, strong forgetting and the highest model utility. Left, bottom: its closed-form edit applies in seconds, up to ∼180× faster than SimNPO, a gap that widens on the larger 7–8B corpus benchmarks (single NVIDIA H100). Right: GROM replaces iterative fine-tuning with a single gradient-free closed-form edit W +P to one target… view at source ↗
Figure 2
Figure 2. Overview of GROM. From gradient-free forward passes over the forget and retain data we collect the per-token [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. 3D PCA diagnostic of the LM-head patch. Hollow [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 32 canonical work pages

  1. [1]

    Belrose, N.; Ostrovsky, I.; McKinney, L.; Furman, Z.; Smith, L.; Halawi, D.; Biderman, S.; and Steinhardt, J. 2023. Eliciting Latent Predictions from Transformers with the Tuned Lens. arXiv preprint arXiv:2303.08112

    arXiv 2023

  2. [2]

    Bourtoule, L.; Chandrasekaran, V.; Choquette-Choo, C.; Jia, H.; Travers, A.; Zhang, B.; Lie, D.; and Papernot, N. 2021. Machine Unlearning. In Proceedings of the IEEE Symposium on Security and Privacy . IEEE S&P 2021

    work page 2021

  3. [3]

    Cao, Y.; and Yang, J. 2015. Towards Making Systems Forget with Machine Unlearning. In Proceedings of the IEEE Symposium on Security and Privacy . IEEE S&P 2015

    work page 2015

  4. [4]

    Carlini, N.; Ippolito, D.; Jagielski, M.; Lee, K.; Tramer, F.; and Zhang, C. 2022. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations

    work page 2022

  5. [5]

    Carlini, N.; Tramer, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T.; Song, D.; Erlingsson, U.; Oprea, A.; and Raffel, C. 2021. Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium ( USENIX Security 21)

    work page 2021

  6. [6]

    Chen, J.; and Yang, D. 2023. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:2310.20150

    arXiv 2023

  7. [7]

    Dai, D.; Dong, L.; Hao, Y.; Sui, Z.; Chang, B.; and Wei, F. 2022. Knowledge Neurons in Pretrained Transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 8493--8502

    work page 2022

  8. [8]

    R.; Lin, H.; Belkin, M.; Huerta, R.; and Vuli \'c , I

    Dong, Y. R.; Lin, H.; Belkin, M.; Huerta, R.; and Vuli \'c , I. 2025. UNDIAL : Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models. In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ArXiv:2402.10052

    arXiv 2025

Show all 58 references
  1. [9]

    C.; Kolter, J

    Dorna, V.; Mekala, A.; Zhao, W.; McCallum, A.; Lipton, Z. C.; Kolter, J. Z.; and Maini, P. 2025. OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics. arXiv preprint arXiv:2506.12618

    2025

  2. [10]

    Duan, M.; Suri, A.; Mireshghallah, N.; Min, S.; Shi, W.; Zettlemoyer, L.; Tsvetkov, Y.; Choi, Y.; Evans, D.; and Hajishirzi, H. 2024. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841

    2024 arXiv

  3. [11]

    Eldan, R.; and Russinovich, M. 2023. Who’s harry potter? approximate unlearning for LLMs

    2023

  4. [12]

    Fan, C.; Liu, J.; Lin, L.; Jia, J.; Zhang, R.; Mei, S.; and Liu, S. 2025. Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems

    2025

  5. [13]

    Fang, J.; Jiang, H.; Wang, K.; Ma, Y.; Jie, S.; Wang, X.; He, X.; and Chua, T.-S. 2024. AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models. arXiv preprint arXiv:2410.02355

    2024 arXiv

  6. [14]

    Geva, M.; Caciularu, A.; Wang, K.; and Goldberg, Y. 2022. Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 30--45

    2022

  7. [15]

    Geva, M.; Schuster, R.; Berant, J.; and Levy, O. 2021. Transformer Feed-Forward Layers Are Key-Value Memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 5484--5495

    2021

  8. [16]

    Y.; Valiant, G.; and Zou, J

    Ginart, A.; Guan, M. Y.; Valiant, G.; and Zou, J. 2019. Making AI Forget You: Data Deletion in Machine Learning. In Advances in Neural Information Processing Systems. NeurIPS 2019

    2019

  9. [17]

    Guo, C.; Goldstein, T.; Hannun, A.; and Van Der Maaten, L. 2019. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030

    2019 arXiv

  10. [18]

    Hager, W. W. 1989. Updating the Inverse of a Matrix. SIAM Review, 31(2): 221--239

    1989

  11. [19]

    Hase, P.; Bansal, M.; Kim, B.; and Ghandeharioun, A. 2023. Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models. In Advances in Neural Information Processing Systems (NeurIPS)

    2023

  12. [20]

    Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  13. [21]

    J.; Van Der Sloot, B.; and Borgesius, F

    Hoofnagle, C. J.; Van Der Sloot, B.; and Borgesius, F. Z. 2019. The European Union general data protection regulation: what it is and what it means. Information & Communications Technology Law, 28(1): 65--98

    2019

  14. [22]

    S.; and Smith, V

    Hu, S.; Fu, Y.; Wu, Z. S.; and Smith, V. 2024. Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning. arXiv preprint arXiv:2406.13356. ICLR 2025

    2024 arXiv

  15. [23]

    Y.; Zhou, W.; Wang, F.; Morstatter, F.; Zhang, S.; Poon, H.; and Chen, M

    Huang, J. Y.; Zhou, W.; Wang, F.; Morstatter, F.; Zhang, S.; Poon, H.; and Chen, M. 2024. Offset unlearning for large language models. arXiv preprint arXiv:2404.11045

    2024 arXiv

  16. [24]

    T.; Wortsman, M.; Gururangan, S.; Schmidt, L.; Hajishirzi, H.; and Farhadi, A

    Ilharco, G.; Ribeiro, M. T.; Wortsman, M.; Gururangan, S.; Schmidt, L.; Hajishirzi, H.; and Farhadi, A. 2022. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089

    2022 arXiv

  17. [25]

    A.; Chaudhuri, K.; and Zou, J

    Izzo, Z.; Smart, M. A.; Chaudhuri, K.; and Zou, J. 2021. Approximate data deletion from machine learning models. In International conference on artificial intelligence and statistics, 2008--2016. PMLR

    2021

  18. [26]

    R.; Liu, S.; and Chang, S

    Ji, J.; Liu, Y.; Zhang, Y.; Liu, G.; Kompella, R. R.; Liu, S.; and Chang, S. 2024. Reversing the forget-retain objectives: An efficient llm unlearning framework from logit difference. Advances in Neural Information Processing Systems, 37: 12581--12611

    2024

  19. [27]

    Jia, J.; Zhang, Y.; Zhang, Y.; Liu, J.; Runwal, B.; Diffenderfer, J.; Kailkhura, B.; and Liu, S. 2024. Soul: Unlocking the power of second-order optimization for llm unlearning. arXiv preprint arXiv:2404.18239

    2024 arXiv

  20. [28]

    Jin, Z.; Cao, P.; Wang, C.; He, Z.; Yuan, H.; Li, J.; Chen, Y.; Liu, K.; and Zhao, J. 2024. RWKU : Benchmarking Real-World Knowledge Unlearning for Large Language Models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2024

  21. [29]

    W.; and Liang, P

    Koh, P. W.; and Liang, P. 2017. Understanding Black-box Predictions via Influence Functions. In Proceedings of the 34th International Conference on Machine Learning (ICML), 1885--1894

    2017

  22. [30]

    Levy, O.; Seo, M.; Choi, E.; and Zettlemoyer, L. 2017. Zero-Shot Relation Extraction via Reading Comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL)

    2017

  23. [31]

    D.; Dombrowski, A.-K.; Goel, S.; Phan, L.; et al

    Li, N.; Pan, A.; Gopal, A.; Yue, S.; Berrios, D.; Gatti, A.; Li, J. D.; Dombrowski, A.-K.; Goel, S.; Phan, L.; et al. 2024. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218

    2024 arXiv

  24. [32]

    Lin, C.-Y. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74--81

    2004

  25. [33]

    Lin, Y.; Yang, C.; Xiang, Z.; Song, Y.; and Su, J. 2026. ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models. In Proceedings of the 43rd International Conference on Machine Learning (ICML)

    2026

  26. [34]

    Liu, B.; Liu, Q.; and Stone, P. 2022. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, 243--254. PMLR

    2022

  27. [35]

    Y.; Xu, X.; Li, H.; et al

    Liu, S.; Yao, Y.; Jia, J.; Casper, S.; Baracaldo, N.; Hase, P.; Yao, Y.; Liu, C. Y.; Xu, X.; Li, H.; et al. 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence, 1--14

    2025

  28. [36]

    C.; and Kolter, J

    Maini, P.; Feng, Z.; Schwarzschild, A.; Lipton, Z. C.; and Kolter, J. Z. 2024. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121

    2024 arXiv

  29. [37]

    Mekala, A.; Dorna, V.; Dubey, S.; Lalwani, A.; Koleczek, D.; Rungta, M.; Hasan, S.; and Lobo, E. 2025. Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models. In Rambow, O.; Wanner, L.; Apidianaki, M.; Al-Khalifa, H.; Eugenio, B. D.; and Sc...

    2025

  30. [38]

    J.; and Belinkov, Y

    Meng, K.; Bau, D.; Andonian, A. J.; and Belinkov, Y. 2022. Locating and Editing Factual Associations in GPT . In Advances in Neural Information Processing Systems

    2022

  31. [39]

    S.; Andonian, A

    Meng, K.; Sharma, A. S.; Andonian, A. J.; Belinkov, Y.; and Bau, D. 2023. Mass-Editing Memory in a Transformer. In International Conference on Learning Representations

    2023

  32. [40]

    nostalgebraist. 2020. Interpreting GPT : The Logit Lens. https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens

    2020

  33. [41]

    Pawelczyk, M.; Neel, S.; and Lakkaraju, H. 2023. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv:2310.07579

    2023 arXiv

  34. [42]

    D.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Ermon, S.; Manning, C. D.; and Finn, C. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems. NeurIPS 2023

    2023

  35. [43]

    C.; and Liu, H

    Ren, J.; Xing, Y.; Cui, Y.; Aggarwal, C. C.; and Liu, H. 2025. SoK: Machine Unlearning for Large Language Models. arXiv preprint arXiv:2506.09227

    2025 arXiv

  36. [44]

    Rosen, J. 2011. The right to be forgotten. Stan. L. Rev. Online, 64: 88

    2011

  37. [45]

    Shi, W.; Ajith, A.; Xia, M.; Huang, Y.; Liu, D.; Blevins, T.; Chen, D.; and Zettlemoyer, L. 2024 a . Detecting Pretraining Data from Large Language Models. In International Conference on Learning Representations (ICLR)

    2024

  38. [46]

    A.; and Zhang, C

    Shi, W.; Lee, J.; Huang, Y.; Malladi, S.; Zhao, J.; Holtzman, A.; Liu, D.; Zettlemoyer, L.; Smith, N. A.; and Zhang, C. 2024 b . Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460

    2024 arXiv

  39. [47]

    S.; and Smith, V

    Thaker, P.; Maurya, Y.; Hu, S.; Wu, Z. S.; and Smith, V. 2024 a . Guardrail baselines for unlearning in llms. arXiv preprint arXiv:2403.03329

    2024 arXiv

  40. [48]

    S.; and Smith, V

    Thaker, P.; Maurya, Y.; Hu, S.; Wu, Z. S.; and Smith, V. 2024 b . Position: LLM Unlearning Benchmarks are Weak Measures of Progress. arXiv preprint arXiv:2410.02879. SaTML 2025

    2024 arXiv

  41. [49]

    Thudi, A.; Deza, G.; Chandrasekaran, V.; and Papernot, N. 2022. Unrolling sgd: Understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), 303--319. IEEE

    2022

  42. [50]

    Wang, B.; Zi, Y.; Sun, Y.; Zhao, Y.; and Qin, B. 2024. Rkld: Reverse kl-divergence-based knowledge distillation for unlearning personal information in large language models. arXiv preprint arXiv:2406.01983

    2024 arXiv

  43. [51]

    A.; Zhang, C.; Zettlemoyer, L.; Li, K.; and Henderson, P

    Wei, B.; Shi, W.; Huang, Y.; Smith, N. A.; Zhang, C.; Zettlemoyer, L.; Li, K.; and Henderson, P. 2024. Evaluating copyright takedown methods for language models. Advances in Neural Information Processing Systems, 37: 139114--139150

    2024

  44. [52]

    Xu, X.; Yue, X.; Liu, Y.; Ye, Q.; Hu, H.; and Du, M. 2025. Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs. arXiv preprint arXiv:2505.16831

    2025 arXiv

  45. [53]

    Yao, Y.; Xu, X.; and Liu, Y. 2024 a . Large Language Model Unlearning. In Advances in Neural Information Processing Systems. NeurIPS, arXiv:2310.10683

    2024 arXiv

  46. [54]

    Yao, Y.; Xu, X.; and Liu, Y. 2024 b . Large language model unlearning. Advances in Neural Information Processing Systems, 37: 105425--105475

    2024

  47. [55]

    A.; Kamath, G.; Kulkarni, J.; Lee, Y

    Yu, D.; Naik, S.; Backurs, A.; Gopi, S.; Inan, H. A.; Kamath, G.; Kulkarni, J.; Lee, Y. T.; Manoel, A.; Wutschitz, L.; et al. 2021. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500

    2021 arXiv

  48. [56]

    Zhang, B.; Chen, Z.; Shen, C.; and Li, J. 2024 a . Verification of Machine Unlearning is Fragile. In Forty-first International Conference on Machine Learning

    2024

  49. [57]

    Zhang, R.; Lin, L.; Bai, Y.; and Mei, S. 2024 b . Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning. In Proceedings of the Conference on Language Modeling (COLM). ArXiv:2404.05868

    2024 arXiv

  50. [58]

    Zhang, Z.; Wang, F.; Li, X.; Wu, Z.; Tang, X.; Liu, H.; He, Q.; Yin, W.; and Wang, S. 2025. Catastrophic Failure of LLM Unlearning via Quantization. In The Thirteenth International Conference on Learning Representations

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.