Pith. sign in

REVIEW 4 cited by

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.22086 v3 pith:UBVH3FUI submitted 2024-10-29 cs.LG cs.CL

Zhiqi Bu , Xiaomeng Jin , Bhanukiran Vinzamuri , Anil Ramakrishna , Kai-Wei Chang , Volkan Cevher , Mingyi Hong This is my paper · ORCID
classification cs.LGcs.CL
keywords unlearningoptimizationdifferencegradientlearningmachinemulti-taskngdiff
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspective, framing it as a regularized multi-task optimization problem, where one task optimizes a forgetting objective and another optimizes the model performance. In particular, we introduce a normalized gradient difference (NGDiff) algorithm, enabling us to have better control over the trade-off between the objectives, while integrating a new, automatic learning rate scheduler. We provide a theoretical analysis and empirically demonstrate the superior performance of NGDiff among state-of-the-art unlearning methods on the TOFU and MUSE datasets while exhibiting stable training.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding

    cs.LG 2025-11 reject novelty 6.0 of 10

    LLM unlearning methods that pass greedy-decoding benchmarks leak forgotten facts when the model is sampled repeatedly, and the new leak@k metric quantifies this.

  2. SoK: Machine Unlearning for Large Language Models

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A new taxonomy for LLM unlearning distinguishes removal-intended from suppression-intended methods, and argues that gradient ascent methods functionally behave like suppression.

  3. Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning

    cs.CL 2025-06 reject novelty 5.0 of 10

    SU uses two assistant models trained on different data splits to score tokens, then unlearns only tokens whose scores diverge, claiming better retain-set utility with comparable forget quality.

  4. LLM Unlearning for Cyber Defense: A Survey on Methods, Challenges, and Emerging Threats

    cs.LG 2026-06 conditional novelty 4.0 of 10

    Most gradient-based LLM unlearning methods achieve behavioral suppression, not true forgetting, and current benchmarks cannot certify that knowledge has been removed.

Pith tools