Pith. sign in

REVIEW 3 cited by

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.22086 v3 pith:UBVH3FUI submitted 2024-10-29 cs.LG cs.CL

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate

Zhiqi Bu , Xiaomeng Jin , Bhanukiran Vinzamuri , Anil Ramakrishna , Kai-Wei Chang , Volkan Cevher , Mingyi Hong This is my paper
classification cs.LG cs.CL
keywords unlearningoptimizationdifferencegradientlearningmachinemulti-taskngdiff
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspective, framing it as a regularized multi-task optimization problem, where one task optimizes a forgetting objective and another optimizes the model performance. In particular, we introduce a normalized gradient difference (NGDiff) algorithm, enabling us to have better control over the trade-off between the objectives, while integrating a new, automatic learning rate scheduler. We provide a theoretical analysis and empirically demonstrate the superior performance of NGDiff among state-of-the-art unlearning methods on the TOFU and MUSE datasets while exhibiting stable training.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding

    cs.LG 2025-11 reject novelty 6.0

    LLM unlearning methods that pass greedy-decoding benchmarks leak forgotten facts when the model is sampled repeatedly, and the new leak@k metric quantifies this.

  2. OFMU: Optimization-Driven Framework for Machine Unlearning

    cs.LG 2025-09 unverdicted novelty 6.0

    A penalty-based bi-level optimization framework for machine unlearning that decorrelates forget and retention gradients via inner maximization and restores utility via outer minimization, with convergence guarantees a...

  3. LLM Unlearning for Cyber Defense: A Survey on Methods, Challenges, and Emerging Threats

    cs.LG 2026-06 conditional novelty 4.0

    Most gradient-based LLM unlearning methods achieve behavioral suppression, not true forgetting, and current benchmarks cannot certify that knowledge has been removed.