pith. sign in

arxiv: 2605.21539 · v1 · pith:WSCAMYALnew · submitted 2026-05-20 · 💻 cs.LG

DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models

Xuyang Zhong , Qizhang Li , Yiwen Guo , Chen Liu This is my paper

Pith reviewed 2026-05-22 00:46 UTC · model grok-4.3

classification 💻 cs.LG
keywords machine unlearninglarge language modelsoptimizer statesgradient conflictforgetting and retainingsafety alignmentmulti-task learningquantized optimization
0
0 comments X

The pith

DualOptim+ separates shared and objective-specific optimizer states in LLMs and bridges them adaptively using the direction of gradient conflict between forgetting and retaining.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces DualOptim+ to handle the tension in machine unlearning where an optimizer must erase specific knowledge while preserving everything else. It keeps a single base state that holds representations common to both forgetting and retaining goals and adds separate delta states that hold only the differences each goal needs. The optimizer then decides how strongly to connect or separate these states according to whether the two gradients point in similar or opposing directions. A quantized 8-bit version is also provided to cut memory use. Experiments on both made-up and real unlearning scenarios plus safety and multi-task settings show the approach produces better simultaneous performance on all goals than standard methods.

Core claim

DualOptim+ maintains a base optimizer state that captures representations shared across forgetting and retaining objectives together with separate delta states that store only the objective-specific residuals; the framework then adaptively adjusts the degree of sharing versus decoupling by measuring the directional conflict between the forgetting and retaining gradients, yielding improved trade-offs without added instability. The 8-bit variant preserves this behavior at lower memory cost.

What carries the argument

Adaptive bridging mechanism that measures directional conflict between forgetting and retaining gradients to decide how much to share the base state versus keep the delta states decoupled.

If this is right

  • The same base-plus-delta structure can be applied to any training run that must simultaneously satisfy conflicting objectives such as safety alignment and capability retention.
  • Memory cost can be lowered by switching to the 8-bit quantized variant while keeping the same unlearning performance.
  • The method extends directly to real-world unlearning requests where only a subset of training data must be forgotten.
  • Multi-task learning settings benefit because the shared base state prevents one task from erasing progress on another.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same conflict-measurement idea could be tested in continual learning to decide when to freeze versus update shared parameters.
  • If gradient conflict can be computed cheaply, the technique might generalize to any optimizer that currently uses a single set of states for multiple loss terms.
  • One could measure whether the benefit scales with model size by repeating the fictitious unlearning experiments on models larger than those tested here.

Load-bearing premise

The directional conflict between forgetting and retaining gradients can be measured reliably enough to decide when and how to bridge the shared base state with the decoupled delta states without creating new training instabilities.

What would settle it

Run the same unlearning benchmarks with and without the conflict-based bridging rule; if the version that ignores the measured conflict direction matches or exceeds DualOptim+ on the joint forgetting-plus-retaining metrics, the adaptive bridging step is not carrying the claimed benefit.

Figures

Figures reproduced from arXiv: 2605.21539 by Chen Liu, Qizhang Li, Xuyang Zhong, Yiwen Guo.

Figure 2
Figure 2. Figure 2: Comparison of cosine similarity over time steps. (a) Similarity between the update terms for forgetting and retaining of different methods using targeted unlearning objective (Yuan et al., 2025). (b) Similarity between forgetting and retaining gradients of targeted and untargeted unlearning objectives. For better visualization, the similarity is calculated based on the exponential moving average (EMA) of t… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of unlearning metrics and losses over time steps. We plot the (a) forget efficacy, i.e., the average of targeted forget efficacy and untargeted forget efficacy, (b) model utility, (c) forgetting loss, and (d) retaining loss. Note that the results are collected when forgetting 5% data of TOFU using IDK+GD loss function. The model is TOFU-finetuned Phi 1.5. D. Comparison with Federated Learning Me… view at source ↗
read the original abstract

We propose DualOptim+, a novel optimization framework for improving machine unlearning in large language models. It introduces a base state to capture common representations shared by forgetting and retaining objectives and delta states to preserve objective-specific residuals. This architecture allows the optimizer to adaptively bridge shared and decoupled states based on the directional conflict between forgetting and retaining gradients. We further introduce DualOptim+ 8bit, a quantized variant that reduces memory overhead without compromising performance. Extensive experiments across fictitious and real-world unlearning, safety alignment, and multi-task learning tasks demonstrate that DualOptim+ consistently achieves a superior trade-off between different objectives. Codes are available at https://github.com/CityU-MLO/DualOptimPlus.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes DualOptim+, a novel optimizer for machine unlearning in LLMs. It maintains a shared base state capturing common representations across forgetting and retaining objectives, plus objective-specific delta states for residuals. The optimizer adaptively bridges these states using a measure of directional conflict between the forgetting and retaining gradients. A quantized 8-bit variant (DualOptim+ 8bit) is also introduced to reduce memory use. Extensive experiments on fictitious and real-world unlearning, safety alignment, and multi-task learning are reported to show superior objective trade-offs, with code released at the cited GitHub repository.

Significance. If the central empirical claims hold under scrutiny, the work offers a practical mechanism for managing gradient conflicts in LLM unlearning without requiring fully separate optimizers. The base-plus-delta architecture and adaptive bridging rule constitute a clear methodological contribution, and the public code plus quantized variant support reproducibility and deployment considerations. The result, if robust, would be relevant to privacy-preserving and safety-aligned LLM training pipelines.

major comments (2)
  1. [§4.2] §4.2 (Adaptive Bridging Rule): the directional conflict signal (inner product or cosine between forgetting and retaining gradients) is used to decide the bridging weight, but the manuscript does not report layer-wise statistics or variance across mini-batches. In high-dimensional LLM spaces most parameters contribute little to either objective; if the signal is near zero or dominated by a few layers, the adaptive rule collapses to a fixed mixture and the claimed advantage over non-adaptive baselines disappears. A concrete test (e.g., histogram of conflict values per layer or ablation with noise-injected gradients) is required to establish that the measurement is load-bearing rather than incidental.
  2. [Table 2, Figure 3] Table 2 and Figure 3 (Unlearning Trade-off Results): the reported superiority is shown via aggregate metrics, yet no error bars, multiple random seeds, or data-selection criteria are provided. Without these, it is impossible to judge whether the observed gains are statistically reliable or sensitive to the particular fictitious/real-world splits used. This directly affects the central claim that DualOptim+ “consistently achieves a superior trade-off.”
minor comments (2)
  1. [§3.1] §3.1: the notation for base state θ_base and delta states Δ_forget, Δ_retain is introduced without an explicit update equation; adding the precise optimizer step (e.g., how the bridged state is formed) would improve clarity.
  2. [§5.3] §5.3 (Quantized Variant): memory savings are stated for DualOptim+ 8bit, but no corresponding wall-clock or convergence curves are shown; a brief comparison table would help readers assess the performance-memory trade-off.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback and recommendation for major revision. We address each major comment point by point below, providing the strongest honest responses possible based on the manuscript. Revisions have been made to incorporate additional analyses and clarifications where the comments identify gaps in the current presentation.

read point-by-point responses

  1. Referee: [§4.2] §4.2 (Adaptive Bridging Rule): the directional conflict signal (inner product or cosine between forgetting and retaining gradients) is used to decide the bridging weight, but the manuscript does not report layer-wise statistics or variance across mini-batches. In high-dimensional LLM spaces most parameters contribute little to either objective; if the signal is near zero or dominated by a few layers, the adaptive rule collapses to a fixed mixture and the claimed advantage over non-adaptive baselines disappears. A concrete test (e.g., histogram of conflict values per layer or ablation with noise-injected gradients) is required to establish that the measurement is load-bearing rather than incidental.

    Authors: We agree that layer-wise statistics and variance analysis are needed to confirm the adaptive bridging rule is not incidental. In the revised manuscript we add histograms of per-layer cosine similarities (the directional conflict measure) computed over multiple mini-batches on the primary unlearning benchmarks. These plots show clear variation both across layers and across batches, with many layers exhibiting non-negligible conflict values rather than near-zero signals. We further include a controlled ablation that injects isotropic noise into the gradients to reduce effective conflict; DualOptim+ retains its advantage over fixed-mixture baselines under this perturbation, indicating that the adaptive rule remains load-bearing. revision: yes

  2. Referee: [Table 2, Figure 3] Table 2 and Figure 3 (Unlearning Trade-off Results): the reported superiority is shown via aggregate metrics, yet no error bars, multiple random seeds, or data-selection criteria are provided. Without these, it is impossible to judge whether the observed gains are statistically reliable or sensitive to the particular fictitious/real-world splits used. This directly affects the central claim that DualOptim+ “consistently achieves a superior trade-off.”

    Authors: We accept that the absence of error bars and seed reporting limits assessment of statistical reliability. The revised manuscript now reports all Table 2 and Figure 3 results as means over three independent random seeds together with standard deviations; error bars are added to the figure. We also expand the experimental setup section with explicit criteria for constructing the fictitious and real-world unlearning splits (including how samples were selected and balanced), thereby addressing sensitivity to particular data partitions. revision: yes

Circularity Check

0 steps flagged

No circularity: DualOptim+ is a constructive empirical method with independent experimental validation

full rationale

The paper defines DualOptim+ as a novel optimizer architecture with base states for shared representations and delta states for objective-specific residuals, plus an adaptive bridging rule driven by measured directional conflict between forgetting and retaining gradients. This is a direct method proposal rather than a derivation that reduces to its own inputs by construction. No equations or claims in the provided text equate a 'prediction' or result to a fitted parameter or self-referential definition. Experiments on fictitious/real-world unlearning, safety alignment, and multi-task tasks supply external empirical support, and the 8-bit quantized variant is presented as a practical extension without circular reduction. The approach is self-contained against benchmarks and does not rely on load-bearing self-citations or uniqueness theorems imported from prior author work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 2 invented entities

Review based on abstract only; no explicit free parameters, axioms, or invented entities detailed beyond the high-level description of base and delta states as new architectural components.

invented entities (2)
  • base state no independent evidence
    purpose: capture common representations shared by forgetting and retaining objectives
    Introduced as core part of the optimizer architecture to handle shared information.
  • delta states no independent evidence
    purpose: preserve objective-specific residuals
    Introduced to handle differences between forgetting and retaining goals.

pith-pipeline@v0.9.0 · 5655 in / 1194 out tokens · 39107 ms · 2026-05-22T00:46:52.404340+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages · 3 internal anchors

  1. [1]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    URL https://api.semanticscholar. org/CorpusID:248118878. Bianchi, F., Suzgun, M., Attanasio, G., Rottger, P., Juraf- sky, D., Hashimoto, T., and Zou, J. Safety-tuned llamas: Lessons from improving the safety of large language mod- els that follow instructions. InThe Twelfth International Conference on Learning Representations, 2024. Bourtoule, L., Chandra...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  2. [2]

    org/CorpusID:239998651

    URL https://api.semanticscholar. org/CorpusID:239998651. Dang, Q.-V . Right to be forgotten in the age of machine learning. InAdvances in Digital Science: ICADS 2021, pp. 403–411. Springer, 2021. Deng, Z., Liu, C. Y ., Pang, Z., He, X., Feng, L., Xuan, Q., Zhu, Z., and Wei, J. GUARD: Generation-time LLM unlearning via adaptive restriction and detection. I...

    work page doi:10.18653/v1/2021.acl-long.250 2021

  3. [3]

    Textbooks Are All You Need II: phi-1.5 technical report

    URL https://aclanthology.org/2025. emnlp-main.283/. Jia, J., Liu, J., Ram, P., Yao, Y ., Liu, G., Liu, Y ., Sharma, P., and Liu, S. Model sparsity can simplify machine unlearning.Advances in Neural Information Processing Systems, 36:51584–51605, 2023. Jordan, K., Jin, Y ., Boza, V ., You, J., Cesista, F., New- house, L., and Bernstein, J. Muon: An optimiz...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/p17-1147 2025

  4. [4]

    org/CorpusID:237532606

    URL https://api.semanticscholar. org/CorpusID:237532606. Liu, Z., Zhu, T., Tan, C., and Chen, W. Learning to refuse: Towards mitigating privacy risks in llms.arXiv preprint arXiv:2407.10058, 2024. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains...

    work page arXiv 2024

  5. [5]

    NumGLUE:

    URL https://openreview.net/forum? id=6lE4dQXaUcb. Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., and Kolter, J. Z. Tofu: A task of fictitious unlearning for llms. InFirst Conference on Language Modeling, 2024. Mishra, S., Mitra, A., Varshney, N., Sachdeva, B., Clark, P., Baral, C., and Kalyan, A. NumGLUE: A suite of fundamental yet challenging mat...

    work page doi:10.18653/v1/2022.acl-long.246 2024

  6. [6]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    URL https://api.semanticscholar. org/CorpusID:239616091. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models.arXiv preprint arXiv:2307.09288, 2023. Wang, P., Lu, S., Yin, H., Yang, B., Zhu, T., and Dai, C. Fedcm: client ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.24963/ijcai.2025/706 2023

  7. [7]

    12 DualOptim+ Yao, Y ., Xu, X., and Liu, Y

    URL https://openreview.net/forum? id=vQLUAkl5SG. 12 DualOptim+ Yao, Y ., Xu, X., and Liu, Y . Large language model unlearn- ing. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: //openreview.net/forum?id=8Dy42ThoNe. Yuan, X., Pang, T., Du, C., Chen, K., Zhang, W., and Lin, M. A closer look at machine unlearn...

    work page 2024

  8. [8]

    Zhong, X., Luo, H., and Liu, C

    URL https://openreview.net/forum? id=MXLBXjQkmb. Zhong, X., Luo, H., and Liu, C. Dualoptim: Enhancing efficacy and stability in machine unlearning with dual op- timizers. InThe Thirty-ninth Annual Conference on Neu- ral Information Processing Systems, 2025. URL https: //openreview.net/forum?id=77zz0JTNjn. Zou, A., Phan, L., Wang, J., Duenas, D., Lin, M., ...

    work page 2025

  9. [9]

    13 DualOptim+ A

    URL https://openreview.net/forum? id=IbIB8SBKFV. 13 DualOptim+ A. Pseudo-code of DualOptim+ with Muon The pseudo-code of DualOptim+ integrated in Muon (Jordan et al., 2024) is shown in Algorithm 2. Algorithm 2DualOptim+ with Muon 1: Input:parameter θ, learning rate η, momentum factor β, forget objective Lf , retain objective Lr, total steps N, forget freq...

    work page 2024

  10. [10]

    Convergence ofB t Based on the update rule (5), we have the following equation: B(Ff+Fr)T =β Ff+Fr B(Ff+Fr)(T−1) + (1−β)   FfX t=1 βFf+Fr−tgf,(Ff+Fr)(T−1)+t + FrX t=1 βFr−tgr,(Ff+Fr)(T−1)+F f+t   (8) Based on Assumption 3.1, we can conclude that limT→∞ B(Ff+Fr)T exists, so we let XB = lim T→∞ B(Ff+Fr)T . We consider the equation above, take the expect...

    work page

  11. [11]

    Convergence of∆ f When t∈((F f +F r)(T−1) +F f ,(F f +F r)T] , ∆f is not updated, so ∆f,(Ff+Fr)T = ∆f,(Ff+Fr)(T−1)+F f . Based on the update rule (6), we have the following equation: ∆f,(Ff+Fr)T =β Ff ∆f,(Ff+Fr)(T−1) + (1−β)· FfX t=1 βFf −t gf,(Ff+Fr)(T−1)+t − bB(Ff+Fr)(T−1)+t−1 (10) 14 DualOptim+ WhenT→ ∞, and for1≤t≤F f , according to (9), we have: lim ...

    work page

  12. [12]

    Convergence of∆ r Similarly to (10), we have: ∆r,(Ff+Fr)T =β Fr∆r,(Ff+Fr)(T−1) + (1−β)· FrX t=1 βFr−t gr,(Ff+Fr)(T−1)+F f+t − bB(Ff+Fr)(T−1)+F f+t−1 (13) WhenT→ ∞, and for1≤t≤F r, according to (9), we have: lim T→∞ B(Ff+Fr)(T−1)+F f+t−1 =β Ff+t−1XB + (1−β)   FfX k=1 βFf+t−1−k ·mG+ t−1X k=1 βt−1−k ·nG   =β Ff+t−1XB +β t−1(1−β Ff )mG+ (1−β t−1)nG (14) W...

    work page arXiv 1973

  13. [13]

    with a rank of 8 to Llama 2. As shown in Table 13, the performance gains from both DualOptim and DualOptim+ are less pronounced than in full-parameter unlearning, but DualOptim+ and its 8bit variant exhibit the best performance in most cases. Furthermore, the results indicate that the performance gap between LoRA and full-parameter tuning widens as the vo...

    work page arXiv 2015