Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Model Unlearning via Sparse Autoencoder Subspace Guided Projections

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that sparse-autoencoder feature subspaces can guide parameter-space unlearning, giving better forgetting and jailbreak robustness than gradient-based or inference-time steering baselines.

desk verdict SSPU is a genuinely new unlearning method, but its headline gain over RMU is inflated by selecting the SAE layer on the evaluation set itself; worth refereeing, not yet convincing. read the letter →

arxiv 2505.24428 v1 pith:DUUMFKS3 submitted 2025-05-30 cs.CL cs.LG

Xu Wang , Zihao Li , Benyou Wang , Yan Hu , Difan Zou This is my paper · ORCID
classification cs.CLcs.LG
keywords machineunlearningsparseautoencoderssubspaceprojectionlargelanguagemodelsjailbreakrobustnessWMDP-Cyberrepresentationalmisdirectionknowledgeretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SSPU is an attempt to make LLM unlearning interpretable and controllable by using sparse autoencoder (SAE) features as a geometric map of what should be forgotten. The paper argues that instead of steering activations at inference time or applying unconstrained gradient ascent, one can build two subspaces from SAE decoder vectors—one aligned with the target topic, one orthogonal to it—and fine-tune a small set of weights so that forget-set activations are projected into the irrelevant subspace while retained activations stay frozen. On gemma-2-2b-it, this lowers WMDP-Cyber accuracy from 27.13% (the strongest baseline, RMU) to 23.91%, keeps MMLU, TruthfulQA, and GSM8K near their original levels, and reduces accuracy under four jailbreak prompt types. A sympathetic reader would take away that interpretable activation subspaces can supervise parameter-space updates, yielding a controllable forgetting-retention trade-off.

What carries the argument

The load-bearing object is the pair of orthonormal subspaces $U_{\mathrm{reg}}$ and $U_{\perp}$ built by QR decomposition of selected sparse autoencoder decoder columns: $U_{\mathrm{reg}}$ spans the top-$K$ forget-relevant features and $U_{\perp}$ spans the bottom-$K$ forget-irrelevant features. The unlearning loss $\|h^u_f - c\|_2^2$ drives forget-batch activations toward a random control vector $c = \gamma U_{\perp}U_{\perp}^{\top}r / \|U_{\perp}U_{\perp}^{\top}r\|_2$ lying entirely in $U_{\perp}$, while the regularization loss $\sum_p \|(I - U_{\mathrm{reg}}U_{\mathrm{reg}}^{\top})(p - p_0)\|_2^2$ restricts parameter changes to the relevant subspace. Together these make unlearning a projection problem rather than an open-ended gradient search.

What would settle it

Compute the mean squared activation of the bottom-$K$ features on the retain benchmarks (MMLU, TruthfulQA, GSM8K); if any 'irrelevant' feature responds strongly to a retained task, $U_{\perp}$ overlaps with retained knowledge and SSPU's projection loss should degrade that task, which can be checked from the paper's own activation-score formulas without retraining.

Watch

Extended reading notes

Core claim

The central claim is that the directions a sparse autoencoder uses to represent a topic can be turned into coordinates for unlearning: align the weight update with the forget-relevant subspace, drive forget activations into the orthogonal 'irrelevant' subspace, and the model forgets the target knowledge while preserving unrelated capabilities. The paper supports this with a three-stage pipeline—data-driven layer and feature selection, QR-based subspace construction, and a constrained objective combining unlearn, retain, and subspace-regularization losses—and reports that SSPU outperforms the strongest baseline (RMU) by 3.22 percentage points on WMDP-Cyber forgetting, raises average utility by 2.88 points over RMU, and lowers jailbreak attack success rates below both RMU and inference-time SAE steering.

Load-bearing premise

The load-bearing premise is that the bottom-$K$ SAE features define an 'irrelevant' subspace $U_{\perp}$ whose directions are semantically unrelated to everything the model should retain; the paper shows these features activate weakly on the forget corpus but does not show they are neutral on the retain distribution, so if $U_{\perp}$ overlaps with retained-task directions, the unlearning loss will erode utility beyond what the retain loss can compensate.

Editorial extensions

If this is right

  • If SSPU is correct, unlearning gains an interpretable substrate: each parameter update can be traced to the SAE feature directions it suppresses, a property gradient-only methods lack.
  • Weight-space unlearning guided by SAE subspaces should resist jailbreak prompts better than inference-time steering, because the target knowledge is edited in the weights rather than shunted at inference time.
  • The reported 3.22-point WMDP-Cyber improvement over RMU with better average utility implies the forgetting-retention trade-off is not fixed by the baseline method; subspace constraints can move it.
  • Because the pipeline selects layer and features from data, the same recipe should transfer to other target topics and models, provided a suitable SAE is available.
  • The method needs a forget corpus and a separate retain corpus; entangled or non-separable corpora are outside the demonstrated setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An open check is whether $U_{\perp}$ is truly neutral for retained tasks: the paper verifies bottom-$K$ features activate weakly on the forget set, but not that they are uninformative for MMLU, TruthfulQA, or GSM8K; a retain-side activation score would settle it.
  • The same subspace-projection recipe could be applied to other unlearning targets, such as copyrighted text or personally identifying information, whenever an SAE feature set can be identified; the paper does not test this.
  • Constraining updates to span($U_{\mathrm{reg}}$) may cap achievable forgetting; topics needing larger weight movement than the relevant subspace permits would likely require iterative subspace re-estimation, a regime the paper does not explore.
  • The jailbreak evaluation uses rephrased multiple-choice questions; free-form generation could reveal partial or indirect recall that MCQ accuracy does not capture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SSPU (SAE-Guided Subspace Projection Unlearning), a method that uses sparse autoencoder features to identify subspaces relevant and irrelevant to a forget topic, then fine-tunes the model by steering forget activations into the 'irrelevant' subspace while constraining weight updates to the 'relevant' subspace. The authors evaluate SSPU on the WMDP-Cyber benchmark for forgetting and on MMLU, TruthfulQA, and GSM8K for retention, reporting a 3.22% improvement in forgetting over the best baseline (RMU) and improved resistance to jailbreak prompts. The method is presented as a three-stage pipeline with layer/feature selection, QR-based subspace construction, and a regularized objective.

Significance. If established, the result is significant: it combines the interpretability of SAE features with weight-space unlearning, potentially offering a better forgetting-retention trade-off and adversarial robustness than either pure gradient-based or inference-time steering methods. The paper makes a clear contribution by proposing a concrete, mechanistically motivated pipeline and by reporting experiments on a standard unlearning benchmark with multiple utility checks. The jailbreak robustness evaluation is a worthwhile addition. However, the central empirical claim is currently compromised by test-set selection in the layer choice and by the absence of error bars; these issues must be addressed before the claimed advantage can be considered reliable.

major comments (3)
  1. [§4.2, §4.3, Table 1] The layer selection procedure in §4.2 measures unlearning strength on the WMDP-Cyber multiple-choice test set and then chooses layer 3 for all subsequent experiments. Table 1 then reports SSPU's accuracy on that same test set. This is a post-hoc selection on the evaluation metric: the method's key architectural choice is tuned to maximize forgetting on the benchmark, while the baselines are not given a comparable selection step. The reported 3.22% advantage over RMU may therefore reflect selection bias rather than a genuine improvement. Please select the layer on a validation split or use cross-validation, and report results across all candidate layers to show robustness.
  2. [Appendix A, Table 1] All experiments appear to use a single random seed (seed 42, as stated in Appendix A). With no error bars or multiple runs, the 3.22% WMDP-Cyber gap between SSPU and RMU cannot be distinguished from noise, especially given the small absolute difference. Please report means and standard deviations over at least 3–5 seeds, and state whether the reported numbers are averages or single runs.
  3. [§3.2, §3.3, Limitations] The 'irrelevant' subspace U⊥ is constructed from the bottom-K SAE features ranked by forget activation, but the paper only demonstrates that these features activate weakly on the forget set (Figure 3, center). It does not show that they are neutral with respect to the retain distribution (MMLU, TruthfulQA, GSM8K). If U⊥ contains directions that are important for retained capabilities, steering forget activations into U⊥ could cause collateral damage that the retain loss in Eq. (5) cannot fully compensate. The Limitations section itself concedes that the method 'does not explicitly guarantee that unrelated capabilities outside this subspace remain entirely unaffected.' Please provide evidence about the retain-set activation of the selected bottom-K features, or show that the utility benchmarks are stable across different choices of K or across seeds.
minor comments (5)
  1. [Abstract] The phrase 'reduces harmful knowledge accuracy by 3.22%' is ambiguous because it reads as an absolute reduction rather than a relative one; please clarify that it is a 3.22 percentage-point reduction on the WMDP-Cyber accuracy metric.
  2. [§4.2] The layer selection uses 'top-K features by sparsity' but the actual ranking criterion in §3.1 is based on mean squared activation and an importance ratio; the wording is inconsistent.
  3. [Figure 3] The jailbreak robustness results are only presented graphically; please include the exact numerical values (or a table) so that the claimed 13.59% and 2.83% reductions can be verified.
  4. [§4.4] The jailbreak datasets are constructed by rewriting WMDP-Cyber items, but there is no human validation or automatic check that the rewrites preserve the original answer; please discuss potential validity concerns.
  5. [Algorithm 1] The projection operator in Eq. (6) and the regularization term Lreg are applied to the parameter difference p - p0, but the derivation in Appendix B writes Lreg(p) = ||(I - UregUreg^T) p||^2; please make the notation consistent.

Circularity Check

1 steps flagged · score 4.0 of 10

One validation step is circular by construction; layer selection on the test set further biases the headline number, though the core SSPU derivation is not circular.

  1. self definitional [Section 3.1 (Eqs. 1-2) and Section 4.2, Figure 3 center]
    "Sbottomfeats = BottomK({1 ≤ j ≤ D}, K). ... Sbottomfeats is the set of K feature indices with the lowest forget_scorej across all D SAE features. ... The top-K features (blue line) exhibit markedly higher mean squared activation in the forget set compared to the bottom-K features (orange line). This demonstrates that ... the bottom-K subspace contains virtually no such information."

    The bottom-K features are defined as the K features with the lowest forget_score on the forget corpus, and then Figure 3 center is presented as evidence that these same features have low mean squared activation on the forget set. The 'demonstration' merely restates the selection criterion; it cannot independently establish that U⊥ is semantically irrelevant to the forget topic. The paper uses this tautological plot to justify treating U⊥ as an 'irrelevant' subspace and as the target for Lunlearn, so the evidential support for that subspace property reduces to its own definition.

full rationale

The core SSPU derivation is self-contained and not circular: the forget/retain subspaces are built from SAE decoder columns ranked by activation statistics on the forget and retain corpora (Eqs. 1-2), and the unlearning objective (Eqs. 3-7) is an optimization over those subspaces; no benchmark label is used to fit the loss, and no output constant is fitted. The cited SAE (Gemma Scope), RMU, and related baselines are external or independently specified, and the authors' own prior circuit-analysis paper is only mentioned in Related Work, not load-bearing. The one genuinely circular validation is in §4.2/Fig. 3: bottom-K features are selected as the features with the lowest forget-set activation, and then the paper 'demonstrates' they have low forget-set activation; this is the selection criterion restated and cannot independently establish that U⊥ is irrelevant to the forget topic. Separately, the §4.2 layer choice is made by measuring accuracy drop on the WMDP-Cyber test set, and Table 1 then reports SSPU's forgetting on that same test set; this is a test-set-selection/overfitting concern for the headline 3.22% gain over RMU, but it is not a derivation-level circularity. The paper's own Limitations note that SSPU 'does not explicitly guarantee that unrelated capabilities outside this subspace remain entirely unaffected.' Because the central method still has independent empirical content, the circularity score is moderate rather than high.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities beyond the SAE features and subspaces, which come from the existing Gemma Scope SAE. The central claim depends on several free hyperparameters (gamma, alpha, lambda_reg, K, tau, layer) and on the domain assumption that SAE bottom features are semantically irrelevant to retained knowledge. No code or data release is provided.

free parameters (6)
  • steering coefficient gamma = 200
    Controls the norm of the control vector c in Eq. (3); chosen by hand in Appendix A.
  • retention weight alpha = 50
    Balances the retain loss in Eq. (5); set in Appendix A.
  • subspace regularization weight lambda_reg = 1e-4
    Weights the parameter-update constraint in Eq. (6); set in Appendix A.
  • subspace size K = 1024
    Number of top and bottom SAE features used to build Ureg and U⊥; no sensitivity analysis is provided.
  • percentile threshold tau
    Used to filter features by importance ratio in Section 3.1; its value is not specified in the paper.
  • feature extraction layer ell = 3
    Selected in Section 4.2 by measuring unlearning strength on the WMDP-Cyber test set, a post-hoc choice on the target metric.
assumptions (4)
  • domain assumption SAE features are semantically interpretable and their activation scores separate topic-relevant from topic-irrelevant directions.
    The entire feature-selection pipeline in Section 3.1 relies on this; if false, the subspaces are meaningless.
  • domain assumption The bottom-K features form an 'irrelevant' subspace U⊥ that is safe to push forget activations into without harming retained knowledge.
    Eq. (4) drives forget activations toward U⊥; the paper only shows these features fire weakly on the forget set, not that they are neutral for retain tasks.
  • standard math Left-multiplying (I - Ureg Ureg^T) onto the weight delta projects out update components in the relevant activation subspace, assuming the weight matrix's input dimension matches the residual-stream dimension.
    Used in Eq. (6); dimensionally valid if p is a matrix whose rows align with residual-stream features, but the paper does not explicitly state this.
  • ad hoc to paper Measuring steering-based unlearning strength on the WMDP-Cyber test set is a valid way to select the SAE layer for the final method.
    Section 4.2 uses the test set to choose layer 3, then Table 1 reports final results on the same set, implicitly assuming no leakage from this selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model Unlearning via Sparse Autoencoder Subspace Guided Projections." pith.science (2026). https://pith.science/paper/DUUMFKS3

@misc{pith2026250524428,
  author       = {Pith},
  title        = {Pith review of: Model Unlearning via Sparse Autoencoder Subspace Guided Projections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DUUMFKS3}},
  note         = {Machine review of arXiv:2505.24428}
}
read the original abstract

Large language models (LLMs) store vast amounts of information, making them powerful yet raising privacy and safety concerns when selective knowledge removal is required. Existing unlearning strategies, ranging from gradient-based fine-tuning and model editing to sparse autoencoder (SAE) steering, either lack interpretability or fail to provide a robust defense against adversarial prompts. We propose SAE-Guided Subspace Projection Unlearning (SSPU), a novel framework that leverages SAE features to drive targeted updates in the model's parameter space, enabling precise, interpretable, and robust unlearning. SSPU's three-stage pipeline performs data-driven layer and feature selection, subspace construction via QR decomposition, and constrained optimization that controls activations into an "irrelevant" subspace while preserving retained knowledge. Overall, we use SAE features to construct a subspace that supervises unlearning, refining the loss and adding a regularization term to guide interpretable parameter updates. In experiments on the WMDP-Cyber forget set and three utility benchmarks (MMLU, TruthfulQA, GSM8K), SSPU reduces harmful knowledge accuracy by 3.22% compared to the strongest baseline. It also improves adversarial robustness, lowering malicious accuracy under jailbreak prompts compared to baselines. Our findings expose the limitations of prior unlearning methods and demonstrate how interpretable subspace-guided optimization can achieve robust, controllable model behavior.

Figures

Figures reproduced from arXiv: 2505.24428 by the authors.

Figure 1
Figure 1. Three-stage overview of our SSPU: SAE–Guided Subspace Projection Unlearning. (a) Feature Selection: extract SAE activations on forget and retain examples, compute activation scores, and select the top- and bottom-ranked latent dimensions. (b) Subspace Construction: collect decoder vectors for the selected features and perform QR decomposition to obtain orthonormal bases for the relevant and irrelevant subspaces. (c)… view at source ↗
Figure 2
Figure 2. Overview of our experimental framework. Left: the datasets used for unlearning, including WMDP–Cyber as the forget corpus and WikiText as the retain corpus. Center: four unlearning methods–Gradient Ascent (GA), Negative Preference Optimization (NPO), Represen￾tation Misdirection Unlearning (RMU), and SAE-based unlearning–shown with their core update formulas. Right: four metrics for unlearning. Forgetting Ability on… view at source ↗
Figure 3
Figure 3. Layer-wise unlearning effectiveness, feature selection analysis and jailbreak robustness. Left: Layer-wise unlearning effectiveness measured on the WMDP–Cyber test set by steering the top-10, top-50, and top-100 SAE-extracted features at six different layers of the gemma-2b-it model. Center: Mean squared activation strength on the forget set for the top-10 (blue) versus bottom-10 (orange) SAE-extracted features. Rig… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories

    cs.AI 2025-12 conditional novelty 6.0 of 10

    RippleBench automatically generates questions at increasing semantic distance from unlearned topics and shows all eight tested unlearning methods degrade accuracy that recovers only slowly with distance.

Reference graph

Works this paper leans on

28 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [3]

    URL https://arxiv.org/abs/2502.12520. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems,

    arXiv

  2. [5]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang

    URL https://arxiv.org/abs/2503.01854. Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. Transactions on Machine Learning Research,

    arXiv

  3. [7]

    URL https://doi.org/10.48550/arXiv.2404. 18239. Dahyun Jung, Jaehyung Seo, Jaewook Lee, Chanjun Park, and Heuiseok Lim. CoME: An unlearning- based approach to conflict-free model editing. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguis...

    doi:10.48550/arxiv.2404

  4. [8]

    URL https://aclanthology.org/2025.naacl-long.325/

    Association for Computational Linguistics. URL https://aclanthology.org/2025.naacl-long.325/. Matthew Khoriaty, Andrii Shportko, Gustavo Mercier, and Zach Wood-Doughty. Don’t forget it! conditional sparse autoencoder clamping works for unlearning,

    work page 2025

  5. [9]

    Edward Kim

    URL https://arxiv.org/ abs/2503.11127. Edward Kim. Nevermind: Instruction override and moderation in large language models,

    arXiv

  6. [10]

    Nevermind: Instruction Override and Moderation in Large Language Models

    URL https://arxiv.org/abs/2402.03303. Hyoseo Kim, Dongyoon Han, and Junsuk Choe. Negmerge: Consensual weight negation for strong machine unlearning. In Adaptive Foundation Models: Evolving AI for Personalized and Efficient Learning,

    work page Pith review arXiv

  7. [11]

    Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D

    URL https://doi.org/10.18653/v1/2024.naacl-long.228. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew Bo Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu...

    doi:10.18653/v1/2024.naacl-long.228 2024

  8. [12]

    doi: 10.18653/v1/2024.blackboxnlp-1

    Association for Computational Linguistics. doi: 10.18653/v1/2024.blackboxnlp-1

    doi:10.18653/v1/2024.blackboxnlp-1 2024

Show all 28 references
  1. [15]

    Christopher J Lynch, Erik J Jensen, Virginia Zamponi, Kevin O’Brien, Erika Frydenlund, and Ross Gore

    URL https://arxiv.org/abs/2502.15910. Christopher J Lynch, Erik J Jensen, Virginia Zamponi, Kevin O’Brien, Erika Frydenlund, and Ross Gore. A structured narrative prompt for prompting narratives from large language models: sentiment assessment of chatgpt-generated narratives a...

    arXiv

  2. [17]

    Aashiq Muhamed, Jacopo Bonato, Mona Diab, and Virginia Smith

    URL https://doi.org/10.48550/arXiv.2403.08295. Aashiq Muhamed, Jacopo Bonato, Mona Diab, and Virginia Smith. Saes Can improve unlearning: Dynamic sparse autoencoder guardrails for precision unlearning in llms,

    doi arXiv

  3. [18]

    David Pape, Sina Mavali, Thorsten Eisenhofer, and Lea Schönherr

    URL https: //arxiv.org/abs/2504.08192. David Pape, Sina Mavali, Thorsten Eisenhofer, and Lea Schönherr. Prompt obfuscation for large language models,

    arXiv

  4. [19]

    Johnny Lin

    URL https://aclanthology.org/2024.blackboxnlp-1.19/. Johnny Lin. Neuronpedia: Interactive reference and tooling for analyzing neural networks,

    2024

  5. [20]

    Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju

    URL https://arxiv.org/abs/2409.11026. Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few-shot unlearners. In Forty-first International Conference on Machine Learning,

    arXiv

  6. [21]

    Nicholas Pochinkov and Nandi Schoots

    URL https://openreview.net/forum?id=GKcwle8XC9. Nicholas Pochinkov and Nandi Schoots. Dissecting language models: Machine unlearning via selective pruning. CoRR, abs/2403.01267,

    arXiv

  7. [22]

    URL https://doi.org/10.48550/arXiv.2403.01267

    doi: 10.48550/ARXIV .2403.01267. URL https://doi.org/10.48550/arXiv.2403.01267. Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders,

    doi arXiv

  8. [23]

    Mark Russinovich and Ahmed Salem

    URL https://arxiv.org/abs/2407.14435. Mark Russinovich and Ahmed Salem. Obliviate: Efficient unmemorization for protecting intellectual property in large language models,

    arXiv

  9. [24]

    12 Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang

    URL https://arxiv.org/abs/2502.15010. 12 Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang. Knowledge unlearning for llms: Tasks, methods, and challenges,

    arXiv

  10. [25]

    URL https://arxiv.org/abs/2311. 15766. Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. CoRR, abs/2308.10248,

    arXiv

  11. [26]

    Xu Wang, Yan Hu, Wenyu Du, Reynold Cheng, Benyou Wang, and Difan Zou

    URL https://doi.org/10.48550/arXiv.2308.10248. Xu Wang, Yan Hu, Wenyu Du, Reynold Cheng, Benyou Wang, and Difan Zou. Towards understanding fine-tuning mechanisms of LLMs via circuit analysis. In ICLR 2025 Workshop on Building Trust in Language Models and Applications , 2025a. ...

    2025 doi arXiv

  12. [27]

    Yuanshun Yao, Xiaojun Xu, and Yang Liu

    URL https://arxiv.org/abs/2502.11190. Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems,

    arXiv

  13. [28]

    org/abs/2503.03710

    URL https://arxiv. org/abs/2503.03710. A E XPERIMENTAL PARAMETER SETTINGS All unlearning experiments operate on the same subset of model parameters (the MLP up-projection weights) in layers [1,2,3] and parameter indices

  14. [29]

    relevant

    A fixed random seed of 42 ensures reproducibil- ity. Gradient Ascent (GA). We fine-tune with a learning rate of 3 × 10−5 over a single epoch and up to 500 update batches. A linear warmup of 20 steps is used, and gradients are clipped to a norm of 1.0. The objective combines a ...

    2025

  15. [2016]

    URL https://arxiv.org/abs/1609.07843. Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev,...

    arXiv

  16. [2021]

    Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy

    URL https://arxiv.org/ abs/2110.14168. Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy. Applying sparse autoencoders to unlearn knowledge in language models. In Neurips Safe Generative AI Workshop 2024 ,

    2024 arXiv

  17. [2022]

    doi: 10.18653/v1/2022.acl-long.229

    Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.229. URL https://aclanthology.org/2022.acl-long.229/. Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding-corrupted prompts. In The Thirty-eighth A...

    2022 doi

  18. [2023]

    doi: 10.18653/v1/2023

    Association for Computational Linguistics. doi: 10.18653/v1/2023. acl-long.805. URL https://aclanthology.org/2023.acl-long.805/. Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power...

    2023 arXiv

  19. [2024]

    Chein-I Chang

    URL https://arxiv.org/abs/2406.12038. Chein-I Chang. Orthogonal subspace projection (osp) revisited: A comprehensive study and analysis. IEEE transactions on geoscience and remote sensing, 43(3):502–518,

    arXiv

  20. [2025]

    Karuna Bhaila, Minh-Hao Van, and Xintao Wu

    URL https://arxiv.org/abs/2501.04952. Karuna Bhaila, Minh-Hao Van, and Xintao Wu. Soft prompting for unlearning in large language models,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.