Pith. sign in

REVIEW 5 major objections 5 minor 47 references

This paper claims that standard MLLM unlearning methods create 'knowledge holes'—severe degradation on benign inputs resembling forgotten content—and that its SPAR method restores 98% of vanilla response quality on LLaVA-1.5-7B while achiev

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:39 UTC pith:7KVEZNEO

load-bearing objection Useful but overclaimed: the knowledge-hole probe is a real addition, SPAR works on one model, and the paper's own Table 1 contradicts its 'systematic' and '0.00% ASR' claims. the 5 major comments →

arxiv 2608.01849 v1 pith:7KVEZNEO submitted 2026-08-03 cs.AI

Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models

Junxiang You , Junkai Chen , Yuhao He , Ruiqi Liu , Zhetao Guo , Shu Wu This is my paper
classification cs.AI
keywords knowledge holesmachine unlearningmultimodal large language modelssafety alignmentsingular value decompositionnull-space projectionresponse qualityattack success rate
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper establishes that current multimodal unlearning benchmarks miss a systematic failure: after removing unsafe content, models collapse on benign inputs that share generic response patterns with the forgotten material. The authors construct a probing benchmark that transplants structural patterns from forget-set responses onto harmless topics, and show that representative unlearning methods lose 50–80% of response quality on these probes even while standard utility benchmarks look healthy. They then introduce SPAR, which shields generic activation patterns from the forget loss using a frozen-anchor SVD filter and reinforces pattern handling through an entity-masked enhancement loss. On LLaVA-1.5-7B, SPAR recovers over 98% of the original response quality while keeping attack success rate at 0%. The same method fails on Qwen2.5-VL-3B, which the paper attributes to limited hidden dimensionality and strong safety alignment that misclassifies refusal behavior as a generic pattern.

Core claim

In the paper's own terms, knowledge holes are a measurable, systematic consequence of commonly used MLLM unlearning approaches: forget losses penalize everything in the hidden states that correlates with the forget set, including neutral generic patterns shared with benign tasks, so models become excessively refusal-prone on unrelated-but-similar inputs. The paper's central discovery is that this collateral damage can be largely decoupled from actual forgetting. SPAR computes a frozen SVD basis from a retain-fine-tuned reference model, filters the top-k singular directions (assumed to encode generic syntactic structure) out of the trainable model's activations before applying the forget loss

What carries the argument

The load-bearing mechanism is Anchored SVD inside the Anchored Forget Loss: singular vectors are computed once from a frozen reference model fine-tuned on the retain set, so the trainable model cannot rotate harmful content into the filtered directions. Top-k of these right singular vectors are treated as generic structural patterns; per-token projections onto them are normalized into weights, used to subtract the corresponding components from the trainable hidden states before the forget loss. The companion Abstracted Enhancement Loss masks entity tokens and zeroes the image, then applies cross-entropy only on the remaining non-entity positions, reinforcing generic competence without reintr

Load-bearing premise

The method assumes that in the hidden states, the top singular directions overwhelmingly encode generic syntactic structure while content-specific semantics live only in trailing directions; if that spectral separation fails—as the paper's own Qwen2.5-VL-3B results suggest—the anchored filter either shields harmful content or fails to protect benign patterns.

What would settle it

Compute, on Qwen2.5-VL-3B, the cosine similarity between the top-k anchored singular vectors and the embedding of refusal words ('sorry', 'cannot'); if these exceed similarity to generic function words, the anchor is protecting refusal behavior, which would explain the 80.33% ASR.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Standard utility benchmarks (MMVet, POPE, VizWiz) are insufficient to certify unlearned MLLMs; adjacent-input probes of this kind should be part of any safety evaluation.
  • Forgetting strength can be kept at 0% ASR while restoring benign response quality to near-vanilla level on models with sufficient hidden dimension, suggesting the two objectives are not inherently in conflict.
  • The success of anchored (frozen) SVD over standard SVD shows that unlearning methods must prevent the model from gaming the protection subspace by rotating its features.
  • The Qwen2.5-VL-3B failure implies a minimum representational capacity for null-space projection approaches to work, and safety-aligned models may absorb refusal patterns into the protected generic directions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • By symmetry, visual-side knowledge holes likely exist: unlearning weapon images may degrade recognition of visually similar benign objects (tools, sports equipment), which the paper lists as future work but never tests.
  • The spectral-separation assumption—top singular vectors = syntax, trailing = content—could be tested directly by measuring how well principal directions of forget vs. retain hidden states separate in different architectures; if separation degrades with hidden size, a principled threshold for SPAR-like methods could be derived.
  • The failure mode on Qwen suggests a testable extension: applying SPAR to a 3B-scale model without strong safety alignment should show whether the culprit is dimensionality or the alignment-induced refusal patterns.
  • If the frozen anchor is obtained from the retain-fine-tuned model, SPAR's effectiveness may depend on retain-set coverage of the generic patterns; probing with out-of-coverage patterns would bound the method's generality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper defines "knowledge holes" in unlearned multimodal LLMs as severe degradation on benign inputs that share generic patterns with the forget set, constructs a probing benchmark for this phenomenon, and proposes SPAR (Selective Protection with Anchored Regularization), which combines anchored-SVD activation filtering, entity-masked enhancement, and retain-set regularization. On LLaVA-1.5-7B the method is reported to recover near-vanilla response quality while achieving 0% ASR on SafeEraser. On Qwen2.5-VL-3B the method fails (ASR 80.33%), and the paper frames this as a boundary condition. The central claims are that knowledge holes are a systematic consequence of common unlearning methods across architectures and that SPAR bridges them while preserving forgetting quality.

Significance. The benchmark and the proposed method target a real blind spot in MLLM unlearning evaluation: standard utility benchmarks may miss localized, pattern-level degradation. If the LLaVA-1.5-7B results are reproducible, the paper demonstrates a useful diagnostic tool and a mitigation idea (anchored SVD to separate generic structure from content) that is mechanistically plausible. The ablation showing that standard SVD fails while anchored SVD succeeds (Table 2) is a valuable empirical point. However, the paper's headline claims are broader than the evidence: Table 1 contains a direct counterexample to "systematic" knowledge holes, and the abstract's unqualified 0.00% ASR is contradicted by the same table. The contribution is therefore currently supported mainly for one architecture and a subset of baselines, not as the general phenomenon claimed.

major comments (5)
  1. [§5, Table 1] The claim that knowledge holes are "a systematic consequence of unlearning, independent of model architecture, parameter scale, or the specific forget loss used" is contradicted by the paper's own data. On Qwen2.5-VL-3B, KL-Min has Res. Q 7.13 vs. vanilla 7.19 (relative drop <1%) and RR 3.67% vs. 3.46%, i.e., no knowledge hole by the paper's own definition. Because "systematic" is the abstract's central claim and the basis for the proposed evaluation paradigm, this internal counterexample must be addressed by either restricting the claim or explaining why this exception is not informative.
  2. [Abstract and §1, Table 1] The abstract states SPAR achieves "0.00% attack success rate" without qualification, but Table 1 reports SPAR on Qwen2.5-VL-3B with ASR 80.33%. The Limitations section (p. 11) acknowledges the failure on Qwen due to compact hidden dimension and safety alignment. The abstract and introduction should scope the 0.00% claim to LLaVA-1.5-7B, or report both results. As written, the headline claim is misleading.
  3. [§4, Anchored Forget Loss (Eqs. 1–6)] The method relies on the spectral assumption that top singular vectors of hidden states encode generic syntactic structure while content-specific semantics lie in trailing directions. No direct evidence is provided for this decomposition. The Qwen results in Appendix C, Table 6 (ASR rises from 18.67% at k=0 to 80.33% at k=2) are consistent with over-filtering, but they do not independently establish what the top-k singular vectors capture. The authors should provide spectral analysis, e.g., projection of benign vs. harmful token activations onto the top-k basis or an ablation that measures content leakage into the filtered subspace.
  4. [§3, Appendix A, D] The benchmark uses GPT-4o both to generate the probing prompts/patterns and to judge ASR, RR, and Res. Q, and SPAR hyperparameters are selected using those same judged metrics (Tables 2, 5, 6). This creates a circularity risk: part of the reported gain may reflect alignment with GPT-4o's evaluation preferences rather than generic knowledge preservation. The manual inspection of prompts and the small spot-check for binary decisions are helpful, but the paper should report inter-annotator agreement on Res. Q, or run the final LLaVA comparison with an independent judge or human evaluation.
  5. [Table 1, NPO rows] NPO collapses to 0.00 across MMVet, POPE, VizWiz, RR, and Res. Q on both models, with Res. Q 1.00. This indicates the model is effectively destroyed, so including NPO in the "all baselines uniformly suffer from severe knowledge holes" pattern inflates the systematicity claim: a fully collapsed model trivially has low Res. Q. NPO should either be diagnosed and excluded, or its catastrophic collapse should be analyzed separately. The current presentation treats it as another instance of knowledge holes, which is not substantiated.
minor comments (5)
  1. [Abstract] "SPAR recovers over 98% of vanilla response quality compared to below 50% for standard baselines" is only true on LLaVA. On Qwen, KL-Min retains 7.13/7.19 = 99% of vanilla response quality, so the "below 50%" characterization does not generalize.
  2. [§5, Existence of Knowledge Holes] The sentence "The same pattern holds on Qwen2.5-VL-3B" immediately follows a discussion of LLaVA and is immediately qualified by PO's numbers, but the KL-Min row in Table 1 contradicts the claim. Please rephrase to describe per-method, per-architecture patterns rather than a uniform phenomenon.
  3. [Appendix B, Table 4] The Qwen-specific SPAR hyperparameters (alpha=300, beta=2.0, lambda=0.05) are very different from LLaVA's (alpha=600, beta=1.0, lambda=0.5). The manuscript does not discuss why lambda must be 0.05, nor whether the Qwen failure can be attributed to this choice. A sentence explaining the selection procedure would improve reproducibility.
  4. [§4, Eq. (5)] The notation H⊙(W_r 1^T_d) is a bit compressed; W_r is defined as an L-vector, so W_r 1^T_d is L×d. Clarify that the rows of H are scaled by w^r_j.
  5. [Appendix D] The Res. Q template asks the judge to rate on a 1–10 scale, but the main text says the score is "on a scale of 1 to 10" (p. 5). Fine, but consider explicitly stating whether partial scores (e.g., 7.5) are allowed and how the three repeated queries are averaged.

Circularity Check

0 steps flagged

No significant circularity; SPAR's evaluation is aligned with its training objective but not equivalent to it.

full rationale

The paper's derivation is not circular in the technical sense. The knowledge-hole benchmark is constructed by extracting generic patterns from forget-set responses and transplanting them onto benign content, which is an operationalization of the stated definition, not a logical tautology. SPAR's AEL reinforces non-entity token prediction on entity-masked forget-set samples; although this shares a pattern source with the benchmark, SPAR is not trained on the probe prompts themselves, and the evaluation measures raw model responses via a separate GPT-4o judgment. No equation in the paper reduces the reported Res.Q to the AEL loss (Eq. 7 is a token-level cross-entropy; Res.Q is a subjective 1-10 rating on open-ended responses). Hyperparameters are tuned on the reported metrics, which is a standard fairness concern but not a circularity. The paper's 'systematic consequence' claim is internally contradicted by Table 1 (KL-Min on Qwen2.5-VL-3B shows Res.Q 7.13 vs vanilla 7.19, i.e., no severe knowledge hole), and the abstract's unqualified 0.00% ASR is contradicted by SPAR's 80.33% ASR on Qwen; however, overgeneralization and internal inconsistency are correctness/evidence concerns, not circular derivation. Self-citations to SafeEraser and the authors' prior unlearning works are normal and not load-bearing for the theoretical claims.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central results rest on five explicit domain assumptions, most importantly the spectral-separation claim that makes Anchored SVD work, and on six tuned hyperparameters. The paper introduces no new physical or conceptual entity beyond the benchmark and method artifacts. The absence of released code and the reliance on GPT-4o for both construction and judging increase the burden on these assumptions.

free parameters (6)
  • SPAR alpha (retain loss weight) = 600 (LLaVA), 300 (Qwen)
    Swept over {200,600,1800}; alpha=600 chosen because alpha=200 gives Res. Q 3.24 and alpha=1800 raises ASR to 66.67% (Appendix C).
  • SPAR beta (AFL weight) = 1.0 (LLaVA), 2.0 (Qwen)
    Swept over {0.5,1.0,2.0}; beta=0.5 collapses forgetting (ASR 73.22%) and beta=1.0 gives the best utility balance.
  • SPAR lambda (AEL weight) = 0.5 (LLaVA), 0.05 (Qwen)
    Swept over {0.1,0.5,1.0}; lambda=1.0 raises ASR to 1.00% on LLaVA, so 0.5 is chosen as stable.
  • k (number of filtered singular vectors) = 2 (both models)
    Swept over {1,2,4}; k=4 raises ASR to 0.56% on LLaVA, and on Qwen increasing k from 0 to 2 raises ASR from 18.67% to 80.33%, showing strong architecture interaction.
  • target layer for SVD/filtering = layer 8 (inherited from RMU config)
    No ablation on layer choice is reported; the layer is taken from the RMU baseline configuration.
  • entity detection rules (POS tagger plus ~150-word function-word list) = curated list and tagger
    Heuristic definition of entity tokens for AEL; no ablation or measurement of entity-detection accuracy is reported.
axioms (5)
  • domain assumption Top singular vectors of hidden states encode generic structure; content-specific information resides in trailing directions.
    Stated in Section 4 (Anchored Forget Loss); no spectral evidence is given, and the Qwen failure shows it is not universally valid.
  • domain assumption A reference model fine-tuned only on the retain set yields clean generic patterns uncontaminated by harmful content.
    Section 4: the frozen reference is the SVD anchor; if retain fine-tuning still encodes harmful patterns, the anchor is polluted.
  • domain assumption Zero-image input produces well-behaved visual encoder activations that do not conflict with training.
    Section 4 Visual-Dropout and the ablation remark that the pretrained manifold is smooth enough for zero inputs; this is asserted, not demonstrated.
  • domain assumption Probing knowledge holes by transplanting patterns from the forget set onto benign topics captures the intended form of collateral damage.
    Benchmark section: pattern extraction uses GPT-4o with manual inspection, but the mapping from extracted pattern to knowledge hole is a design choice, and the judge is also GPT-4o.
  • domain assumption GPT-4o response quality scores are a valid proxy for benign-input degradation.
    Used for Res. Q and RR; only a small spot-check (over 95% agreement for binary decisions) is reported, with no variance or calibration details.

pith-pipeline@v1.3.0-daily-deepseek · 19088 in / 13674 out tokens · 129256 ms · 2026-08-04T19:39:51.648595+00:00 · methodology

0 comments
read the original abstract

Machine unlearning offers a promising approach to remove unsafe content from Multimodal Large Language Models (MLLMs), yet ensuring the precision of unlearning remains a persistent challenge. One reason is that current MLLM unlearning evaluation paradigms suffer from a critical blind spot: they assess model utility through benchmarks whose representations are distant from the forget set, failing to capture knowledge holes---severe degradation on benign adjacent inputs. To probe knowledge holes in unlearned MLLMs, we construct a benchmark that captures unintended degradation on benign inputs sharing generic patterns with the forget set, and confirm through controlled experiments that they are a systematic consequence of commonly used approaches. Furthermore, to bridge this gap, we propose Selective Protection with Anchored Regularization, which protects generic patterns via anchored activation filtering while reinforcing them through entity-abstracted enhancement. Our experiments on SafeEraser demonstrate that SPAR recovers over 98% of vanilla response quality compared to below 50% for standard baselines---while achieving 0.00% attack success rate and competitive model utility. These results underscore the necessity of more fine-grained evaluation for trustworthy MLLM unlearning.

Figures

Figures reproduced from arXiv: 2608.01849 by Junkai Chen, Junxiang You, Ruiqi Liu, Shu Wu, Yuhao He, Zhetao Guo.

Figure 1
Figure 1. Figure 1: Illustration of knowledge holes in unlearned [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Construction of our knowledge hole probing benchmark. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Relative degradation (∆) of two unlearning meth￾ods with respect to the vanilla model. Red bars indicate the drop in Res. Q (knowledge hole severity); the others indicate the drop in Model Utility. 4 Method To mitigate knowledge holes, we propose Selective Protec￾tion with Anchored Regularization (SPAR). SPAR draws on the insight that the top singular vectors of hidden states en￾code universal patterns, wh… view at source ↗
Figure 4
Figure 4. Figure 4: Overview of SPAR. The framework introduces two key loss terms: LAFL and LAEL. AFL Pathway (top): forget-set samples are passed through a frozen reference model, whose activations undergo Anchored SVD to produce a basis Vref. The trainable model’s activations H are then filtered against Vref to remove generic patterns, yielding Hs for computing LAFL. AEL Pathway (middle): forget-set samples are processed wi… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 10 canonical work pages · 3 internal anchors

  1. [1]

    2024 , url =

    Haotian Liu and Chunyuan Li and Yuheng Li and Yong Jae Lee , title =. 2024 , url =. doi:10.1109/CVPR52733.2024.02484 , timestamp =

    arXiv 2024

  2. [2]

    Qwen2.5-VL Technical Report , journal =

    Shuai Bai and Keqin Chen and Xuejing Liu and Jialin Wang and Wenbin Ge and Sibo Song and Kai Dang and Peng Wang and Shijie Wang and Jun Tang and Humen Zhong and Yuanzhi Zhu and Ming. Qwen2.5-VL Technical Report , journal =. 2025 , url =. doi:10.48550/ARXIV.2502.13923 , eprinttype =. 2502.13923 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.13923 2025

  3. [3]

    Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench , booktitle =

    Zheyuan Liu and Guangyao Dou and Mengzhao Jia and Zhaoxuan Tan and Qingkai Zeng and Yongle Yuan and Meng Jiang , editor =. Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench , booktitle =. 2025 , url =. doi:10.18653/V1/2025.NAACL-LONG.207 , timestamp =

    doi:10.18653/v1/2025.naacl-long.207 2025

  4. [4]

    CoRR , volume =

    Zhaopan Xu and Pengfei Zhou and Weidong Tang and Jiaxin Ai and Wangbo Zhao and Xiaojiang Peng and Kai Wang and Yang You and Wenqi Shao and Hongxun Yao and Kaipeng Zhang , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2503.12545 , eprinttype =. 2503.12545 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.12545 2025

  5. [5]

    CoRR , volume =

    Ruiqi Zhang and Licong Lin and Yu Bai and Song Mei , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2404.05868 , eprinttype =. 2404.05868 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2404.05868 2024

  6. [6]

    Choquette

    Lucas Bourtoule and Varun Chandrasekaran and Christopher A. Choquette. Machine Unlearning , booktitle =. 2021 , url =. doi:10.1109/SP40001.2021.00019 , timestamp =

    arXiv 2021

  7. [7]

    Lipton and J

    Pratyush Maini and Zhili Feng and Avi Schwarzschild and Zachary C. Lipton and J. Zico Kolter , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2401.06121 , eprinttype =. 2401.06121 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2401.06121 2024

  8. [8]

    Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen

    Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen. LoRA: Low-Rank Adaptation of Large Language Models , booktitle =. 2022 , url =

    2022

  9. [9]

    Li and Ann

    Nathaniel Li and Alexander Pan and Anjali Gopal and Summer Yue and Daniel Berrios and Alice Gatti and Justin D. Li and Ann. The. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  10. [10]

    MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities , booktitle =

    Weihao Yu and Zhengyuan Yang and Linjie Li and Jianfeng Wang and Kevin Lin and Zicheng Liu and Xinchao Wang and Lijuan Wang , editor =. MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities , booktitle =. 2024 , url =

    2024

  11. [11]

    arXiv preprint arXiv:2305.10355 , year=

    Evaluating object hallucination in large vision-language models , author=. arXiv preprint arXiv:2305.10355 , year=

    Pith/arXiv arXiv

  12. [12]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Vizwiz grand challenge: Answering visual questions from blind people , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  13. [13]

    SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning , booktitle =

    Junkai Chen and Zhijie Deng and Kening Zheng and Yibo Yan and Shuliang Liu and PeiJun Wu and Peijie Jiang and Jia Liu and Xuming Hu , editor =. SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning , booktitle =. 2025 , url =. doi:10.18653/V1/2025.FINDINGS-ACL.731 , timestamp =

    work page doi:10.18653/v1/2025.findings-acl.731 2025

  14. [14]

    and Laub, A

    Klema, V. and Laub, A. , journal=. The singular value decomposition: Its computation and some applications , year=

  15. [15]

    Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection , booktitle =

    Shauli Ravfogel and Yanai Elazar and Hila Gonen and Michael Twiton and Yoav Goldberg , editor =. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection , booktitle =. 2020 , url =. doi:10.18653/V1/2020.ACL-MAIN.647 , timestamp =

    doi:10.18653/v1/2020.acl-main.647 2020

  16. [16]

    Behrens and Louis L

    Richard T. Behrens and Louis L. Scharf , title =. 1994 , url =. doi:10.1109/78.286957 , timestamp =

    work page doi:10.1109/78.286957 1994

  17. [17]

    Null Space Projection Based Adaptive Beamforming in the Presence of Array Imperfections , journal =

    Yang. Null Space Projection Based Adaptive Beamforming in the Presence of Array Imperfections , journal =. 2009 , url =. doi:10.1587/TRANSCOM.E92.B.2762 , timestamp =

    work page doi:10.1587/transcom.e92.b.2762 2009

  18. [18]

    2025 , eprint=

    Erasing CLIP Memories: Non-Destructive, Data-Free Zero-Shot class Unlearning in CLIP Models , author=. 2025 , eprint=

    2025

  19. [19]

    CoRR , volume =

    Ashish Mishra and Gyanaranjan Nayak and Tarun Kumar and Arpit Shah and Suparna Bhattacharya and Martin Foltin , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.14113 , eprinttype =. 2512.14113 , timestamp =

    work page doi:10.48550/arxiv.2512.14113 2025

  20. [20]

    Fengli Wu and Vaidehi Patil and Jaehong Yoon and Yue Zhang and Mohit Bansal , title =. Trans. Mach. Learn. Res. , volume =. 2026 , url =

    2026

  21. [21]

    Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning

    Yuhang Wang and Zhenxing Niu and Haoxuan Ji and Guangyu He and Linlin Zhang and Haichang Gao , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.05909 , eprinttype =. 2605.05909 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.05909 2026

  22. [22]

    ArXiv , year=

    Machine Unlearning via Null Space Calibration , author=. ArXiv , year=

  23. [23]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Oplora: Orthogonal projection lora prevents catastrophic forgetting during parameter-efficient fine-tuning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  24. [24]

    CoRR , volume =

    OpenAI , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2410.21276 , eprinttype =. 2410.21276 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.21276 2024

  25. [25]

    2025 , url =

    Na Li and Chunyi Zhou and Yansong Gao and Hui Chen and Zhi Zhang and Boyu Kuang and Anmin Fu , title =. 2025 , url =. doi:10.1109/TNNLS.2025.3530988 , timestamp =

    arXiv 2025

  26. [26]

    Not All Tokens Are Meant to Be Forgotten , booktitle =

    Xiangyu Zhou and Yao Qiang and Saleh Zare Zade and Douglas Zytko and Prashant Khanduri and Dongxiao Zhu , editor =. Not All Tokens Are Meant to Be Forgotten , booktitle =. 2026 , url =. doi:10.1609/AAAI.V40I44.41156 , timestamp =

    work page doi:10.1609/aaai.v40i44.41156 2026

  27. [27]

    Findings of the Association for Computational Linguistics,

    Hao Zheng and Zirui Pang and Ling Li and Zhijie Deng and Yuhan Pu and Zhaowei Zhu and Xiaobo Xia and Jiaheng Wei , editor =. Findings of the Association for Computational Linguistics,. 2026 , url =

    2026

  28. [28]

    CoRR , volume =

    Chenlu Ding and Jiancan Wu and Leheng Sheng and Fan Zhang and Yancheng Yuan and Xiang Wang and Xiangnan He , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2510.04217 , eprinttype =. 2510.04217 , timestamp =

    doi:10.48550/arxiv.2510.04217 2025

  29. [29]

    CoRR , volume =

    Hongji Li and Junchi Yao and Manjiang Yu and Priyanka Singh and Xue Li and Di Wang and Lijie Hu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.17911 , eprinttype =. 2512.17911 , timestamp =

    work page doi:10.48550/arxiv.2512.17911 2025

  30. [30]

    MultiDelete for Multimodal Machine Unlearning , booktitle =

    Jiali Cheng and Hadi Amiri , editor =. MultiDelete for Multimodal Machine Unlearning , booktitle =. 2024 , url =. doi:10.1007/978-3-031-72940-9\_10 , timestamp =

    doi:10.1007/978-3-031-72940-9 2024

  31. [31]

    CoRR , volume =

    Myeongseob Ko and Hoang Anh Just and Charles Fleming and Ming Jin and Ruoxi Jia , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2511.00030 , eprinttype =. 2511.00030 , timestamp =

    work page doi:10.48550/arxiv.2511.00030 2025

  32. [32]

    Large Language Model Unlearning , booktitle =

    Yuanshun Yao and Xiaojun Xu and Yang Liu , editor =. Large Language Model Unlearning , booktitle =. 2024 , url =

    2024

  33. [33]

    Proceedings of the 32nd ACM International Conference on Multimedia , pages=

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models , author=. Proceedings of the 32nd ACM International Conference on Multimedia , pages=

  34. [34]

    Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering , booktitle =

    Pan Lu and Swaroop Mishra and Tanglin Xia and Liang Qiu and Kai. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering , booktitle =. 2022 , url =

    2022

  35. [35]

    The Thirteenth International Conference on Learning Representations,

    Yingzi Ma and Jiongxiao Wang and Fei Wang and Siyuan Ma and Jiazhao Li and Jinsheng Pan and Xiujun Li and Furong Huang and Lichao Sun and Bo Li and Yejin Choi and Muhao Chen and Chaowei Xiao , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  36. [36]

    2026 , eprint=

    Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Models , author=. 2026 , eprint=

    2026

  37. [37]

    CoRR , volume =

    Ronen Eldan and Mark Russinovich , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2310.02238 , eprinttype =. 2310.02238 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.02238 2023

  38. [38]

    MM-SafetyBench:

    Xin Liu and Yichen Zhu and Jindong Gu and Yunshi Lan and Chao Yang and Yu Qiao , editor =. MM-SafetyBench:. Computer Vision -. 2024 , url =. doi:10.1007/978-3-031-72992-8\_22 , timestamp =

    doi:10.1007/978-3-031-72992-8 2024

  39. [39]

    From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Models

    Dunyuan Xu and Xikai Yang and Yaoqian Li and Jinpeng Li and Pheng. From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Models , journal =. 2025 , url =. doi:10.48550/ARXIV.2508.04192 , eprinttype =. 2508.04192 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2508.04192 2025

  40. [40]

    Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models , booktitle =

    Xin Zou and Yizhou Wang and Yibo Yan and Yuanhuiyi Lyu and Kening Zheng and Sirui Huang and Junkai Chen and Peijie Jiang and Jia Liu and Chang Tang and Xuming Hu , editor =. Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models , booktitle =. 2025 , url =

    2025

  41. [41]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,

    Shangyu Xing and Fei Zhao and Zhen Wu and Tuo An and Weihao Chen and Chunhui Li and Jianbing Zhang and Xinyu Dai , editor =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,. 2024 , url =. doi:10.18653/V1/2024.EMNLP-MAIN.67 , timestamp =

    doi:10.18653/v1/2024.emnlp-main.67 2024

  42. [42]

    Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models , booktitle =

    Jiaqi Li and Qianshan Wei and Chuanyi Zhang and Guilin Qi and Miaozeng Du and Yongrui Chen and Sheng Bi and Fan Liu , editor =. Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models , booktitle =. 2024 , url =

    2024

  43. [43]

    MultiDelete for Multimodal Machine Unlearning

    Jiali Cheng and Hadi Amiri , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2311.12047 , eprinttype =. 2311.12047 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2311.12047 2023

  44. [44]

    Manning and Stefano Ermon and Chelsea Finn , editor =

    Rafael Rafailov and Archit Sharma and Eric Mitchell and Christopher D. Manning and Stefano Ermon and Chelsea Finn , editor =. Direct Preference Optimization: Your Language Model is Secretly a Reward Model , booktitle =. 2023 , url =

    2023

  45. [45]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization=

    2025

  46. [46]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Reefknot: A comprehensive benchmark for relation hallucination evaluation, analysis and mitigation in multimodal large language models , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  47. [47]

    arXiv preprint arXiv:2606.00105 , year=

    Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning , author=. arXiv preprint arXiv:2606.00105 , year=

    Pith/arXiv arXiv