Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Targeted Forgetting of Image Subgroups in CLIP Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that CLIP can selectively forget a specific image subgroup—such as one airplane manufacturer or one character design—without access to the pre-training data, while retaining strong zero-shot performance on other…

desk verdict A useful new problem and a promising three-stage method, but the central layer-selection mechanism is never validated in isolation and the Fisher definition has a formal ambiguity. read the letter →

arxiv 2506.03117 v1 pith:Y7ZJQM3R submitted 2025-06-03 cs.CV

Zeliang Zhang , Gaowen Liu , Charles Fleming , Ramana Rao Kompella , Chenliang Xu This is my paper
classification cs.CV
keywords machineunlearningCLIPsubgroupforgettingzero-shotlearningFisherinformationLoRAmodelmergingvision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foundation models like CLIP inherit unwanted knowledge from noisy internet training data, but pruning a single subgroup of images—say, Mickey Mouse or Boeing airplanes—while leaving the rest of the class and the model's zero-shot abilities intact has been an open problem. This paper claims a three-stage recipe solves it without access to the pre-training data: forget by fine-tuning selected layers on the target subgroup, remind by fine-tuning on a distribution-aligned retaining set with a moving average of parameters, and restore zero-shot performance by merging the unlearned model with the original. The key insight is to use a relative Fisher information ratio, comparing the layer sensitivity on the forgetting set against the retaining set, to choose which layers to modify with low-rank adapters. Experiments on ImageNet subgroups, CIFAR-10 classes, and generated image styles report that the target subgroup is erased while substantial zero-shot performance on unseen datasets is preserved, in contrast to baseline unlearning methods that collapse under this setting.

What carries the argument

The load-bearing object is the relative Fisher information ratio of Eq. (2): for each layer $\ell$, the expected second derivative of the image-text cosine similarity with respect to that layer's parameters, evaluated on the forgetting set, divided by the same quantity on the retaining set. A high ratio marks a layer that matters more for the target subgroup than for the similar subgroup, so LoRA adapters applied only to the top-ranked layers are supposed to erase the target while leaving the model's global behaviour intact. Around this selection sit the forgetting loss (Eq. 3), which minimizes the image-text similarity of forgetting images against their coarse label; the distribution-alignment perturbation of Eq. (4) plus the EMA update of Eq. (5) in the reminding stage; and the calibration-set-optimized model soup of Eq. (6) in the restore stage.

What would settle it

Run the method on a superclass with four subgroups where one target subgroup's images are visually near-identical to another retained subgroup (for example, two breeds of dogs); if the relative-Fisher-selected layer fine-tuning drives the retained subgroup's accuracy below the original on that subgroup while also erasing the target, the layer-selection heuristic is over-forgetting rather than precisely targeted. The decisive test is whether the method's retain accuracy on the hardest similar subgroup stays within a few points of the original when the target is fully erased.

Watch

Extended reading notes

Core claim

On the paper's terms, the central discovery is that fine-grained subgroup unlearning in CLIP can be achieved by three coordinated moves. First, compute, for each layer of the image encoder, the ratio of the second-order sensitivity of the CLIP contrastive loss on the forgetting set to that on a manually constructed retaining set of similar images; this relative Fisher information (Eq. 2) ranks the layers whose LoRA fine-tuning on the forgetting set (Eq. 3) erases the target subgroup while sparing similar subgroups. Second, before the reminding fine-tuning, optimize small per-image perturbations so that the retaining batch's pre-BatchNorm statistics match the stored BN statistics of the original CLIP (Eq. 4), then fine-tune with an exponential moving average of parameters (Eq. 5) to recover knowledge at risk of over-forgetting. Third, merge the unlearned model with the original by a convex combination whose coefficient is chosen on a small calibration set (Eq. 6), restoring zero-shot accuracy. The paper supports this with retrieval and classification experiments showing the target subgroup is no longer retrieved, while other subgroups, other ImageNet classes, and datasets like Food, STL, and ObjectNet retain most of their original accuracy.

Load-bearing premise

The method assumes the relative Fisher information ratio, a second-order sensitivity measure comparing the forgetting set against the retaining set, identifies exactly the layers whose low-rank fine-tuning erases only the target subgroup's knowledge. Nothing in the paper proves this ranking is correct, so if it is wrong the method will either leave target knowledge in place or damage unrelated knowledge.

Editorial extensions

If this is right

  • Targeted erasure of specific visual subgroups (e.g., a trademarked character, a person's likeness, a protected attribute) becomes possible without retraining CLIP or touching its billion-scale pre-training data.
  • Because the forgetting, reminding, and restoring stages only need a small forgetting set and a small similar retaining set, the approach is practical for deployment-scale cleanups of foundation models.
  • The model-merging step composes: models that forget different classes can be merged into a single model that forgets all of them, as the paper demonstrates for ship, airplane, and cat on CIFAR-10.
  • If the method generalizes, unlearning becomes a post-hoc, data-efficient operation rather than a retraining burden, changing how content-removal requests can be honored for deployed vision-language models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The layer-selection logic is not validated against an independent measure of what a layer stores, so a natural next test is to compare relative Fisher selection against causal intervention probes that measure which layers actually control the target subgroup's embeddings.
  • The method leaves open whether the erased knowledge is truly gone or merely suppressed: one test is whether few-shot fine-tuning on the forgetting subgroup quickly reintroduces the target behaviour, which would imply the unlearning is shallow.
  • A similar three-stage recipe—select, perturb, merge—might transfer to other multimodal encoders whose BatchNorm or normalization statistics encode training-domain information, though the paper only studies CLIP.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a three-stage method for targeted subgroup forgetting in CLIP image encoders without access to the pretraining corpus. Stage one computes a per-layer relative Fisher information ratio (Eq. 2) between the forgetting set and a manually constructed retaining set, selects high-ratio layers, and applies LoRA fine-tuning to minimize cosine similarity on the forgetting set. Stage two aligns the retaining set's batch statistics to the model's stored BatchNorm statistics and fine-tunes with EMA (Eqs. 4-5). Stage three merges the unlearned model with the original model by a coefficient α tuned on a calibration set (Eq. 6). Experiments on ImageNet-Breeds, CIFAR-10, and style subsets with RN50/RN101 report target forgetting, retain accuracy, and zero-shot accuracy on Food, STL, ObjectNet, and retrieval results. The stated claim is that the method forgets specified subgroups while preserving strong zero-shot performance and outperforms six baselines.

Significance. The setting is timely and practical: fine-grained subgroup forgetting in CLIP without pretraining data addresses a real gap, since most baselines either need pretraining access or forget whole classes. The paper includes a broad baseline comparison, two backbones, multiple forgetting scenarios, retrieval-based checks, and ablations of the reminding and restoring stages. If the specialized layer selection is truly necessary, the contribution is significant for CLIP unlearning. However, the central mechanism is not isolated by any ablation, and several methodological ambiguities (formal definition of Eq. 2, missing hyperparameters, selection-evaluation overlap) currently prevent me from endorsing the central claim as established.

major comments (4)
  1. [Section 4.1, Eq. (2)] The relative Fisher information ratio is written as a Hessian of the unnormalized dot product g_img(x)·g_txt(t) with respect to layer parameters. For a ReLU-based encoder this quantity is zero almost everywhere unless the L2 normalization used in CLIP embeddings is explicitly part of the differentiated function; the paper does not state what is actually computed. In addition, no derivation or validation is provided for the claim that layers with high relative Fisher information are the ones whose LoRA adaptation erases only the target subgroup. Please specify the exact differentiable objective (including normalization), state whether an empirical Fisher (squared-gradient) surrogate is used, and justify the ranking.
  2. [Section 5.5] The central contribution of the method is the layer selection in Section 4.1, yet Section 5.5 ablates only the number of reminding steps and the merging coefficient α. There is no experiment with a random layer ranking, with all layers adapted, or with an inverse ranking. Without such an ablation, the reported results cannot distinguish whether relative Fisher information is responsible for the targeted forgetting or whether the reminding and restoring stages alone repair any damage from LoRA fine-tuning. This is a load-bearing gap; please add these ablations.
  3. [Section 5.1 and Appendix A.1] For the proposed method the paper never reports the hyperparameters used in the final experiments: LoRA rank, the number/identity of selected layers, the learning rates for the forgetting and reminding stages, the EMA decay α in Eq. (5), the number of reminding steps, and the batch sizes. Table 4 varies reminding steps but the final setting is not identified, and Table 5 only reports the merging α for one run. Without these details the experiments are not reproducible and the comparison to baselines is difficult to assess. Please provide a complete configuration table.
  4. [Section 5.5, Table 5, and Eq. (6)] The merging coefficient α is selected by maximizing accuracy on a calibration set D_m that 'may include portions of the retain dataset not used in fine-tuning,' and the reported retain accuracy Accr is measured on the retain set. If D_m overlaps with the evaluation retain set, the reported retain accuracy is optimistically biased because the same data is used for selection and evaluation. Please report the exact split and use a held-out portion of the retain set (or cross-validation) for all reported Accr values.
minor comments (5)
  1. [Table 2] The RN50 'airplane' row contains the malformed entry '50.183,888.9'; please correct the formatting.
  2. [Section 5.1] The phrase 'EMMN [12],and CLIP-LIP' contains a missing space after the comma.
  3. [Eq. (6)] The arg max is written over θ, but θ is fully determined by α; the optimization variable should be α.
  4. [Section 4.2] The distribution-alignment step relies on BatchNorm statistics (Eq. 4); this is only meaningful for CNN-based CLIP backbones such as RN50/RN101. Please state explicitly that the method in its current form does not directly extend to ViT-based CLIP models, or adapt the alignment step.
  5. [Section 5.5] The terms 'reminding' and 'retaining' are used interchangeably for the same stage (e.g., the heading says 'reminding' while the text says 'retaining'); unify the terminology.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the three-stage pipeline is an empirical construction, and the central zero-shot results are supported by independent out-of-domain evaluations.

full rationale

The paper does not present a mathematical derivation from first principles; it proposes an empirical three-stage procedure. The relative Fisher information in Eq. (2) is a layer-selection heuristic, not a fitted parameter disguised as a prediction. The forgetting stage optimizes the target subgroup's similarity directly, so target accuracy dropping is the intended optimization objective rather than a predicted outcome. The restoring stage in Eq. (6) selects the merging coefficient on a calibration set that may overlap the retain set, and retain accuracy is indeed reported on retain-derived data; however, the paper's central claim of preserved zero-shot capability is also validated on unseen datasets (Food, STL, ObjectNet, and ImageNet) that are not used in any training or calibration stage, so the result does not reduce to the fitted inputs. The self-citations (references [31], [63], [64]) appear only as motivational examples of CLIP biases and are not load-bearing for the method. The potential technical issue that the Hessian in Eq. (2) may be zero almost everywhere for a ReLU-based encoder is a correctness or implementation ambiguity, not circularity. Overall, no claim in the paper is equivalent by construction to its inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several unstated hyperparameters (alpha, steps, LoRA config) and on three domain assumptions about the validity of relative Fisher layer selection, BN statistics as a distribution proxy, and model merging. No new physical or architectural entities are introduced.

free parameters (5)
  • model_merging_alpha = not stated per experiment; ablation uses 0.65
    Interpolation coefficient between unlearned and original CLIP parameters (Eq. 6), chosen by maximizing accuracy on a calibration set drawn from the retain set. The main experimental value is not reported.
  • reminding_steps = ablated: 5,10,20,30,40,50,100; main value not stated
    Number of fine-tuning steps in the reminding stage; Table 4 shows a sharp trade-off between retain accuracy and zero-shot capacity. The value used in main tables is not given.
  • EMA_decay = not stated
    Decay factor alpha in Eq. (5) for exponential moving average of parameters during reminding; its value is not reported.
  • LoRA_configuration = not stated
    Rank, learning rate, and the set of selected layers for LoRA in the forgetting stage are not disclosed.
  • learning_rates = not stated
    Learning rates for forgetting and reminding fine-tuning stages are not given (baselines use 1e-6).
assumptions (4)
  • ad hoc to paper Relative Fisher information ratio (Eq. 2) selects layers whose LoRA fine-tuning removes only the target subgroup's knowledge.
    Central heuristic for the forgetting stage; no derivation or independent validation is provided.
  • domain assumption CLIP's batch-normalization running statistics encode the global distribution of the pre-training dataset.
    Used in Eq. (4) to align the retaining set distribution with the pretraining distribution.
  • domain assumption Linear interpolation of model weights (model soups) preserves accuracy and restores zero-shot performance.
    Borrowed from Wortsman et al. [56] and used in Eq. (6) without further justification.
  • domain assumption Zero-shot classification accuracy on the target subgroup is an adequate proxy for whether knowledge has been forgotten.
    All evaluations rely on classification accuracy ratios; no membership-inference or feature-level test is performed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Targeted Forgetting of Image Subgroups in CLIP Models." pith.science (2026). https://pith.science/paper/Y7ZJQM3R

@misc{pith2026250603117,
  author       = {Pith},
  title        = {Pith review of: Targeted Forgetting of Image Subgroups in CLIP Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7ZJQM3R}},
  note         = {Machine review of arXiv:2506.03117}
}
read the original abstract

Foundation models (FMs) such as CLIP have demonstrated impressive zero-shot performance across various tasks by leveraging large-scale, unsupervised pre-training. However, they often inherit harmful or unwanted knowledge from noisy internet-sourced datasets, compromising their reliability in real-world applications. Existing model unlearning methods either rely on access to pre-trained datasets or focus on coarse-grained unlearning (e.g., entire classes), leaving a critical gap for fine-grained unlearning. In this paper, we address the challenging scenario of selectively forgetting specific portions of knowledge within a class, without access to pre-trained data, while preserving the model's overall performance. We propose a novel three-stage approach that progressively unlearns targeted knowledge while mitigating over-forgetting. It consists of (1) a forgetting stage to fine-tune the CLIP on samples to be forgotten, (2) a reminding stage to restore performance on retained samples, and (3) a restoring stage to recover zero-shot capabilities using model souping. Additionally, we introduce knowledge distillation to handle the distribution disparity between forgetting, retaining samples, and unseen pre-trained data. Extensive experiments on CIFAR-10, ImageNet-1K, and style datasets demonstrate that our approach effectively unlearns specific subgroups while maintaining strong zero-shot performance on semantically similar subgroups and other categories, significantly outperforming baseline unlearning methods, which lose effectiveness under the CLIP unlearning setting.

Figures

Figures reproduced from arXiv: 2506.03117 by the authors.

Figure 1
Figure 1. In the task of CLIP unlearning, classical machine unlearn [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our method to unlearn the CLIP on a subgroup of images. We first compute the relative fisher information on the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Retrieval results on the ImageNet-1K dataset before and after the unlearning process in the CLIP model. We highlight the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Retrieval results of the CLIP pre- and post-unlearning process. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Examples of stylized images of the sports car in the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 47 canonical work pages

  1. [1]

    Which pretrain samples to rehearse when finetuning pretrained models?arXiv preprint arXiv:2402.08096, 2024

    Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh, and Ankur Taly. Which pretrain samples to rehearse when finetuning pretrained models?arXiv preprint arXiv:2402.08096, 2024. 2

    arXiv 2024

  2. [2]

    Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models.Advances in neural information processing systems, 32, 2019

    Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models.Advances in neural information processing systems, 32, 2019. 5

    work page 2019

  3. [3]

    Evaluating ma- chine unlearning via epistemic uncertainty.arXiv preprint arXiv:2208.10836, 2022

    Alexander Becker and Thomas Liebig. Evaluating ma- chine unlearning via epistemic uncertainty.arXiv preprint arXiv:2208.10836, 2022. 2

    arXiv 2022

  4. [4]

    Un- supervised feature learning and deep learning: A review and new perspectives.CoRR, abs/1206.5538, 1(2665):2012, 2012

    Yoshua Bengio, Aaron C Courville, and Pascal Vincent. Un- supervised feature learning and deep learning: A review and new perspectives.CoRR, abs/1206.5538, 1(2665):2012, 2012. 1

    arXiv 2012

  5. [5]

    Into the laion’s den: Investigating hate in multi- modal datasets.Advances in Neural Information Processing Systems, 36, 2024

    Abeba Birhane, Sanghyun Han, Vishnu Boddeti, Sasha Luc- cioni, et al. Into the laion’s den: Investigating hate in multi- modal datasets.Advances in Neural Information Processing Systems, 36, 2024. 2

    work page 2024

  6. [6]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13, pages 446–461. Springer, 2014. 5

    2014

  7. [7]

    Machine unlearning

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141–159. IEEE, 2021. 1

    work page 2021

  8. [8]

    Single layer single gradient unlearning.arXiv preprint arXiv:2407.11867,

    Zikui Cai, Yaoteng Tan, and M Salman Asif. Single layer single gradient unlearning.arXiv preprint arXiv:2407.11867,

    arXiv

Show all 64 references
  1. [9]

    On catastrophic inheritance of large foundation models.arXiv preprint arXiv:2402.01909, 2024

    Hao Chen, Bhiksha Raj, Xing Xie, and Jindong Wang. On catastrophic inheritance of large foundation models.arXiv preprint arXiv:2402.01909, 2024. 1

    2024 arXiv

  2. [10]

    Data-efficient language-supervised zero- shot learning with self-distillation

    Ruizhe Cheng, Bichen Wu, Peizhao Zhang, Peter Vajda, and Joseph E Gonzalez. Data-efficient language-supervised zero- shot learning with self-distillation. InCVPR, pages 3119– 3124, 2021. 1

    2021

  3. [11]

    Efficient model updates for approximate unlearning of graph-structured data

    Eli Chien, Chao Pan, and Olgica Milenkovic. Efficient model updates for approximate unlearning of graph-structured data. InThe Eleventh International Conference on Learning Repre- sentations, 2022. 1

    2022

  4. [12]

    Zero-shot machine unlearning.IEEE Transactions on Information Forensics and Security, 18:2345– 2354, 2023

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Zero-shot machine unlearning.IEEE Transactions on Information Forensics and Security, 18:2345– 2354, 2023. 2, 5

    2023

  5. [13]

    Style injection in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injection in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8795–8805, 2024. 5

    2024

  6. [14]

    Machine unlearning: fisher infomation matrix and selective forgetting in deep networks

    Lluc Clavera Comas. Machine unlearning: fisher infomation matrix and selective forgetting in deep networks. B.S. thesis, Universitat Polit`ecnica de Catalunya, 2024. 3

    2024

  7. [15]

    An analysis of single-layer networks in unsupervised feature learning

    Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011. 5

    2011

  8. [16]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5

    2009

  9. [17]

    Who’s harry pot- ter? approximate unlearning in llms.arXiv preprint arXiv:2310.02238, 2023

    Ronen Eldan and Mark Russinovich. Who’s harry pot- ter? approximate unlearning in llms.arXiv preprint arXiv:2310.02238, 2023. 2

    2023 arXiv

  10. [18]

    Zero-shot machine unlearn- ing at scale via lipschitz regularization.arXiv preprint arXiv:2402.01401, 2024

    Jack Foster, Kyle Fogarty, Stefan Schoepf, Cengiz ¨Oztireli, and Alexandra Brintrup. Zero-shot machine unlearn- ing at scale via lipschitz regularization.arXiv preprint arXiv:2402.01401, 2024. 2, 5

    2024 arXiv

  11. [19]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9304– 9312, 2020. 2

    2020

  12. [20]

    Bayesian variational federated learning and unlearning in decentral- ized networks

    Jinu Gong, Osvaldo Simeone, and Joonhyuk Kang. Bayesian variational federated learning and unlearning in decentral- ized networks. In2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), pages 216–220. IEEE, 2021. 1

    2021

  13. [21]

    Domain watermark: Effective and harmless dataset copyright protection is closed at hand

    Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effective and harmless dataset copyright protection is closed at hand. NeurIPS, 36, 2024. 1

    2024

  14. [22]

    Calip: Zero-shot enhancement of clip with parameter-free attention

    Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xu- peng Miao, Xuming He, and Bin Cui. Calip: Zero-shot enhancement of clip with parameter-free attention. InAAAI, pages 746–754, 2023. 1

    2023

  15. [23]

    Adaptive machine un- learning.NeurIPS, 34:16319–16330, 2021

    Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Chris Waites. Adaptive machine un- learning.NeurIPS, 34:16319–16330, 2021. 1

    2021

  16. [24]

    Researchers found child abuse material in the largest ai image generation dataset, 2024

    Kris Holt. Researchers found child abuse material in the largest ai image generation dataset, 2024. Accessed: 2024- 10-25. 3

    2024

  17. [25]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 4

    2021 arXiv

  18. [26]

    Learning to unlearn for robust machine unlearning.arXiv preprint arXiv:2407.10494, 2024

    Mark He Huang, Lin Geng Foo, and Jun Liu. Learning to unlearn for robust machine unlearning.arXiv preprint arXiv:2407.10494, 2024. 2

    2024 arXiv

  19. [27]

    Unified gradient-based machine unlearning with remain geometry enhancement.arXiv preprint arXiv:2409.19732, 2024

    Zhehao Huang, Xinwen Cheng, JingHao Zheng, Haoran Wang, Zhengbao He, Tao Li, and Xiaolin Huang. Unified gradient-based machine unlearning with remain geometry enhancement.arXiv preprint arXiv:2409.19732, 2024. 1

    2024 arXiv

  20. [28]

    Exponential moving average versus moving exponential average.Mathematische Semesterberichte, 58: 97–107, 2011

    Frank Klinker. Exponential moving average versus moving exponential average.Mathematische Semesterberichte, 58: 97–107, 2011. 5

    2011

  21. [29]

    Namboodiri

    Alexey Kravets and Vinay P. Namboodiri. Zero-shot CLIP class forgetting via text-image space adaptation.Transactions on Machine Learning Research, 2025. 2, 3, 5

    2025

  22. [30]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5

    2009

  23. [31]

    A whac-a-mole dilemma: Shortcuts come in multi- ples where mitigating one amplifies others

    Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim. A whac-a-mole dilemma: Shortcuts come in multi- ples where mitigating one amplifies others. InCVPR, pages 20071–20082, 2023. 1

    2023

  24. [32]

    Model spar- sity can simplify machine unlearning.Advances in Neural Information Processing Systems, 36, 2024

    Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, PRANAY SHARMA, Sijia Liu, et al. Model spar- sity can simplify machine unlearning.Advances in Neural Information Processing Systems, 36, 2024. 1, 2, 5

    2024

  25. [33]

    Fisher information-based efficient curriculum federated learning with large language models.arXiv preprint arXiv:2410.00131, 2024

    Ji Liu, Jiaxiang Ren, Ruoming Jin, Zijie Zhang, Yang Zhou, Patrick Valduriez, and Dejing Dou. Fisher information-based efficient curriculum federated learning with large language models.arXiv preprint arXiv:2410.00131, 2024. 3

    2024 arXiv

  26. [34]

    Unlearning with fisher masking.arXiv preprint arXiv:2310.05331, 2023

    Yufang Liu, Changzhi Sun, Yuanbin Wu, and Aimin Zhou. Unlearning with fisher masking.arXiv preprint arXiv:2310.05331, 2023. 3

    2023 arXiv

  27. [35]

    Improved fine-tuning by better leveraging pre-training data.Advances in Neural Information Processing Systems, 35:32568–32581, 2022

    Ziquan Liu, Yi Xu, Yuanhong Xu, Qi Qian, Hao Li, Xi- angyang Ji, Antoni Chan, and Rong Jin. Improved fine-tuning by better leveraging pre-training data.Advances in Neural Information Processing Systems, 35:32568–32581, 2022. 2

    2022

  28. [36]

    A tutorial on fisher information.Journal of Mathematical Psychology, 80:40–55, 2017

    Alexander Ly, Maarten Marsman, Josine Verhagen, Raoul PPP Grasman, and Eric-Jan Wagenmakers. A tutorial on fisher information.Journal of Mathematical Psychology, 80:40–55, 2017. 3

    2017

  29. [37]

    Deep unlearning via randomized conditionally independent hessians

    Ronak Mehta, Sourav Pal, Vikas Singh, and Sathya N Ravi. Deep unlearning via randomized conditionally independent hessians. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10422– 10431, 2022. 1

    2022

  30. [38]

    Variational bayesian unlearning.Advances in Neural Information Processing Systems, 33:16025–16036, 2020

    Quoc Phong Nguyen, Bryan Kian Hsiang Low, and Patrick Jaillet. Variational bayesian unlearning.Advances in Neural Information Processing Systems, 33:16025–16036, 2020. 1

    2020

  31. [39]

    Continual lifelong learning with neural networks: A review.Neural networks, 113:54–71,

    German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review.Neural networks, 113:54–71,

  32. [40]

    Safe-clip: Re- moving nsfw concepts from vision-and-language models

    Samuele Poppi, Tobia Poppi, Federico Cocchi, Marcella Cor- nia, Lorenzo Baraldi, Rita Cucchiara, et al. Safe-clip: Re- moving nsfw concepts from vision-and-language models. In Proceedings of the European Conference on Computer Vision,

  33. [41]

    What to pre-train on? efficient intermediate task selection.arXiv preprint arXiv:2104.08247, 2021

    Clifton Poth, Jonas Pfeiffer, Andreas R ¨uckl´e, and Iryna Gurevych. What to pre-train on? efficient intermediate task selection.arXiv preprint arXiv:2104.08247, 2021. 2

    2021 arXiv

  34. [42]

    Does training ai violate copyright law?Berke- ley Tech

    Jenny Quang. Does training ai violate copyright law?Berke- ley Tech. LJ, 36:1407, 2021. 1

    2021

  35. [43]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  36. [44]

    Clip for all things zero-shot sketch-based image retrieval, fine-grained or not

    Aneeshan Sain, Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Clip for all things zero-shot sketch-based image retrieval, fine-grained or not. InCVPR, pages 2765–2775, 2023. 1

    2023

  37. [45]

    Breeds: Benchmarks for subpopulation shift.arXiv preprint arXiv:2008.04859, 2020

    Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopulation shift.arXiv preprint arXiv:2008.04859, 2020. 5

    2008 arXiv

  38. [46]

    Forget me now: Fast and exact unlearning in neighborhood-based recommendation

    Sebastian Schelter, Mozhdeh Ariannezhad, and Maarten de Rijke. Forget me now: Fast and exact unlearning in neighborhood-based recommendation. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2011–2015,

    2011

  39. [47]

    Laion-5b: An open large-scale dataset for training next gener- ation image-text models.NeurIPS, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next gener- ation image-text models.NeurIPS, 35:25278–25294, ...

    2022

  40. [48]

    Remember what you want to for- get: Algorithms for machine unlearning.Advances in Neural Information Processing Systems, 34:18075–18086, 2021

    Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to for- get: Algorithms for machine unlearning.Advances in Neural Information Processing Systems, 34:18075–18086, 2021. 1

    2021

  41. [49]

    The boy who survived: Removing harry potter from an llm is harder than reported.arXiv preprint arXiv:2403.12082, 2024

    Adam Shostack. The boy who survived: Removing harry potter from an llm is harder than reported.arXiv preprint arXiv:2403.12082, 2024. 2

    2024 arXiv

  42. [50]

    Towards foundation models for scientific ma- chine learning: Characterizing scaling and transfer behavior

    Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific ma- chine learning: Characterizing scaling and transfer behavior. NeurIPS, 36, 2024. 1

    2024

  43. [51]

    Fast yet effective machine unlearning

    Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Sys- tems, 2023. 2

    2023

  44. [52]

    Unrolling sgd: Understanding factors in- fluencing machine unlearning

    Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Understanding factors in- fluencing machine unlearning. In2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 303–

  45. [53]

    Machine unlearning via algorithmic stability

    Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. Machine unlearning via algorithmic stability. InConference on Learning Theory, pages 4126–4142. PMLR,

  46. [54]

    Machine unlearning of features and labels

    Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. In30th Annual Network and Distributed System Security Sym- posium, NDSS 2023, San Diego, California, USA, February 27 - March 3, 2023. The Internet Society, 2023. 2, 5

    2023

  47. [55]

    Im- proving clip fine-tuning performance

    Yixuan Wei, Han Hu, Zhenda Xie, Ze Liu, Zheng Zhang, Yue Cao, Jianmin Bao, Dong Chen, and Baining Guo. Im- proving clip fine-tuning performance. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5439–5449, 2023. 2

    2023

  48. [56]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing infer- ence time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Re- becca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Ko- rnblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing ...

    2022

  49. [57]

    One-shot machine unlearning with mnemonic code

    Tomoya Yamashita, Masanori Yamada, and Takashi Shibata. One-shot machine unlearning with mnemonic code. InThe 16th Asian Conference on Machine Learning (Conference Track). 3

  50. [58]

    Arcane: An efficient architecture for exact machine unlearning

    Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. InIJCAI, page 19, 2022. 1

    2022

  51. [59]

    Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hier- archy

    Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hier- archy. InProceedings of the 2020 conference on fairness, accountability, and transparency, pages ...

    2020

  52. [60]

    Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective.NeurIPS, 36, 2024

    Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective.NeurIPS, 36, 2024. 4

    2024

  53. [61]

    Towards certified unlearning for deep neural networks.arXiv preprint arXiv:2408.00920, 2024

    Binchi Zhang, Yushun Dong, Tianhao Wang, and Jundong Li. Towards certified unlearning for deep neural networks.arXiv preprint arXiv:2408.00920, 2024. 1

    2024 arXiv

  54. [62]

    Graph unlearning with efficient partial re- training

    Jiahao Zhang. Graph unlearning with efficient partial re- training. InCompanion Proceedings of the ACM on Web Conference 2024, pages 1218–1221, 2024. 1

    2024

  55. [63]

    Discover and mitigate multiple biased subgroups in image classifiers

    Zeliang Zhang, Mingqian Feng, Zhiheng Li, and Chenliang Xu. Discover and mitigate multiple biased subgroups in image classifiers. InCVPR, 2024. 1

    2024

  56. [64]

    Can clip count stars? an empirical study on quantity bias in clip

    Zeliang Zhang, Zhuo Liu, Mingqian Feng, and Chenliang Xu. Can clip count stars? an empirical study on quantity bias in clip. InEMNLP (Findings), 2024. 1 Appendix A. Experiment details A.1. Implementations In our work, we implement the baseline methods under the setting of CLIP...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.