Adversarial Mixup Unlearning

Yixuan Tang; Yi Yang; Zhuoyi Peng

REVIEW 3 major objections 6 minor 14 references

Adversarial Mixup Unlearning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MixUnlearn makes unlearning mimic retraining by regularizing on adversarial mixup samples.

desk verdict Solid empirical unlearning paper with a real class-level contribution, but the mechanism story is under-tested and the data-level gains over simple mixup are mostly within error bars. read the letter →

arxiv 2502.10288 v1 pith:SZWDEFXB submitted 2025-02-14 cs.LG

Zhuoyi Peng , Yixuan Tang , Yi Yang This is my paper · ORCID

classification cs.LG

keywords machineunlearningcatastrophicmixupaugmentationadversarialtrainingcontrastivelosslabel-agnosticdataprivacy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that catastrophic unlearning—the failure mode where deleting one piece of data also damages knowledge the model should keep—can be overcome by regularizing the unlearner with synthesized mixup samples that inhabit the region where forgetting and retention interfere. It proposes MixUnlearn, a generator-unlearner framework in which a lightweight adversarial mixer creates hard interpolations between data to be forgotten and data to be retained, and contrastive losses then teach the unlearner to remove only the target information while preserving behavior on remaining and unseen data. The intended payoff is a practical approximate-unlearning method: a model that behaves like one retrained without the forgotten data, without the cost of full retraining, in both label-aware and label-agnostic settings. A sympathetic reading of the experiments is that MixUnlearn consistently lands closer to Retrain than existing unlearning baselines across four benchmark datasets and an ImageNet/ViT setup.

What carries the argument

The load-bearing object is the generator-unlearner loop. A MixBlock module—a learnable attention-based mixer with roughly 66K parameters—takes one forgetting sample and one remaining sample and produces a mixed sample $x^{\mathrm{mix}}_{ij} = g(x_i, x_j, \lambda)$ with $\lambda$ sampled from a Beta distribution. An adversarial contrastive loss trains this generator to produce hard samples that push the unlearner to reveal the forgetting sample's target while losing knowledge from the remaining sample. The unlearner is then optimized with two contrastive losses: $L_{\mathrm{mix}}$ applies the reverse objective on the synthetic mixed samples, and $L_{\mathrm{real}}$ reinforces forgetting and retention on the original samples, combined as $L_{\mathrm{unlearn}} = L_{\mathrm{mix}} + \omega L_{\mathrm{real}}$. In the label-agnostic variant, unavailable labels are replaced by sharpened predictions of the initial model, which concentrates the pseudo-target distribution. This mechanism is designed so that the unlearner sees exactly the interpolation zone where the paper argues catastrophic interference lives.

What would settle it

Train a classifier on two well-separated classes, run MixUnlearn to delete one class completely, and then measure test accuracy on the remaining class and membership-inference attack success rate against a model retrained without the deleted class. If the unlearned model drifts from Retrain on these metrics even though the mixup samples cover the interpolation zone, the claim that interpolated samples capture the catastrophic region is false. A sharper test is to introduce remaining-data points that are far from any convex combination of forgotten and remaining training samples and check whether the adversarial mixer can still generate the hard cases that cause catastrophic loss.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that MixUnlearn outperforms existing state-of-the-art unlearning techniques in both label-agnostic and label-aware setups, and that its adversarial mixup regularization overcomes catastrophic effects so the unlearned model approximates a model retrained without the forgotten data. Concretely, the paper reports that its label-aware variant reaches remaining-class test accuracies close to Retrain on CIFAR-10 and SVHN—for example 87.10% and 93.95% in class-level unlearning—while driving the forgotten-class accuracy to roughly zero, and that its label-agnostic variant handles semi-supervised and noisy-label conditions without explicit labels. It also claims that the method is efficient: the learnable mixer has about 66,000 parameters and is updated only every few iterations, so the whole procedure is faster than teacher-student baselines and far cheaper than retraining.

Load-bearing premise

The load-bearing premise is that mixup samples formed by interpolating between forgotten and remaining data actually mimic the points where forgetting and retention conflict, so that regularizing the unlearner on these synthetic samples transfers to unseen remaining data.

Editorial extensions

If this is right

Approximate unlearning can approach Retrain behavior in class-level and data-level deletion tasks, including membership-inference attack success rates close to retraining, on CIFAR-10, SVHN, MNIST, and Fashion-MNIST.
Deletion can be made label-agnostic: no labels for forgotten or remaining data are needed, which extends unlearning to semi-supervised and noisy-label settings.
A small adversarial mixer with about 66K parameters is enough to produce the hard examples, and because it is updated only every few iterations, the overhead stays below teacher-student unlearning baselines.
The learnable generator adds a real gain over vanilla mixup in this setting: replacing the generator with fixed-ratio interpolation degrades both remaining-class utility and forgetting completeness in the reported ablations.
The method transfers beyond small images: the reported ImageNet/ViT experiments show the same pattern of high remaining-class accuracy with near-total forgetting of targeted classes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit is that unlearning difficulty is governed by feature-space geometry: MixUnlearn should need fewer mixup samples when classes are well separated and more when decision regions heavily overlap, a prediction that could be tested by varying class similarity.
The same generator-unlearner pattern could be carried to other modalities by defining the mixing function in an embedding space rather than raw input space—for example interpolating text or tabular representations—provided the interpolation remains semantically meaningful.
The paper leaves open whether samples outside the interpolation region, such as adversarial perturbations of remaining data, can also trigger catastrophic loss; if they can, a future extension would need to generate those cases as well.
Because the ablations show that the real-sample loss is essential, one actionable extension is to anneal its weight during unlearning, starting with strong retention and gradually increasing the mixup regularization.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the quick read: this is a workmanlike empirical paper that will be a useful data point in approximate unlearning, but the story it tells about why it works is not supported by its own ablations. The class-level gains are real; the data-level gains over the simple L-Mix baseline are mostly within error bars, so the 'significantly outperforms' claim in the abstract is too strong for the whole paper.

What's genuinely new: combining a learnable adversarial mixup generator (borrowed from AdAutoMix) with contrastive unlearning losses that push the unlearner to forget the forgetting sample and retain the remaining sample is, as far as I know, novel. The framework handles both label-aware and label-agnostic settings, which is practically useful. The empirical footprint is solid: four datasets plus an ImageNet/ViT extension, five seeds, released code, and ablations showing that both Lreal and Lmix matter. The authors also honestly flag the open question of when to stop unlearning. Good.

The soft spots, in rough order of severity. First, the mechanism claim. The paper says mixup samples from Df x Dr simulate the catastrophic overlap region and that this is why regularization works. But the ablations always mix a forgetting sample with a remaining sample, both in the full method and in the 'w/o MB' vanilla mixup versions. Nobody tests whether mixing two remaining samples (or random pairs) gives the same boost. If it does, the observed gains are generic smoothing and the adversarial generator's cross-set objective has no special role. I agree with the stress-test note: this is a real gap in the evidence. It doesn't kill the empirical contribution, but it means the 'simulate the data susceptible to catastrophic effects' narrative is an assertion, not a demonstrated fact.

Second, hyperparameter selection. They search alphas, three temperatures, a sharpen temperature, omega, and unlearning epochs over wide grids, with no described validation procedure. On small datasets, picking the best from a big grid can account for exactly the kind of gains they report. This is a moderate concern that should have been addressed.

Third, the significance claim is inflated for data-level unlearning. On CIFAR-10, SVHN, MNIST, and Fashion-MNIST, the data-level deltas over L-Mix are on the order of 0.1-0.2 points, well within standard deviations. The class-level gains are larger (e.g., +4 points on CIFAR-10 agnostic), so the method has real value there, but the paper should have reported the data-level results as comparable rather than part of a blanket superiority claim.

Who is this for? Researchers working on approximate unlearning, especially those looking for empirical baselines in the mixup-for-unlearning space. It deserves a serious referee: it is a careful, reproducible empirical study with a testable central claim, even though that claim remains untested. I would not cite it as evidence for the catastrophic-overlap mechanism, but I would cite it as an empirical unlearning method with competitive class-level performance.

Referee Report

3 major / 6 minor

Summary. The paper proposes MixUnlearn, a generator-unlearner framework for approximate machine unlearning. A MixBlock generator creates hard mixup examples by mixing one Forgetting sample with one Remaining sample, and an adversarial contrastive objective trains the generator to produce samples that make the unlearner reveal Forgetting information while losing Remaining knowledge. The unlearner is then regularized by two contrastive losses, one on the mixed samples (Eq. 5) and one on real data (Eq. 6), to forget the target data while retaining utility. Experiments are reported on CIFAR-10, SVHN, MNIST, and FASHION-MNIST, with additional ImageNet/ViT results, in both label-aware and label-agnostic settings, and compared against a wide range of baselines including Retrain, LAF, LAF+R, GLI, and a proposed L-Mix baseline. The paper claims that MixUnlearn significantly outperforms existing state-of-the-art unlearning techniques and overcomes catastrophic unlearning.

Significance. If the claimed effect is real, MixUnlearn is a practically useful approximate unlearning method: it preserves retention utility while erasing targeted data, works without labels, and is relatively cheap due to the lightweight 66K-parameter MixBlock. The paper has notable strengths: code is released; experiments use five seeds and report error bars; the ablation study covers several components; hyperparameter sensitivity is analyzed; robustness checks on noisy-label and semi-supervised settings are included; and the ImageNet/ViT extension broadens the evidence base. However, the central mechanism of the method, that cross-set Df-by-Dr mixup specifically simulates catastrophic overlap, is not isolated by the ablations, and the 'significant' improvement claim is not uniformly supported by the data-level results. These issues are fixable but require additional experiments and more careful claims.

major comments (3)

[Section 5.5, Tables 3 and 4] The ablation study does not isolate the claimed mechanism that mixing Forgetting with Remaining samples is what generates the benefit. The 'w/o MB' ablation replaces the learned MixBlock with vanilla mixup but still mixes Df with Dr samples; it never tests a Remaining-only mixup (or a random-pair mixup) under the same losses Lmix plus Lreal. Consequently, the observed gains over LAF could be due to generic interpolation smoothing of decision boundaries rather than to the paper's proposed 'catastrophic overlap' mechanism. This is load-bearing because the adversarial generator's cross-set objective (Eq. 3) and the central narrative in Sections 1 and 4 depend on Df-by-Dr pairing being special. I ask the authors to add an ablation that mixes two Remaining samples (and, ideally, a Remaining-with-random-class mixing) using the same Lmix and Lreal losses, and to report Testr, Testf, Trainr, and Trainf for those variants.
[Section 5.4, Table 2] The abstract and Section 5.4 claim that MixUnlearn 'significantly outperforms existing state-of-the-art unlearning techniques' and 'achieves significantly greater gains' than L-Mix, but the data-level (Basic) label-agnostic results do not support the word 'significantly' in that setting. For example, on CIFAR-10, Ours has Test 84.82±1.39 versus L-Mix 84.56±1.46, Trainr 79.18±0.98 versus 79.01±1.78, and Trainf 78.48±1.25 versus 79.65±2.21; all differences are within one standard deviation, and similar overlap appears for SVHN and MNIST. The manuscript reports no significance tests (e.g., paired tests across the five seeds). Either provide such tests and restrict the 'significant' claim to the configurations where it holds, or temper the claim in the abstract and Section 5.4.
[Section A.1 and Section 5.1] The hyperparameter selection protocol is not specified. Appendix A.1 lists large search grids for alpha, tau_gen, tau_mix, tau_real, omega, and the sharpen temperature T, and Appendix A.3 states that baseline-specific parameters are tuned 'to optimize performance,' but the manuscript does not state whether this tuning is performed on a validation split or on the same test metrics reported in Tables 1 and 2. If the reported test metrics are used to select hyperparameters, the comparisons are overoptimistic relative to a fair deployment scenario. Please specify the selection criterion and, if possible, show that the ranking of methods is stable under alternative reasonable hyperparameter choices.

minor comments (6)

[Figure 1 caption] The caption contains a typo: 'sythesize' should be 'synthesize'.
[Throughout] The module name is typeset as 'M ixBlock' in Section 4.1 and elsewhere; this should be 'MixBlock' consistently.
[Equation 3 and Section 4.1] The definition of SimLoss as (1 - cosine similarity) appears only after Eq. 3, which makes the sign of the adversarial objective hard to follow; please state the definition before Eq. 3 and briefly explain the sign convention in the text.
[Appendix A.2] The sentence 'we train two 18-layer ResNet models on the CIFAR datasets' should say 'on CIFAR-10 and SVHN' for precision, since only CIFAR-10 is used.
[Appendix A.8, Figure 7 caption] The caption and the surrounding text repeat the same sentence about the time cost comparison on an NVIDIA GeForce RTX 3090; please remove the duplication.
[Figure 8 caption] The caption says 'lambda is set as 0.5,' but the paper elsewhere samples lambda from a Beta distribution; please clarify whether this is a fixed value for visualization only.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the unlearning objectives are empirical and the central claims are validated against the external Retrain baseline.

full rationale

MixUnlearn's derivation is an empirical training procedure rather than a formal chain that reduces to its inputs. The generator (Eq. 3) and unlearner (Eqs. 5-7) are optimized with contrastive losses whose targets are either available labels or Sharpen(fD(x)) from the initial model; the reported metrics (Testr, Testf, ASR, Trainr, Trainf) are measured against an independently retrained model, so no fitted parameter is renamed as a prediction. The mechanism claim that Df x Dr mixup samples simulate the data vulnerable to overlapping catastrophic effects is motivated by the Figure 1 toy example and is not fully isolated in the ablations, since the w/o MB ablation retains cross-set mixing; however, a missing control or an unproven mechanism is an experimental-support gap, not circularity. There are no load-bearing self-citations: the borrowed MixBlock module comes from external prior work (Qin et al., 2024), and the comparisons are to external baselines including the Retrain gold standard. Acknowledged limitations, such as the fixed unlearning-epoch termination rule (Appendix A.3) and the pragmatic ImageNet/ViT subset (Appendix A.11), do not make the argument circular. Overall, the central claim has independent empirical content.

Assumptions & free parameters 7 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical, so free parameters are mostly hyperparameters. There is no derivation with hidden free constants. The learned MixBlock weights (66K) are trained on the same data, which is normal but should be counted as a fitted component; their contribution to the result is not separated from the hyperparameters.

free parameters (7)

Beta distribution alpha for lambda = searched in {0.3, 0.5, 0.75, 1, 1.5}
Controls the mix ratio distribution for the generator and unlearner losses; tuned per dataset without a described validation split.
tau_gen = searched in {0.05, 0.1, 0.5, 1, 5}
Temperature in the generator contrastive loss Eq. 3; tuned per dataset.
tau_mix = searched in {1, 10, 20, 50}
Temperature in the mixup unlearning loss Eq. 5; tuned per dataset.
tau_real = searched in {2, 5, 10, 20, 40}
Temperature in the real-sample unlearning loss Eq. 6; tuned per dataset.
omega = tuned from {0.5, 1, 10, 20, 30}
Weight balancing Lmix and Lreal in Eq. 7; tuned per dataset.
sharpen temperature T = 0.3
Temperature in Sharpen operation for label-agnostic pseudo-targets; fixed by hand.
unlearning epochs = 20 for MNIST/FASHION, 40 for CIFAR/SVHN
Fixed budget used as termination criterion; acknowledged as an open question.

assumptions (3)

domain assumption Mixup regularization improves the unlearner's behavior on intermediate regions between forgetting and remaining data.
The entire method relies on the premise in Section 4 that interpolation samples are where catastrophic unlearning occurs; only a toy example is given, no proof.
domain assumption p(x) = one-hot label (label-aware) or Sharpen(fD(x)) (label-agnostic) provides reliable targets for retention and forgetting.
Eq. 4 uses these pseudo-targets; if the initial model's sharpened predictions are wrong, the unlearner is anchored to the initial model's errors.
domain assumption The adversarial generator-unlearner loop converges to useful hard mixup samples instead of collapsing.
The paper uses alternating optimization with generator updates every four iterations (Section 4.2) but provides no convergence analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adversarial Mixup Unlearning." pith.science (2026). https://pith.science/paper/SZWDEFXB

@misc{pith2026250210288,
  author       = {Pith},
  title        = {Pith review of: Adversarial Mixup Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZWDEFXB}},
  note         = {Machine review of arXiv:2502.10288}
}

read the original abstract

Machine unlearning is a critical area of research aimed at safeguarding data privacy by enabling the removal of sensitive information from machine learning models. One unique challenge in this field is catastrophic unlearning, where erasing specific data from a well-trained model unintentionally removes essential knowledge, causing the model to deviate significantly from a retrained one. To address this, we introduce a novel approach that regularizes the unlearning process by utilizing synthesized mixup samples, which simulate the data susceptible to catastrophic effects. At the core of our approach is a generator-unlearner framework, MixUnlearn, where a generator adversarially produces challenging mixup examples, and the unlearner effectively forgets target information based on these synthesized data. Specifically, we first introduce a novel contrastive objective to train the generator in an adversarial direction: generating examples that prompt the unlearner to reveal information that should be forgotten, while losing essential knowledge. Then the unlearner, guided by two other contrastive loss terms, processes the synthesized and real data jointly to ensure accurate unlearning without losing critical knowledge, overcoming catastrophic effects. Extensive evaluations across benchmark datasets demonstrate that our method significantly outperforms state-of-the-art approaches, offering a robust solution to machine unlearning. This work not only deepens understanding of unlearning mechanisms but also lays the foundation for effective machine unlearning with mixup augmentation.

Figures

Figures reproduced from arXiv: 2502.10288 by the authors.

**Figure 2.** The goal of our generator is to produce hard mixed samples that challenge the unlearner. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Representation distributions in the class-level unlearning on the CIFAR-10. The blue [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Kernel Density Estimate Plots for Loss Distributions. We use the setting of class-level [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Learning Curve of Initial Model. A.3 BASELINES For the gold-standard baseline Retrain, we retrain the CNN models on the MNIST datasets for 20 epochs with a learning rate of 1e-3, while the ResNet models on the CIFAR datasets are retrained for 40 epochs with a learning …

**Figure 6.** Figure 6: Hyperparameter Sensitivity Analysis on CIFAR-10 class-level unlearning. The X-axis [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Time cost comparison for class-level unlearning. Experiments are conducted using [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 8.** Figure 8: Visualization of the Mixed Sample. The experiment is conducted on class-level unlearning [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

**Figure 9.** Figure 9: Unlearning gaps between a focal method and the retrained model in terms of Train [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

**Figure 10.** Figure 10: Kernel Density Estimate Plots for Loss Distributions. We use the setting of class-level [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

**Figure 11.** Figure 11: Time cost comparison for class-level unlearning methods on ImageNet using Vision [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 12 canonical work pages

[4]

These mislabeled samples are then designated as data to be forgotten. The initial model is trained with these noisy labels, while the retrained model is trained with the remaining data (note that remaining data’s labels are clean), creating a more complex unlearning environment (Shen et al., 2024). We assess label-aware methods within this noisy context. ...

work page 2024
[5]

Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms

Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv:1708.07747,

arXiv
[6]

12 Published as a conference paper at ICLR 2025 A A PPENDIX A.1 H YPERPARAMETERS AND IMPLEMENTATION In all experiments, we use a batch size of

work page 2025
[8]

Unroll (Thudi et al.,

retrains the model on smaller data shards from the remaining dataset and aggregates the results 13 Published as a conference paper at ICLR 2025 through ensembling. Unroll (Thudi et al.,

work page 2025
[9]

For RandLabel, we tune the weight for retaining loss from {0.0001, 0.01, 0.1, 1, 10, 100}

For LAF (Shen et al., 2024), we use the best parameters reported in the original paper. For RandLabel, we tune the weight for retaining loss from {0.0001, 0.01, 0.1, 1, 10, 100}. For L-Mix, we tune alpha (i.e., the parameter from the Beta distribution) from {0.3, 0.5, 0.75, 1, 1.5}. Discussion on Terminating the Unlearning Process . We adopt the common pr...

work page 2024
[10]

To obtain initial models, we train two 18-layer ResNet models on the CIFAR datasets for 20 epochs with a learning rate of 5e-5 and train two CNN models on the MNIST datasets for 10 epochs with a learning rate of 1e-3, following Shen et al. (2024). To ensure model convergence, we examine the learning curves of these initial models, as shown in Figure

work page 2024
[11]

This shows that our approach can effectively handle label noise

In the noisy-label scenario, our method demonstrates robust performance, as seen in both the accuracy and ASR (Attack Success Rate) metrics which are close to Retrain. This shows that our approach can effectively handle label noise. In the semi-supervised scenario, where a large percentage of the training data remains unannotated, our method continues to ...

work page 2025
[13]

This comparison is performed using two metrics: training accuracy on the forgotten data (Trainf ) and testing accuracy

To better understand the impact of the data ratio on unlearning performance, we quantify the gap between a focal method (ours or a baseline) and the retrained model. This comparison is performed using two metrics: training accuracy on the forgotten data (Trainf ) and testing accuracy. For a given metric, such as Trainf , we calculate the absolute differen...

work page 2009

Show all 14 references

[14]

vit-b16-224-in21k

as the model of interest. Specifically, we utilize the pre-trained weights of “vit-b16-224-in21k”, which were trained on ImageNet-21K. For our experiments, we use the validation set of ImageNet-1K, consisting of 50,000 images, as the dataset to manipulate the pre- trained Visi...

2016
[2009]

Towards unbounded machine unlearning

11 Published as a conference paper at ICLR 2025 Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. Advances in Neural Information Processing Systems, 36,

2025
[2021]

A survey on mixup augmentations and beyond

Xin Jin, Hongyu Zhu, Siyuan Li, Zedong Wang, Zicheng Liu, Chang Yu, Huafeng Qin, and Stan Z Li. A survey on mixup augmentations and beyond. arXiv preprint arXiv:2409.05202,

arXiv
[2022]

Reading digits in natural images with unsupervised feature learning

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, volume 2011, pp. 4,

2011
[2024]

Convolutional networks for images, speech, and time series

Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, 3361(10):1995,

1995
[3090]

Compared to Retraining, which requires a complete model retraining and is thus highly time- consuming, our method demonstrates significant superiority in efficiency

SISA utilizes 4 GPU devices to facilitate unlearning, whereas other methods use only a single device. Compared to Retraining, which requires a complete model retraining and is thus highly time- consuming, our method demonstrates significant superiority in efficiency. This inef...

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [4]

These mislabeled samples are then designated as data to be forgotten. The initial model is trained with these noisy labels, while the retrained model is trained with the remaining data (note that remaining data’s labels are clean), creating a more complex unlearning environment (Shen et al., 2024). We assess label-aware methods within this noisy context. ...

work page 2024

[2] [5]

Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms

Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv:1708.07747,

arXiv

[3] [6]

12 Published as a conference paper at ICLR 2025 A A PPENDIX A.1 H YPERPARAMETERS AND IMPLEMENTATION In all experiments, we use a batch size of

work page 2025

[4] [8]

Unroll (Thudi et al.,

retrains the model on smaller data shards from the remaining dataset and aggregates the results 13 Published as a conference paper at ICLR 2025 through ensembling. Unroll (Thudi et al.,

work page 2025

[5] [9]

For RandLabel, we tune the weight for retaining loss from {0.0001, 0.01, 0.1, 1, 10, 100}

For LAF (Shen et al., 2024), we use the best parameters reported in the original paper. For RandLabel, we tune the weight for retaining loss from {0.0001, 0.01, 0.1, 1, 10, 100}. For L-Mix, we tune alpha (i.e., the parameter from the Beta distribution) from {0.3, 0.5, 0.75, 1, 1.5}. Discussion on Terminating the Unlearning Process . We adopt the common pr...

work page 2024

[6] [10]

To obtain initial models, we train two 18-layer ResNet models on the CIFAR datasets for 20 epochs with a learning rate of 5e-5 and train two CNN models on the MNIST datasets for 10 epochs with a learning rate of 1e-3, following Shen et al. (2024). To ensure model convergence, we examine the learning curves of these initial models, as shown in Figure

work page 2024

[7] [11]

This shows that our approach can effectively handle label noise

In the noisy-label scenario, our method demonstrates robust performance, as seen in both the accuracy and ASR (Attack Success Rate) metrics which are close to Retrain. This shows that our approach can effectively handle label noise. In the semi-supervised scenario, where a large percentage of the training data remains unannotated, our method continues to ...

work page 2025

[8] [13]

This comparison is performed using two metrics: training accuracy on the forgotten data (Trainf ) and testing accuracy

To better understand the impact of the data ratio on unlearning performance, we quantify the gap between a focal method (ours or a baseline) and the retrained model. This comparison is performed using two metrics: training accuracy on the forgotten data (Trainf ) and testing accuracy. For a given metric, such as Trainf , we calculate the absolute differen...

work page 2009