Siamese Machine Unlearning with Knowledge Vaporization and Concentration

Hengtao He; Jun Zhang; Khaled B. Letaief; Shenghui Song; Songjie Xie

REVIEW 5 major objections 6 minor 40 references

Siamese Machine Unlearning with Knowledge Vaporization and Concentration

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Siamese unlearning vaporizes forgotten-data logits and concentrates the rest.

desk verdict Interesting practical unlearning recipe, but its own ablation contradicts the central mechanism: the vaporization loss can be satisfied by the predictor head alone, and the paper admits this in an appendix. read the letter →

arxiv 2412.01207 v1 pith:XZEDVU2F submitted 2024-12-02 cs.LG

Songjie Xie , Hengtao He , Shenghui Song , Jun Zhang , Khaled B. Letaief This is my paper · ORCID

classification cs.LG

keywords machineunlearningrighttobeforgottenSiamesenetworksknowledgevaporizationconcentrationapproximatemembershipinferenceattackdataaugmentation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Machine unlearning asks how to remove the influence of specific training points from an already-trained model without retraining from scratch. This paper argues that a model retrained on only the remaining data has a recognizable signature: augmented views of forgotten data produce dispersed, high-entropy logits, while augmented views of remaining data stay tightly clustered. On that basis it proposes two complementary objectives—knowledge vaporization, which disperses the logits of forgetting data, and knowledge concentration, which preserves representations of remaining data—implemented through a Siamese network with shared weights and a stop-gradient predictor. The paper claims this reproduces a retrained model's behavior using only a small subset of remaining data and no teacher models, and it reports strong forgetting, preserved utility, and low membership-inference success across full-class, sub-class, and random forgetting on CIFAR-10 and CIFAR-100.

What carries the argument

The carrying mechanism is the Siamese network with a prediction MLP head and stop-gradient, used to define a symmetric cosine-distance loss between two augmented views of the same sample. Knowledge vaporization minimizes this distance for forgetting data, pushing the two views' logits apart so the sample's representations become dispersed and unpredictable; knowledge concentration maximizes the same distance for remaining data, pulling each sample's augmented views together so its learned representation is preserved. The cross-entropy term with adaptively permuted labels, generated by a randomized-response scheme from the per-class unlearning ratio, prevents the process from destroying model utility on partially retained classes. Together these losses implement the paper's central identity: the unlearned model should exhibit the same augmentation-response signature as a model retrained without the forgetting data.

What would settle it

Train retrained models on a dataset where forgotten and remaining samples are semantically well separated, such as synthetic classes with no shared features, and compute the per-sample KL divergence of augmented-view logits; if forgetting samples do not show clearly higher dispersion than remaining samples, the empirical premise collapses. A direct test is to run Siamese unlearning on such a setting and check whether the unlearned model's outputs diverge from the retrained model far more than the paper's CIFAR results, which would show the method rides on the dispersion signature rather than on the loss functions themselves.

Watch

Extended reading notes

Core claim

The paper's central claim is that unlearning can be posed as matching the logit dispersion pattern of a retrained model, rather than approximating its weights. In a retrained model, augmented views of forgetting data scatter widely over the logit space while augmented views of remaining data concentrate, as the paper shows with t-SNE plots and KL-divergence measurements. The proposed Siamese unlearning method enforces this pattern directly: for each forgetting sample it minimizes the cosine similarity between two augmented views through a prediction head with stop-gradient, and for each remaining sample it maximizes that similarity, while a symmetric cross-entropy term keeps the outputs usable for classification. An adaptive label permutation, derived from the per-class fraction of unlearned data via randomized response, sets the target labels for forgetting data so that partially forgotten classes are not destroyed. The result, the paper argues, is an unlearned model whose outputs align with those of the retrained model, achieving the best accuracy-forgetting trade-off among the compared baselines and, in full-class scenarios, zero membership-inference success on forgetting data.

Load-bearing premise

The whole method rests on the observation that in a retrained model the augmented views of forgotten data are much more dispersed than augmented views of remaining data; that observation is demonstrated with t-SNE plots and KL divergences of ten data points per set from a single VGG16-BN model on CIFAR-10, so if the signature does not hold for other architectures, datasets, or augmentation strengths, the vaporization and concentration losses will not reproduce retrained-model behavior.

Editorial extensions

If this is right

If the central claim holds, an unlearning request can be served with access to less than 2% of the remaining dataset, which matters when the full remaining set is unavailable or privacy-sensitive.
Because no teacher model or stored gradient history is needed, the memory footprint of unlearning stays the same as the original model, unlike distillation-based methods.
The method applies to instance-wise and random forgetting, not only whole-class removal, so it covers the common single-user deletion request.
In full-class scenarios the reported 0.00 MIA on forgetting data implies that membership-inference attacks cannot distinguish forgotten points from non-members, a direct privacy benefit.
The approach is robust across simple, contrastive, and Cutout augmentations, suggesting the result does not depend on one particular augmentation choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The augmentation-dispersion signature could itself be turned into a post-hoc unlearning verification test: measure KL divergence of augmented-view logits on a candidate forgotten point and compare it with a reference set of remaining points, without needing retrained checkpoints.
The adaptive label permutation is a natural place to connect unlearning with local differential privacy; treating per-class unlearning ratios as a privacy budget could yield formal guarantees on how much class-level information survives.
If the dispersion signature is architecture-dependent, the method's effectiveness on larger or differently trained models is a direct empirical question; testing on ImageNet-scale models or transformer backbones would settle whether the signature generalizes.
The vaporization objective might combine with parameter-space methods such as Fisher-information damping, targeting the same forgetting samples while limiting collateral damage on shared representations.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth taking seriously, but the central mechanism is not actually demonstrated. What is new here is a concrete recipe: a SimSiam-style Siamese loss that pushes apart logits of augmented views of forgetting data and pulls together views of remaining data, combined with an adaptive label permutation for the cross-entropy term. That combination, plus the practical claims of no teacher model and only a small remaining-data subset, is a genuine contribution to approximate unlearning. The experiments are broad: three scenarios, three architectures, several baselines, and the authors ship code. If the method works as reported, it is a useful addition to the toolbox.

The soft spot is load-bearing. The paper says knowledge vaporization disperses the encoder's output logits, but the loss in Eq. (4) has a stop-gradient on one side and goes through the predictor h on the other. The network can satisfy that loss by changing h alone, leaving f_omega's logits nearly untouched. The paper's own Appendix 8.3 admits this: without cross-entropy terms, optimizing KVC fails to alter the output logits of the forgetting data. That directly undercuts the stated mechanism. Worse, the appendix's admission contradicts the ablation tables, where KVC-only rows are reported as achieving AccDf = 0.00. That is an internal inconsistency a referee would catch immediately. If the tables are right, the appendix is wrong; if the appendix is right, the core conceptual story is wrong and the actual forgetting is likely driven by the permuted-label cross-entropy, making the method a variant of random-label fine-tuning.

There are smaller problems too. All tables show single runs with no error bars. Hyperparameters for baselines were tuned with the test set in the loop. The motivating observation about retrained-model logit dispersion rests on ten data points per set from one VGG16-BN on CIFAR-10, with t-SNE plots and averaged KL divergences—suggestive but thin. The citation pattern is fine and the paper engages the relevant literature.

My take: the paper deserves a serious referee, but it needs major revision. The authors should resolve the contradiction, add multiple seeds, and directly measure whether f_omega's logits actually disperse after optimizing LKV (for example, by removing h from the measurement). If vaporization turns out to be a byproduct of the cross-entropy term, they should reframe the contribution honestly. As written, I would not cite it as a demonstrated mechanism, but I would send it to review with the expectation of heavy revision.

Referee Report

5 major / 6 minor

Summary. Motivated by the observation that, in a retrained-on-remaining-data model, augmented views of forgetting samples produce more dispersed output logits than augmented views of remaining samples, the paper introduces knowledge vaporization and knowledge concentration, operationalized through a Siamese network with a prediction head h and stop-gradient (Eqs. 4-5) plus a symmetric cross-entropy term with adaptively permuted labels for forgetting data (Eqs. 6-9). The combined objective requires no retraining, no teacher model, and only a 1000-sample subset of the remaining data. The method is evaluated on CIFAR-10 and CIFAR-100 with VGG16-BN, ResNet18, and ResNet50 under full-class, sub-class, and random-forgetting scenarios against eight baselines on training/test accuracy and membership-inference metrics. The authors report the best overall trade-off in most settings, MIA down to 0.00, and low runtime, and they include ablations of the vaporization/concentration loss, cross-entropy terms, and adaptive label permutation.

Significance. If the empirical results hold, this is a useful and practical contribution to approximate unlearning: no retraining, no stored teacher models, only about 1000 remaining samples (claimed less than 2% of the training set), and strong reported trade-offs between forgetting, utility, and MIA across full-class, sub-class, and random forgetting on CIFAR-10/100 with three backbones and three augmentation sets. The paper deserves credit for a concrete, falsifiable protocol: the per-class permutation probabilities are computed from the known deletion-request ratios r rather than fitted to test performance, so the design is not circular; the evaluation is unusually broad, including runtimes and an ablation; and source code is provided. The central reservation is that the mechanism described in Section 3.3 is not established by the evidence, parts of the manuscript contradict each other (Appendix 8.3 vs. Tables 7/10), the tables lack error bars, and Eq. (7) is not reproducible as printed.

major comments (5)

[§3.3, Eq. (4); §8.3; Tables 7 and 10] The central mechanistic claim, that Eq. (4) disperses the backbone's augmented-view logits, is contradicted by the manuscript's own text and ablation tables. In LKV each term maximizes cosine distance between p1 = h(fω(x1)) and the stop-gradient copy sg(l2); gradients reach fω only through the auxiliary head h on the non-stopped branch, so the loss can be approximately minimized by changing h's output directions while leaving fω nearly unchanged. Appendix 8.3 concedes exactly this, stating that without cross-entropy terms, "optimizing knowledge vaporization and concentration alone fails to alter the output logits of the forgetting data." However, the KVC-only rows of Table 7 and Table 10 report AccDf=0.00 and TADf=0.00, which requires that fω's logits on Df changed drastically. These two statements cannot both be true. Because the permuted-label cross-entropy term SCE(x, p(y)) in Eq. (9) is the only term with a direct gradient into fω's logits on forgetting data, the observed forgetting could be driven by that term (the RandLab/Amnesiac mechanism) rather than by the claimed vaporization, so the claim in Section 3.3 is not demonstrated as stated. I request a direct measurement of fω's logit dispersion on Df (e.g., the augmented-view KL divergence as in Table 9) before and after unlearning under each ablation configuration, and an ablation that removes LKV while keeping CE and ALP, to isolate the vaporization contribution.
[§4.1 and §8.1, Tables 1-6] All results in Tables 1-6 are single runs without error bars, and Appendix 8.1 states that hyperparameters of all compared methods were tuned to achieve optimal metric values in each scenario. Several headline comparisons are close (Table 5: Ours and SCRUB both report AccDf=100.00 with gap 0.86; Table 2 TADr: BadT 74.80 vs. Ours 73.96; Table 4 TADr: SCRUB 73.26 vs. Ours 72.38), so the claims that the method "consistently outperforms" baselines and achieves the "best trade-off" are not statistically supported as reported. I ask for multiple seeds with means and standard deviations on the main tables, and for the λ and per-scenario epoch values used in the reported results, which are currently absent from Section 8.
[Eq. (7)] The adaptive label permutation probability in Eq. (7) is not reproducible as printed. If the numerator is r_k − 1, then for every r_k < 1 and K > 2 the probability of keeping the true label is negative (e.g., r_k = 0 with K = 10 gives −1/8), which contradicts the text's statement that r_k = 0 causes no permutation; if r_k^{-1} was intended, the notation must be corrected. The denominator also appears to be K − 1 rather than k − 1. Because the ablation (Table 7) credits ALP with improving the AccDf gap from −3.92 to −0.28 relative to the retrained model, the exact rule matters for reproducing the results; please restate Eq. (7) unambiguously and state precisely what was implemented.
[Algorithm 1 vs. §3.3 and §8.1] Algorithm 1 is inconsistent with the restricted-access claim. The Input line names the remaining-data subset "Sf" (presumably Sr), and lines 5-8 draw the remaining-data mini-batches from Dr, whereas Section 3.3 and Appendix 8.1 state that the method uses only a 1000-sample subset Sr (claimed to be less than 2% of the dataset). As printed, the algorithm contradicts the paper's headline advantage of not needing full remaining-data access; please correct both the source of the mini-batches and the input variable name.
[§3.2 and Appendix A, Table 9] The motivating observation, that augmented views of forgetting data are more dispersed than those of remaining data in retrained models, is evidenced only by t-SNE plots and KL divergences computed on 10 data points per set from a single VGG16-BN model on CIFAR-10 (Appendix A). In the random-forgetting scenario with contrastive augmentation, the reported KL values are 1.6926 for Dr vs. 1.7256 for Df, which is effectively no dispersion gap, yet the method's random-unlearning results are among its strongest (Tables 5-6). This suggests the dispersion signature is neither robust nor necessary for the method's success, leaving the design rationale of Section 3.2 unverified across architectures, datasets, and scenarios. I would like a quantitative check of the dispersion gap on the ResNet models and on CIFAR-100, and a statement of how the method behaves when the gap is weak or absent.

minor comments (6)

[§3.3, 'Memory efficiency' bullet] The phrase "without the need for additional neural networks" is inaccurate because the method itself trains an auxiliary predictor MLP h (Section 7, two fully connected layers with batch normalization); the claim should be qualified as "no teacher model" or "h is discarded after unlearning."
[§4.2, first paragraph] The sentence "which means it effectively erases the knowledge of Df and maintains the utility on Df" should read "maintains the utility on Dr"; as printed it is self-contradictory.
[Table 7] The column alignment of the checkmarks in Table 7 is ambiguous in the current rendering; in particular, the row described in the main text as CE-only (yielding 0 on AccDf) does not visibly match the row with AccDf=100.00(20.18). Please ensure each checkmark is placed under the correct column.
[Tables 5-6] In the random-forgetting scenario, the notation Df is used for the whole test set ("we only consider the test accuracy on the entire test dataset Df"), which conflicts with the definition of Df as forgotten training data in Section 3.1; rename the test set to avoid confusion.
[§4.2 and Table 2] The phrase "state-of-the-art" is too strong given that individual metrics favor some baselines (e.g., Table 2 TADr: BadT 74.80 vs. Ours 73.96; Table 4 TADr: SCRUB 73.26 vs. Ours 72.38); "best overall trade-off" is a more accurate description.
[Figure 5] Please state whether the reported runtimes include the per-method hyperparameter search described in Appendix 8.1 and whether all methods were evaluated on the same hardware.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Siamese unlearning objective is an empirically motivated heuristic evaluated against retrained models, not a derivation from fitted inputs or self-citations.

full rationale

The paper does not claim a formal derivation of the retrained model from first principles, and no fitted quantity is renamed as a prediction. The motivating observation in Section 3.2 and Appendix A—that augmented views of forgetting data are more dispersed in retrained models—is presented as direct empirical evidence (t-SNE plots and KL divergences measured on retrained VGG16-BN models), not as an input that is later recovered by the objective. The adaptive label permutation in Eq. (7) uses the known per-class forgetting ratios r, which are given inputs of the unlearning request, not parameters fitted to the target metrics. The final objective Eqs. (8)-(9) is a proposed training loss whose success is assessed by held-out evaluations (AccDf, TADr/TADf, MIA) against an independently retrained ground-truth model; the reported forgetting and utility numbers are not forced by construction. The manuscript's Appendix 8.3 admits that optimizing knowledge vaporization and concentration alone 'fails to alter the output logits of the forgetting data' and attributes this to the optimization neglecting fω; this is an internal limitation about the mechanism of the loss, not a circular step, since the full objective includes the cross-entropy terms and the overall method is validated empirically against baselines. There are no load-bearing self-citations and no invocation of an author-derived uniqueness theorem. The contribution is an approximate-unlearning heuristic with experimental support, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an empirical signature (dispersion/concentration of augmented-view logits), a heuristic loss design, and several tuned hyperparameters. No new physical or mathematical entities are postulated; the 'vaporization/concentration' terms are conceptual labels for existing cosine-similarity operations.

free parameters (4)

lambda (trade-off weight) = not reported
Hyperparameter in Eq. (8)-(9) balancing KVC/ALP losses and cross-entropy; no sensitivity analysis or chosen value is given in the paper.
size of remaining-data subset Sr = 1000 samples (<2% of CIFAR-10/100)
The method is evaluated with a fixed subset of 1000; no ablation shows how performance depends on this size.
predictor hidden dimension = 64
MLP head hidden size is fixed to 64; no ablation.
unlearning epochs T and learning rate = learning rate 0.0001, momentum 0.9, weight decay 0.0001; T not stated precisely (figures show ~17 epochs)
These are chosen hyperparameters; no sensitivity analysis is reported.

assumptions (4)

domain assumption Each data sample has an underlying representation eta_x to which augmented views converge (Eq. 1)
Invoked to motivate knowledge vaporization; eta_x is never estimated or verified.
domain assumption Retrained models disperse augmented views of forgetting data more than those of remaining data
Based on 10 points per set from one VGG16-BN model on CIFAR-10 (Figure 1, Table 9); assumed to generalize.
standard math Stop-gradient on one branch stabilizes the Siamese optimization and prevents collapse
Standard practice from SimSiam [8]; the paper relies on it without re-derivation.
domain assumption Cross-entropy with permuted labels guides the network toward useful predictions while forgetting
The paper asserts this in Section 3.3 and shows ablations, but no proof that the permutation schedule is optimal.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Siamese Machine Unlearning with Knowledge Vaporization and Concentration." pith.science (2026). https://pith.science/paper/XZEDVU2F

@misc{pith2026241201207,
  author       = {Pith},
  title        = {Pith review of: Siamese Machine Unlearning with Knowledge Vaporization and Concentration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZEDVU2F}},
  note         = {Machine review of arXiv:2412.01207}
}

read the original abstract

In response to the practical demands of the ``right to be forgotten" and the removal of undesired data, machine unlearning emerges as an essential technique to remove the learned knowledge of a fraction of data points from trained models. However, existing methods suffer from limitations such as insufficient methodological support, high computational complexity, and significant memory demands. In this work, we propose the concepts of knowledge vaporization and concentration to selectively erase learned knowledge from specific data points while maintaining representations for the remaining data. Utilizing the Siamese networks, we exemplify the proposed concepts and develop an efficient method for machine unlearning. Our proposed Siamese unlearning method does not require additional memory overhead and full access to the remaining dataset. Extensive experiments conducted across multiple unlearning scenarios showcase the superiority of Siamese unlearning over baseline methods, illustrating its ability to effectively remove knowledge from forgetting data, enhance model utility on remaining data, and reduce susceptibility to membership inference attacks.

Figures

Figures reproduced from arXiv: 2412.01207 by the authors.

**Figure 1.** t-SNE visualization of logit outputs from retrained models in two scenarios: (a) full-class forgetting and (b) random forgetting. Dots in different colors represent augmented views of different data points. The left panel shows the visualization for the augmented views of forgetting data, while the right panel displays those of remaining data. In both scenarios, it can be observed that the augmented views of forg… view at source ↗

**Figure 2.** Illustration of the concepts of knowledge concentration [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The proposed Siamese network for unlearning. Two augmented views of one data point are processed by a network [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Visualization of logits from Siamese unlearned models for scenarios: (a) full-class, (b) sub-class, and (c) random forgetting. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Runtime of the evaluated methods on CIFAR-100 dataset [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Test accuracy on Df and Dr using Siamese unlearning with (a) solely knowledge concentration and (b) jointly knowledge concentration and vaporization. tration require validation. We conduct experiments exclusively focusing on knowledge concentration within Dr of CIFAR-…

**Figure 7.** Figure 7: t-SNE visualization of logit outputs from retrained models with the simple data augmentation in three scenarios: (a) full-class forgetting, (b) sub-class forgetting, and (c) random forgetting. 8. Experiment Details 8.1. Training Settings Pretraining In the experiments,…

**Figure 8.** Figure 8: t-SNE visualization of logit outputs from retrained models with the contrastive data augmentation in three scenarios: (a) full-class forgetting, (b) sub-class forgetting, and (c) random forgetting. 60 40 20 0 20 40 60 40 20 0 20 40 60 forgetting data class 9 class 9 cl…

**Figure 9.** Figure 9: t-SNE visualization of logit outputs from retrained models with the cutout data augmentation in three scenarios: (a) full-class forgetting, (b) sub-class forgetting, and (c) random forgetting. Unlearning. To reproduce the compared methods, we re-implement them based on…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 26 canonical work pages

[1]

Fully-convolutional siamese networks for object tracking

Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully-convolutional siamese networks for object tracking. In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part II 14, pages 850–865. Springer, 2016. 3

work page 2016
[2]

Machine unlearning

Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141–159. IEEE, 2021. 1, 2

2021
[3]

Signature verification using a” siamese” time delay neural network

Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard S¨ackinger, and Roopak Shah. Signature verification using a” siamese” time delay neural network. Advances in neural information processing systems, 6, 1993. 3

work page 1993
[4]

Towards making systems for- get with machine unlearning

Yinzhi Cao and Junfeng Yang. Towards making systems for- get with machine unlearning. In 2015 IEEE symposium on security and privacy, pages 463–480. IEEE, 2015. 2

2015
[5]

Graph unlearning

Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC conference on com- puter and communications security , pages 499–513, 2022. 2

work page 2022
[6]

Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary

Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7766–7775, 2023. 1, 2, 3

work page 2023
[7]

A simple framework for contrastive learning of visual representations

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 2, 3, 5

work page 2020
[8]

Exploring simple siamese rep- resentation learning

Xinlei Chen and Kaiming He. Exploring simple siamese rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 15750–15758, 2021. 3, 4

work page 2021

Show all 40 references

[9]

Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher

Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 7210–7217, 2023. 1, 3, 5

2023
[10]

Zero-shot machine unlearning

Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security , 18: 2345–2354, 2023. 4

2023
[11]

Improved regular- ization of convolutional neural networks with cutout

Terrance DeVries and Graham W Taylor. Improved regular- ization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017. 5

2017 arXiv
[12]

Fast machine unlearning without retraining through selective synaptic dampening

Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12043–12051, 2024. 5

2024
[13]

Making ai forget you: Data deletion in ma- chine learning

Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in ma- chine learning. Advances in neural information processing systems, 32, 2019. 2

2019
[14]

Eternal sunshine of the spotless net: Selective forgetting in deep networks

Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9304– 9312, 2020. 1, 2, 3, 5

2020
[15]

Amne- siac machine learning

Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amne- siac machine learning. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 11516–11524, 2021. 1, 2, 3, 5

2021
[16]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 4, 5

2016
[17]

Siamese neural networks for one-shot image recognition

Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, pages 1–30. Lille, 2015. 3

2015
[18]

Understanding black-box predictions via influence functions

Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International confer- ence on machine learning, pages 1885–1894. PMLR, 2017. 1, 3

2017
[19]

Learning multiple layers of features from tiny images

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5

2009
[20]

Towards unbounded machine unlearn- ing

Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearn- ing. Advances in neural information processing systems, 36,
[21]

Erm-ktp: Knowledge-level machine un- learning via knowledge transfer

Shen Lin, Xiaoyu Zhang, Chenyang Chen, Xiaofeng Chen, and Willy Susilo. Erm-ktp: Knowledge-level machine un- learning via knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20147–20155, 2023. 2

2023
[22]

New insights and perspectives on the natural gradient method

James Martens. New insights and perspectives on the natural gradient method. Journal of Machine Learning Research, 21 (146):1–76, 2020. 1

2020
[23]

A survey of machine unlearning

Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022. 1

2022 arXiv
[24]

Very deep convo- lutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 4, 5

2014 arXiv
[25]

Systematic evaluation of pri- vacy risks of machine learning models

Liwei Song and Prateek Mittal. Systematic evaluation of pri- vacy risks of machine learning models. In 30th USENIX Se- curity Symposium (USENIX Security 21), pages 2615–2632,
[26]

Deepface: Closing the gap to human-level perfor- mance in face verification

Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level perfor- mance in face verification. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 1701–1708, 2014. 3

2014
[27]

Tarun, Vikram S

Ayush K. Tarun, Vikram S. Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Sys- tems, 35(9):13046–13055, 2024. 4

2024
[28]

On the necessity of auditable algorithmic definitions for machine unlearning

Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Pa- pernot. On the necessity of auditable algorithmic definitions for machine unlearning. In 31st USENIX Security Sympo- sium (USENIX Security 22), pages 4007–4022, 2022. 3 9

2022
[29]

What makes for good views for contrastive learning? Advances in neural informa- tion processing systems, 33:6827–6839, 2020

Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in neural informa- tion processing systems, 33:6827–6839, 2020. 2

2020
[30]

Machine unlearning via algorithmic stability

Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Ra- man Arora. Machine unlearning via algorithmic stability. In Conference on Learning Theory, pages 4126–4142. PMLR,
[31]

Demys- tifying the draft eu artificial intelligence act—analysing the good, the bad, and the unclear elements of the proposed ap- proach

Michael Veale and Frederik Zuiderveen Borgesius. Demys- tifying the draft eu artificial intelligence act—analysing the good, the bad, and the unclear elements of the proposed ap- proach. Computer Law Review International, 22(4):97–112,
[32]

The eu general data protection regulation (gdpr)

Paul V oigt and Axel V on dem Bussche. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10(3152676):10– 5555, 2017. 1

2017
[33]

Machine unlearning of features and la- bels

Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and la- bels. arXiv preprint arXiv:2108.11577, 2021. 5

2021 arXiv
[34]

Randomized response: A survey tech- nique for eliminating evasive answer bias

Stanley L Warner. Randomized response: A survey tech- nique for eliminating evasive answer bias. Journal of the American statistical association, 60(309):63–69, 1965. 5 10

1965
[35]

Using the CIFAR-10 dataset, we train the VGG16-BN solely on the remaining data across three unlearning scenarios:

Empirical Analysis on the Retrained Models To find an effective approach to approximate retrained models, we conduct experiments to empirically examine the behaviors of retrained models on both forgetting and remaining data. Using the CIFAR-10 dataset, we train the VGG16-BN so...
[36]

Full-class unlearning: The classes automobile, cat, and truck are removed from the CIFAR-10 dataset
[37]

Sub-class unlearning: We focus on the automobile class, randomly selecting 90% of its training data as forgetting data
[38]

Moreover, we utilize data augmentation and observe the response of the retrained models toward the augmentations of data points

Random unlearning: 10% of the training data points are designated as forgetting points and excluded from the dataset. Moreover, we utilize data augmentation and observe the response of the retrained models toward the augmentations of data points. To comprehensively investigate...
[39]

Predictor design

Implementation Details The unlearning procedures of the proposed Siamese unlearning method are summarized in Algorithm 1. Predictor design. The predictor h in the proposed Siamese unlearning method comprises an MLP head consisting of two fully connected layers with a batch nor...
[40]

99.86 0.00 93.58 0.00 0.03

Experiment Details 8.1. Training Settings Pretraining In the experiments, pretraining of the original and retrained models is necessary to evaluate the performance of unlearning. For the three unlearning scenarios, both the original and retrained models on the CIFAR-10 dataset...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Fully-convolutional siamese networks for object tracking

Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully-convolutional siamese networks for object tracking. In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part II 14, pages 850–865. Springer, 2016. 3

work page 2016

[2] [2]

Machine unlearning

Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141–159. IEEE, 2021. 1, 2

2021

[3] [3]

Signature verification using a” siamese” time delay neural network

Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard S¨ackinger, and Roopak Shah. Signature verification using a” siamese” time delay neural network. Advances in neural information processing systems, 6, 1993. 3

work page 1993

[4] [4]

Towards making systems for- get with machine unlearning

Yinzhi Cao and Junfeng Yang. Towards making systems for- get with machine unlearning. In 2015 IEEE symposium on security and privacy, pages 463–480. IEEE, 2015. 2

2015

[5] [5]

Graph unlearning

Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC conference on com- puter and communications security , pages 499–513, 2022. 2

work page 2022

[6] [6]

Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary

Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7766–7775, 2023. 1, 2, 3

work page 2023

[7] [7]

A simple framework for contrastive learning of visual representations

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 2, 3, 5

work page 2020

[8] [8]

Exploring simple siamese rep- resentation learning

Xinlei Chen and Kaiming He. Exploring simple siamese rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 15750–15758, 2021. 3, 4

work page 2021