Pith. sign in

REVIEW 4 major objections 6 minor 44 references

RUB: Evaluating Residual Knowledge in Unlearned Models

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Existing machine-unlearning methods can pass standard verification metrics while still allowing an adversary to construct inputs that resurface information the model was supposed to forget.

desk verdict One clean class-wise unlearning result buried under an overclaimed instance-wise and generative narrative; deserves peer review but needs the confounded parts reworked. read the letter →

arxiv 2504.14798 v2 pith:73WVNQ3P submitted 2025-04-21 cs.LG cs.CV

Hao Xuan , Xingyu Li This is my paper · ORCID
classification cs.LGcs.CV
keywords machineunlearningrobustresidualknowledgeadversarialattackverificationbenchmarkmembershipinferenceimage-to-imagegenerationpartialforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that machine-unlearning verification is currently measuring only whether the unlearning procedure was executed, not whether the forgotten information is actually gone. It introduces Robust Unlearning as a formal standard, and a benchmark (RUB) built around the Unlearning Mapping Attack (UMA), an adversarial probe that optimizes small input perturbations so that an unlearned model reproduces the original model's outputs for forget-set samples. Across classification and image generation, the authors report that existing unlearning methods—several of them passing standard unlearning-accuracy and membership-inference checks—remain vulnerable to UMA, sometimes reverting to near-original behavior under image perturbations of 8/255. If the finding is right, current practice for certifying privacy and content removal in deep learning is weaker than assumed.

What carries the argument

The load-bearing object is the Unlearning Mapping Attack (UMA): an optimization problem that maps the original model's forgotten outputs back into inputs. For each forget-set sample x with original output f(x;θ), UMA minimizes a loss L(f_u(δ;θ_u), f(x;θ)) over perturbations δ starting from random noise, using projected gradient descent with sign-gradient updates and bounded perturbation strength for discriminative tasks (8/255 or 16/255) and unbounded for generation. The work this does is twofold: it turns the paper's Robust Unlearning criterion—output distance greater than a threshold ε1 for every forget input and every allowed δ—into a concrete, optimizable verification target, and it doubles as a training signal, since adding a maximization step over such perturbations to the unlearning loss makes the model empirically more robust. UMA is also the piece that makes the paper's negative result: because the attack does not touch the unlearning procedure or the model parameters, it is invisible to process-based verification.

What would settle it

Train a reference model from scratch on the retain set only, with no exposure to forget-set samples, and run UMA with the paper's own bounds and hyperparameters. The paper's Table 2 already reports 99.98 percent unlearning accuracy under an 8/255 attack for a retrained CIFAR-10 baseline; if a model that never saw the forget samples produces the same attack success as unlearned models, then UMA success is not sufficient evidence of residual knowledge, and the benchmark would need a control group.

Watch

Extended reading notes

Core claim

The paper's central claim is that verification metrics currently used to certify machine unlearning—unlearning accuracy and membership-inference score—do not measure whether forgotten information is actually gone. It introduces the Unlearning Mapping Attack (UMA), which, for each forget-set sample x, searches for a perturbation δx such that the unlearned model's output on δx matches the original model's output on x. On class-wise unlearning the attack succeeds mainly on methods that stray from the retrained baseline; on instance-wise unlearning it succeeds against every baseline tested, including retraining from scratch, at perturbation bounds of 8/255 and 16/255. In generative experiments, UMA restores masked-region reconstructions that the unlearned model had stopped producing, with per-pixel L1 distances to the original model's output dropping to under two intensity levels. The authors conclude that unlearned models retain residual knowledge that standard checks miss.

Load-bearing premise

Minimizing the distance between the unlearned model's output on a perturbed input and the original model's output is treated as evidence that the model still remembers the forgotten sample, yet a model retrained without that sample can show the same small distance.

Editorial extensions

If this is right

  • Unlearning accuracy and membership-inference scores should not be treated as certificates of forgetting; a model can score near-perfectly on both and still be defeated by bounded perturbed inputs.
  • Instance-level unlearning is a harder regime than class-level unlearning: UMA succeeds against every tested instance-wise unlearning method, including retraining, while class-wise retraining resists it.
  • In generative unlearning, the attack restores visual content that unlearning had suppressed; the paper's L1 measurements put the attacked outputs within roughly a third of an intensity level per pixel of the original model's reconstruction for the I2I method.
  • Two defenses show initial promise within the paper's framework: adversarial training that adds a UMA loss term, and test-time purification with a variational autoencoder, both reducing attack success at some cost in clean accuracy or compute.
  • Because UMA is computed from the pre- and post-unlearning models directly, it provides a unified robustness check across classification, image-to-image reconstruction, and text-to-image synthesis, rather than a method-specific test.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If a retrained model that never saw the forget samples also succeeds under UMA—which the paper's Table 2 already shows for instance-level CIFAR-10—then attack success alone does not prove memory; this is an inference because the paper still reads UMA success as residual knowledge.
  • The mapping objective is continuous and pixel-based, so adapting it to discrete tokens or prompts would be a natural extension for language models, where the same residual-knowledge question is open.
  • UMA's projected-gradient-descent solver is local, so the paper's unbounded generative attacks give no certified failure cases; pairing the benchmark with certified robustness bounds, such as Lipschitz guarantees on the unlearned model, would make the Robust Unlearning standard testable in both directions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes 'Robust Unlearning,' a principle requiring unlearned models to be indistinguishable from retrained counterparts and resistant to adversarial attempts to resurface forgotten information. It introduces the Unlearning Mapping Attack (UMA), which optimizes input perturbations so that an unlearned model's output matches the original model's output on forget-set samples. The authors evaluate UMA on class-wise and instance-wise image classification (CIFAR-10, CIFAR-100, Tiny-ImageNet) and on image-to-image generative unlearning (MAE-based), reporting that existing unlearning methods remain vulnerable under UMA even when they pass standard verification metrics. They also present preliminary defenses based on adversarial unlearning and input purification. The class-wise experiments include a retrained baseline that is robust to UMA, while the instance-wise experiments show that both unlearned and retrained models are highly vulnerable to UMA.

Significance. If the central claim is established, the paper provides a valuable benchmark and a concrete attack for auditing machine unlearning, and its class-wise results with a retrained control are a meaningful empirical contribution. The authors are explicit about releasing code and checkpoints, and the formal definition of Robust Unlearning is a useful conceptual step. However, the broad conclusion that 'existing unlearning techniques remain vulnerable' is currently supported only in the class-wise setting; the instance-wise and generative results are confounded by the fact that retrained models exhibit the same UMA vulnerability. The significance of the work therefore depends on whether the authors can, in revision, separate residual-knowledge effects from ordinary generalization effects.

major comments (4)
  1. [Section 5.2, Table 2] The instance-wise results do not support the claim that UMA recovers residual forgotten knowledge. The retrained baseline, which never saw the forget-instance training data, already has UA=93.78 on CIFAR-10 without any attack and UA=99.98 under an 8/255 attack; the attacked unlearned models reach essentially identical values (e.g., FT 99.98, IU 99.98, l1-sparse 99.98). Consequently, UMA success on unlearned models is indistinguishable from UMA success on a model that genuinely forgot the data. To support the residual-knowledge interpretation, the paper must report the excess vulnerability relative to the retrained baseline (or otherwise isolate memorization from class-level generalization), for instance by testing whether the found perturbations are specifically aligned with the identity of the forget samples.
  2. [Section A.3, Table 7 and Figures 6-7] The generative experiments have the same confound as the instance-wise discriminative experiments. The small L1(I3,I1) values after attack (e.g., 48,317 for the I2I forget set, versus 1,140,778 without attack) are not compared against a retrained generator that never saw the forget images. A normally trained image-to-image model can plausibly reconstruct masked regions from image priors, so the small post-attack L1 distance may reflect ordinary reconstruction ability rather than the resurfacing of memory of the specific forget images. A retrained control is necessary before claiming that UMA bypasses unlearning in generative models.
  3. [Section 3.2, Definition 4] The paper explicitly acknowledges that the retrained baseline does not satisfy the absolute Robust Unlearning criterion, yet the abstract and conclusion present UMA vulnerability as evidence that unlearning methods 'remain vulnerable.' In the instance-wise setting, since the gold-standard retrained model is equally vulnerable, the absolute criterion cannot separate unlearning-specific failures from baseline model behavior. The authors should either redefine Robust Unlearning relative to a retrained model (for example, requiring that the unlearned model be no more vulnerable than retraining) or restrict the residual-knowledge claim to the class-wise setting where the retrained control is robust.
  4. [Section 4.1, Eq. (3)] The UMA objective minimizes ||fu(δx;θu) - f(x;θ)|| for each forget sample x. Because this objective is minimized to nearly the same degree for a retrained model (as shown in Table 2), the objective itself does not isolate residual knowledge; it measures the ease with which any model can be made to reproduce original-model outputs under bounded perturbations. The paper should characterize what distinguishes residual-knowledge-driven success from generalization-driven success, for example by comparing UMA loss on forget versus retain samples after matching the retrained baseline, or by examining whether the found δx is semantically related to the forgotten instance.
minor comments (6)
  1. [Section 1, Contributions] There is a typo in the contributions list: 'the first empirical metho asses' should be 'the first empirical method to assess.'
  2. [Algorithm 1 and Eq. (5)] The step size is denoted η in Algorithm 1 but α in Eq. (5); please unify the notation.
  3. [Section 5.3] The phrase 'test-time sample pontification' should be 'test-time sample purification.'
  4. [Table 3 caption] The caption uses 'Unbound' for attack strength; please define explicitly that this means the perturbation is not clipped, and state the range used in the experiments.
  5. [Appendix A.3] The appendix reports L1 distances but does not provide the UMA hyperparameters used for generative tasks (number of steps, step size, loss function); please add these for reproducibility.
  6. [Section 4.1] The statement that 'under ideal conditions (e.g., if the optimization objective is convex), UMA provides a theoretical guarantee of robustness' is vacuous without specifying ε1; please restate as a consistency property with the definition rather than a formal guarantee.

Circularity Check

2 steps flagged · score 6.0 of 10

UMA's Eq. (3) is the same distance condition used to define Robust Unlearning, and the paper's own retrained baseline fails it too, so the residual-knowledge conclusion is partly definitional and uncontrolled.

  1. self definitional [Section 4.1, Eq. (3); Section 3.2, Definition 4]
    "The UMA formulation in (3) aligns with the definition of robust unlearning. If for every x∈Du we find an optimal δx to minimize the difference, and the minimum difference is still larger than ε1, we can conclude that the unlearned model is robust with respect to ε1. ... If UMA successfully recovers forgotten knowledge, it demonstrates that residual information persists despite unlearning efforts."

    Definition 4 defines Robust Unlearning as ||fu(δx,θu)−f(x,θ)||>ε1 for all δx, and Eq. (3) is exactly the minimization of that same distance. Thus 'vulnerability to UMA' is the same inequality that defines non-robustness; calling UMA success 'residual knowledge' imports the conclusion into the operational metric. The attack is not validated against any independent ground-truth label of residual knowledge: success is the minimized loss itself, so the headline finding is partially true by construction.

  2. fitted input called prediction [Section 5.2, Table 2; Section 3.2]
    "For instance-level unlearning, as shown in Table 2, all baseline methods display limited robustness against unlearning mapping attacks. While the retraining method performs the best, it still lacks sufficient robustness, even with ϵ = 8/255. ... even models trained without specific data may still generate corresponding information (Shumailov et al., 2024)."

    The instance-wise claim that UMA uncovers residual forgotten knowledge is not separated from ordinary generalization. The paper's own retrained control, which never saw the forget instances, also yields UA ≈ 93.78 without attack and ≈ 99.98 under UMA at 8/255 on CIFAR-10 (Table 2). Since the attack 'succeeds' on a model with no residual knowledge of the forget set, UMA success cannot by itself evidence residual knowledge. The central conclusion reduces to 'models are attackable under this objective,' which is also true of the baseline; the extra step attributing attackability to residual traces is assumed, not measured. Generative results (Appendix A.3) have the same structure: L1(I3,I1) is minimized by construction, with no retrained control.

full rationale

The paper couples Robust Unlearning (Definition 4) and UMA (Eq. 3) by construction: the robust condition requires ||fu(δx,θu)−f(x,θ)||>ε1 for every δx, and UMA minimizes exactly this quantity. Consequently, 'vulnerability to UMA' is equivalent to 'failure of Robust Unlearning' by definition. The further claim that UMA success reveals residual forgotten knowledge is not independently grounded: in the instance-wise discriminative experiments the retrained model, which never saw the forget examples, also yields UA ≈ 93.78 (no attack) and ≈ 99.98 (8/255) on CIFAR-10, so the attack flags ordinary generalization as residual memory. The paper acknowledges this in Section 3.2 but does not use the retrained control to adjust the instance-wise or generative conclusions. Generative evaluation (Appendix A.3) similarly defines success as small L1(I3,I1) after optimizing δ to minimize that same L1, making the low distances partly tautological. The class-wise results (Table 1) are the one setting where retraining is robust, providing some independent content and preventing a higher score. No load-bearing self-citations were found.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the threat model established by the authors: the original model's output is the ground truth for forgotten knowledge, and a bounded PGD search is the adversary. The perturbation bounds and optimizer settings are hand-chosen hyperparameters that influence all reported success rates. No new physical or external entities are postulated.

free parameters (3)
  • Adversarial perturbation bound epsilon = 8/255 and 16/255 for classification; unbounded for generation
    Chosen by the authors to preserve major feature information (Section 5.1); the measured vulnerability levels depend on this bound.
  • UMA step size eta = 1/255 default (ablated 0.1 to 16/255)
    Selected via ablation study (Appendix A.1); it controls attack convergence and success.
  • UMA iterations T = 100 default (ablated 0 to 192)
    Selected via ablation study (Appendix A.1); more iterations increase attack efficacy.
assumptions (4)
  • domain assumption Standard machine unlearning definition (Definition 1, Cao and Yang 2015) requiring the unlearned model to align with a retrained model on the retain set and diverge on the forget set by epsilon_1.
    The paper builds its Robust Unlearning framework on this standard starting definition and does not justify it beyond citing prior work.
  • ad hoc to paper The pre-unlearning model's output f(x;theta) is the correct reference for determining whether forgotten information has been resurfaced.
    Equation (3) and Definition 4 use f(x;theta) as the attack target and robustness reference; this is a specific modeling choice of this paper's threat model.
  • domain assumption PGD-based optimization is a representative or sufficient probe of the set of all possible perturbations delta_x.
    The paper acknowledges the optimization is non-convex and cannot explore all perturbations (Section 4.2), yet interprets attack failures as strengthening the evidence of robustness.
  • domain assumption For generative tasks, the attack input can be unbounded and only output similarity to the original model's output matters.
    Stated in Section 3.1 and 5.2; this threat model excludes input realism or semantic similarity constraints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RUB: Evaluating Residual Knowledge in Unlearned Models." pith.science (2026). https://pith.science/paper/73WVNQ3P

@misc{pith2026250414798,
  author       = {Pith},
  title        = {Pith review of: RUB: Evaluating Residual Knowledge in Unlearned Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73WVNQ3P}},
  note         = {Machine review of arXiv:2504.14798}
}
read the original abstract

Machine Unlearning (MUL) has emerged as a key mechanism for privacy protection and content regulation, yet current techniques often fail to guarantee the complete removal of sensitive information. While most existing works focus on verifying the execution of unlearning, they overlook the critical question of whether models remain robust against adversarial attempts to recover forgotten knowledge. In this work, we advocate for the principle of Robust Unlearning, which requires models to be both indistinguishable from retrained counterparts and resilient against diverse adversarial threats. To instantiate this principle, we propose a unified benchmark, RUB (Robust Unlearning Benchmark), that systematically evaluates the robustness of unlearning algorithms across classification, image-to-image reconstruction, and text-to-image synthesis. Within this framework, we introduce the Unlearning Mapping Attack (UMA) as a generalizable method to detect residual information, and demonstrate how existing attack strategies can be adapted into this framework as long as they conform to the generic UMA framework. Our experiments across discriminative and generative tasks reveal that state-of-the-art unlearning methods remain vulnerable under these evaluations, even when passing standard verification metrics. By positioning robustness as the central criterion and providing a benchmark for adversarial evaluation, we hope RUB paves the way toward more reliable and secure unlearning practices. The codebase and model checkpoints in RUB will be published.

Figures

Figures reproduced from arXiv: 2504.14798 by the authors.

Figure 1
Figure 1. An illustration of the malicious post-MUL attack. With knowledge of the pre- and post-unlearning models, the attacker at￾tempts to recover forgotten information from the unlearned model by injecting carefully-designed noise into the query. with its practical use of these models. By contrast, gener￾ative models allow for unconstrained δx, as the attack fo￾cuses exclusively on the outputs generated from crafted in￾put… view at source ↗
Figure 2
Figure 2. Unlearning Mapping Attack on image generation unlearning. I2I (Li et al., 2024) unlearning method is tested here. Recon￾structed images are from ImageNet1k dataset. Ground Truth Unlearned Output Masked Attack Input Attacked Output Ground Truth Unlearned Output Masked Attack Input Attacked Output Forget Set Retain Set Masked Input Masked Input [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Unlearning Mapping Attack on image generation unlearning. SalUn (Fan et al., 2024) unlearning method is tested here. Reconstructed images are from ImageNet1k dataset. tion quality. For performance comparison, IIS is higher the better; while lower FID indicates good generation quality. Please refer to the Appendix A.3 for more information. 5.2. Experiment Result In [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Ablation on attack step size. The experiments are done on CIFAR10 using SalUn (Fan et al., 2024) as the baseline unlearning algorithm. All experiments have a fixed number of steps of 100 and an attack strength of 16/255. No Atk 8/255 16/255 UA MIA UA MIA UA MIA FT+vae …
Figure 6
Figure 6. Figure 6: Examples of the generated images using I2I (Li et al., 2024) unlearning methods. Ground truth, I0, Masked Input, Im, Output before Unlearning, I1, Output after Unlearning, I2, UMA Attacked Output, I3, are represented here as discussed in Section A.3 13 [PITH_FULL_IMAG…
Figure 7
Figure 7. Figure 7: Examples of the generated images using SalUn (Fan et al., 2024) unlearning methods. Ground truth, I0, Masked Input, Im, Output before Unlearning, I1, Output after Unlearning, I2, UMA Attacked Output, I3, are represented here as discussed in Section A.3 14 [PITH_FULL_I…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 15 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N

    Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp.\ 141--159. IEEE, 2021

    2021

  3. [3]

    and Yang, J

    Cao, Y. and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, pp.\ 463--480, 2015. doi:10.1109/SP.2015.35

    doi:10.1109/sp.2015.35 2015

  4. [4]

    Membership inference attacks from first principles

    Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tramer, F. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp.\ 1897--1914. IEEE, 2022

    2022

  5. [5]

    Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation

    Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., and Liu, S. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In The Twelfth International Conference on Learning Representations, 2024

    2024

  6. [6]

    Model inversion attacks that exploit confidence information and basic countermeasures

    Fredrikson, M., Jha, S., and Ristenpart, T. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pp.\ 1322--1333, 2015

    work page 2015

  7. [7]

    Verifi: Towards verifiable federated unlearning

    Gao, X., Ma, X., Wang, J., Sun, Y., Li, B., Ji, S., Cheng, P., and Chen, J. Verifi: Towards verifiable federated unlearning. IEEE Transactions on Dependable and Secure Computing, 2024

    work page 2024

  8. [8]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    work page 2020

Show all 44 references
  1. [9]

    Amnesiac machine learning

    Graves, L., Nagisetty, V., and Ganesh, V. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 11516--11524, 2021

    2021

  2. [10]

    Verifying in the dark: Verifiable machine unlearning by using invisible backdoor triggers

    Guo, Y., Zhao, Y., Hou, S., Wang, C., and Jia, X. Verifying in the dark: Verifiable machine unlearning by using invisible backdoor triggers. IEEE Transactions on Information Forensics and Security, 19: 0 708--721, 2024. doi:10.1109/TIFS.2023.3328269

    2024

  3. [11]

    Probing unlearned diffusion models: A transferable adversarial attack perspective

    Han, X., Yang, S., Wang, W., Li, Y., and Dong, J. Probing unlearned diffusion models: A transferable adversarial attack perspective. arXiv preprint arXiv:2404.19382, 2024

    2024 arXiv

  4. [12]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  5. [13]

    A., Chaudhuri, K., and Zou, J

    Izzo, Z., Smart, M. A., Chaudhuri, K., and Zou, J. Approximate data deletion from machine learning models. In International Conference on Artificial Intelligence and Statistics, pp.\ 2008--2016. PMLR, 2021

    2008

  6. [14]

    A., Dullerud, N., Thudi, A., Chandrasekaran, V., and Papernot, N

    Jia, H., Yaghini, M., Choquette-Choo, C. A., Dullerud, N., Thudi, A., Chandrasekaran, V., and Papernot, N. Proof-of-learning: Definitions and practice. In 2021 IEEE Symposium on Security and Privacy (SP), pp.\ 1039--1056. IEEE, 2021

    2021

  7. [15]

    Model sparsity can simplify machine unlearning

    Jia, J., Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., Sharma, P., and Liu, S. Model sparsity can simplify machine unlearning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=0jZH883i34

    2023

  8. [16]

    Koh, P. W. and Liang, P. Understanding black-box predictions via influence functions. In International conference on machine learning, pp.\ 1885--1894. PMLR, 2017

    2017

  9. [17]

    Towards unbounded machine unlearning

    Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. Towards unbounded machine unlearning. Advances in neural information processing systems, 36, 2024

    2024

  10. [18]

    Machine unlearning for image-to-image generative models

    Li, G., Hsu, H., Chen, C.-F., and Marculescu, R. Machine unlearning for image-to-image generative models. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=9hjVoPWPnh

    2024

  11. [19]

    The right to be forgotten in federated learning: An efficient realization with rapid retraining

    Liu, Y., Xu, L., Yuan, X., Wang, C., and Li, B. The right to be forgotten in federated learning: An efficient realization with rapid retraining. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications, pp.\ 1749--1758. IEEE, 2022

    2022

  12. [20]

    Backdoor attacks via machine unlearning

    Liu, Z., Wang, T., Huai, M., and Miao, C. Backdoor attacks via machine unlearning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 14115--14123, 2024

    2024

  13. [21]

    Towards deep learning models resistant to adversarial attacks

    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  14. [22]

    Mehta, R., Pal, S., Singh, V., and Ravi, S. N. Deep unlearning via randomized conditionally independent hessians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10422--10431, 2022

    2022

  15. [23]

    Pardau, S. L. The california consumer privacy act: Towards a european style privacy regime in the united states. 2018

    2018

  16. [24]

    O., Cohen, N., Mittal, G., and Hegde, C

    Pham, M., Marshall, K. O., Cohen, N., Mittal, G., and Hegde, C. Circumventing concept erasure methods for text-to-image generative models. In The Twelfth International Conference on Learning Representations, 2023

    2023

  17. [25]

    Towards understanding and enhancing robustness of deep learning models against malicious unlearning attacks

    Qian, W., Zhao, C., Le, W., Ma, M., and Huai, M. Towards understanding and enhancing robustness of deep learning models against malicious unlearning attacks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 1932--1942, 2023

    1932

  18. [26]

    Regulation, G. D. P. General data protection regulation (gdpr). Intersoft Consulting, Accessed in October, 24 0 (1), 2018

    2018

  19. [27]

    Improved techniques for training gans

    Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016

    2016

  20. [28]

    Membership inference attacks against machine learning models

    Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

    2017

  21. [29]

    A., and Anderson, R

    Shumailov, I., Shumaylov, Z., Kazhdan, D., Zhao, Y., Papernot, N., Erdogdu, M. A., and Anderson, R. J. Manipulating sgd with data ordering attacks. Advances in Neural Information Processing Systems, 34: 0 18021--18032, 2021

    2021

  22. [30]

    Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai

    Shumailov, I., Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M., Yona, I., Howard, H., and Bagdasaryan, E. Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai. arXiv preprint arXiv:2407.00106, 2024

    2024 arXiv

  23. [31]

    M., Song, L., Wagh, S., and Mittal, P

    Sommer, D. M., Song, L., Wagh, S., and Mittal, P. Towards probabilistic verification of machine unlearning. arXiv preprint arXiv:2003.04247, 2020

    2003 arXiv

  24. [32]

    K., Chundawat, V

    Tarun, A. K., Chundawat, V. S., Mandal, M., and Kankanhalli, M. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  25. [33]

    On the necessity of auditable algorithmic definitions for machine unlearning

    Thudi, A., Jia, H., Shumailov, I., and Papernot, N. On the necessity of auditable algorithmic definitions for machine unlearning. In 31st USENIX Security Symposium (USENIX Security 22), pp.\ 4007--4022, 2022

    2022

  26. [34]

    Ring-a-bell! how reliable are concept removal methods for diffusion models? In International Conference on Learning Representations, 2024

    Tsai, Y.-L., Hsu, C.-Y., Xie, C., Lin, C.-H., Chen, J.-Y., Li, B., Chen, P.-Y., Yu, C.-M., and Huang, C.-Y. Ring-a-bell! how reliable are concept removal methods for diffusion models? In International Conference on Learning Representations, 2024

    2024

  27. [35]

    Machine unlearning of features and labels

    Warnecke, A., Pirch, L., Wressnegger, C., and Rieck, K. Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577, 2021

    2021 arXiv

  28. [36]

    Large language model unlearning

    Yao, Y., Xu, X., and Liu, Y. Large language model unlearning. arXiv preprint arXiv:2310.10683, 2023

    2023 arXiv

  29. [37]

    Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models

    Yuan, H., Jin, Z., Cao, P., Chen, Y., Liu, K., and Zhao, J. Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models. arXiv preprint arXiv:2408.10682, 2024

    2024 arXiv

  30. [38]

    Verification of machine unlearning is fragile

    Zhang, B., Chen, Z., Shen, C., and Li, J. Verification of machine unlearning is fragile. In Forty-first International Conference on Machine Learning, 2024

    2024

  31. [39]

    To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images

    Zhang, Y., Jia, J., Chen, X., Chen, A., Zhang, Y., Liu, J., Ding, K., and Liu, S. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now. In European Conference on Computer Vision, pp.\ 385--403. Springer, 2025

    2025

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  34. [42]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  35. [43]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  36. [44]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.