Pith. sign in

REVIEW 3 major objections 5 minor 32 references

Privacy-Aware Lifelong Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PALL claims one network can learn tasks sequentially, then delete any task so exactly that it becomes indistinguishable from a model that never saw that task's data, by resetting the task's subnetwork and replaying the others' memories.

desk verdict PALL is a solid engineering contribution to continual learning with unlearning, but the 'exact' guarantee is asserted rather than proven, and the stress-test concern about later-task masks and initializations is on point. read the letter →

arxiv 2505.10941 v1 pith:SF3MZT5R submitted 2025-05-16 cs.LG

Ozan Özdenizci , Elmar Rueckert , Robert Legenstein This is my paper · ORCID
classification cs.LG
keywords privacy-awarelifelonglearningexacttaskunlearningtask-incrementalsparsesubnetworksknowledgetransfercatastrophicforgettingmachineepisodicmemoryrehearsal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single fixed-capacity neural network can do lifelong learning and machine unlearning at the same time: it acquires tasks one after another without forgetting, and on request it removes a task's influence so completely that the resulting model behaves as if that task's data had never been seen. Its method, PALL, gives each task a sparse subnetwork inside one architecture, selected by learned importance scores on top of a frozen set of previously trained parameters, so the task's information is confined to a known slice of the network. Unlearning resets that slice to its initial random values and briefly retrains the few shared parameters that later tasks borrowed, using stored exemplars and logits. The empirical case is that this yields exact unlearning by construction while matching or beating independent per-task models at a fraction of the memory cost. If the guarantee holds, PALL would give a practical route from right-to-be-forgotten regulations to deployed sequential learners without per-task model copies or full retraining.

What carries the argument

The load-bearing object is the task-specific binary subnetwork mask $m_t \in \{0,1\}^d$, built at a per-layer connectivity rate $\alpha$ by keeping the largest $|s_t|$ entries of a learned importance-score vector $s_t$, trained by a straight-through estimator while parameter updates are masked by the cumulative cover $M_{i-1}$ of all previous tasks. This mask is what claims to confine a task's information to one identifiable slice of the network. The second mechanism is the reset-and-replay unlearning operator: reinitialize $\theta \odot m_t$ from $\varphi(\cdot)$ and retrain only the affected shared parameters — those in $m_t \cap m_\tau$ for later tasks $\tau > t$ — for $N_f$ iterations on episodic buffers of exemplars and stored logits, following the rehearsal objective of Eq. (6) with $\beta = 0.5$. The third is bookkeeping: the union mask $M_i = \bigvee_{j \in \Omega_i} m_j$ plus the reinitialization of all unused parameters after each learning request are what the argument relies on to stop information from leaking outside the designated subnetwork.

What would settle it

Train PALL on task A then task B, unlearn A, and run a from-scratch model on B alone with the same algorithm; if any input-dependent test — a membership-inference classifier over A's data, a linear probe on intermediate activations, or a distributional distance between output logits — tells the unlearned model apart from the from-scratch model at better than chance, the exact-unlearning guarantee of Eq. (1) fails. A cheaper first check: compute whether the reinitialized-and-retrained parameters $\theta \odot m_A$ respond to gradients from $A$'s data more than fresh samples from $\varphi(\cdot)$ do, which would indicate residual information in the 'deleted' weights.

Watch

Extended reading notes

Core claim

The central discovery, stated on the paper's own terms, is that exact task unlearning can be a design property of the learning algorithm rather than a post-hoc repair. Because each task's gradient updates are masked to touch only the parameters not yet used by earlier tasks, and because all unused parameters are reinitialized after every learning request, the paper argues that everything learned from task $t$ resides in the parameters $\theta \odot m_t$ selected by that task's binary mask. An unlearning request for $t$ therefore resets $\theta \odot m_t$ to fresh values from the initialization distribution $\varphi(\cdot)$, deletes the task's mask and memory buffer, and runs $N_f$ rehearsal iterations over the small intersection of $m_t$ with later tasks' masks, which are the only parameters whose reset could hurt surviving tasks. The result, the paper argues, is a model distributed as $L(\theta_0, D_{[\tau \in \Omega_i]})$ — what a learner that never saw $D_t$ would have produced — so an adversary cannot distinguish it from a from-scratch model. Empirically this yields chance-level accuracy on unlearned tasks, zero catastrophic forgetting ($F_l = 0$) across all settings, accuracy that matches or exceeds the independent-model baseline in the longest scenarios (86.11 versus 86.80 on 100-task TinyImageNet), and a 4.3$\times$ to 12$\times$ smaller inference model size than training separate models per task.

Load-bearing premise

The whole guarantee rests on one assumption: everything a forgotten task taught the network stays inside its own subnetwork's parameters, so resetting those parameters and briefly retraining the few shared ones truly reproduces the model that would have existed without that task — and the paper offers no proof that later tasks, which were built on top of the forgotten task's weights, carry no lingering trace of it.

Editorial extensions

If this is right

  • An unlearned task's test accuracy drops to chance immediately after the reset, giving a cheap operational signal (the paper's $A_u$ metric) that deletion happened; the paper stresses the real guarantee is by design, not by audit.
  • Because only 1–4% of the network's parameters need retraining per unlearning and a request completes in 1–3 seconds on the tested setups, exact unlearning becomes cheap enough to invoke repeatedly in adaptive sequences — the paper stresses up to 50 interleaved requests over 100 tasks without collapse.
  • Inference cost stays at one base model plus one binary mask per surviving task (about 0.69 MB added per task on the 100-task ViT experiment), where the independent-model alternative adds a full network per task and reaches roughly 2 GB.
  • As the connectivity rate $\alpha$ shrinks, the method still functions with very small task subnetworks (54K parameters per task at 99% sparsity) and outperforms all independent-subnetwork variants in the 40- and 100-task comparisons, which is what makes the scheme usable when the number of tasks is large or unknown.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exactness guarantee inherits the confinement assumption, and that assumption is testable: later tasks' masks and optimization paths start from weights trained on the forgotten task, so an adversarial audit — a classifier trying to tell the unlearned model apart from a from-scratch model trained only on the surviving tasks — would be a sharper check than the chance-level $A_u$ the paper reports
  • The same mask bookkeeping suggests a route to finer-grained deletion: if per-example parameter effects could be separated inside a subnetwork, sample-level right-to-be-forgotten requests would follow; as designed, two samples inside one task cannot be told apart, so PALL's granularity is inherently the whole task.
  • In open-ended task streams with unknown $T$, the fixed connectivity $\alpha$ trades capacity against forgetting resolution; an adaptive $\alpha$ that grows subnetworks when unlearning frees parameters is a direct, testable extension the paper leaves open.
  • The paper's check that retrained parameters differ from their pre-unlearning values (mean absolute differences around 0.002–0.02) shows the weights moved, but not that they carry no exploitable trace of $D_t$; probing the reset-then-retrained weights with $D_t$'s gradients or a linear probe would settle whether deletion is truly complete.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. PALL is a task-incremental lifelong learning algorithm that trains task-specific sparse subnetworks in a single fixed-capacity network, freezes used parameters to prevent forgetting, permits forward transfer by sharing frozen parameters, and upon a task-unlearning request resets the subnetwork parameters optimized on that task and retrains shared parameters using an episodic buffer. The paper defines exact task unlearning through distributional equality with from-scratch training on the remaining tasks (Eq. 1), claims this guarantee by design, and reports experiments on sequential CIFAR and TinyImageNet with ResNet and ViT architectures, comparing with continual learning and unlearning baselines.

Significance. If the exact-unlearning guarantee were established, this would be a notable advance because it would combine privacy-aware unlearning with fixed-capacity lifelong learning, a combination that is largely absent from the existing literature. The paper's empirical strengths are substantial: 20-seed repetitions, multiple architectures, ablations on memory buffer size and retraining duration, worst-case metrics, training-time comparisons, and released code. The no-forgetting claim (Fl = 0) follows structurally from parameter freezing and is credible. However, the central exact-unlearning claim is not supported by the algorithm as written or by the empirical protocol; the paper's own caveat in Section 4.2 that chance-level Au does not necessarily correspond to exact unlearning, and that a privacy audit is left for future work, underlines this gap.

major comments (3)
  1. [Section 3.3, Eq. (1), Algorithm 1 lines 18-30] Eq. (1) requires U(theta_{i-1}, t) and L(theta0, D_{<=t} \ D_t) to be equal in distribution. For any later task tau > t, the mask m_tau is obtained by optimizing scores s_tau (Eqs. 4-5) on frozen shared parameters inherited from the D_t-trained model, and the private parameters of tau are optimized from that same initialization. The unlearning branch resets theta ⊙ m_t and runs Nf iterations of Eq. (6), but it does not resample m_tau or retrain the later-task private parameters from a D_t-free starting point. Consequently, the output of U is not distributionally equal to a fresh run of L on the remaining tasks unless the later masks and private parameters are independent of D_t, which is exactly the proposition that forward transfer denies. No proof of this independence is supplied, so the 'exact unlearning guarantees by design' claim in the Abstract and Section 1 is not established.
  2. [Section 4.2 and Table 1] The empirical evaluation does not measure the distributional condition in Eq. (1). Table 1 lists 'Exact' in the Au column for PALL but reports no Au values, and Section 4.2 explicitly states that chance-level Au does not necessarily correspond to exact unlearning and that a privacy audit is left for future work. Since the exact-unlearning guarantee is the paper's load-bearing differentiator, the paper should either provide a concrete test of distributional closeness to the from-scratch retrained model or explicitly downgrade the claim to approximate unlearning and revise the abstract, title, and contribution statements accordingly.
  3. [Section 3.3, paragraph beginning 'Given a task unlearning request'] The statement 'We can facilitate this exactly by resetting the parameters theta_{i-1} ⊙ m_t' is circular as a justification of exactness: exactness is precisely the property to be proven, not a consequence of resetting. The surrounding argument also relies on two unstated assumptions: (i) all influence of D_t is confined to the parameters directly optimized on D_t, so resetting them removes the task's influence; and (ii) the Nf-step rehearsal in Eq. (6) on other tasks' memory buffers cannot reintroduce task-t information through shared or later-task parameters. These assumptions need to be stated explicitly and argued for, or the exactness claim must be weakened.
minor comments (5)
  1. [Section 3.3, notation] The sentence introducing the submask, 'We indicate the parameter indices which are specifically trained using data from Dt with the submask mt, and mt - mt correspond to the shared, frozen parameter indices from previous tasks', uses the same symbol for the full mask and the trained-only part; please introduce a distinct notation, for example n_t, for the trained-only submask.
  2. [Eq. (6)] In Eq. (6), the first expectation draws (x,y,z) from B_tau but only (x,y) is used; either write (x,y) for the first term or define the notation so that the logit z is used only in the second term.
  3. [Algorithm 1, line 21] The instruction 'mt ← Retrieve the submask of mt indicating the params trained via Dt or Bt' is confusing because both sides appear identical; please clarify which stored mask is being read and how the learned submask is identified relative to the full task mask.
  4. [Table 1] The Au column for PALL says 'Exact' rather than reporting a number; please report the observed Au values so the reader can at least check the weak empirical signal that unlearned tasks yield chance-level accuracy.
  5. [Section 2.2 and Section 3.2] The paper cites the adaptive unlearning setting of Gupta et al. (2021) in related work but does not discuss how PALL behaves under adaptively chosen unlearning requests; a brief statement on this point would help position the contribution.

Circularity Check

2 steps flagged · score 6.0 of 10

Exact-unlearning guarantee is asserted by identifying parameter reset with distributional equality, and Table 1 labels PALL 'Exact' via the disclaimed Au proxy.

  1. self definitional [Section 3.1 Eq. (1); Section 3.3 unlearning paragraph; Algorithm 1 lines 18-30]
    "Given a task unlearning request Ri = (t, U), our goal is to update the model parameters θi−1, such that the new model does not possess any information about Dt, i.e., none of its parameters have been optimized with the data observed from task t. We can facilitate this exactly by resetting the parameters θi−1 ⊙ mt, by sampling new values from the initialization distribution ϕ(.)."

    Eq. (1) defines exact task unlearning distributionally: U(θt, τ) =p L(θ0, D≤t∖Dτ). The Section 3.3 argument instead takes 'none of its parameters have been optimized with Dt' as the content of exact unlearning and then asserts that resetting θi−1⊙mt realizes it. These are not equivalent: masks and private parameters of later tasks were optimized using shared weights that were themselves trained on Dt, so after resetting only θi−1⊙mt the model is not in general distributed as L(θ0, D≤t∖Dτ). The claimed 'by design' guarantee is therefore the operational resetting criterion restated, not a consequence of Eq. (1); the distributional requirement is never derived.

  2. renaming known result [Section 4.2 Evaluation Metrics; Section 5.1 Table 1 discussion]
    "Our method satisfies exact unlearning (i.e., random classification Au on unlearned tasks), no catastrophic forgetting (Fl = 0), and achieves Al metrics very close to, or higher than training independent models with exact unlearning guarantees."

    The paper's own Section 4.2 says 'a lower Au does not necessarily correspond to an exact unlearning guarantee. We include Au only to evaluate inexact unlearning baselines through a weak measure.' Yet the only evidence presented for PALL's exactness in Table 1 is the label 'Exact' together with this identification of exact unlearning with chance-level Au. The empirical validation thus substitutes a proxy that the paper itself disclaims for the formal property in Eq. (1); the table entry 'Exact' is the guarantee-by-construction claim restated rather than a measured or externally audited outcome.

full rationale

The non-circular parts of the paper are substantial: task-learning accuracy (Al), forgetting (Fl, Fu), runtime, and memory comparisons against Independent, PackNet, WSN, ER, DER++ and other baselines are genuine benchmark results and do not reduce to the paper's inputs. No load-bearing self-citation chain or imported uniqueness theorem is present; WSN and DER++ are external prior work. The circularity is confined to the central privacy claim. Section 3.3 reduces exact unlearning to resetting the parameters identified with Dt, and Section 5.1/Table 1 then treat random classification on unlearned tasks as satisfying the formal Eq. (1) definition, despite Section 4.2's explicit caveat that Au does not imply an exact guarantee. The headline 'exact task unlearning guarantees by design' is therefore the operational resetting procedure renamed as a guarantee, rather than a demonstrated instance of Eq. (1).

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method depends on a small number of hyperparameters (α, β, Nf, buffer size) selected by design or light ablations; they are listed explicitly. The strongest load-bearing axiom is the confinement assumption: that a task's data influences only the parameters directly trained on it, so resetting them suffices for exact unlearning under Eq. (1). This axiom is ad hoc to the paper and not proven. The task-incremental assumption (task ID known) is stated and scoped. No invented entities are introduced; the subnetwork masks and memory buffers are algorithmic structures.

free parameters (4)
  • alpha (subnetwork connectivity rate) = 1/T default (variants 0.01 to 0.2)
    Controls the fraction of parameters per task subnetwork. Chosen as 1/T based on the number of tasks, not tuned on the target metric; ablations show robustness across 0.01-0.2.
  • beta (rehearsal regularization weight) = 0.5
    DER++ weight for logit distillation in the retraining objective Eq. (6); selected via ablation in Appendix A.3.2.
  • Nf (retraining iterations after unlearning) = 50 (10 for long-horizon experiments)
    Number of gradient steps in Eq. (6); ablation shows Nf=50 better than Nf=0 and almost as good as Nf=100.
  • episodic memory buffer size = 500 (S-CIFAR), 1000 (S-TinyImageNet)
    Total exemplars stored for rehearsal; larger buffers improve Al at the cost of memory; chosen by design.
assumptions (4)
  • domain assumption Task-incremental setting: task IDs are available to the learner at train and test time.
    The method relies on task IDs to select the subnetwork mask; class- and domain-incremental settings are excluded (Section 6).
  • ad hoc to paper All information from a task's data is contained in the parameters directly optimized on that task, so resetting them removes the task's influence.
    Central to the exact unlearning claim (Sec 3.3, Eq. 5 and Algorithm 1 lines 18-30); not proven and questionable under forward transfer because later tasks' masks and optimization trajectories depend on shared weights trained on the unlearned task.
  • ad hoc to paper Rehearsal retraining on other tasks' episodic memory buffers does not reintroduce information about the unlearned task.
    Relies on buffers of remaining tasks containing no samples from D_t, which holds by construction, but the retrained parameters' distribution is not shown to match a from-scratch model.
  • standard math Straight-through estimator for binarized mask optimization is a valid gradient approximation.
    Standard technique from prior work (Ramanujan et al. 2020); used in Eq. (4)-(5).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Privacy-Aware Lifelong Learning." pith.science (2026). https://pith.science/paper/SF3MZT5R

@misc{pith2026250510941,
  author       = {Pith},
  title        = {Pith review of: Privacy-Aware Lifelong Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SF3MZT5R}},
  note         = {Machine review of arXiv:2505.10941}
}
read the original abstract

Lifelong learning algorithms enable models to incrementally acquire new knowledge without forgetting previously learned information. Contrarily, the field of machine unlearning focuses on explicitly forgetting certain previous knowledge from pretrained models when requested, in order to comply with data privacy regulations on the right-to-be-forgotten. Enabling efficient lifelong learning with the capability to selectively unlearn sensitive information from models presents a critical and largely unaddressed challenge with contradicting objectives. We address this problem from the perspective of simultaneously preventing catastrophic forgetting and allowing forward knowledge transfer during task-incremental learning, while ensuring exact task unlearning and minimizing memory requirements, based on a single neural network model to be adapted. Our proposed solution, privacy-aware lifelong learning (PALL), involves optimization of task-specific sparse subnetworks with parameter sharing within a single architecture. We additionally utilize an episodic memory rehearsal mechanism to facilitate exact unlearning without performance degradations. We empirically demonstrate the scalability of PALL across various architectures in image classification, and provide a state-of-the-art solution that uniquely integrates lifelong learning and privacy-aware unlearning mechanisms for responsible AI applications.

Figures

Figures reproduced from arXiv: 2505.10941 by the authors.

Figure 1
Figure 1. Illustration of PALL. Task-specific subnetworks obtained after learning are indicated by [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Evaluations with different number of unlearning requests [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages

  1. [3]

    Results are averaged over 20 random seeds. Episodic MemoryPT t=1|Bt| Method Exact Unlearning S-CIFAR10 S-CIFAR100 Al↑ F l↓ Fu↓ Al↑ F l↓ Fu↓ − Sequential ✗ 70.71 13.86 0.0 35.35 13.43 0.0 200 ER ✗ 84.09 4.83 2.56 50.08 6.69 8.14 DER++ ✗ 89.23 3.09 1.03 59.82 6.99 1.43 PALL(β = 0.0) ✓ 93.91 0.0 2.36 71.68 0.0 0.83 PALL(β = 0.5) ✓ 93.95 0.0 0.67 71.90 0.0 0....

    work page 2018

  2. [4]

    On tiny episodic memories in continual learning

    18 Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc’Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486,

    arXiv 1902

  3. [7]

    Safe: Machine unlearning with shard graphs

    15 Yonatan Dukler, Benjamin Bowman, Alessandro Achille, Aditya Golatkar, Ashwin Swami- nathan, and Stefano Soatto. Safe: Machine unlearning with shard graphs. arXiv preprint arXiv:2304.13169,

    arXiv

  4. [8]

    Making AI forget you: Data deletion in machine learning

    3 11 Published as a conference paper at ICLR 2025 Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making AI forget you: Data deletion in machine learning. Advances in Neural Information Processing Systems, 32,

    work page 2025

  5. [9]

    Towards adversarial evaluations for inexact machine unlearning

    3 Shashwat Goel, Ameya Prabhu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640,

    arXiv

  6. [10]

    Continual learning via neural pruning.arXiv preprint arXiv:1903.04476,

    3 Siavash Golkar, Michael Kagan, and Kyunghyun Cho. Continual learning via neural pruning.arXiv preprint arXiv:1903.04476,

    arXiv 1903

  7. [13]

    icarl: Incremental classifier and representation learning

    5, 10 Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001–2010,

    work page 2001

  8. [14]

    Progressive neural networks

    2 Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671,

    arXiv

Show all 32 references
  1. [15]

    Membership inference at- tacks against machine learning models

    2 Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18. IEEE,

    2017

  2. [18]

    A comprehensive survey of forgetting in deep learning beyond continual learning

    18 13 Published as a conference paper at ICLR 2025 Zhenyi Wang, Enneng Yang, Li Shen, and Heng Huang. A comprehensive survey of forgetting in deep learning beyond continual learning. arXiv preprint arXiv:2307.09218,

    2025 arXiv

  3. [19]

    Lifelong learning with dynamically expandable networks

    3 Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547,

    arXiv

  4. [20]

    Legonet: A fast and exact unlearning architecture

    3 Sihao Yu, Fei Sun, Jiafeng Guo, Ruqing Zhang, and Xueqi Cheng. Legonet: A fast and exact unlearning architecture. arXiv preprint arXiv:2210.16023,

    arXiv

  5. [21]

    To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods

    2 Dawen Zhang, Shidong Pan, Thong Hoang, Zhenchang Xing, Mark Staples, Xiwei Xu, Lina Yao, Qinghua Lu, and Liming Zhu. To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods. arXiv preprint arXiv:2302.03350,

    arXiv

  6. [22]

    3 14 Published as a conference paper at ICLR 2025 A A PPENDIX A.1 E XPERIMENTAL SETUP Datasets: We experimented with S-CIFAR10: 5 tasks × 2 classes, S-CIFAR100: 10 tasks × 10 classes (Krizhevsky, 2009), and S-TinyImageNet in various sequential configurations: 20 tasks × 10 cla...

    2025

  7. [23]

    We did not have trainable parameters in the batch-norm or layer-norm layers of these models

    based on the ViT-T/8 specifications from (Steiner et al., 2022), consisting of 12 encoder layers with 3 heads, a model width of 192, an MLP dimensionality of 768, and a patch size of 8x8, resulting in 5,408,064 trainable parameters. We did not have trainable parameters in the ...

    2022

  8. [24]

    regularization parameter of 1.0 and temperature parameter of 2.0. For learning with selective forgetting (LSF) (Shibata et al., 2021), we also use a combination of EWC and LwF for learning as performed in the original work, with the default LSF regularization weight of

    2021

  9. [25]

    Baseline methods LwF and LSF both perform a preliminary classifier layer training phase at the be- ginning of task learning, which is then followed by complete model training. We also implemented these following the original methods, by splitting the number of epochs into two ...

    2025

  10. [26]

    Note that we keep the final classifier layer structure of the models hardwired based on the associated class neurons, such that there was no score optimization performed

    for ϕ(.) while initializing the model parameters and importance scores at the beginning of training, as well as during unlearning-related parameter and 16 Published as a conference paper at ICLR 2025 score resetting steps. Note that we keep the final classifier layer structure...

    2025

  11. [27]

    Therefore, a mem- ory buffer and parameter retraining is necessary to balance exact unlearn- ing

    does not perform well (i.e., Al ↑ /F max u ↓: 93.35/3.41 for S-CIFAR10, and 70.24/5.31 for S-CIFAR100). Therefore, a mem- ory buffer and parameter retraining is necessary to balance exact unlearn- ing. We could also achieve better per- formances after unlearning by using longe...

    2017

  12. [29]

    method upon task unlearning requests. In our main experiments, ER was adapted to perform approximate unlearning, by finetun- ing model parameters using episodic memories towards misclassification on the task to be unlearned. In this new setting, for any unlearning request, we ...

    2025

  13. [31]

    ER (Retraining)

    Al↑ Au↓ Fl↓ Fu↓ Al↑ Au↓ Fl↓ Fu↓ Independent 95.19 Exact 0.0 0.0 73.22 Exact 0.0 0.0 PALL(α= 1/T) 94.34 Exact 0.0 0.60 72.35 Exact 0.0 0.40 ER (Retraining) 78.80Exact 2.05 7.53 34.50 Exact 4.07 20.40 ER 87.88 58.48 3.45 1.61 57.63 42.69 4.67 7.91 Table A5 show that this approac...

    2025

  14. [32]

    based on the models from Table A6. We did not in- clude DER++, since the training time of this algorithm was too far out of the range from the other methods, and also rapidly increased with the number of tasks (e.g., 3.04 sec/epoch when 10 tasks are learned, 6.19 sec/epoch for...

    2025

  15. [100]

    ExactUnlearning Nu= 3 Nu= 5 Nu= 10 Nu= 20 Nu= 50 Al Fl Fu Al Fl Fu Al Fl Fu Al Fl Fu Al Fl Fu Sequential ✗ 52.46 1.71 0.0 51.86 1.72 0.0 53.01 1.73 0.0 54.15 1.73 0.0 51.91 1.80 0.0EWC ✗ 79.53 0.43 0.0 79.57 0.43 0.0 79.53 0.43 0.0 79.59 0.43 0.0 79.17 0.44 0.0DER++ ✗ 76.58 0....

  16. [2015]

    A unified framework for continual learning and machine unlearning

    1, 3 Romit Chatterjee, Vikram Chundawat, Ayush Tarun, Ankur Mali, and Murari Mandal. A unified framework for continual learning and machine unlearning. arXiv preprint arXiv:2408.11374 ,

    arXiv

  17. [2017]

    Continual learning and private unlearning

    2, 6, 15 12 Published as a conference paper at ICLR 2025 Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pp. 243–254. PMLR,

    2025

  18. [2018]

    Machine unlearning

    2 Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159. IEEE,

    2021

  19. [2019]

    When machine unlearning jeopardizes privacy

    2, 6, 7, 16, 18 Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC Con- ference on Computer and Communications Security, pp. 896–911,

    2021

  20. [2020]

    Towards making systems forget with machine unlearning

    2, 6, 7, 9, 10, 16, 18 Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy (SP), pp. 463–480. IEEE,

    2015

  21. [2021]

    Inexact unlearning needs more careful evaluations to avoid a false sense of privacy

    3 Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. arXiv preprint arXiv:2403.01218,

    arXiv

  22. [2022]

    Continual learning and catas- trophic forgetting

    1, 2 Gido M van de Ven, Nicholas Soures, and Dhireesha Kudithipudi. Continual learning and catas- trophic forgetting. arXiv preprint arXiv:2403.05175,

    arXiv

  23. [2023]

    An image is worth 16x16 words: Transformers for image recognition at scale

    3 Alexey Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

    2010 arXiv

  24. [2024]

    Continual learning with hypernetworks

    2 Johannes V on Oswald, Christian Henning, Benjamin F Grewe, and Jo ˜ao Sacramento. Continual learning with hypernetworks. arXiv preprint arXiv:1906.00695,

    1906 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.