{"id":"9a564d4b-125d-44ad-8116-6526bc86f535","arxiv_id":"2505.12239","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper derives a closed-form recursive update that removes the influence of forgotten samples from a linear classifier exactly, yielding a model identical to retraining on the remaining data.","lead":"This paper introduces a method for exactly removing the influence of specific training samples from a continually learned model, without storing or revisiting the original training data. Machine learning services that must honor user deletion requests could use it, if the model is built from a frozen pre-trained backbone and a linear classifier.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Exactness is only w.r.t. the linear head on a frozen backbone; if the pre-trained backbone has seen the forgotten samples, ACU does not erase that knowledge, so the 'exact forgetting' claim is scoped too broadly.","rationale":"The reader's verdict and weakest-assumption analysis are sound. I verified the core algebra: with T_{i-1}^{-1} equal to the retained Gram matrix plus gamma I and T_i^{-1}=T_{i-1}^{-1}-F_i^T F_i, the Woodbury update (5) is correct, and Theorem 1 follows; the appendix's swapped ˇD_i/ˆD_i symbols in Eqs. (43)-(44) are typos, not a broken derivation. The experiments' zero gaps are consistent with the identity. The one place where the central claim can be over-read is the target of exactness. Equation (2) optimizes only over W, and Section 3.1 fixes the backbone, so 'retrained on the retained set' actually means retraining the linear classifier on the retained set with the unchanged pre-trained features. If the pre-trained backbone itself was exposed to the forget data, the final model still contains that knowledge, and the privacy claim is too strong. Section 5 acknowledges the limitation but counters with an unsupported assertion about certified privacy guarantees in public PTMs, and Appendix E.1 avoids the issue by using disjoint base/CL partitions. Because the paper discloses the condition, the appropriate verdict stays conditional: the contribution is a clean head-only exact unlearning method for analytic CL with a trusted pre-trained backbone, not general exact forgetting for any PTM-based continual learner. No change to the reader's CONDITIONAL verdict is needed; the abstract and title should be reworded to state the condition explicitly.","tokens_in":21615,"tokens_out":17105,"duration_ms":173052,"concrete_test":"Pre-train or fine-tune the feature backbone on the same samples that will later be designated as the forget set (e.g., train a CIFAR-10 backbone on CIFAR-10 training images, then select some of those images as ˇD). Run the ACU CL/CU pipeline and compare the final model against the true from-scratch retrained model, retraining both backbone and head only on the retained set, using parameter distance and MIA on ˇD. If the distance or MIA is materially nonzero, ACU does not achieve exact forgetting in the standard from-scratch sense; it only matches a head-only retrained model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theorem (Theorem 1, Section 3.4) is a correct algebraic identity: given W0 and T0 from ridge regression (Eqs. 3 and 4) on fixed features, the recursive updates (Eqs. 5 and 6) reproduce the classifier retrained on the retained set. The load-bearing gap is not in the algebra but in what the target represents. The target Wi is retrained only on the retained set on top of a frozen backbone; it is not a model retrained from scratch on the retained data. Section 5 explicitly acknowledges that ACU does not unlearn knowledge acquired during pre-training, and Section 3.1 assumes the backbone is never modified. However, the abstract and title claim 'exact forgetting' with privacy preservation, which extends further. If an open-source pre-trained backbone was trained on data overlapping the current forget set, the backbone still encodes the forgotten samples after ACU. Section 5's assertion that public PTMs come with 'certified security and privacy guarantees' is unsupported. Appendix E.1 deliberately uses disjoint base and CL partitions, so the experiments never expose this failure mode. The exactness result is therefore conditional not only on a frozen backbone but also on that backbone having no memory of the forgotten data; otherwise ACU removes only the head's contribution, and the privacy-promise framing is unmet.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Continual Unlearning (CU), a setting in which a model produced by a continual-learning phase must sequentially remove the influence of designated samples, with access only to the forgotten samples and not to the retained set. The proposed method, Analytic Continual Unlearning (ACU), assumes a frozen pre-trained feature extractor and a linear analytic classifier trained by ridge regression. ACU maintains a d_F x d_F Knowledge Tracking Matrix T_i = (sum over retained set of f^T f + gamma I)^{-1}, which is updated by a Woodbury-type recursion using only the forgetting features, together with the recursive model update W_i = (I + T_i sum over forget set of f^T f) W_{i-1} - T_i sum over forget set of f^T y (Eqs. (5)-(6)). Theorem 1 claims that this update exactly reproduces the ridge solution retrained on the retained set. Experiments on CIFAR-10 and CIFAR-100 compare ACU with Finetuning, SCRUB, NegGrad, WoodFisher, RandomLabel, and others on parameter, accuracy, MIA, and runtime gaps, reporting zero deltas for ACU across all metrics.","tokens_in":21888,"tokens_out":9959,"duration_ms":99417,"significance":"The central algebraic identity is correct and useful: for a fixed feature map, ACU gives an exact, closed-form, sequential decremental update that needs neither retained data nor gradient iteration, with O(d_F^3 + |forget set| d_F^2) per-request cost and O(d_F^2) memory. If the stated assumptions hold, this is a clean result for the analytic-CL community and a meaningful step toward efficient unlearning services. The paper is also commendable for separating the recursive update from the oracle re-trained target, which makes the exactness claim falsifiable by direct comparison, and for reporting cumulative efficiency under 25-50 requests. The main caveats are that the exactness statement is scoped to the linear head on a frozen backbone, and that the privacy-preservation claims go beyond what is actually proven.","major_comments":[{"comment":"The title and abstract claim 'exact forgetting' and 'privacy preservation' without qualification, but Theorem 1 and Eq. (2) concern only the linear analytic classifier on a frozen pre-trained backbone. Section 5 acknowledges that ACU does not unlearn knowledge acquired during pre-training and instead asserts that public pre-trained models come with 'certified security and privacy guarantees'; no citation or argument supports this assertion, and a backbone trained on data overlapping the forget set will retain that information after ACU. Please qualify the claims, e.g., 'exact unlearning of the analytic head on a frozen backbone,' and either remove or support the certified-guarantee sentence.","section":"Abstract, Section 3.1, Section 5"},{"comment":"The statement that the Knowledge Tracking Matrix has rank R <= d_F << N and therefore 'cannot be inverted to recover the original dataset, thereby preserving historical data privacy' is an informal privacy claim. For gamma > 0, T_i is nonsingular, so its rank is d_F; the compression is in the dimension of the representation, not in rank. Whether T_i leaks membership or features depends on the feature map, the attacker's auxiliary information, and the privacy definition. As written, the privacy guarantee is not established. Please provide a formal privacy model and proof, or weaken the statement to say that ACU does not store raw samples and does not require access to the retained set.","section":"Section 3.2"},{"comment":"The experimental setting is ambiguous on a load-bearing point: Appendix E.1 states that 'both the pre-trained base model and the original model are trained for 300 epochs using the SGD optimizer,' but Theorem 1 requires the initial W_0 to be exactly the ridge solution in Eq. (3). If W_0 is obtained by SGD, the ACU update is not guaranteed to match the re-trained model, and the zero gaps in Table 1 need an explanation. If W_0 is instead the analytic solution, this should be stated explicitly. In addition, if the 'optimal re-trained model' in Table 1 is computed by the same closed form (8), the zero deltas are a verification of the algebra rather than an independent empirical test; please clarify the construction of both models.","section":"Appendix E.1, Table 1"}],"minor_comments":[{"comment":"In the inductive step from i to i+1, the sums over forgotten features are written over the i-th forgetting set instead of the (i+1)-th forgetting set; the final equality is correct only after changing the index from D_check_i to D_check_{i+1} throughout that display.","section":"Appendix A, Lemma 3, Eq. (36)"},{"comment":"The identity F_check_i^T F_check_i = sum_{j in D_hat_i} f_j^T f_j is incorrect; the sum should be over the forgetting set D_check_i. The same wrong retained-set index appears again in the final line of Eq. (44), where the theorem statement and Eq. (6) use D_check_i. Please correct both occurrences and re-check all subsequent set indices.","section":"Appendix A, Theorem 1 proof, Eqs. (43)-(44)"},{"comment":"The word 'reversible' should be 'invertible' in the statement of the Woodbury identity, and the sentence 'So (18) can be rewritten as' appears to refer to the wrong displayed equation; the intended manipulation concerns Eq. (22).","section":"Appendix A, Lemma 2"},{"comment":"The stated computational complexities for W_i and T_i appear to be swapped: the term O(d_F^3 + n d_F^2 + n^2 d_F + n^3) corresponds to the Woodbury update of T_i, while the update of W_i has cost O(d_F^3 + n d_F^2 + n d_C d_F + d_C d_F), with n = |D_check_i|. Please correct the assignment.","section":"Appendix B"},{"comment":"The recursive formula for the CL phase uses the symbol F_check_i in the update, but the learning set is denoted D_tilde_p and the feature matrix F_tilde_p; please use a consistent subscript so the recursion is over p.","section":"Appendix C, Eq. (45)"}],"recommendation":"major_revision","confidential_remarks":"The reported zero deltas for ACU in Table 1 are exact by construction only under the analytic-W0 assumption, so the paper should make explicit that this is a sanity check of Theorem 1 rather than an independent empirical validation. The baseline comparison is also tilted: most baselines are approximate unlearning methods designed for a different setting, and several are allowed to access the retained data, while ACU is exact for a narrower model class. This may be acceptable for an efficiency and feasibility study, but the framing should acknowledge it. The Section 5 claim about 'certified security and privacy guarantees' of public pre-trained models should be checked carefully before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the algebraic core is correct but it is a standard ridge-regression downdate. What's new is the packaging: a Continual Unlearning problem for PTM-based continual learning, with a clean recursive scheme that removes samples from the analytic classifier's normal equations without touching the retained set. The authors deserve credit for stating the problem and for recognizing that Woodbury does the job. The theorem's equivalence is derived from first principles and no fitted parameters enter it; that's real.\n\nWhere it gets soft: 'exact forgetting' is exact only with respect to the linear head on a frozen backbone. If the pre-trained backbone has seen the forgotten samples, ACU does not erase that knowledge. The paper admits this in Section 5, but the abstract and title sell it as exact forgetting with privacy preservation, and Section 5's appeal to 'certified security and privacy guarantees' of public PTMs is unsupported. The appendix proof also has visible slips: Eq. (36) sums over the wrong request and Eq. (43) uses the retained set for F_i^T F_i instead of the forgetting set. Both are fixable; the final identity is a known Woodbury application. But they need fixing before publication. The experiments are what you'd expect from an exact identity: ACU's zero gaps are automatic, not empirical evidence, and the paper gives no code, so reproducibility rests on the formulas. The privacy claim for the Knowledge Tracking Matrix is informal—low rank is not a formal privacy guarantee.\n\nWho gets value: people working on machine unlearning for PTM-based CL, and analytic learning folks. It is a subfield-level contribution, not a breakthrough, but it is a legitimate one. Recommendation: send it to peer review; ask the authors to correct the appendix errors, release code, and reframe the privacy claim as head-only unlearning conditional on a clean backbone.","headline":"A correct but narrow exact-unlearning identity for frozen-feature ridge classifiers, packaged as a new problem; referee it, but require the authors to fix the appendix algebra and scale back the backbone-unlearning claims.","tokens_in":22413,"tokens_out":2375,"would_cite":true,"duration_ms":24968,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a continual-learning classifier with a frozen pre-trained feature extractor can erase specific training data exactly, using only the forgotten samples, via a recursive closed-form update proven equivalent to…","keywords":["machine unlearning","continual unlearning","continual learning","analytic learning","pre-trained models","closed-form least squares","knowledge tracking matrix","exact forgetting"],"falsifier":"Pre-train the feature backbone on the same distribution as the data slated for forgetting, run ACU to unlearn a subset, and then probe the backbone's features with a membership-inference classifier trained to tell forgotten samples from never-seen ones on the basis of feature statistics. If this probe separates the two groups with accuracy clearly above chance, the influence of the forgotten data survives in the deployed feature extractor, which contradicts the service-level claim that ACU erases it; under exact forgetting the probe should perform at chance.","tokens_in":21438,"feed_emoji":"🧹","tokens_out":11708,"duration_ms":108218,"temperature":0.7,"pith_summary":"The paper introduces continual unlearning (CU), the setting where, after continual learning, forgetting requests arrive sequentially, and claims that a model built from a frozen pre-trained feature extractor plus a ridge-regression classifier can answer each request exactly using only the samples to be forgotten. It identifies gradient-based updates as the root cause of why existing unlearning methods either need to revisit the retained data (violating CL privacy) or trade fidelity for efficiency, and proposes Analytic Continual Unlearning (ACU) as a gradient-free alternative. ACU maintains a compressed knowledge tracking matrix and applies a recursive update that provably reproduces, after every request, the classifier that would have been obtained by retraining on the remaining data. If the claim holds, exact privacy-compliant forgetting becomes a lightweight algebraic operation that degrades neither under frequent requests nor without access to historical data.","feed_headline":"One closed-form update deletes data exactly, no retained set","feed_subtitle":"Each request needs only the forgotten samples and exactly matches a model retrained on everything else.","key_machinery":"The knowledge tracking matrix $T_i = (\\sum_{j\\in\\hat{D}_i} f_j^\\top f_j + \\gamma I)^{-1}$, a $d_F\\times d_F$ matrix that compresses the entire retained set into a second-order statistic, is the engine of the method; it is updated in closed form through the Woodbury matrix identity (Lemma 2), which converts the removal of the forgetting set's feature gram matrix $\\check{F}_i^\\top\\check{F}_i$ into a recursive update involving only $\\check{F}_i$. On top of it, the classifier update in Theorem 1 re-weights the previous model by the forgotten features' covariance and subtracts the forgotten features' target contribution, a two-term 'amplify and erase' decomposition that the paper derives by rewriting the retained-set solution. The enabling structure is the frozen pre-trained backbone, which makes feature extraction gradient-free and keeps the whole pipeline analytic, so that data influence is an explicit algebraic quantity rather than a trace left in SGD trajectories.","core_discovery":"On its own terms, the paper's central discovery is Theorem 1: if the classifier is the analytic least-squares solution $W_0 = (\\sum_j f_j^\\top f_j + \\gamma I)^{-1} \\sum_j f_j^\\top y_j$ trained during the continual learning phase, and the knowledge tracking matrix $T_0$ stores the corresponding inverse-covariance term, then for each unlearning request $i$ the recursive update $$W_i = \\left(I + T_i \\sum_{j\\in \\check{D}_i} f_j^\\top f_j\\right) W_{i-1} - T_i \\sum_{j\\in \\check{D}_i} f_j^\\top y_j, \\qquad T_i = T_{i-1} + T_{i-1}\\check{F}_i^\\top (I - \\check{F}_i T_{i-1} \\check{F}_i^\\top)^{-1} \\check{F}_i T_{i-1}$$ is exactly equivalent to the ridge-regression classifier retrained from scratch on the retained set $\\hat{D}_i = D \\setminus \\bigcup_{k\\le i}\\check{D}_k$. The update is interpretable: the first term amplifies the knowledge that must stay, the second erases the knowledge that must go. Because both updates use only the forgotten features $\\check{F}_i$, the retained data never needs to be revisited, and because the tracking matrix is a compressed second-order statistic, it cannot be unrolled to recover the original samples. Experiments confirm the theorem numerically: ACU attains exactly zero deviation from the re-trained model on parameter distance, retained/forgetting/test accuracy, and membership-inference indicators, while gradient-based baselines degrade sharply as requests accumulate.","pith_inferences":["The mechanism is really inverse-covariance maintenance: the same two-term recursion should apply verbatim to any ridge-regression head trained on frozen features, whether linear probes, kernel regressors, or federated models, making sequential exact deletion a generic algebraic service rather than a CL-specific construction.","The guarantee covers only the classifier; if the frozen backbone has memorised the forgotten inputs, a membership-inference probe on the feature vectors themselves should still separate them, so a natural extension pairs ACU with low-rank representation corrections and measures residual leakage after both stages.","Exactness holds in real arithmetic; in practice the recursion's accumulated floating-point drift in $T_i$ over hundreds of requests is unmeasured, and the deviation from the retrained model would set a safe request-count envelope for deployment.","If the theorem stands, request-fragmentation denial-of-service attacks lose their point, as each fragment costs the same small closed-form operation, and the adversarial frontier shifts entirely to the backbone, which is why the authors' suggestion of an adjustable backbone is the consequential open direction."],"forward_implications":["Each forgetting request costs $O(d_F^3)$ plus terms depending only on the size of the forgetting set, independent of how much data the model has seen, so even a stream of adversarial requests arriving one fragment at a time stays cheap to serve.","After every request the model equals the classifier that never saw the forgotten data, which means membership-inference attacks on the classifier should perform at chance and the unlearning can be verified without trusting the service provider.","The model remains in its optimal analytic form after each request, so the system can alternate freely between learning and forgetting phases without resets or retraining.","Any continual learner built on a frozen pre-trained extractor plus an analytic classifier inherits exact forgetting as a corollary, provided it maintained the tracking matrix recursively during learning."],"supporting_citations":[{"why":"Supplies the random-projections plus frozen pre-trained extractor plus analytic classifier approach that ACU extends from continual learning to unlearning.","marker":"[7]"},{"why":"Provides the exemplar-free analytic continual-learning formulation whose final closed-form solution is ACU's starting model W0.","marker":"[15]"},{"why":"Introduces analytic class-incremental learning with absolute memorization and privacy protection, the direct precedent for the compressed knowledge tracking matrix.","marker":"[26]"},{"why":"Supplies the blockwise recursive Moore-Penrose inverse whose Woodbury-style recursion underlies both tracking-matrix updates.","marker":"[49]"},{"why":"Contributes the SCRUB and NegGrad+ baselines that ACU must outperform in unlearning effectiveness and model fidelity.","marker":"[11]"},{"why":"NegGrad is the only baseline that also avoids the retained set, so ACU is compared against it on equal privacy terms.","marker":"[23]"},{"why":"The finetuning baseline represents the retained-set re-training approach ACU claims to make unnecessary.","marker":"[17]"},{"why":"Provides the membership-inference attack used as the privacy metric for how close the unlearned model is to the re-trained one.","marker":"[57]"}],"fun_headline_variants":["Exact unlearning via closed-form recursive update","Forget precisely: no retained data, no gradients","One analytic step removes data, matches retrain","Gradient-free continual unlearning with exactness","Closed-form erasure for continual learning models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's guarantee rests on the assumption that the pre-trained feature extractor is frozen and never modified during learning or unlearning, so erasing the linear classifier's dependence on the forgotten samples removes all of their influence on the deployed model; the paper states this in Section 3.1 and flags in Section 5 that if the backbone itself holds domain-specific or private knowledge, ACU does not erase it.","fun_headline_variants_meta":{"raw":{"variants":["Exact unlearning via closed-form recursive update","Forget precisely: no retained data, no gradients","One analytic step removes data, matches retrain","Gradient-free continual unlearning with exactness","Closed-form erasure for continual learning models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1532,"prompt_tokens":1182,"completion_tokens":350,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":798,"completion_tokens_details":{"reasoning_tokens":279}},"tokens_in":798,"tokens_out":350,"duration_ms":3963,"temperature":1.0,"reasoning_tokens":279,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:38:03.736453+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pre-train the feature backbone on the same distribution as the data slated for forgetting, run ACU to unlearn a subset, and then probe the backbone's features with a membership-inference classifier trained to tell forgotten samples from never-seen ones on the basis of feature statistics. If this probe separates the two groups with accuracy clearly above chance, the influence of the forgotten data survives in the deployed feature extractor, which contradicts the service-level claim that ACU erases it; under exact forgetting the probe should perform at chance.","supporting_citations":[{"cited_title":"RanPAC: Random projections and pre-trained models for continual learning","cited_arxiv_id":null,"evidence_quote":"Supplies the random-projections plus frozen pre-trained extractor plus analytic classifier approach that ACU extends from continual learning to unlearning."},{"cited_title":"GACL: Exemplar-free generalized analytic continual learning","cited_arxiv_id":null,"evidence_quote":"Provides the exemplar-free analytic continual-learning formulation whose final closed-form solution is ACU's starting model W0."},{"cited_title":"ACIL: Analytic class-incremental learning with absolute memorization and privacy protection","cited_arxiv_id":null,"evidence_quote":"Introduces analytic class-incremental learning with absolute memorization and privacy protection, the direct precedent for the compressed knowledge tracking matrix."},{"cited_title":"Blockwise recursive moore–penrose inverse for network learning","cited_arxiv_id":null,"evidence_quote":"Supplies the blockwise recursive Moore-Penrose inverse whose Woodbury-style recursion underlies both tracking-matrix updates."},{"cited_title":"Towards un- bounded machine unlearning","cited_arxiv_id":null,"evidence_quote":"Contributes the SCRUB and NegGrad+ baselines that ACU must outperform in unlearning effectiveness and model fidelity."},{"cited_title":"Unrolling SGD: Understanding Factors Influencing Machine Unlearning","cited_arxiv_id":null,"evidence_quote":"NegGrad is the only baseline that also avoids the retained set, so ACU is compared against it on equal privacy terms."},{"cited_title":"Machine un- learning of features and labels","cited_arxiv_id":null,"evidence_quote":"The finetuning baseline represents the retained-set re-training approach ACU claims to make unnecessary."},{"cited_title":"Membership inference attacks against machine learning models","cited_arxiv_id":null,"evidence_quote":"Provides the membership-inference attack used as the privacy metric for how close the unlearned model is to the re-trained one."}],"review_version":1}