{"id":"3170ee7f-ed3c-43f8-a039-036711ac63b1","arxiv_id":"2412.18355","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"FedTA freezes a pre-trained ViT and trains small tail anchors to keep class features in place, reducing forgetting in federated continual learning.","lead":"Federated continual learning lets many devices keep learning from new data over time, but non-identical data across devices and tasks causes the shared model to forget old knowledge. This paper proposes FedTA, which freezes a pre-trained vision model and attaches small trainable 'tail anchors' to features so that class representations stay fixed, showing large gains on CIFAR-100 and ImageNet-R.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Input Enhancement tokens before the frozen ViT break the paper's own premise of a shared feature space, so local and global prototypes may live in mismatched spaces, undermining Tail Anchors.","rationale":"The reader's weakest assumption correctly identifies the reliance on a shared, stable feature space, but attributes the risk to non-IID data or task drift. My concern is more direct and internal: FedTA's own Input Enhancement deliberately inserts client-specific learnable tokens before the frozen ViT, so the 'frozen' features are not actually fixed. This is not a speculation about external data shift; it is a property of the method's architecture. If the resulting feature-space shift is large, the global prototypes are not valid in every client's space, and the tail-anchor mechanism loses its geometric meaning. This is the most load-bearing concern because every component (Tail Anchor, contrastive loss, global prototype selection) depends on the premise of a common space. The proposed test would quantify the shift and the cross-client validity of the prototypes, directly settling whether the premise holds in the paper's own experimental setting. I keep the reader's CONDITIONAL verdict because the empirical results might still be robust even if the theoretical justification is flawed, but the condition should include this specific verification.","tokens_in":12884,"tokens_out":3981,"duration_ms":37473,"concrete_test":"Take a held-out set of images from classes shared by at least two clients (e.g., the 25 public classes on CIFAR-100). After the first round of local training, for each client compute Fout for these images using that client's trained Input Enhancement and the frozen ViT. Compute the average pairwise cosine distance between clients' Fout vectors for the same image. Compare this to the average distance between different classes' Fout within a single client. If the inter-client distance is comparable to or larger than the inter-class distance, the common-feature-space premise fails and global prototypes are mismatched. Additionally, test global prototype generality: classify held-out features from client A using the global prototype selected from client B; if accuracy drops by more than 10% versus using A's own prototype, the anchors are not shared.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central premise (Sec. 3.3) is that all clients share the same frozen pre-trained ViT, 'ensuring that they have the same knowledge/feature space.' FedTA's own Input Enhancement (Sec. 4.1) violates this premise: each client trains client-specific IE tokens that are concatenated to the patch embeddings (Eq. 2) and processed by the frozen ViT. Because the ViT's self-attention mixes these extra tokens with image patches, the output feature Fout for the same image differs across clients and across tasks, even though the ViT weights are frozen. The local prototypes (Eq. 7) are computed in these client-specific spaces, and Best Global Prototype Selection (Sec. 4.4) picks one client's prototype as the global anchor. Using that vector in the contrastive loss (Eq. 5) on another client's F_TA assumes the two are in the same vector space; if the IE-induced shift is large, the anchor is not a fixed point and the claim that FedTA 'preserves the relative positions of features' is unsupported. The paper never measures or bounds this shift.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedTA, a federated continual learning (FCL) method that combines a frozen pretrained Vision Transformer (ViT) with trainable \"input enhancement\" tokens and trainable \"tail anchors.\" The core idea is that, because the ViT is frozen, all clients share the same feature extractor; the method then mixes tail anchors with the frozen output features to keep class features near fixed global prototypes, thereby preventing what the authors call parameter-forgetting and output-forgetting. The server also performs selective input knowledge fusion (SIKF) on the input enhancement knowledge bases and best global prototype selection (BGPS) on uploaded local prototypes. Experiments on CIFAR-100 and ImageNet-R compare FedTA against several FL, FL+CL, and FCL baselines, reporting accuracy and two knowledge-retention metrics, plus ablations and t-SNE visualizations.","tokens_in":13236,"tokens_out":3806,"duration_ms":35985,"significance":"If the claims are validated, FedTA would be a useful contribution to FCL: it avoids replay data, keeps communication costs small, and reports strong gains over existing methods while using a frozen pretrained backbone. The paper also addresses an important and under-studied problem, spatial-temporal catastrophic forgetting, and proposes a mechanism that is conceptually interesting: fixing feature positions via learnable anchors that are mixed with frozen features. The use of a frozen ViT to sidestep parameter forgetting is a promising direction. The authors are also to be credited for including ablations, retention metrics, and an efficiency/privacy discussion. However, the current manuscript has substantial reproducibility and conceptual gaps: the central mixing operation is never defined mathematically, the input enhancement component appears to violate the paper's own premise of a shared feature space, and the experimental section reports no error bars or hyperparameter values. These issues must be resolved before the empirical claims can be fully assessed.","major_comments":[{"comment":"The central operation of FedTA, mixing a tail anchor with the output feature to form F_TA, is never defined. The text says \"it will be mixed with Fout to form a new feature F_TA\" but provides no equation, no description of whether the mixing is a weighted sum, concatenation, or attention-like combination, and no statement of the resulting dimension. This makes Eqs. (5), (6), and (7) ambiguous and the method unreproducible. Please provide the exact operation, and also specify how F_TA is fed to the classifier for the cross-entropy loss in Eq. (6).","section":"Sec. 4.2, Eq. (5), Eq. (7)"},{"comment":"The premise stated in Sec. 3.3 is that all clients share the same frozen pretrained ViT, \"ensuring that they have the same knowledge/feature space.\" However, Sec. 4.1 introduces client-specific input enhancement tokens that are concatenated to the patch embeddings in Eq. (2) and then processed by the frozen ViT. Because self-attention mixes these tokens with image patches, the output feature Fout for the same image will differ across clients and across tasks, even with frozen ViT weights. Consequently, local prototypes computed via Eq. (7) live in client-dependent feature spaces, and using a global prototype selected from one client's space in the contrastive loss (Eq. (5)) on another client assumes that the IE-induced shift is negligible. The paper never measures or bounds this shift. This is load-bearing for the claim that FedTA preserves feature positions and enables a common feature space. Please address this directly, e.g., by analyzing the shift empirically or by modifying the method so that prototypes are computed in a truly shared space.","section":"Sec. 3.3 vs. Sec. 4.1, Eq. (2)"},{"comment":"The main experimental result consists of single-run accuracy numbers with no error bars, no standard deviations, and no number of seeds. Given that the reported gains over the strongest baseline (FedMGP) are sometimes modest, and that the ablation comparisons (e.g., Ours vs. Ours-w/o SIKF on ImageNet-R) are close, the reader cannot assess whether the differences are statistically meaningful. In addition, hyperparameters are not reported: the loss weights lambda1, lambda2, lambda3 in Eqs. (3) and (6), the temperature tau in Eq. (5), the threshold Th in Sec. 4.4, the knowledge base size M and selected count N in Eq. (2), and the optimizer settings are all absent. Please provide these details and multiple-seed results with error bars.","section":"Table 1 and Sec. 5.1"},{"comment":"The ablation for Selective Input Knowledge Fusion does not support the claim that SIKF is beneficial. On ImageNet-R, Ours-w/o SIKF achieves 80.0/80.5/81.1/82.9/81.7 across tasks, while full FedTA achieves 81.5/78.8/79.2/80.6/85.0. Thus at tasks 2, 3, and 4 the variant without SIKF is better. The text admits this \"sometimes falls below\" but the claim in the abstract and contributions that SIKF is a novel and effective component is not substantiated. Please either provide a more thorough analysis (e.g., with variance, additional datasets, or a corrected mechanism) or temper the claim.","section":"Table 1 and Sec. 5.3"},{"comment":"The Best Global Prototype Selection criterion is not well justified. Eq. (9) defines M_ij via a function \"dis\", which is elsewhere used as distance/cosine similarity, but then states that M_ij = 1 for prototypes of the same class; this is inconsistent (if it is a distance, same-class entries should be 0; if it is a similarity, the text should say so and define the range). Moreover, Eq. (10) selects the prototype with the lowest average similarity as the global anchor, but the paper provides no argument why the least representative prototype in this sense is the best anchor point for a class. This is a core component, so please clarify the definition and provide an empirical or theoretical justification.","section":"Sec. 4.4, Eqs. (9) and (10)"}],"minor_comments":[{"comment":"The label \"PositionDift\" appears to be a typo for \"Position Drift.\"","section":"Fig. 1"},{"comment":"The notation for keys is confusing: K_ie^s, K_ie^in, and the summation over \"Kie_s\" are not clearly defined. Please restate the surrogate loss and specify what is being summed over.","section":"Eq. (3)"},{"comment":"The privacy discussion states that \"If we randomly mix Tail Anchor with features, this issue will be resolved,\" but random mixing would appear to destroy the method's ability to place features at fixed class-specific positions. Please clarify this statement.","section":"Sec. 5.4"},{"comment":"The t-SNE figure labels the number of input enhancement tokens and tail anchors but does not state which dataset, which task, or which classes are shown, nor whether the \"same samples\" are from old tasks. Please make the figure self-contained.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is closely related to the authors' prior work on federated class-incremental learning (e.g., refs. [38] and [39]), and the strongest baseline FedMGP is also from the same group. The authors should make sure the comparison is fair and that the novelty relative to these prior works is clearly delineated. The paper does not currently release code or provide seed-level results, which is particularly important given the missing definitions of the core mixing operation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"One thing to know up front: FedTA is a prompt-style FCL method that freezes a pre-trained ViT and mixes learnable 'tail anchors' with the output features; it reports large accuracy and retention gains over baselines on CIFAR-100 and ImageNet-R. The interesting idea is that you can condition the frozen features with a small trainable vector and then pull the result toward a server-selected global prototype. That is a clean way to think about output-side forgetting, and avoiding replay data is a real plus for privacy.\n\nThe paper does some things well. The experimental setup is genuinely hard: five clients, five tasks, non-IID splits, and separate private/public classes. The retention metrics from the FCL survey are the right way to measure forgetting. The ablation shows the tail anchor is load-bearing, and the efficiency numbers (about 250K trainable params) make the method plausible for edge devices.\n\nThe soft spots are real. The biggest one is the paper's own premise. The authors argue that a shared frozen ViT guarantees a common feature space across clients, but their Input Enhancement adds client-specific trainable tokens to the patch embeddings. Those tokens go through the self-attention layers, so the output features for the same image are different for each client and each task. The local prototypes are computed in these client-specific spaces, and the server then picks one client's prototype as the global anchor for another client. That only works if the shift induced by the input tokens is small, and the paper never measures or bounds it. This is a load-bearing gap, even though the strong empirical results suggest the shift might be small in practice.\n\nNext, the paper is thin on reporting: no error bars, no hyperparameter values for the loss weights, temperature, or threshold, and the 'mix' operation for the tail anchor is described in words but never defined as an equation. The 'first to address spatial-temporal heterogeneity' claim is also overstated, since the authors cite FedMGP and their own survey, both of which tackle the same setting. And the Selective Input Knowledge Fusion ablation is actually worse on ImageNet-R, so its contribution is not clear.\n\nThe circularity concern is minor: the contrastive loss pulls features toward the anchors, so the anchor-position claim is partly by construction, but the held-out accuracy and retention numbers stand independently.\n\nWho should read it: anyone working on rehearsal-free FCL or prompt-based continual learning. The idea is worth discussing even if the verification is incomplete. I would send it to review, but with a clear request for hyperparameters, code or at least error bars, and a direct analysis of the cross-client feature-alignment problem. Without that, the central claim about a common feature space remains an assertion rather than a result.","headline":"FedTA is a prompt-based FCL method with strong reported gains, but its client-specific input tokens undercut the paper's central claim of a shared frozen-ViT feature space.","tokens_in":13655,"tokens_out":5438,"would_cite":true,"duration_ms":45642,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that spatial-temporal catastrophic forgetting in federated continual learning can be controlled by freezing a pre-trained ViT and mixing learnable Tail Anchors with frozen output features to hold every class at a fixed…","keywords":["federated continual learning","spatial-temporal data heterogeneity","catastrophic forgetting","pre-trained Vision Transformer","tail anchor","input enhancement","global prototype selection","knowledge retention"],"falsifier":"Train FedTA with clients using different pre-trained backbones (or the same backbone pre-trained on different data), so local features are not in one shared space, and check whether the server-selected global prototypes still lift final-task accuracy; a large drop would confirm that the method's gains depend on the shared-feature-space assumption rather than on the anchor mechanism itself.","tokens_in":12746,"feed_emoji":"⚓","tokens_out":5508,"duration_ms":49452,"temperature":0.7,"pith_summary":"Federated continual learning couples two sources of data shift: each client holds a different private slice of classes (spatial heterogeneity), and every client's data arrives as a sequence of new tasks (temporal heterogeneity). The paper argues that this input-level heterogeneity moves both the parameters and the output features of a model, and names those two failure modes parameter-forgetting and output-forgetting. To stop them, it proposes Federated Tail Anchor (FedTA): keep a pre-trained Vision Transformer frozen, prepend small trainable input tokens, and append trainable class-specific tail anchors to the output features so every class has a fixed target position in feature space. In experiments on CIFAR-100 and ImageNet-R, FedTA keeps accuracy high across five tasks and five clients and keeps temporal and spatial knowledge retention around 98 percent.","feed_headline":"Frozen ViT plus tail anchors stops forgetting across clients and tasks","feed_subtitle":"Mixing learnable anchors with a frozen transformer keeps earlier and local knowledge at about 98 percent retention","key_machinery":"The Tail Anchor is the load-bearing object: a set of key-value pairs, one learnable vector per class, with a key used to select the right anchor by cosine similarity to the frozen output feature. The anchor is concatenated (mixed) with the output feature to form a new feature, and a contrastive loss ties that mixed feature to a global prototype fixed by the server. Because the anchor's relative position is defined against a frozen global prototype, it can pull drifted features back to their original class location even when inputs change across clients and tasks. Input Enhancement tokens and the server-side knowledge fusion play a supporting role: they let the frozen ViT handle new classes and let the server merge heterogeneous local tokens without touching the backbone.","core_discovery":"The central claim is that spatial-temporal catastrophic forgetting in federated continual learning can be reduced to a feature-position problem and solved without updating the backbone. FedTA uses a frozen pre-trained ViT shared by all clients, which removes parameter-forgetting because the feature extractor never changes. Input Enhancement adds a small set of trainable tokens to the patch embeddings so the frozen model adapts to downstream classes; Selective Input Knowledge Fusion distills these tokens on the server; Best Global Prototype Selection picks, for each class, the local prototype least similar to all others and fixes it as a global anchor. During local training, the learned Tail Anchor is mixed with the frozen output features and pulled toward that anchor by contrastive loss. The paper reports that FedTA outperforms FCL baselines, that the ablation removing the tail anchor causes the largest accuracy drop, and that t-SNE visualizations show feature positions stay stable across tasks and clients.","pith_inferences":["If the shared-feature-space premise is right, FedTA's recipe transfers to any frozen encoder, so the same anchor mechanism could be used for federated continual learning of text or audio by changing only the encoder.","A testable extension is to replace the server's lowest-similarity prototype rule with a maximum-margin rule (pick the prototype that maximizes separation from all other classes); the paper's adjacency-matrix machinery already computes the needed similarities.","The method's reliance on a single global pre-trained model suggests a natural stress test: apply FedTA in a setting where clients receive the same architecture but with different pre-trained weights, and measure how quickly anchor alignment degrades.","If output-forgetting is truly the main failure mode, then even without input enhancement, the tail anchor should retain most of the benefit; an ablation that isolates the anchor from Input Enhancement would separate the value of position-fixing from the value of adapter tokens."],"forward_implications":["Clients only communicate input-enhancement tokens and class prototypes; if FedTA holds, the backbone never needs to be sent, which cuts communication and lets heterogeneous clients share one frozen encoder.","Forgetting is controlled by fixing feature positions rather than replaying old data, so FedTA avoids the privacy and storage costs of replay-based FCL baselines.","Because the tail anchor is the component that holds features in place, any new task can be added by learning a new anchor vector, making class-incremental growth a matter of adding key-value pairs.","The reported gap between FedTA and the strongest baseline (e.g., final-task accuracy 89.4 versus 82.2 on CIFAR-100) is driven mostly by the tail anchor, per the ablation, so future FCL methods should treat feature-position anchoring as a first-class mechanism."],"supporting_citations":[{"why":"Supplies the frozen pre-trained ViT that gives clients a shared feature space.","marker":"[4]"},{"why":"Defines spatial-temporal catastrophic forgetting and the knowledge-retention metrics the experiments use.","marker":"[35]"},{"why":"Shows learnable prompts can adapt a frozen ViT in continual learning, the basis for Input Enhancement.","marker":"[32]"},{"why":"Provides the complementary-prompt baseline FedDualP and further evidence that prompt-style input adaptation works with frozen ViTs.","marker":"[33]"},{"why":"The GLFC federated class-incremental baseline whose accuracy FedTA is compared against.","marker":"[3]"},{"why":"The FedMGP baseline, the strongest prior method in the accuracy table that FedTA outperforms.","marker":"[39]"},{"why":"The MFCL data-free FCL baseline, representative of replay-free methods FedTA avoids.","marker":"[1]"}],"fun_headline_variants":["Tail anchors fix federated learning's double forgetting","FedTA: freezing ViT, adding anchors, forgetting goes away","Frozen features plus learned anchors stop spatial-temporal drift","One frozen ViT, tail anchors: clients never forget","Feature-position trick solves federated continual forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All clients share one frozen pre-trained ViT whose feature space is the same for every client and every task, so anchors and prototypes computed from local features remain valid for features produced later by other clients; if that shared space drifts or was never aligned, the anchors would be attached to the wrong coordinates.","fun_headline_variants_meta":{"raw":{"variants":["Tail anchors fix federated learning's double forgetting","FedTA: freezing ViT, adding anchors, forgetting goes away","Frozen features plus learned anchors stop spatial-temporal drift","One frozen ViT, tail anchors: clients never forget","Feature-position trick solves federated continual forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1373,"prompt_tokens":915,"completion_tokens":458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":381}},"tokens_in":531,"tokens_out":458,"duration_ms":4800,"temperature":1.0,"reasoning_tokens":381,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:46:18.825991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FedTA with clients using different pre-trained backbones (or the same backbone pre-trained on different data), so local features are not in one shared space, and check whether the server-selected global prototypes still lift final-task accuracy; a large drop would confirm that the method's gains depend on the shared-feature-space assumption rather than on the anchor mechanism itself.","supporting_citations":[{"cited_title":"Federated continual learning via knowledge fu- sion: A survey","cited_arxiv_id":null,"evidence_quote":"Defines spatial-temporal catastrophic forgetting and the knowledge-retention metrics the experiments use."},{"cited_title":"Federated class-incremental learn- ing","cited_arxiv_id":null,"evidence_quote":"The GLFC federated class-incremental baseline whose accuracy FedTA is compared against."},{"cited_title":"Personalized federated continual learning via multi-granularity prompt","cited_arxiv_id":null,"evidence_quote":"The FedMGP baseline, the strongest prior method in the accuracy table that FedTA outperforms."},{"cited_title":"A data-free ap- proach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks","cited_arxiv_id":null,"evidence_quote":"The MFCL data-free FCL baseline, representative of replay-free methods FedTA avoids."}],"review_version":1}