{"id":"7c7b282e-f041-4826-9598-2da8b3cca31c","arxiv_id":"2412.02509","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A frozen vision transformer plus trainable attention-steering blocks and EWC achieves competitive continual learning accuracy without rehearsal memory.","lead":"FCL-ViT is a continual learning vision transformer that keeps a pretrained backbone frozen and learns small blocks that steer attention toward the task at hand. It reports strong final-task accuracy on ImageNet-100 and CIFAR-100 without replaying old images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Comparison conflates task-incremental with class-incremental: FCL-ViT uses task IDs and per-task heads, while quoted baselines from [7] are class-incremental; SOTA claim may rest on an easier protocol.","rationale":"The paper's architecture is clearly described, and the ablation study (Figure 4) gives some evidence that TSBs reduce forgetting. However, the central claim of state-of-the-art performance depends on an apples-to-apples comparison with benchmark methods. The reader's concern about backbone capacity is plausible, but the more fundamental issue is the evaluation protocol: Section 3.1 and 3.2 explicitly use the task index and per-task linear heads, which makes the setting task-incremental. The baselines from [7] (iCaRL, WA, DER, DyTox+) are standard class-incremental methods that do not receive task IDs. Therefore, the reported improvements could simply reflect the easier task-aware setting, not the proposed feedback mechanism. This is a correctness risk that cannot be resolved by hyperparameter tuning or code release; it requires re-evaluating the model under the class-incremental protocol or demonstrating that the quoted baselines also use task IDs. Since the paper does neither, the SOTA claim is not currently supported.","tokens_in":10860,"tokens_out":6924,"duration_ms":65557,"concrete_test":"Run a class-incremental version of FCL-ViT in which the task index is not provided at inference and the model must choose among all classes by concatenating the task-specific LBs and taking the highest logit across all tasks. Recompute the CIFAR-100 10-task Last accuracy. If it falls below the quoted DER (65.22) or DyTox+ (62.06) results, the central SOTA claim fails. As a second check, consult the original DyTox paper to verify that the quoted numbers in Table 2 were obtained without task IDs.","verdict_should_be":"REJECT","load_bearing_attack":"FCL-ViT is evaluated in a task-incremental (task-aware) setting, whereas the benchmark numbers quoted from [7] are class-incremental. Section 3.1 states that the model is evaluated 'using the task index i', and Section 3.2 adds a new Linear Block per task and selects the classifier via 'the respective LB for the task', requiring the task ID at inference. iCaRL, WA, DER, and DyTox+ do not use task IDs; they classify among all classes seen so far. Thus, FCL-ViT only needs to discriminate classes within the current task, which is a fundamentally easier problem. The paper does not acknowledge this protocol mismatch, so the claimed state-of-the-art Last accuracy may be an artifact of the easier evaluation rather than the proposed TSB/TAB mechanism. This concern is independent of the backbone-capacity issue and affects every reported comparison in Tables 1 and 2.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FCL-ViT, a Vision Transformer architecture for continual learning that combines a frozen ViT backbone (TABs) with trainable Task-Specific Blocks (TSBs) and Elastic Weight Consolidation (EWC). Inference proceeds in two phases: a generic feature extraction pass and a task-specific pass in which TSB outputs are injected through cross-attention into the TABs. A new linear classifier is added per task, and no rehearsal memory is used. The method is evaluated on ImageNet-100 and CIFAR-100 under several task splits, compared with iCaRL, WA, DER, and DyTox+, and ablated by removing the TSBs and by varying the EWC strength. The paper claims that FCL-ViT surpasses state-of-the-art continual learning performance while retaining few trainable parameters.","tokens_in":11100,"tokens_out":6260,"duration_ms":62306,"significance":"If the claims are substantiated, FCL-ViT is a useful contribution to rehearsal-free continual learning: the architecture is simple, the trainable parameter count is low, and the core ablation (with vs. without TSBs) shows a large improvement (71.80 vs. 53.75 Last accuracy on ImageNet-100). The paper also reports stable performance across increasing numbers of tasks, which is a desirable property. However, the headline state-of-the-art claim is currently not supported by the experimental comparison because of a protocol mismatch and unverified baseline comparability. The mechanism itself is plausible, but the empirical evidence needs to be placed on a sound footing. No equation is fitted to the target results, so circularity is not a concern; the main risks are experimental comparability and missing details.","major_comments":[{"comment":"The evaluation protocol is task-incremental for FCL-ViT but class-incremental for all quoted baselines. Section 3.1 states that the model is evaluated 'using the task index i', and Section 3.2 adds a Linear Block per task and selects the classifier via the task-specific head ŷ_k = H̃_k z. By contrast, iCaRL, WA, DER, and DyTox+ operate without task IDs and must discriminate among all classes seen so far. FCL-ViT only needs to discriminate within the current task, which is an easier problem. The reported Last accuracy comparisons are therefore not apples-to-apples, and the abstract's 'surpasses state-of-the-art' claim is not supported by Tables 1 and 2. The authors should either evaluate FCL-ViT in a class-incremental protocol or compare against task-aware baselines and explicitly restrict the claim to task-incremental learning.","section":"§3.1–3.2, Tables 1–2"},{"comment":"All baseline numbers are imported from [7] without disclosing the backbone used for each method. FCL-ViT uses a ViT-Base backbone with embedding dimension D=768. If iCaRL, WA, DER, or DyTox+ used smaller backbones (e.g., ViT-Small), the reported gains could be due to backbone capacity rather than the proposed TSB/TAB mechanism. The #TP column counts only trainable parameters and does not account for the frozen backbone's capacity. Please report the backbone specification for every baseline and, ideally, rerun at least DyTox+ with the same ViT-Base backbone so that the comparison is controlled for architecture capacity.","section":"§4.1, Tables 1–2"},{"comment":"The ablation that removes TSBs and trains all ViT weights with EWC shows a drop from 71.80 to 53.75 on ImageNet-100. This demonstrates that TSBs help relative to that particular full-fine-tuning baseline, but it does not show that the two-phase mechanism improves over a much simpler task-aware linear probe on the frozen ViT-Base features with per-task heads. Since the TABs are frozen and the per-task LB already provides task-specific classification, adding a linear-probe baseline is necessary to attribute the performance to attention tuning rather than to the quality of the frozen backbone features. This baseline is absent and is load-bearing for the paper's explanatory claim.","section":"§4.2"},{"comment":"Several experimental details required for reproducibility are missing: the value of α in Eq. (5) is never reported; the EWC Fisher Information Matrix computation is unspecified (number of samples used, diagonal vs. full approximation, and whether it is recomputed after each task); and the ImageNet-100 results in Table 1 have no variance or number of runs. Given the small differences in some comparisons (e.g., CIFAR-100 10-task Last: 65.02 vs. 65.22), these omissions prevent verification of the claimed improvements.","section":"§3.2 and §4.1"}],"minor_comments":[{"comment":"The sentence defining θ_i,j and θ_{i-1,j} says they are trained for tasks T_i and T_{i+1}, respectively, which is inconsistent with the penalty term (θ_i,j,l − θ_{i−1,j,l})² that compares consecutive tasks; please correct to T_i and T_{i−1}.","section":"§3.2, Eq. (4)"},{"comment":"The abstract says TABs and TSBs 'operate in both phases and are responsible for tuning the TABs attention'; this is ambiguous because only the TSBs tune the TAB attention. Please reword for clarity.","section":"Abstract and §1"},{"comment":"The text says FCL-ViT 'outperforms other models for all CIFAR100 splits', but Table 2 shows that DER has higher Last accuracy on the 10-task split (65.22 vs. 65.02); please reconcile the claim with the table.","section":"§4.1, Table 2 and Figure 2"},{"comment":"The reported inference slowdown of 'approximately 92% slower' is given without measurement details; please state the hardware, batch size, and evaluation setup used for timing.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper does not release code. Given the small margins in some comparisons and the protocol mismatch, code or a detailed experimental appendix would materially help the review. The most important request is a matched-protocol evaluation: either class-incremental evaluation of FCL-ViT or task-aware baselines for the comparisons in Tables 1 and 2."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing you should know: the central SOTA claim does not hold up under a protocol mismatch. FCL-ViT needs the task index at inference — Section 3.1 says so explicitly, and Section 3.2 adds a per-task Linear Block selected by task ID. iCaRL, WA, DER, and DyTox+ are quoted from class-incremental benchmarks where no task ID is used. Comparing a task-aware method against class-incremental baselines is apples-to-oranges, and the paper never acknowledges the difference. The stress-test note is right, and it affects every number in Tables 1 and 2. This is not a minor flaw; it is the foundation of the abstract's \"surpasses state-of-the-art\" claim.\n\nNow the credit. The architecture itself is clearly described and genuinely new in the CL context: a frozen ViT backbone (TABs) used in a two-phase inference, where TSBs generate cross-attention keys/values to steer the features toward the current task, with EWC to regularize the TSBs. The idea is borrowed from TOAST but the application to continual learning is original. The ablation in Figure 4 is the strongest part of the paper — removing the TSBs drops Imagenet-100 last accuracy from 71.8 to 53.75, which shows the mechanism matters. The authors are also honest about the 92% inference slowdown and the absence of replay memory.\n\nThe other weaknesses are real but secondary. Baselines are imported from [7] without backbone sizes, so part of the gap could be capacity. There is no frozen-feature linear-probe baseline, which is the obvious control for a method that adds trainable modules on top of a frozen backbone. Hyperparameters like the loss weight alpha and EWC FIM details are underspecified. The explanation that EWC works better in a higher-dimensional transformer space is post hoc and untested. But the TSB effect is measured directly, so the core mechanism survives the critique.\n\nWho gets value? Readers working on task-incremental learning with transformers, or on rehearsal-free parameter-efficient CL, will find the architecture worth studying. Do not use the benchmark tables as evidence until the comparison is redone against task-aware baselines (e.g., task-aware DyTox, prompt methods with task IDs) and the missing ablations and hyperparameters are added.\n\nI would not desk-reject this. The method is plausible and the ablation is real, but the evaluation needs major surgery. Send it to review with instructions to fix the protocol comparison and the missing baselines. The authors should also share code if they want the numbers taken seriously.","headline":"The architecture is real and the TSB ablation is convincing, but the SOTA claim compares a task-aware method to class-incremental baselines, and that mismatch is load-bearing.","tokens_in":11566,"tokens_out":2995,"would_cite":false,"duration_ms":32277,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-phase attention tuning beats CL rivals without replay memory","keywords":["continual learning","vision transformer","attention tuning","catastrophic forgetting","elastic weight consolidation","rehearsal-free","task-specific features","feedback mechanism"],"falsifier":"Retrain iCaRL, WA, DER, and DyTox+ on the same frozen ViT-Base backbone and training schedule as FCL-ViT and compare Last accuracy; if FCL-ViT no longer leads under matched backbone capacity, the central superiority claim collapses. A second check: replace the TSB steering vectors with a fixed linear projection of phase-1 features; if accuracy stays high, the task-specific cross-attention mechanism is not doing the load-bearing work.","tokens_in":10677,"feed_emoji":"🎯","tokens_out":7707,"duration_ms":69396,"temperature":0.7,"pith_summary":"The paper tries to show that continual learning can be handled without replay memory by making a Vision Transformer's attention itself task-aware. The proposed FCL-ViT runs every image through the same frozen Transformer twice: the first pass produces generic features, small Task-Specific Blocks convert those features into attention-steering vectors, and the second pass uses cross-attention to produce a representation tuned to the current task. The authors report that this design, regularized by Elastic Weight Consolidation, achieves top Last-task accuracy on ImageNet-100 with 10 tasks and stable average accuracy on CIFAR-100 across 10, 20, and 50 tasks while training only about 14.23 million parameters and storing no old examples. If true, the central payoff is a rehearsal-free continual classifier whose forgetting is controlled by attention steering rather than by memory or parameter expansion.","feed_headline":"Two-phase attention tuning beats CL rivals without replay memory","feed_subtitle":"FCL-ViT freezes its Transformer, steers attention per task, and holds ~67% final-task accuracy on CIFAR-100.","key_machinery":"The load-bearing identity is the two-phase attention replay over one frozen Transformer. In phase 1, patch embeddings pass through $d$ Tunable self-Attention Blocks in self-attention mode, giving $\\mathbf{r} = \\mathbf{f}_d$. Task-Specific Blocks map $\\mathbf{r}$ through stacked linear-dropout layers into steering vectors $\\mathbf{g}_j$, one per depth. In phase 2 the same blocks switch to cross-attention, with $\\mathbf{Q}_j = \\mathbf{W}_{qj}\\mathbf{f}_{j-1}$ and $\\mathbf{K}_j = \\mathbf{W}_{kj}\\mathbf{g}_j$, $\\mathbf{V}_j = \\mathbf{W}_{vj}\\mathbf{g}_j$, so each layer's attention is steered by task-specific information. The trainable parameters are the TSB matrices $\\mathbf{H}^1_j, \\mathbf{H}^2_j$ and per-task linear classifiers; Elastic Weight Consolidation penalizes changes to important TSB weights using the diagonal Fisher Information Matrix. This is what lets the architecture adapt the same frozen backbone to each new task instead of expanding or replaying.","core_discovery":"The central claim is that catastrophic forgetting can be substantially reduced by giving a frozen Vision Transformer a feedback loop that retunes attention to the task at hand. FCL-ViT's 12 Tunable self-Attention Blocks (TABs) stay frozen; during phase 1 they act as ordinary self-attention and yield generic feature vector $\\mathbf{r}$. Task-Specific Blocks (TSBs), each a two-linear-layer module with dropout, read $\\mathbf{r}$ and emit per-block steering vectors $\\mathbf{g}_j$. In phase 2 the same TABs run again as cross-attention layers: the query comes from the current layer's input, the key and value come from $\\mathbf{g}_j$, and the final vector $\\mathbf{z}$ is what the per-task classifier sees. Training updates only the TSB weights and the current task's linear classifier, with an Elastic Weight Consolidation penalty that protects TSB parameters important for earlier tasks. The paper reports Last-task Top-1 accuracy of 71.80% on ImageNet-100 (10 tasks), above DyTox+ (69.10%) and DER (66.70%), and stable Last accuracy of 65.02%, 67.61%, and 67.72% on CIFAR-100 for 10, 20, and 50 tasks, all without rehearsal memory; removing the TSBs drops ImageNet-100 Last accuracy to 53.75%.","pith_inferences":["Editorial inference: because the backbone is frozen and steering is a generic cross-attention operation, the same two-phase design should transfer to other Transformer backbones (smaller or larger ViTs, Swin-style models), but the paper only evaluates ViT-Base.","Editorial inference: the reported roughly 92% inference slowdown means the second pass is a real deployment cost; a natural extension would train a lightweight head to predict the TSB steering vectors directly from phase-1 features, collapsing the two passes into one.","Editorial inference: the BLAZE wildfire experiment shows a single domain-shift sequence, so the no-rehearsal claim has not yet been stress-tested on longer or more heterogeneous task orders; a multi-domain curriculum would be a harder test."],"forward_implications":["No-exemplar continual learning becomes competitive with replay-based methods: FCL-ViT reports Last accuracy 71.80% on ImageNet-100 10-task and stable 65-68% on CIFAR-100 across 10/20/50 tasks without storing any previous samples.","Task count can grow without architectural expansion: the 12 frozen backbone blocks and the TSBs are reused across all tasks; each new task adds only one linear classifier, so trainable-parameter growth is much smaller than in dynamic-expansion methods like DER.","The TSB mechanism, not the regularizer alone, is responsible for the forgetting control: removing the TSBs and fine-tuning the whole ViT with only EWC drops ImageNet-100 Last accuracy from 71.80% to 53.75%.","The method's accuracy is unusually stable as tasks increase from 10 to 50 on CIFAR-100 (Last 65.02% to 67.61% to 67.72%), whereas comparator methods degrade, so its advantage is expected to be largest in long task sequences."],"supporting_citations":[{"why":"Supplies the Elastic Weight Consolidation regularizer that protects TSB parameters important for previous tasks.","marker":"[16]"},{"why":"Provides the attention-steering feedback idea that FCL-ViT adapts from transfer learning to continual learning via cross-attention tuning.","marker":"[22]"},{"why":"Serves as the main transformer-based CL competitor (DyTox) and as the source of the benchmark numbers reported in Tables 1 and 2.","marker":"[7]"},{"why":"Is the dynamic expandable representation baseline that FCL-ViT outperforms on ImageNet-100 Last accuracy while using far fewer trainable parameters.","marker":"[28]"},{"why":"Defines the replay-based iCaRL baseline and the Avg/Last evaluation protocol the paper follows.","marker":"[20]"},{"why":"Is the classifier-rebalancing baseline (WA) compared on both ImageNet-100 and CIFAR-100 splits.","marker":"[30]"},{"why":"Introduces the Vision Transformer whose pretrained weights initialize the frozen TAB backbone.","marker":"[5]"}],"fun_headline_variants":["FCL-ViT steers frozen ViT attention to beat continual learning rivals","Task-tuned attention feedback lets frozen ViT top CL benchmarks","No replay, no forgetting: FCL-ViT's attention tuning leads CL","Frozen transformer, dynamic attention: FCL-ViT outdoes CL SOTA","Feedback attention tuning: FCL-ViT sets new CL record without memory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the competing methods' benchmark scores were measured with backbones of comparable size to FCL-ViT's frozen ViT-Base, so the reported gains reflect the attention-tuning mechanism rather than a larger pretrained backbone.","fun_headline_variants_meta":{"raw":{"variants":["FCL-ViT steers frozen ViT attention to beat continual learning rivals","Task-tuned attention feedback lets frozen ViT top CL benchmarks","No replay, no forgetting: FCL-ViT's attention tuning leads CL","Frozen transformer, dynamic attention: FCL-ViT outdoes CL SOTA","Feedback attention tuning: FCL-ViT sets new CL record without memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0009,"raw_usage":{"total_tokens":3922,"prompt_tokens":1041,"completion_tokens":2881,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":2779}},"tokens_in":657,"tokens_out":2881,"duration_ms":18166,"temperature":1.0,"reasoning_tokens":2779,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:22:30.806415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain iCaRL, WA, DER, and DyTox+ on the same frozen ViT-Base backbone and training schedule as FCL-ViT and compare Last accuracy; if FCL-ViT no longer leads under matched backbone capacity, the central superiority claim collapses. A second check: replace the TSB steering vectors with a fixed linear projection of phase-1 features; if accuracy stays high, the task-specific cross-attention mechanism is not doing the load-bearing work.","supporting_citations":[{"cited_title":"Overcoming catastrophic forgetting in neural net- works","cited_arxiv_id":null,"evidence_quote":"Supplies the Elastic Weight Consolidation regularizer that protects TSB parameters important for previous tasks."},{"cited_title":"Dytox: Transformers for continual learning with dynamic token expansion, in: Proceedings IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Serves as the main transformer-based CL competitor (DyTox) and as the source of the benchmark numbers reported in Tables 1 and 2."},{"cited_title":"Der: Dynamically expandable repre- sentation for class incremental learning, in: Proceedings IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Is the dynamic expandable representation baseline that FCL-ViT outperforms on ImageNet-100 Last accuracy while using far fewer trainable parameters."},{"cited_title":"icarl: Incremental classifier and representation learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Defines the replay-based iCaRL baseline and the Avg/Last evaluation protocol the paper follows."},{"cited_title":"Maintaining discriminationandfairnessinclassincrementallearning,in:Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Is the classifier-rebalancing baseline (WA) compared on both ImageNet-100 and CIFAR-100 splits."}],"review_version":1}