{"id":"e38c7893-9bbf-4160-83bc-e14d121ef920","arxiv_id":"2412.12654","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"CALA learns a class-specific logit correction from fake incremental tasks and applies it to real new classes, giving small accuracy gains on three FSCIL benchmarks.","lead":"The paper proposes CALA, a plug-in module that rescales the output scores of a frozen classifier to reduce confusion between old and new classes in few-shot class-incremental learning. Tests on three benchmarks show small gains, but the main state-of-the-art comparison uses a stronger base model than the competing methods and the theoretical justification is flawed.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The mini-ImageNet SOTA claim is confounded: CALA's backbone gives session-0 83.98 and its scalar-α baseline 56.65, both already above prior best 54.04, so the 57.75 result is not shown to come from the class-aware adapter.","rationale":"I treat the central claim as the empirical SOTA claim in Table 2 plus the method's mechanism. The weakest link is the comparison protocol. In Table 2, all non-CALA rows have session-0 accuracy between 72.17 and 80.47, while CALA and LA-fscil start at 83.98; the base-session gap is as large as the final reported gain. Table 4 is a within-backbone experiment: applying CALA to SAVC's reproduced backbone improves last-session accuracy by only +1.02, to about 55.06, below both LA-fscil (56.65) and CALA standalone (57.75). The authors' LA-fscil baseline also uses an 'optimal' α chosen after seeing the test sessions, so the diagnostic may be optimistic. A matched-backbone run is therefore the decisive missing experiment. I do not claim the adapter does nothing; the plug-and-play numbers suggest a small gain. But the paper's headline SOTA claim is not supported as written. The reader's stated weakest assumption was transfer of the mixup-trained adapter; I partially agree, but Table 4 provides some transfer evidence, whereas the backbone confound is directly quantified in the paper. I also note the theory in Section 3.4 does not establish Fisher consistency and Eq. 9 contains an unreported μ; these add to the REJECT case but are secondary to the controlled-comparison problem. The absence of released code amplifies the reproducibility risk.","tokens_in":15012,"tokens_out":14751,"duration_ms":143541,"concrete_test":"Run the CALA adapter on the exact SAVC checkpoint (or exact SAVC backbone pretraining protocol) used for the ⋄ row in Table 4, and compare last-session mini-ImageNet accuracy to Table 2's 57.75; also run LA-fscil with that same SAVC backbone. If SAVC+CALA lands near 55.06 (i.e., only the +1.02 plug-in gain) and LA-fscil with the SAVC backbone lands near 55.6, the claimed 57.75 SOTA margin is a backbone artifact and the headline comparison should be withdrawn or reframed as a small plug-in gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing problem is in the empirical core: the state-of-the-art comparison in Table 2 (Section 4.3) is not a controlled comparison of methods. CALA and LA-fscil start at 83.98% base-session accuracy, whereas the reproduced baselines begin at 72.17–80.47%; the base-accuracy gap is comparable to or larger than the claimed final gain. LA-fscil—a single optimally tuned scalar added to all novel logits, with no class-aware adapter—already reaches 56.65% in the last session, 2.61 points above SAVC's 54.04%. Table 4 (Section 4.5) then shows that plugging CALA into the actual reproduced SAVC backbone improves it by only +1.02 points, i.e., to about 55.06%, which is below both LA-fscil and CALA's own 57.75%. Therefore the headline 'state-of-the-art accuracy' is largely inherited from an unmatched backbone/pre-training protocol, not produced by the class-aware adapter. This does not prove the adapter is useless—it may contribute a small gain—but it means the paper's central claim that experiments 'consistently validate the effectiveness' of CALA as a SOTA method is unsupported as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CALA (Class-Aware Logit Adapter) for few-shot class-incremental learning. After training a frozen backbone on base classes, the method runs a pseudo-incremental stage in which fake novel classes are generated by mixup of base-class images. A two-layer MLP is trained to map the cosine-similarity vector between each fake novel prototype and the base prototypes to a scalar logit bonus beta_c; at real incremental sessions the frozen adapter computes beta for each novel class and adds gamma*beta to the novel-class logits before softmax. The manuscript claims that this class-aware logit adjustment reduces novel-class confusion, yields state-of-the-art accuracy on mini-ImageNet, CIFAR-100, and CUB-200, and serves as a plug-and-play module for existing incremental-frozen methods.","tokens_in":15324,"tokens_out":6073,"duration_ms":57005,"significance":"If the transfer from mixup-generated pseudo-novel classes to real novel classes holds, CALA would be a simple and useful classifier-level module that could be added to many FSCIL methods without retraining the backbone. The paper includes several useful diagnostic elements: an FPR analysis, confusion-matrix visualizations, t-SNE plots, and plug-and-play experiments on three baselines. However, the evidence as presented is not sufficient to support the central state-of-the-art claim. The main Table 2 comparison is confounded by an unmatched base-session accuracy gap, the mini-ImageNet SOTA margin largely disappears when CALA is added to a reproduced baseline in Table 4, the theoretical Fisher-consistency argument in Section 3.4 is derived from a loss that omits the actual adapter terms, and the similarity/regularizer analysis in Section 4.5 is partly circular because Eq. 9 directly enforces the reported relationship. The core idea remains plausible, but the paper currently overclaims what it demonstrates.","major_comments":[{"comment":"The state-of-the-art comparison is not a controlled comparison of methods. CALA and LA-fscil both begin at 83.98% base-session accuracy, whereas the compared methods in Table 2 start at 72.17% to 80.47%. The claimed final-session margin (57.75% versus SAVC's 54.04%) is therefore largely inherited from the stronger backbone/base-training protocol rather than from the class-aware adapter. This is confirmed by Table 4, where adding CALA to the reproduced SAVC backbone improves the last session by only +1.02 points, to about 55.06%, which is below both LA-fscil (56.65%) and CALA's own headline number. To support the SOTA claim, the authors must compare against baselines trained under the same protocol or report gains relative to matched base-session accuracy.","section":"§4.3, Table 2"},{"comment":"The Fisher-consistency analysis is not about CALA as defined. Equation (11) is obtained by dropping the logit-adjustment terms and the regularization term from Eq. (9), reducing the loss to log(1 + sum_{k neq j} e^{z_k - z_j}) for a single class. Moreover, z_j is produced by the frozen backbone and is not optimized by L_CALA; the trainable parameters are the adapter weights phi. The argument that increasing z_j decreases the loss therefore does not establish any property of the adapter or of the class-aware balancing factor. Either derive the consistency statement for the actual L_CALA including the beta terms and regularizer, or remove this theory subsection as unsupported.","section":"§3.4, Eq. (9)–(11)"},{"comment":"The load-bearing transfer assumption is asserted but not validated. The adapter is trained on fake novel classes constructed by linearly mixing base-class images with lambda in [0.4, 0.6], and the same trained mapping is then applied to real incremental sessions. Because mixup features are convex combinations of base features, their cosine similarities to base prototypes may be systematically different from those of genuine novel classes, which would miscalibrate beta. The paper should provide direct evidence that the pseudo-task similarity geometry matches the real-task geometry, for example by comparing the distribution of novel-to-base similarity scores between pseudo-novel and real-novel classes, or by evaluating the adapter on held-out tasks before claiming generalization.","section":"§3.3, Eq. (1), and §3.4–3.5"},{"comment":"The empirical claim that larger novel-to-base similarity leads to larger beta_c is to a large degree enforced by the loss rather than discovered by the adapter. Equation (9) includes the regularization term sum_c (beta_c - mu ||S_c||_2), which explicitly drives beta_c toward a scaled version of the similarity norm. Presenting Fig. 7 as evidence that 'the greater the similarity, the larger the adjustment' is therefore circular unless the same relationship is shown when this regularizer is absent or its strength is varied. Please report an ablation with mu = 0 or otherwise separate the learned component of beta from the component imposed by the regularizer.","section":"§4.5, Fig. 7, and §3.4, Eq. (9)"}],"minor_comments":[{"comment":"The notation [0, alpha_tilde] is not defined; please state explicitly that 0 is a B-dimensional zero vector and that the bracket denotes concatenation, and clarify the distinction between the scalar alpha_c and the vector alpha_tilde.","section":"§3.3, Eq. (5)"},{"comment":"Equation (6) defines S_tilde_c as a vector of B cosine similarities, while Eq. (9) uses ||S_tilde_c||_2; please clarify the norm notation. Also, since the input to the MLP is softmax-normalized in Eq. (7) but the regularizer uses the L2 norm of the unnormalized vector, the relationship between the two quantities should be stated explicitly.","section":"§3.4, Eq. (6) and Eq. (9)"},{"comment":"Several hyperparameters of the pseudo-training stage are missing: the number of pseudo-incremental sessions T_tilde, the number of pseudo-classes per session, the number of mixup samples per class, and the termination criterion for the 'while not done' loop in Algorithm 1. The 20 epochs mentioned in the text should be tied to the loop structure.","section":"§4.2 and Algorithm 1"},{"comment":"The CIFAR-100 and CUB-200 results are presented only as performance curves without numerical per-session values or standard deviations. Please provide tables or numeric listings so that the claimed consistent improvement can be checked.","section":"§4.3, Fig. 3"},{"comment":"The 'Improvement with CALA' rows report only deltas; the actual accuracies of the CALA-augmented baselines are not shown. This makes it impossible to see whether the final accuracies are close to the baselines, and the very large harmonic-mean gains should be accompanied by the separate base and novel accuracies used to compute them.","section":"§4.5, Table 4"},{"comment":"The text following the Figure 3 caption contains a long run of '/uni000000...' escape-like tokens. If this text is present in the submitted manuscript it must be removed; if it is an extraction artifact, the figure and caption should be re-checked in the source file.","section":"Figure 3 caption area"},{"comment":"The statement that the red lines 'better fit the black dashed line' after CALA is a visual claim. Please add a quantitative measure of logit imbalance, such as the average base-versus-novel logit gap, rather than relying on inspection of the plot.","section":"§4.4, Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The reader's rejection is understandable: the headline SOTA result in Table 2 is not a controlled comparison, and the theory subsection does not analyze the actual loss. However, the plug-and-play experiments in Table 4 do show small, consistent gains on three reproduced baselines, which suggests the core module may have some value. I therefore recommend a major revision that requires the authors to either redo the comparisons under matched protocols or substantially scale back the SOTA claims, to fix or remove the Fisher-consistency discussion, and to provide explicit evidence for the mixup-to-real transfer assumption."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CALA is a legitimate incremental idea: a small MLP that maps a novel class's cosine similarity to base prototypes into a logit bonus, trained on mixup-generated pseudo tasks. That is a sensible extension of logit adjustment to FSCIL, and the plug-and-play results in Table 4 are the most credible part—consistent small gains around one point on the last session across three reproduced baselines. I believe those numbers.\n\nThe problem is the main SOTA claim. In Table 2, CALA and LA-fscil start at 83.98 base accuracy while every reproduced baseline starts at 72–80. That 3.5-point gap at session 0 is larger than the claimed final-session advantage over SAVC. LA-fscil is just one tuned scalar added to all novel logits, and it already gets 56.65, ahead of SAVC's 54.04. So the 57.75 for CALA is almost entirely inherited from the stronger backbone or training protocol, not from the class-aware adapter. The stress-test note is right: Table 4 shows CALA on the reproduced SAVC backbone gains only +1.02, to 55.06, below both LA-fscil and CALA's own reported 57.75. That does not prove the adapter is useless, but it means the paper's central claim is unsupported as written.\n\nThe theory section 3.4 is also not about CALA. Eq. 11 drops the adjustment term and the regularizer from Eq. 9; it is just the plain softmax cross-entropy for one class. The Fisher consistency argument never engages with the actual loss. That is a real gap, though fixable. Also, Eq. 9's regularizer is ambiguous—is it (β_c − μ||S_c||₂)² or without the square?—and μ is never reported in the implementation details. Minor but annoying.\n\nThe transfer from mixup-fake novel classes to real novel classes is asserted rather than demonstrated. That is the load-bearing premise, and the paper gives no evidence that the confusion geometry matches. Still, that is a typical assumption in FSCIL pseudo-task training, not a fatal flaw on its own.\n\nWho is this for? People working on prototype-based FSCIL might find the plug-in idea worth a look, and the comparison failure is a useful cautionary tale about base-session matching. But as a claim to state-of-the-art, it does not hold. I would want matched-backbone comparisons, a corrected theory statement, and released code before taking the headline seriously.\n\nRecommendation: send it to review. There is enough of a kernel here, and a good reviewer could force the authors to fix the confound and the theory. Expect heavy revision; if the authors cannot show gains over a single scalar on matched backbones, the method's contribution shrinks to a small plug-in improvement.","headline":"Plausible plug-in idea for FSCIL, but the headline SOTA claim is confounded by an unmatched backbone and the theory section skips the actual loss.","tokens_in":15850,"tokens_out":2956,"would_cite":false,"duration_ms":25260,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a class-aware logit adapter, trained on pseudo-incremental tasks, generates per-class logit balancing factors that correct confused novel classes, and reports state-of-the-art FSCIL accuracy on three benchmarks.","keywords":["few-shot class-incremental learning","logit adjustment","class-aware adapter","prototype similarity","mixup","pseudo-incremental learning","plug-and-play module","novel class confusion"],"falsifier":"One concrete falsifier: build a FSCIL split whose novel classes are deliberately dissimilar from every base class (low cosine prototype similarity), and compare CALA's $\\beta$ values against a class-agnostic constant. If the adapter still issues large bonuses to these dissimilar classes, or the accuracy gain vanishes, the learned similarity-to-bonus mapping does not generalize from mixup-fake to real novel classes.","tokens_in":14813,"feed_emoji":"🎯","tokens_out":7256,"duration_ms":58153,"temperature":0.7,"pith_summary":"This paper tries to establish that the residual error in few-shot class-incremental learning (FSCIL) is a class-sensitive bias at the logit level, and that it can be corrected by a learned class-aware logit adapter. The typical incremental-frozen framework trains a backbone on abundant base classes and then freezes it, leaving novel classes systematically confused with the base classes they resemble. CALA adds a small adapter that, for each novel class, maps the cosine similarity between that class's prototype and all base-class prototypes to a scalar logit bonus; the bonus is added to the novel-class logit before the softmax. The adapter is trained offline in pseudo-incremental sessions whose fake novel classes are made by mixing base-class images, then frozen and applied to real incremental sessions. If the transfer from fake to real classes holds, CALA is a plug-and-play classifier-level correction for any frozen-backbone FSCIL method, and the paper reports consistent improvements on mini-ImageNet, CIFAR-100, and CUB-200.","feed_headline":"Plug-in logit adapter lifts few-shot incremental accuracy to 57.75%","feed_subtitle":"Class-aware logit bonus rescues novel classes confused with base classes, improving FSCIL baselines without retraining.","key_machinery":"The central object is the class-aware logit adapter, a two-hidden-layer MLP $g_\\phi$ that maps a normalized prototype-similarity vector to a single scalar. The similarity vector for a novel class $c$ is $S_c = [\\mathrm{sim}(p_c,p_0^1),\\dots,\\mathrm{sim}(p_c,p_0^B)]$, where $p_c$ is the prototype of the novel class, $p_0^b$ are the frozen base-class prototypes in the classifier $W_0$, and $\\mathrm{sim}$ is cosine similarity. The scalar output $\\beta_c = g_\\phi(\\sigma(S_c))$ is added element-wise to the novel-class logits as $\\hat{z} = z + [0,\\gamma \\beta]$, so that classes resembling base classes receive larger corrections. The machinery also includes the pseudo-incremental training loop: mixup fabricates fake novel classes from disjoint base-class pairs, their prototypes are computed with the frozen backbone, the MLP is trained to predict logit bonuses, and the loss $L_{\\mathrm{CALA}} = -\\sum_j \\tilde{Y}_j \\log \\tilde{p}_j + \\sum_c (\\beta_c - \\mu \\|S_c\\|_2)$ ties the bonus to the magnitude of the similarity vector. This coupling between similarity and bonus is what carries the transfer from pseudo-trained to real incremental sessions.","core_discovery":"At the level of the paper's own argument, CALA's discovery is that the confusion between novel and base classes in FSCIL is not uniform across novel classes, so a single class-agnostic logit adjustment factor $\\alpha$ is insufficient. The paper proposes that the required correction for a novel class $c$ should grow with its similarity to base classes, and implements this as a class-aware adapter $g_\\phi$ that takes the similarity vector $S_c = [\\cos(p_c,p_0^1),\\dots,\\cos(p_c,p_0^B)]$ and outputs a scalar $\\beta_c$. The adjusted logit for the novel part is $\\hat{z} = z + [0,\\gamma \\beta]$, with $\\gamma$ controlling the overall strength. The adapter is trained by constructing fake novel classes via mixup of base-class instances, computing their prototypes, and optimizing a logit-adjustment loss whose simplified form $L_{\\mathrm{CALA}} = \\log(1+\\sum_{k\\neq j} e^{z_k-z_j})$ shows Fisher consistency: raising the target novel-class logit lowers the loss. In the real FSCIL stage the adapter is frozen and applied to genuine novel-class prototypes. On mini-ImageNet the method reports last-session accuracy 57.75% against a previous best of 54.04%, with gains also reported on CIFAR-100 and CUB-200.","pith_inferences":["Editorial inference: if the mixup-to-real transfer is robust, the same prototype-similarity-to-bonus mapping could be applied to other prototype-based continual learning settings with frozen backbones, such as online or task-free continual learning, where class confusion is likewise driven by feature-space overlap.","Editorial inference: the adapter's $\\beta$ could double as an uncertainty signal; classes whose $\\beta$ is high are those most entangled with base classes, so abstention or a separate rejection rule on those classes might further improve novel-class precision.","Editorial inference: the need to tune $\\gamma$ suggests a testable extension in which the adapter also outputs $\\gamma$, or $\\gamma$ is set per session from the empirical similarity distribution, removing the global hyperparameter search."],"forward_implications":["On mini-ImageNet, CALA raises last-session accuracy to 57.75%, up from the 54.04% of the strongest quoted baseline, and improves average accuracy over all sessions.","As a plug-and-play module, CALA improves three existing incremental-frozen methods (CEC, BiDist, SAVC) on mini-ImageNet, with gains concentrating in later sessions; the last-session harmonic mean of base and novel accuracy rises by 9.60, 9.19, and 3.97 points, respectively.","Because the correction is applied at the classifier level, the feature space and backbone are untouched, so CALA can be added to any prototype-based incremental-frozen FSCIL method without retraining the backbone.","The learned $\\beta_c$ grows with novel-to-base similarity, matching the paper's motivation that more similar novel classes need stronger correction; the similarity analysis across datasets and within mini-ImageNet classes supports this monotone relationship.","The class-aware logit-adjustment loss is Fisher consistent: for a single novel class the loss simplifies to $\\log(1+\\sum_{k\\neq j} e^{z_k-z_j})$, so raising the correct novel-class logit always decreases the loss."],"supporting_citations":[{"why":"supplies the logit-adjustment loss and Fisher-consistency reasoning that CALA's class-aware loss adapts for FSCIL","marker":"[3]"},{"why":"provides the mixup operation used to fabricate fake novel classes for pseudo-training","marker":"[42]"},{"why":"defines the FSCIL setting and the dataset splits and baselines the paper follows","marker":"[28]"},{"why":"identifies the novel-class confusion problem and the feature-level calibration approach that CALA contrasts with at the classifier level","marker":"[34]"},{"why":"supplies an incremental-frozen framework and baseline used for comparisons and plug-and-play tests","marker":"[44]"},{"why":"provides a forward-compatible baseline whose results CALA builds on and exceeds","marker":"[45]"},{"why":"provides a strong baseline and the prior best last-session accuracy that CALA reports surpassing","marker":"[26]"},{"why":"provides a continually evolved classifier baseline used both as comparison and as plug-and-play test bed","marker":"[41]"}],"fun_headline_variants":["Class-aware logit adapter boosts few-shot incremental accuracy","Similarity-based logit rectification lifts FSCIL to 57.75%","CALA: light-weight adapter fixes novel-base confusion","Plug-in adapter improves few-shot class-incremental learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fake novel classes created by mixing base-class images trigger the same kind of classification mistakes as real novel classes, so the adapter's learned rule for how much to boost a class generalizes to classes it never saw.","fun_headline_variants_meta":{"raw":{"variants":["Class-aware logit adapter boosts few-shot incremental accuracy","Similarity-based logit rectification lifts FSCIL to 57.75%","CALA: light-weight adapter fixes novel-base confusion","Plug-in adapter improves few-shot class-incremental learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000413,"raw_usage":{"total_tokens":2199,"prompt_tokens":1070,"completion_tokens":1129,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":1058}},"tokens_in":686,"tokens_out":1129,"duration_ms":9035,"temperature":1.0,"reasoning_tokens":1058,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:51:38.654006+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete falsifier: build a FSCIL split whose novel classes are deliberately dissimilar from every base class (low cosine prototype similarity), and compare CALA's $\\beta$ values against a class-agnostic constant. If the adapter still issues large bonuses to these dissimilar classes, or the accuracy gain vanishes, the learned similarity-to-bonus mapping does not generalize from mixup-fake to real novel classes.","supporting_citations":[{"cited_title":"Few-shot class- incremental learning via class-aware bilateral distillation","cited_arxiv_id":null,"evidence_quote":"supplies an incremental-frozen framework and baseline used for comparisons and plug-and-play tests"},{"cited_title":"Learning imbalanced datasets with label- distribution-aware margin loss","cited_arxiv_id":null,"evidence_quote":"supplies the logit-adjustment loss and Fisher-consistency reasoning that CALA's class-aware loss adapts for FSCIL"},{"cited_title":"mixup: Beyond empirical risk minimiza- tion","cited_arxiv_id":null,"evidence_quote":"provides the mixup operation used to fabricate fake novel classes for pseudo-training"},{"cited_title":"Few-shot class- incremental learning","cited_arxiv_id":null,"evidence_quote":"defines the FSCIL setting and the dataset splits and baselines the paper follows"},{"cited_title":"Few-shot class-incremental learning via training-free prototype calibration","cited_arxiv_id":null,"evidence_quote":"identifies the novel-class confusion problem and the feature-level calibration approach that CALA contrasts with at the classifier level"},{"cited_title":"Forward compatible few-shot class-incremental learning","cited_arxiv_id":null,"evidence_quote":"provides a forward-compatible baseline whose results CALA builds on and exceeds"},{"cited_title":"Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning","cited_arxiv_id":null,"evidence_quote":"provides a strong baseline and the prior best last-session accuracy that CALA reports surpassing"},{"cited_title":"Few-shot incremental learning with contin- ually evolved classifiers","cited_arxiv_id":null,"evidence_quote":"provides a continually evolved classifier baseline used both as comparison and as plug-and-play test bed"}],"review_version":1}