Pith. sign in

REVIEW 3 major objections 5 minor 61 references

MoTE: Mixture of Task-specific Experts for Pre-Trained ModelBased Class-incremental Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MoTE claims that an exemplar-free ensemble of per-task adapters, filtered by task-scope reliability and fused with confidence-weighted features, achieves the highest average accuracy on five class-incremental benchmarks.

desk verdict A solid, incremental improvement over EASE with a heuristic filtering trick that works on the tested benchmarks; worth reviewing, but the 'SOTA' phrasing overreaches. read the letter →

arxiv 2506.11038 v1 pith:D4L72JTU submitted 2025-05-21 cs.LG

Linjie Li , Zhenyu Wu , Yang Ji This is my paper · ORCID
classification cs.LG
keywords Class-incrementallearningPre-trainedmodelsMixtureofexpertsAdaptersPrototypeclassifierExemplar-freecontinualPrompt-basedmethodsTask-specific
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Class-incremental learning asks a model to keep learning new classes from a stream without forgetting old ones, and this paper attacks the version in which no old examples are stored. MoTE trains one small adapter per task on a frozen pre-trained vision transformer, discards the task classifiers, and classifies by cosine similarity to per-class mean prototypes. At inference it applies a routing-like step: an adapter is kept only if its top prediction falls inside its own task's label set, and kept adapters' features are merged with weights derived from confidence and self-confidence. The paper reports that this scheme beats prompt-based and adapter-based baselines on CIFAR100, CUB200, ImageNet-A, ImageNet-R, and VTAB, runs roughly 30 percent faster than the closest per-task-adapter competitor, and stores about $1/n$ of that competitor's prototype memory. A limited-adapter variant maps how accuracy depends on the number of adapters.

What carries the argument

The load-bearing object is a per-task adapter that acts as an expert over a frozen pre-trained transformer, together with a prototype pool $P^j_t = \frac{1}{|D_j|}\sum_{x\in D_j} f(x; A_t(x))$ for each class. At inference, every adapter's feature is compared against all prototypes by cosine similarity. Expert filtering keeps adapter $k$ only if its top-1 predicted class $c_k$ lies in that task's label set $C_k$. The kept experts' features are then fused as $F_{\text{mix}} = \sum_i w_i F_i$ with weights $w_i = z_i^{(1)} + \gamma s_i$, where $z_i^{(1)}$ is the top logit, $s_i = (z_i^{(1)} - z_i^{(2)})/z_i^{(1)}$ is the self-confidence score, and $\gamma$ is set adaptively to $z_i^{(1)}$ instead of being tuned as a global hyperparameter. This combination of scope-based filtering and confidence/self-confidence weighting is what emulates a routing layer without training one.

What would settle it

Run the CIFAR100 B0-Inc10 protocol with tasks deliberately made semantically overlapping (e.g., grouping visually similar classes together) and measure, for each adapter, the fraction of out-of-task test images whose top-1 cosine-similarity class lies inside that adapter's own label set; if that fraction is high and MoTE's accuracy drops to the level of the unfiltered-average ablation, the reliability filter is not the source of the gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that the dimensional misalignment that cripples naively expanded adapter ensembles can be avoided by treating each adapter as an isolated task expert and postponing all interaction to inference. Training a new adapter never touches old adapters or the backbone, so old-task representations are literally frozen; the only cross-task operation is a weighted mixture of the experts' features at test time. The paper reports that this scheme reaches the highest average accuracy on every benchmark setting it tests, with the largest gains on ImageNet-A, and that it does so without storing any exemplars, while running roughly 30 percent faster than EASE and storing about $1/n$ of EASE's prototype memory for $n$ tasks. On average forgetting, MoTE does not claim the lowest numbers; the paper attributes that to the frozen backbone's zero-shot floor keeping weak methods stable. The Adapter-Limited variant makes the secondary claim that accuracy as a function of adapter count is not linear, and that a single adapter already exploits much of the pre-trained model's transferable ability.

Load-bearing premise

The method assumes that an adapter trained on one task will not confidently place images from other tasks into its own classes; when that fails, the filtering rule keeps a wrong expert and the weighted fusion is polluted.

Editorial extensions

If this is right

  • Any new task can be added by training one adapter on its data; previous adapters and prototypes are untouched, so old-task representations cannot be overwritten by later training.
  • Task identity does not need to be known at test time; the scope-based filtering rule recovers it from prediction patterns, so the method works under task-agnostic inference.
  • Because only reliable experts participate, inference cost grows more slowly than a full ensemble; MoTE reports roughly 30 percent faster inference than EASE as the number of tasks grows.
  • Storing one mean prototype per real class instead of pseudo-prototypes for every old class on every new adapter cuts prototype memory by a factor of about $1/n$ relative to EASE.
  • Adapter-Limited MoTE indicates that even a single adapter retains the pre-trained model's zero-shot baseline, so the accuracy gain from adding experts is modest on easy datasets and larger on harder ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the reliability filter holds on more overlapping task distributions than the tested benchmarks, the same architecture plausibly extends to cross-dataset streams, where the paper's preliminary experiment shows smaller but still positive gains.
  • Beyond the paper: a softer, learned gate could replace the hard task-scope threshold and would let the framework handle class overlap or multi-label samples, which the paper explicitly leaves out of scope.
  • Beyond the paper: because each adapter is trained in isolation and never revised, MoTE's experts cannot repair each other's blind spots; a testable extension is a post-hoc calibration pass that reweights experts using only stored prototypes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MoTE, an exemplar-free class-incremental learning method built on a frozen ViT backbone. MoTE trains one lightweight adapter per task as a task-specific expert, computes class prototypes per task, and at inference time evaluates all adapters on the test sample. It then filters out experts whose top-1 prototype match falls outside their own task's label set, and fuses the features of the remaining experts with weights derived from confidence and a self-confidence score. A variant, Adapter-Limited MoTE, restricts the number of adapters and constructs prototypes for new classes through existing adapters. The method is evaluated on CIFAR100, CUB, ImageNet-A, ImageNet-R, and VTAB under several incremental protocols, and the paper reports higher average accuracy than L2P, DualPrompt, CODA-Prompt, SimpleCIL, Adam variants, and EASE, while also claiming roughly 30% faster inference and lower prototype memory than EASE. A short cross-dataset experiment is included in the conclusion.

Significance. If the empirical claims hold, MoTE would be a simple and efficient addition to the PTM-based CIL toolbox: it avoids training a routing network, uses only prototypes, and the code is publicly released. The paper has several concrete strengths: main results are averaged over five seeds, the ablation study shows monotonically increasing accuracy as components are added, and the inference-time and memory comparisons against EASE are useful. The cross-dataset exploration also goes beyond the standard single-domain benchmark. However, the central claim of state-of-the-art average accuracy is weakened by the omission of two strong and standard baselines, RanPAC and SLCA, and the core expert-filtering mechanism relies on an assumption about cross-adapter feature comparability that is not directly validated. These issues are fixable but are load-bearing for the paper's main conclusions.

major comments (3)
  1. [§5.2, Table 1] The claim that 'MoTE performs best under all experimental settings' is not supported by the comparison as presented, because RanPAC [55] and SLCA [54]—both standard PTM-based CIL methods cited in the related work—are absent from Table 1. Given that the margins over EASE are small (e.g., 92.95 vs 92.40 on CIFAR100 B0-Inc10 and 67.26 vs 64.36 on INA B0-Inc20), including these baselines under the same ViT-B/16-IN21K backbone could change the ranking. The authors should add these methods and report Avg and AF for all settings, or explicitly restrict the claim to the methods compared.
  2. [§4.2, Eq. (8)] The expert filtering rule is underspecified and mixes incompatible feature spaces. The text says each expert's output is compared with 'all class prototypes,' but the formula c_k = argmax cos(F_k, P^j_k) uses the notation of Eq. (8), where P^j_t is defined only for classes of task t using adapter A_t. If c_k is computed only over task k's own prototypes, then c_k ∈ C_k by construction and no expert could ever be filtered; if c_k is computed over all prototypes from all tasks, then the scores mix features generated by different adapters, so the top-1 match is not a well-defined task-membership test. The paper's own Table 4 shows TIA of only 61.23% on INA B0-Inc20, indicating that the filtering criterion is frequently wrong on that benchmark. To justify the core mechanism, the paper should clarify the notation, report the retention rate of off-task experts, and compare the filtering step against an oracle task filter or against unfiltered fusion with a controlled amount of noise.
  3. [§5.1.4 and §3.2] Reproducibility is incomplete. The paper does not specify the adapter bottleneck size r, which layers of the ViT receive adapters, or the exact form of the 'parallel' adapter used as the default. In particular, Eq. (6) includes an extra additive h_msa term that is not part of the standard parallel adapter formula and is not explained. Since Table 5 shows that the sequential and parallel embedding choices give different results, the exact default architecture must be stated precisely. The training schedule is given, but the per-task adapter hyperparameters are missing, and no random seed is reported for the adapter initialization beyond the five benchmark seeds.
minor comments (5)
  1. [§5.3.1, Table 3] The prose says configuration #3 is 'weighted mixing of all experts based on their confidence levels,' but in Table 3 filtering is enabled for #3, #4, and #5. Because filtering and weighting are coupled throughout the table, the individual contribution of confidence weighting versus self-confidence weighting is not cleanly isolated. A configuration with weighting but without filtering would make the ablations more informative.
  2. [§5.4.3, Figure 10] There is an inconsistency between the text and the figure caption: the text says the visualization uses CIFAR B0-Inc5, ImageNet-R B0-Inc5, and VTAB B0-Inc5, while the caption for panels (g)-(i) says VTAB B0-Inc10. Please align the protocol names.
  3. [§4.4 and §5.4.4] The Adapter-Limited MoTE prototype-construction rule is described only verbally and the experimental results in Figure 11 are shown without error bars or a table of exact values. Adding the formal definition of the merging operation and reporting means and standard deviations over the five seeds would make the adapter-count trade-off analysis reproducible and easier to interpret.
  4. [§4.3, Eq. (10)] The self-confidence score s_i is defined as (z_1st - z_2nd)/z_1st, but the paper does not define the logits z_i used in the prototype classifier. If these are cosine similarities, they can be negative, and the ratio becomes ill-conditioned when the top score is near zero or negative. Please specify the exact logit definition and discuss how negative scores are handled.
  5. [§5.2, AF discussion] The explanation for why MoTE does not achieve the best AF on some benchmarks relies on the zero-shot lower bound of the pre-trained model. This verbal argument would be more convincing if accompanied by per-task forgetting curves or a quantitative comparison of the lower-bound contribution across methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MoTE is an empirically evaluated expert-filtering and weighted-fusion method; its components are tested by ablations, not derived from their own outputs, and the self-citations are not load-bearing.

full rationale

The paper's central claims are benchmark performance comparisons (Table 1, Table 2) and ablations (Table 3, Table 4) of an inference-time mechanism: task-aware expert filtering (§4.2) and confidence/SCS-weighted joint inference (§4.3). These are not derived quantities; they are algorithmic choices whose effects are measured against external baselines such as L2P, DualPrompt, CODA-Prompt, Adam, and EASE. The 'reliable expert' criterion c_k ∈ C_k in §4.2 is an operational definition of reliability, not a parameter fitted to the target accuracy, and the paper's own ablation (#1→#2) tests rather than assumes its benefit. The adaptive setting γ = z1st in Eq. 11 is a fixed design rule, not a fitted prediction. Self-citations ([1], [42]) appear only in background and related-work contexts and are not load-bearing; no uniqueness theorem or prior-work premise is used to force the method. The behavioral assumption about adapter confusion patterns is a correctness premise, not circularity: it is externally testable and does not make the reported results equivalent to the input by construction. Therefore no circular step is exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on standard CIL assumptions (disjoint tasks, known label scopes), on the transferability of a frozen ViT backbone, on prototype-based classification, and on the heuristic that a task adapter's top-1 class membership is a trustworthy task-identity signal. No formal derivation is provided; the results are empirical and depend on the unstated adapter capacity and training hyperparameters.

free parameters (4)
  • Adapter bottleneck size r = not reported
    Capacity of each task expert influences feature quality; the value is not given in Section 5.1.4.
  • Adapter embedding type (parallel vs sequential) = parallel (Par)
    Chosen after comparing Seq and Par in Table 5; affects how adapter outputs enter the backbone.
  • Scaling factor gamma in Eq. 11 = adaptive, set to z_1st
    The adaptive setting was selected after comparing fixed values 0.1, 0.5, 1, 2 in Fig 8, so the choice is informed by benchmark performance.
  • Training schedule (SGD lr, weight decay, epochs, batch size) = lr 0.01, wd 0.005, 20 epochs, batch 48
    These hyperparameters affect adapter convergence and are reported but were presumably tuned; no sensitivity analysis is given.
assumptions (5)
  • domain assumption Pre-trained ViT-B/16-IN21K features transfer to all five downstream datasets with only small adapters.
    Section 3.2 and Section 5.1.4 freeze the PTM and rely on its representations for prototypes and classification.
  • domain assumption Task label sets are disjoint and known at inference; each sample belongs to exactly one task.
    Stated in Section 3.1 and used by expert filtering in Section 4.2 to define C_k.
  • domain assumption Cosine similarity to mean class prototypes is a sufficient classifier for adapted embeddings.
    Prototype classifier in Eq. 8-9, adopted from prior CIL work [21, 31, 32]; assumes each class embedding is unimodal.
  • domain assumption Each task-specific adapter trained with cross-entropy separates its own classes from other tasks' classes well enough for filtering to be accurate.
    The filtering rule in Section 4.2 fails if adapters confidently misclassify out-of-task samples into their own label sets.
  • ad hoc to paper The self-confidence score (top minus second logit divided by top logit) measures expert reliability.
    Eq. 10 introduces SCS as a heuristic; no theory is given beyond ablation results in Table 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MoTE: Mixture of Task-specific Experts for Pre-Trained ModelBased Class-incremental Learning." pith.science (2026). https://pith.science/paper/D4L72JTU

@misc{pith2026250611038,
  author       = {Pith},
  title        = {Pith review of: MoTE: Mixture of Task-specific Experts for Pre-Trained ModelBased Class-incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D4L72JTU}},
  note         = {Machine review of arXiv:2506.11038}
}
read the original abstract

Class-incremental learning (CIL) requires deep learning models to continuously acquire new knowledge from streaming data while preserving previously learned information. Recently, CIL based on pre-trained models (PTMs) has achieved remarkable success. However, prompt-based approaches suffer from prompt overwriting, while adapter-based methods face challenges such as dimensional misalignment between tasks. While the idea of expert fusion in Mixture of Experts (MoE) can help address dimensional inconsistency, both expert and routing parameters are prone to being overwritten in dynamic environments, making MoE challenging to apply directly in CIL. To tackle these issues, we propose a mixture of task-specific experts (MoTE) framework that effectively mitigates the miscalibration caused by inconsistent output dimensions across tasks. Inspired by the weighted feature fusion and sparse activation mechanisms in MoE, we introduce task-aware expert filtering and reliable expert joint inference during the inference phase, mimicking the behavior of routing layers without inducing catastrophic forgetting. Extensive experiments demonstrate the superiority of our method without requiring an exemplar set. Furthermore, the number of tasks in MoTE scales linearly with the number of adapters. Building on this, we further explore the trade-off between adapter expansion and model performance and propose the Adapter-Limited MoTE. The code is available at https://github.com/Franklilinjie/MoTE.

Figures

Figures reproduced from arXiv: 2506.11038 by the authors.

Figure 1
Figure 1. Overview of PTM-based CIL Algorithms, these algorithms freeze the pre-trained [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of research motivation and preliminary experiment. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Illustrative comparison of related methods. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Structure of the adapter and two embedding types: sequential(Seq) and paral [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Illustration of MoTE. Left: For the n-th task, we freeze the pre-trained backbone and train An as the task-specific expert, extracting class prototypes. Middle: During in￾ference, features extracted by different experts are compared with the class prototype pool. Exper…
Figure 5
Figure 5. Figure 5: When multiple reliable experts are identified for a given sample, [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Illustration of Adapter Limited MoTE. The number of adapters is assumed to [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison of different methods across various experimental settings, all initialized with (ViT-B/16-IN21K). 5.2. Benchmark Comparison In this section, we compare the performance of MoTE with other state￾of-the-art methods. To evaluate the robustness of the…
Figure 8
Figure 8. Figure 8: Performance Comparison of Global vs. Adaptive Scaling, all initialized with (ViT-B/16-IN21K) (a) CIFAR100 B0 Inc10 (b) CUB B0 Inc20 (c) INA B0 Inc10 (d) VTAB B0 Inc10 (e) INA B0 Inc4 [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: Comparison of Inference Time with EASE. MoTE and EASE are based on the same PTM (ViT-B/16-IN21K). 5.4.2. Comparison of Inference Time and Memory Costs In this subsection, we present a detailed comparison of EASE and MoTE in terms of inference time and memory consumptio…
Figure 10
Figure 10. Figure 10: Visualization of t-SNE [61] for feature representations from different experts. (a), (b) and (c) correspond to the experimental protocol CIFAR B0-Inc5, (d), (e), and (f) correspond to the INR B0-Inc5, while (g), (h), and (i) correspond to the VTAB B0-Inc10. Weighted M…
Figure 11
Figure 11. Figure 11: Performance comparison of different numbers of adapters under various experimental settings, all initialized with ViT-B/16-IN21K. This improvement is further verified through ablation studies presented in Tab. 3 and Tab. 4. 5.4.4. Evaluation on Adapter-limited MoTE We…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 45 canonical work pages

  1. [55]

    M. D. McDonnell, D. Gong, A. Parvaneh, E. Abbasnejad, A. van den Hengel, Ranpac: Random projections and pre-trained models for con- tinual learning, Advances in Neural Information Processing Systems 36 (2024)

    work page 2024

  2. [54]

    Zhang, L

    G. Zhang, L. Wang, G. Kang, L. Chen, Y. Wei, Slca: Slow learner with classifier alignment for continual learning on a pre-trained model, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19148–19158

    work page 2023

  3. [1]

    Z. Wu, J. Guo, Y. Liu, L. Li, Y. Ji, An iterative resampling deep de- coupling domain adaptation method for class-imbalance bearing fault diagnosis under variant working conditions, Expert Systems with Ap- plications 252 (2024) 124240

    work page 2024

  4. [2]

    Birjali, M

    M. Birjali, M. Kasri, A. Beni-Hssane, A comprehensive survey on sen- timent analysis: Approaches, challenges and trends, Knowledge-Based Systems 226 (2021) 107134

    work page 2021

  5. [3]

    Z. Wu, H. Zhang, J. Guo, Y. Ji, M. Pecht, Imbalanced bearing fault diagnosis under variant working conditions using cost-sensitive deep do- main adaptation network, Expert Systems with Applications 193 (2022) 116459. 30

    work page 2022

  6. [4]

    X. He, K. Zhao, X. Chu, Automl: A survey of the state-of-the-art, Knowledge-based systems 212 (2021) 106622

    work page 2021

  7. [5]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255

    2009

  8. [6]

    Babcock, S

    B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in: Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database sys- tems, 2002, pp. 1–16

    work page 2002

Show all 61 references
  1. [7]

    J. A. Silva, E. R. Faria, R. C. Barros, E. R. Hruschka, A. C. d. Carvalho, J. Gama, Data stream clustering: A survey, ACM Computing Surveys (CSUR) 46 (1) (2013) 1–31

    2013

  2. [8]

    Zhou, Q.-W

    D.-W. Zhou, Q.-W. Wang, Z.-H. Qi, H.-J. Ye, D.-C. Zhan, Z. Liu, Class- incremental learning: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  3. [9]

    L. Wang, X. Zhang, H. Su, J. Zhu, A comprehensive survey of continual learning: theory, methodandapplication, IEEETransactionsonPattern Analysis and Machine Intelligence (2024)

    2024

  4. [10]

    R. M. French, Catastrophic forgetting in connectionist networks, Trends in cognitive sciences 3 (4) (1999) 128–135

    1999

  5. [11]

    J. L. McClelland, B. L. McNaughton, R. C. O’Reilly, Why there are complementary learning systems in the hippocampus and neocortex: in- sights from the successes and failures of connectionist models of learning and memory., Psychological review 102 (3) (1995) 419

    1995

  6. [12]

    2001– 2010

    S.-A.Rebuffi, A.Kolesnikov, G.Sperl, C.H.Lampert, icarl: Incremental classifier and representation learning, in: Proceedings of the IEEE con- ference on Computer Vision and Pattern Recognition, 2017, pp. 2001– 2010

    2017

  7. [13]

    F. M. Castro, M. J. Marín-Jiménez, N. Guil, C. Schmid, K. Alahari, End-to-end incremental learning, in: Proceedings of the European con- ference on computer vision (ECCV), 2018, pp. 233–248. 31

    2018

  8. [14]

    Z. Li, D. Hoiem, Learning without forgetting, IEEE transactions on pattern analysis and machine intelligence 40 (12) (2017) 2935–2947

    2017

  9. [15]

    Douillard, M

    A. Douillard, M. Cord, C. Ollion, T. Robert, E. Valle, Podnet: Pooled outputs distillation for small-tasks incremental learning, in: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part XX 16, Springer, 2020, pp. 86–102

    2020

  10. [16]

    S. Yan, J. Xie, X. He, Der: Dynamically expandable representation for class incremental learning, in: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2021, pp. 3014–3023

    2021

  11. [17]

    F.-Y.Wang, D.-W.Zhou, H.-J.Ye, D.-C.Zhan, Foster: Featureboosting and compression for class-incremental learning, in: European conference on computer vision, Springer, 2022, pp. 398–414

    2022

  12. [18]

    Zhou, Q.-W

    D.-W. Zhou, Q.-W. Wang, H.-J. Ye, D.-C. Zhan, A model or 603 ex- emplars: Towards memory-efficient class-incremental learning, in: The Eleventh International Conference on Learning Representations, 2022

    2022

  13. [19]

    Kirkpatrick, R

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., Overcoming catastrophic forgetting in neural networks, Proceed- ings of the national academy of sciences 114 (13) (2017) 3521–3526

    2017

  14. [20]

    Sun, D.-W

    H.-L. Sun, D.-W. Zhou, H.-J. Ye, D.-C. Zhan, Pilot: A pre- trained model-based continual learning toolbox, arXiv preprint arXiv:2309.07117 (2023)

    2023 arXiv

  15. [21]

    Zhou, Z.-W

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, D.-C. Zhan, Z. Liu, Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need, International Journal of Computer Vision (2024) 1–21

    2024

  16. [22]

    X. Han, Z. Zhang, N. Ding, Y. Gu, X. Liu, Y. Huo, J. Qiu, Y. Yao, A. Zhang, L. Zhang, et al., Pre-trained models: Past, present and future, AI Open 2 (2021) 225–250

    2021

  17. [23]

    Z. Wang, Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, T. Pfister, Learning to prompt for continual learn- ing, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 139–149. 32

    2022

  18. [24]

    Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y. Lee, X. Ren, G. Su, V. Perot, J. Dy, et al., Dualprompt: Complementary prompt- ing for rehearsal-free continual learning, in: European Conference on Computer Vision, Springer, 2022, pp. 631–648

    2022

  19. [25]

    L. Wang, J. Xie, X. Zhang, M. Huang, H. Su, J. Zhu, Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality, Advances in Neural Information Processing Systems 36 (2024)

    2024

  20. [26]

    J. S. Smith, L. Karlinsky, V. Gutta, P. Cascante-Bonilla, D. Kim, A. Ar- belle, R. Panda, R. Feris, Z. Kira, Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning, in: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and P...

    2023

  21. [27]

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, G. E. Hinton, Adaptive mix- tures of local experts, Neural computation 3 (1) (1991) 79–87

    1991

  22. [28]

    Aljundi, P

    R. Aljundi, P. Chakravarty, T. Tuytelaars, Expert gate: Lifelong learn- ing with a network of experts, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3366–3375

    2017

  23. [29]

    J. Yu, Y. Zhuge, L. Zhang, P. Hu, D. Wang, H. Lu, Y. He, Boost- ing continual learning of vision-language models via mixture-of-experts adapters, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23219–23230

    2024

  24. [30]

    Shazeer, A

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hin- ton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, arXiv preprint arXiv:1701.06538 (2017)

    2017 arXiv

  25. [31]

    Zhou, H.-L

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, D.-C. Zhan, Expandable subspace en- semble for pre-trained model-based class-incremental learning, in: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23554–23564

    2024

  26. [32]

    Snell, K

    J. Snell, K. Swersky, R. Zemel, Prototypical networks for few-shot learn- ing, arXiv: Learning,arXiv: Learning (2017). 33

    2017

  27. [33]

    Kendall, Y

    A. Kendall, Y. Gal, What uncertainties do we need in bayesian deep learningforcomputervision?, Advancesinneuralinformationprocessing systems 30 (2017)

    2017

  28. [34]

    Krizhevsky, G

    A. Krizhevsky, G. Hinton, et al., Learning multiple layers of features from tiny images (2009)

    2009

  29. [35]

    C. Wah, S. Branson, P. Welinder, P. Perona, S. Belongie, The caltech- ucsd birds-200-2011 dataset (2011)

    2011

  30. [36]

    Hendrycks, K

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, D. Song, Natural ad- versarial examples, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15262–15271

    2021

  31. [37]

    Hendrycks, S

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al., The many faces of robust- ness: A critical analysis of out-of-distribution generalization, in: Pro- ceedings of the IEEE/CVF international conference on computer vi...

    2021

  32. [38]

    X. Zhai, J. Puigcerver, A. Kolesnikov, P. Ruyssen, C. Riquelme, M. Lu- cic, J. Djolonga, A. S. Pinto, M. Neumann, A. Dosovitskiy, et al., A large-scale study of representation learning with the visual task adapta- tion benchmark, arXiv preprint arXiv:1910.04867 (2019)

    2019 arXiv

  33. [39]

    Y. Liu, Y. Su, A.-A. Liu, B. Schiele, Q. Sun, Mnemonics training: Multi- class incremental learning without forgetting, in: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, 2020, pp. 12245–12254

    2020

  34. [40]

    L. Wang, X. Zhang, K. Yang, L. Yu, C. Li, L. Hong, S. Zhang, Z. Li, Y. Zhong, J. Zhu, Memory replay with data compression for continual learning, arXiv preprint arXiv:2202.06592 (2022)

    2022 arXiv

  35. [41]

    Z. Luo, Y. Liu, B. Schiele, Q. Sun, Class-incremental exemplar compres- sion for class-incremental learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 11371–11380. 34

    2023

  36. [42]

    L. Li, S. Liu, Z. Wu, et al., Tae: Task-aware expandable representation for long tail class incremental learning, arXiv preprint arXiv:2402.05797 (2024)

    2024 arXiv

  37. [43]

    Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, Y. Fu, Large scale incremental learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 374–382

    2019

  38. [44]

    S. Hou, X. Pan, C. C. Loy, Z. Wang, D. Lin, Learning a unified classifier incrementally via rebalancing, in: Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, 2019, pp. 831–839

    2019

  39. [45]

    P. Dhar, R. V. Singh, K.-C. Peng, Z. Wu, R. Chellappa, Learning with- out memorizing, in: Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 2019, pp. 5138–5146

    2019

  40. [46]

    Q. Gao, C. Zhao, B. Ghanem, J. Zhang, R-dfcil: Relation-guided repre- sentation learning for data-free class incremental learning, in: European Conference on Computer Vision, Springer, 2022, pp. 423–439

    2022

  41. [47]

    S. Dong, X. Hong, X. Tao, X. Chang, X. Wei, Y. Gong, Few-shot class- incremental learning via relation knowledge distillation, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 1255–1263

    2021

  42. [48]

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, S.- N. Lim, Visual prompt tuning, in: European Conference on Computer Vision, Springer, 2022, pp. 709–727

    2022

  43. [49]

    X. L. Li, P. Liang, Prefix-tuning: Optimizing continuous prompts for generation, arXiv preprint arXiv:2101.00190 (2021)

    2021 arXiv

  44. [50]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  45. [51]

    S. Chen, C. Ge, Z. Tong, J. Wang, Y. Song, J. Wang, P. Luo, Adapt- former: Adapting vision transformers for scalable visual recognition, Advances in Neural Information Processing Systems 35 (2022) 16664– 16678. 35

    2022

  46. [52]

    D. Lian, D. Zhou, J. Feng, X. Wang, Scaling & shifting your features: A new baseline for efficient model tuning, Advances in Neural Information Processing Systems 35 (2022) 109–123

    2022

  47. [53]

    Zhou, H.-L

    D.-W. Zhou, H.-L. Sun, J. Ning, H.-J. Ye, D.-C. Zhan, Continual learn- ing with pre-trained models: A survey, in: IJCAI, 2024, pp. 8363–8371

    2024

  48. [56]

    Riquelme, J

    C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, N. Houlsby, Scaling vision with sparse mixture of experts, Advances in Neural Information Processing Systems 34 (2021) 8583–8595

    2021

  49. [57]

    Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

    A. Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  50. [58]

    Vaswani, Attention is all you need, Advances in Neural Information Processing Systems (2017)

    A. Vaswani, Attention is all you need, Advances in Neural Information Processing Systems (2017)

    2017

  51. [59]

    Rebuffi, H

    S.-A. Rebuffi, H. Bilen, A. Vedaldi, Learning multiple visual domains with residual adapters, Advances in neural information processing sys- tems 30 (2017)

    2017

  52. [60]

    Paszke, S

    A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch (2017)

    2017

  53. [61]

    Van der Maaten, G

    L. Van der Maaten, G. Hinton, Visualizing data using t-sne., Journal of machine learning research 9 (11) (2008). 36

    2008

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.