Pith. sign in

REVIEW 3 major objections 5 minor 54 references

PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PROL claims that a single lightweight prompt generator, trained only on the first task and frozen, combined with per-class scaler-shifter parameters selected by class-wise keys, is enough to make rehearsal-free online continual learning…

desk verdict Solid empirical OCL paper with a real result, but the 'significantly higher' claim outruns the statistics and the class-key routing mechanism is the right thing to worry about. read the letter →

arxiv 2507.12305 v1 pith:IK6HFR3O submitted 2025-07-16 cs.LG cs.AIcs.CV

M. Anwar Ma'sum , Mahardhika Pratama , Savitha Ramasamy , Lin Liu , Habibullah Habibullah , Ryszard Kowalczyk This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords onlinecontinuallearningrehearsal-freeprompt-basedclass-incrementalcatastrophicforgettingpre-trainedmodelpromptgeneratorscaler-shifter
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PROL, a method for online continual learning—learning from streaming data that can be seen only once—that stores no exemplars and grows no prompt components. PROL trains a single tiny prompt generator on the first task, freezes it, and for each new class learns just 16 scalar parameters that rescale and shift the frozen generator's output. A learnable class-wise key selects which per-class scalers and shifters apply to each input, and an orthogonality loss plus a cross-correlation matrix keep classes separated and the pre-trained backbone's generalization intact. The authors report final average accuracies of 86.32%, 73.50%, 47.72%, and 72.51% on CIFAR100, ImageNet-R, ImageNet-A, and CUB, respectively, outperforming joint-replay baselines on three of the four datasets while using about 0.213M trainable parameters. The claim is that a single frozen general prompt with per-class scalar corrections is sufficient for strong rehearsal-free performance.

What carries the argument

The load-bearing object is the prompt-generation pair $(G_K, G_V)$: a set of 1D convolution kernels, three parameters each, distributed across the MSA heads of the ViT layers, with fewer than 1K parameters total. It is trained only on the first task and frozen. Plasticity is supplied by four learned vectors per class—scalers and shifters for the key and value prompts—each of length $l-1$, so 16 parameters per class at prompt length $l=5$. At inference the class-wise key $K_c$, a learnable embedding in $\mathbb{R}^D$, is selected by cosine similarity to the input feature; the similarity also scales the generator output. The orthogonality loss forces current keys perpendicular to previous keys, and the cross-correlation matrix $M = \frac{1}{B}[f_\theta(x)]^\top [f_{\theta;P}(x)]$ aligns the pre-trained model's feature with the prompted feature to preserve generalization (adapted from SAFE). A hard-soft update rule switches from a constant high learning rate to a cosine-annealed decay once the cross-entropy loss drops below a threshold.

What would settle it

Run PROL on any of the four benchmarks but replace the cosine top-1 key selection with an oracle that uses the ground-truth class to fetch the scalers and shifters; if accuracy rises substantially, the routing is the bottleneck. Alternatively, construct a task ordering that places two near-duplicate classes in different tasks and measure whether accuracy on those classes collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that online continual learning can split the problem into a fixed general component and a lightweight class-specific adaptation. A single prompt generator (fewer than 1K parameters, arranged as 1D convolution kernels across the ViT attention heads) is trained only on the first task and never touched again; it supplies the general knowledge that carries over to all classes. Plasticity comes from four vectors per class—scalers and shifters for the key and value prompts—totaling 16 parameters per class at prompt length 5. At inference, cosine similarity between the input feature and the learnable class-wise key $K_c$ selects the correct scalers and shifters, and the same similarity scales the generator output as a confidence controller. Experimentally, the paper claims this arrangement yields significantly higher final and cumulative average accuracy than the compared replay, joint, and growing-prompt methods on three of four benchmarks, and stronger performance than all rehearsal-free baselines on all four.

Load-bearing premise

That cosine similarity between the input embedding and the learned class-wise keys always picks the right class-specific scalers and shifters at inference; if two classes have nearly collinear embeddings, the wrong per-class parameters are applied, a failure mode the paper does not analyze.

Editorial extensions

If this is right

  • Rehearsal-free online continual learning can match or beat methods that replay stored features or train on all class data jointly, suggesting data-privacy policies need not impose a large accuracy penalty.
  • A generator trained only on the first task appears to carry enough general knowledge for all later tasks, so the expensive part of prompting can be paid once.
  • Per-class scaler-shifter tables keep trainable parameters at about 0.213M, making the approach attractive for settings where model size and streaming throughput matter.
  • The method stays within about 2% accuracy of the closest growing-prompt competitor while using substantially fewer parameters and higher throughput.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the key-routing assumption holds across harder distributions, the class-wise keys could be viewed as trainable prototypes, suggesting that prototype-separation techniques (e.g., equi-angular key placement) could be dropped into PROL's routing without retraining the generator.
  • The frozen-first-task design implies a sensitivity to the base task's size and representativeness that the paper does not test; a skewed first task could cap all later accuracy.
  • An oracle-selection experiment of the kind described as a falsifier would quantify how much accuracy rests on the routing step versus the scale-shift parameters, telling practitioners whether to invest in better key separation or larger keys.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PROL, a rehearsal-free online class-incremental learning (OCIL) method. PROL freezes a single lightweight CNN-based prompt generator after the first task, then modulates the generated prompts per class using 16 scaler/shifter parameters that are selected at inference by cosine similarity between the input embedding and learnable class-wise keys. A cross-correlation loss is used to preserve the pre-trained model's generalization, and a hard-soft update schedule adjusts the learning rate. The method is evaluated on CIFAR100, ImageNet-R, ImageNet-A, and CUB against PTM-based OCIL baselines (RanPAC, RanDumb, MOS) and prompt-based methods (L2P, DualPrompt, ConvPrompt), reporting higher final and cumulative average accuracy in most settings while using roughly 0.213M trainable parameters.

Significance. The paper targets an important practical constraint, rehearsal-free OCIL, and its design is original: a single frozen prompt generator with per-class affine prompt modulation is a novel and potentially efficient mechanism. The empirical study is broad (four datasets, ten tasks, three seeds), and the authors provide code and detailed per-task tables. If the routing mechanism is reliable and the reported gains are statistically robust, PROL would be a strong contribution to parameter-efficient rehearsal-free continual learning. However, two load-bearing issues currently weaken the claims: the orthogonality loss does not actually enforce orthogonality as written, and the class-wise key routing—the mechanism that separates PROL from earlier prompt methods—is never directly validated.

major comments (3)
  1. [§4.3, Eq. (7)] The orthogonality loss is defined as L_ort = (1/B) Σ K_ci · K_c'i, and the text states that the model is optimized so that K_c·K_c' = 0. Minimizing a raw dot product drives the dot product toward negative values, not toward zero, unless the key norms are also controlled. As written, the loss encourages anti-parallel keys rather than orthogonal ones, so the stated objective is not implemented. Please either replace the loss with a squared or absolute dot-product term (e.g., (K_c·K_c')²) or provide evidence that the raw dot product converges to zero under the current formulation. This is not merely a wording issue because the key geometry directly affects the routing reliability claimed in the paper.
  2. [§4.2 and Algorithm 2] The inference procedure selects the top-1 class-wise key K_c by cosine similarity and applies the associated scalers/shifters. The paper itself, in §3(c), identifies that task-wise keys suffer from high similarity, K_t ≈ K_t', leading to wrong task prediction. PROL moves the same mechanism to the class level (100–200 keys) without validating that the failure mode is resolved. The two mitigations are not sufficient: the similarity factor s = cos(x, K_c) only down-weights a prompt when the selected key is dissimilar, and a wrong key that is close to the input still receives high similarity; L_ort in Eq. (7) is computed only between current-batch keys and sampled old keys and does not enforce pairwise separation of all keys, while it also competes with L_sim. No experiment reports key-selection accuracy, key-similarity distributions, or the accuracy gap between Algorithm 2 and an oracle that selects the ground-truth class's scalers/shifters. Without such evidence, the reported gains cannot be attributed to the proposed per-class key routing rather than to the frozen generator and classifier. Please add these analyses.
  3. [§5.1 and Table 1] The paper repeatedly uses the word 'significant' to describe accuracy differences, but no standard deviations, confidence intervals, or significance tests are reported despite running three seeds. This matters because the gaps to the closest competitor, ConvPrompt, are only 1.5–2.6 percentage points on CIFAR100 and ImageNet-R (FAA), which may be within run-to-run noise. Please report per-seed results, variance, and ideally a paired significance test across seeds for the main accuracy comparisons (Table 1 and Figure 3). Without this, the claim of consistent superiority over the closest baselines is not statistically grounded.
minor comments (5)
  1. [Algorithm 2, line 2] The line 'Find top-1 K_c where c ∈ T' is notationally unclear because T is defined in §3(a) as the number of tasks, not the set of all classes. It should read something like c ∈ {1, ..., C} over all learned classes.
  2. [Eq. (6)] The denominator uses an undefined ε; please state that ε is a small positive constant used to avoid division by zero. Also, the similarity factor s = cos(x, K_c) can be negative if the input and selected key are not similar; multiplying G(x) by a negative value may flip the prompt. Please clarify whether s is clipped to [0, 1] or explain the behavior for negative similarities.
  3. [§5.2(a) and Table 1 caption] The text says 'our method outperforms the existing SOTAs with a significant gap, i.e., 2-76% FAA and 2-64% CAA, respectively, except in the CUB dataset,' but a few sentences later it states a 2-63% FAA and 2-52% CAA margin for CUB. These statements are contradictory and should be reconciled. The table caption also contains typos ('Imagnet-R', 'Rhsl').
  4. [Figure 3] The subplot captions appear mismatched: the CIFAR100 column is labeled 'CIFAR100 Average Accuracy' but the adjacent forgetting plots are labeled 'ImageNet-R Average Forgetting' and 'ImageNet-A Average Forgetting' while appearing under the CIFAR100 column. Please check that each subplot's caption corresponds to the correct dataset and metric.
  5. [Table 5] The ablation rows mix notations: 'FT+G+SS+KL_inter+Lintra+Lsim' and later 'FT+G+SS+K+HSUL_inter+Lintra+Lsim+Lort' are inconsistent (K vs KL, 'HSU' vs 'HSUL'). Please use consistent abbreviations for the components and losses.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PROL is an empirical method evaluated against external benchmarks, and no fitted quantity is relabeled as a prediction.

full rationale

The paper's central claim is empirical: PROL reports higher final average accuracy and cumulative average accuracy than RanPAC, RanDumb, MOS, L2P, DualPrompt, and ConvPrompt on CIFAR100, ImageNet-R, ImageNet-A, and CUB. The proposed components are defined independently of the target result: the prompt generator is a 1D CNN kernel set (Section 4.1a), the scalers and shifters are bounded per-class parameters (Section 4.1b and Eq. 1), and the losses in Eqs. 4-9 are standard supervised and regularization objectives. Algorithm 1 and Algorithm 2 specify training and inference, and the reported numbers come from running those algorithms with a fixed ViT-B/16 backbone and the same dataset splits for all methods. Dataset-specific hyperparameters (lambda_2, L_thres, learning rates) are tuned and disclosed in Appendix C; this is normal benchmarking, not fitting the answer into the claim. There is no uniqueness theorem imported from prior work, and no load-bearing step is justified only by a self-citation; the cited methods (ConvPrompt, DualPrompt, SAFE) provide architectural building blocks, but the evaluation is external and comparative. The potential weakness that class-wise key routing may select the wrong scaler/shifter (the paper itself notes in Section 3c that similar task-wise keys cause wrong task prediction, and Algorithm 2 selects the top-1 class key) is a robustness or correctness risk, not circularity, because the reported accuracy is not defined in terms of that routing assumption and the paper does not claim to have derived the accuracy from it. Under the hard rules, no circular step can be exhibited with a specific reduction, so the honest finding is score 0.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The method rests on common supervised-learning assumptions plus several design choices specific to this paper. No new physical or conceptual entities are introduced; the class-wise keys and scaler-shifter parameters are trainable but already enumerated as free parameters.

free parameters (7)
  • lambda2 (inter-task loss weight) = 0.001 (CIFAR100), 0.01 (CUB), 0.03 (ImageNet-R/A)
    Dataset-specific weight tuned by grid search in the supplementary; not derived from first principles.
  • L_thres (hard-soft update threshold) = 0.3 (CIFAR100, CUB), 0.8 (ImageNet-R/A)
    Threshold for switching from hard to soft updates; chosen per dataset without a described validation protocol.
  • Loss weights lambda1, lambda3, lambda4 (and lambda5) = 1.0 each; lambda5 not reported
    Hand-set constants in the joint loss Eq. (9); paper states lambda1, lambda3, lambda4 are 1.0 but does not list lambda5.
  • Learning rate for PROL = not reported; grid search over [0.001, 0.005, 0.01, 0.05, 0.1]
    Section 5.1 says learning rates for all methods are set by grid search, but the per-dataset selected value for PROL is not shown.
  • Prompt length (default) and number of prompted ViT layers = l=5; layers 5 (inferred), tested 3,5,7,9
    Architectural hyperparameters chosen to balance accuracy and throughput; robustness tested in Table 4.
  • E_a and E_b bounds for scalers and shifters = not quantified
    Eq. (1) clamps scalers and shifters to (1 +/- E_a) and +/- E_b, but the actual numeric values are not given in the paper.
  • Cosine annealing maxT and min_lr = maxT=20, min_lr=0.005
    Schedule for the soft-update decay; hand-set in the supplementary material.
assumptions (6)
  • domain assumption A pre-trained ViT-B/16 on ImageNet-21K provides transferable features that prompts can steer to new classes without updating backbone weights.
    Frozen backbone used throughout (Sections 4.4 and 5.1); if false, the accuracy claim collapses.
  • ad hoc to paper The generator trained only on the first task produces useful prompts for all later classes through scaling and shifting.
    Section 4.1(a) freezes the generator after task 1; the paper demonstrates this empirically in Figures 3 and Table 5 but offers no formal argument.
  • ad hoc to paper Orthogonal class-wise keys (K_c dot K_c' = 0) improve classifier discrimination.
    Section 4.3 invokes the vector principle; no proof links key orthogonality to MLP accuracy.
  • domain assumption The cross-correlation matrix M and L_gen preserve PTM generalization.
    Adapted from SAFE [51]; the paper trusts SAFE's mechanism without re-establishing it in the OCL setting.
  • ad hoc to paper Hard-soft update switching based on L_thres stabilizes single-pass learning.
    Section 4.5(e) describes the heuristic; no theoretical or ablative proof beyond Table 5.
  • standard math Standard cross-entropy (Eqs. 4 and 5) is a suitable training objective for the streaming classifier.
    The optimization uses standard cross-entropy; this is a routine assumption, not a novel claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning." pith.science (2026). https://pith.science/paper/IK6HFR3O

@misc{pith2026250712305,
  author       = {Pith},
  title        = {Pith review of: PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IK6HFR3O}},
  note         = {Machine review of arXiv:2507.12305}
}
read the original abstract

The data privacy constraint in online continual learning (OCL), where the data can be seen only once, complicates the catastrophic forgetting problem in streaming data. A common approach applied by the current SOTAs in OCL is with the use of memory saving exemplars or features from previous classes to be replayed in the current task. On the other hand, the prompt-based approach performs excellently in continual learning but with the cost of a growing number of trainable parameters. The first approach may not be applicable in practice due to data openness policy, while the second approach has the issue of throughput associated with the streaming data. In this study, we propose a novel prompt-based method for online continual learning that includes 4 main components: (1) single light-weight prompt generator as a general knowledge, (2) trainable scaler-and-shifter as specific knowledge, (3) pre-trained model (PTM) generalization preserving, and (4) hard-soft updates mechanism. Our proposed method achieves significantly higher performance than the current SOTAs in CIFAR100, ImageNet-R, ImageNet-A, and CUB dataset. Our complexity analysis shows that our method requires a relatively smaller number of parameters and achieves moderate training time, inference time, and throughput. For further study, the source code of our method is available at https://github.com/anwarmaxsum/PROL.

Figures

Figures reproduced from arXiv: 2507.12305 by the authors.

Figure 1
Figure 1. Topology of OCL and prompt-based CL methods (a) OCL Methods without pre-trained model (PTM), (b) OCL methods with [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The proposed PROL diagram. PROL utilizes a single generator (GK, GV ) for all classes in all tasks, learnable scalers (a i K, ai V ), and shifters (b i K, bi V ) parameters asocciated to the class-wise key Ki . The generator is trained only in the first task and frozen afterward, while the scalers, shifters, and class-wise keys are trained in every task. The prompt is generated by scaling and shifting G(x) by the se… view at source ↗
Figure 3
Figure 3. Historical performance plot of the consolidated methods in CIFAR100, Imagnet-R, Imagnet-A, and CUB200 dataset [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Detailed throughput and execution time of consolidated [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 40 canonical work pages

  1. [1]

    Continual lifelong learning in natural language processing: A survey.arXiv preprint arXiv:2012.09823,

    Magdalena Biesialska, Katarzyna Biesialska, and Marta R Costa-Jussa. Continual lifelong learning in natural language processing: A survey.arXiv preprint arXiv:2012.09823,

    arXiv 2012

  2. [2]

    Class-incremental contin- ual learning into the extended der-verse.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5497– 5512, 2022

    Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class-incremental contin- ual learning into the extended der-verse.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5497– 5512, 2022. 2

    work page 2022

  3. [3]

    Dark experience for gen- eral continual learning: a strong, simple baseline.Advances in neural information processing systems, 33:15920–15930,

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for gen- eral continual learning: a strong, simple baseline.Advances in neural information processing systems, 33:15920–15930,

  4. [4]

    New insights on reducing abrupt representation change in online continual learning.arXiv preprint arXiv:2104.05025, 2021

    Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuyte- laars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning.arXiv preprint arXiv:2104.05025, 2021. 6, 7, 11

    arXiv 2021

  5. [5]

    A continual learning survey: Defying for- getting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying for- getting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021. 1

    work page 2021

  6. [6]

    A unified continual learn- ing framework with general parameter-efficient tuning

    Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jian Zhang. A unified continual learn- ing framework with general parameter-efficient tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11483–11493, 2023. 2

    2023

  7. [7]

    Consistent prompting for rehearsal-free continual learning

    Zhanxin Gao, Jun Cen, and Xiaobin Chang. Consistent prompting for rehearsal-free continual learning. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28463–28473, 2024. 1, 2

    work page 2024

  8. [8]

    Not just selection, but exploration: Online class-incremental contin- ual learning via dual view consistency

    Yanan Gu, Xu Yang, Kun Wei, and Cheng Deng. Not just selection, but exploration: Online class-incremental contin- ual learning via dual view consistency. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7442–7451, 2022. 2, 3

    work page 2022

Show all 54 references
  1. [9]

    Online continual learning through mutual information maximization

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. InInter- national conference on machine learning, pages 8109–8126. PMLR, 2022. 6, 7, 11

    2022

  2. [10]

    Dealing with cross-task class discrimination in online continual learning

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 11878–11887, 2023. 2, 3, 6, 7, 11

    2023

  3. [11]

    The many faces of robustness: A critical analysis of out-of-distribution generalization.ICCV, 2021

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kada- vath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization.ICCV...

    2021

  4. [12]

    Natural adversarial examples.CVPR,

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Stein- hardt, and Dawn Song. Natural adversarial examples.CVPR,

  5. [13]

    Non-exemplar online class-incremental con- tinual learning via dual-prototype self-augment and refine- ment

    Fushuo Huo, Wenchao Xu, Jingcai Guo, Haozhao Wang, and Yunfeng Fan. Non-exemplar online class-incremental con- tinual learning via dual-prototype self-augment and refine- ment. InProceedings of the AAAI Conference on Artificial Intelligence, pages 12698–12707, 2024. 2

    2024

  6. [14]

    On- line continual learning for interactive instruction following agents.arXiv preprint arXiv:2403.07548, 2024

    Byeonghwi Kim, Minhyuk Seo, and Jonghyun Choi. On- line continual learning for interactive instruction following agents.arXiv preprint arXiv:2403.07548, 2024. 2, 3

    2024 arXiv

  7. [15]

    Online continual learning on class incremental blurry task configuration with anytime inference.arXiv preprint arXiv:2110.10031, 2021

    Hyunseo Koh, Dahyun Kim, Jung-Woo Ha, and Jonghyun Choi. Online continual learning on class incremental blurry task configuration with anytime inference.arXiv preprint arXiv:2110.10031, 2021. 2

    2021 arXiv

  8. [16]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6, 11

    2009

  9. [17]

    Evolv- ing parameterized prompt memory for continual learning

    Muhammad Rifki Kurniawan, Xiang Song, Zhiheng Ma, Yuhang He, Yihong Gong, Yang Qi, and Xing Wei. Evolv- ing parameterized prompt memory for continual learning. InProceedings of the AAAI Conference on Artificial Intel- ligence, pages 13301–13309, 2024. 1, 2, 4

    2024

  10. [18]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021. 2

    2021 arXiv

  11. [19]

    Prefix-tuning: Optimiz- ing continuous prompts for generation.arXiv preprint arXiv:2101.00190, 2021

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimiz- ing continuous prompts for generation.arXiv preprint arXiv:2101.00190, 2021. 2, 4

    2021 arXiv

  12. [20]

    Inflora: Interference-free low-rank adaptation for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free low-rank adaptation for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23638–23647, 2024. 2

    2024

  13. [21]

    Incremental learning with neural networks for computer vision: a survey.Artificial intelligence review, 56 (5):4557–4589, 2023

    Hao Liu, Yong Zhou, Bing Liu, Jiaqi Zhao, Rui Yao, and Zhiwen Shao. Incremental learning with neural networks for computer vision: a survey.Artificial intelligence review, 56 (5):4557–4589, 2023. 1

    2023

  14. [22]

    Ranpac: Ran- dom projections and pre-trained models for continual learn- ing.Advances in Neural Information Processing Systems, 36:12022–12053, 2023

    Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton Van den Hengel. Ranpac: Ran- dom projections and pre-trained models for continual learn- ing.Advances in Neural Information Processing Systems, 36:12022–12053, 2023. 1, 2, 3, 6, 7, 8, 11

    2023

  15. [23]

    Rethinking momentum knowledge distillation in online continual learning

    Nicolas Michel, Maorong Wang, Ling Xiao, and Toshihiko Yamasaki. Rethinking momentum knowledge distillation in online continual learning. InProceedings of the 41st Inter- national Conference on Machine Learning. JMLR.org, 2024. 2

    2024

  16. [24]

    Gdumb: A simple approach that questions our progress in continual learning

    Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 524–540. Springer, 2020. 2

    2020

  17. [25]

    Ran- dumb: Random representations outperform online continu- ally learned representations.Advances in Neural Informa- tion Processing Systems, 37:37988–38006, 2024

    Ameya Prabhu, Shiven Sinha, Ponnurangam Kumaraguru, Philip Torr, Ozan Sener, and Puneet Dokania. Ran- dumb: Random representations outperform online continu- ally learned representations.Advances in Neural Informa- tion Processing Systems, 37:37988–38006, 2024. 1, 2, 3, 6, 7, 8, 11

    2024

  18. [26]

    icarl: Incremental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. InProceedings of the IEEE con- ference on Computer Vision and Pattern Recognition, pages 2001–2010, 2017. 2

    2001

  19. [27]

    Learning to learn without forgetting by maximizing transfer and minimizing interference.arXiv preprint arXiv:1810.11910, 2018

    Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference.arXiv preprint arXiv:1810.11910, 2018. 6, 7, 11

    2018 arXiv

  20. [28]

    Convolutional prompting meets language models for continual learning

    Anurag Roy, Riddhiman Moulick, Vinay K Verma, Sap- tarshi Ghosh, and Abir Das. Convolutional prompting meets language models for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23616–23626, 2024. 1, 2, 4, 6, 7, 8, 11

    2024

  21. [29]

    Learning equi-angular repre- sentations for online continual learning

    Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, and Jonghyun Choi. Learning equi-angular repre- sentations for online continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  22. [30]

    Bud- geted online continual learning by adaptive layer freezing and frequency-based sampling

    Minhyuk Seo, Hyunseo Koh, and Jonghyun Choi. Bud- geted online continual learning by adaptive layer freezing and frequency-based sampling. InThe Thirteenth Interna- tional Conference on Learning Representations, 2025. 1, 2, 3

    2025

  23. [31]

    Continual learning for real-world autonomous systems: Algorithms, challenges and frameworks.Journal of Intelligent & Robotic Systems, 105 (1):9, 2022

    Khadija Shaheen, Muhammad Abdullah Hanif, Osman Hasan, and Muhammad Shafique. Continual learning for real-world autonomous systems: Algorithms, challenges and frameworks.Journal of Intelligent & Robotic Systems, 105 (1):9, 2022. 1

    2022

  24. [32]

    Continual dif- fusion: Continual customization of text-to-image diffusion with c-lora.arXiv preprint arXiv:2304.06027, 2023

    James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual dif- fusion: Continual customization of text-to-image diffusion with c-lora.arXiv preprint arXiv:2304.06027, 2023. 2

    2023 arXiv

  25. [33]

    Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning. InProceedings of the IE...

    2023

  26. [34]

    Mos: Model surgery for pre-trained model-based class-incremental learning.arXiv preprint arXiv:2412.09441, 2024

    Hai-Long Sun, Da-Wei Zhou, Hanbin Zhao, Le Gan, De- Chuan Zhan, and Han-Jia Ye. Mos: Model surgery for pre-trained model-based class-incremental learning.arXiv preprint arXiv:2412.09441, 2024. 2, 3, 6, 7, 8, 11

    2024 arXiv

  27. [35]

    Con- tinual learning on graphs: A survey.arXiv preprint arXiv:2402.06330, 2024

    Zonggui Tian, Du Zhang, and Hong-Ning Dai. Con- tinual learning on graphs: A survey.arXiv preprint arXiv:2402.06330, 2024. 1

    2024 arXiv

  28. [36]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. 6, 11

    2011

  29. [37]

    Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub- optimality.Advances in Neural Information Processing Sys- tems, 36, 2024

    Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub- optimality.Advances in Neural Information Processing Sys- tems, 36, 2024. 1, 2

    2024

  30. [38]

    A comprehensive survey of continual learning: theory, method and application.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: theory, method and application.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1

    2024

  31. [39]

    Dealing with synthetic data contamination in on- line continual learning

    Maorong Wang, Nicolas Michel, Jiafeng Mao, and Toshihiko Yamasaki. Dealing with synthetic data contamination in on- line continual learning. InThe Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024. 1, 2, 3, 6, 7, 11

    2024

  32. [41]

    Improving plasticity in online continual learning via collaborative learning

    Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23460–23469, 2024. 2

    2024

  33. [42]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning.Advances in Neural Informa- tion Processing Systems, 35:5682–5695, 2022

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning.Advances in Neural Informa- tion Processing Systems, 35:5682–5695, 2022. 1, 2

    2022

  34. [43]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. InEuropean Conference on Computer Vision, pages 631–648. Springer,

  35. [44]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139–149,

  36. [45]

    Online prototype learning for online con- tinual learning

    Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online con- tinual learning. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 18764–18774,

  37. [46]

    1, 2, 3, 6, 7, 8, 11

  38. [47]

    Mitigating catastrophic forget- ting in online continual learning by modeling previous task interrelations via pareto optimization

    Yichen Wu, Hong Wang, Peilin Zhao, Yefeng Zheng, Ying Wei, and Long-Kai Huang. Mitigating catastrophic forget- ting in online continual learning by modeling previous task interrelations via pareto optimization. InForty-first Interna- tional Conference on Machine Learning, 2024. 2, 3

    2024

  39. [48]

    Large scale in- cremental learning

    Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale in- cremental learning. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 374–382, 2019. 2

    2019

  40. [49]

    Layerwise proximal replay: a proximal point method for on- line continual learning

    Jason Yoo, Yunpeng Liu, Frank Wood, and Geoff Pleiss. Layerwise proximal replay: a proximal point method for on- line continual learning. InProceedings of the 41st Interna- tional Conference on Machine Learning. JMLR.org, 2024. 1, 2, 3

    2024

  41. [50]

    Forgetting, ignorance or myopia: Revis- iting key challenges in online continual learning.Advances in Neural Information Processing Systems, 37:58341–58375,

    Wang Xinrui, Chuanxing Geng, Wenhai Wan, Shao-Yuan Li, and Songcan Chen. Forgetting, ignorance or myopia: Revis- iting key challenges in online continual learning.Advances in Neural Information Processing Systems, 37:58341–58375,

  42. [51]

    Safe: Slow and fast parameter- efficient tuning for continual learning with pre-trained mod- els.Advances in Neural Information Processing Systems, 37: 113772–113796, 2025

    Linglan Zhao, Xuerui Zhang, Ke Yan, Shouhong Ding, and Weiran Huang. Safe: Slow and fast parameter- efficient tuning for continual learning with pre-trained mod- els.Advances in Neural Information Processing Systems, 37: 113772–113796, 2025. 2, 5

    2025

  43. [52]

    Slca: Slow learner with classifier align- ment for continual learning on a pre-trained model

    Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slca: Slow learner with classifier align- ment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19148–19158, 2023. 2

    2023

  44. [53]

    F-oal: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning

    Huiping Zhuang, Yuchen Liu, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Yi Wang, and Lap-Pui Chau. F-oal: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning. Advances in Neural Information Processing Systems, 37: 41517–...

    2024

  45. [54]

    Expandable subspace ensemble for pre-trained model- based class-incremental learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model- based class-incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23554–23564, 2024. 2, 3, 6, 11

    2024

  46. [2022]

    1, 2, 3, 4, 6, 7, 8, 11

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.