Pith. sign in

REVIEW 10 cited by

Continual Learning with Pre-Trained Models: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.16386 v2 pith:FDXJ5VJ4 submitted 2024-01-29 cs.LG cs.CV

Da-Wei Zhou , Hai-Long Sun , Jingyi Ning , Han-Jia Ye , De-Chuan Zhan This is my paper · ORCID
classification cs.LGcs.CV
keywords learningdatacontinualknowledgemethodsmodelpre-trainedsurvey
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Nowadays, real-world applications often face streaming data, which requires the learning system to absorb new knowledge as data evolves. Continual Learning (CL) aims to achieve this goal and meanwhile overcome the catastrophic forgetting of former knowledge when learning new ones. Typical CL methods build the model from scratch to grow with incoming data. However, the advent of the pre-trained model (PTM) era has sparked immense research interest, particularly in leveraging PTMs' robust representational capabilities. This paper presents a comprehensive survey of the latest advancements in PTM-based CL. We categorize existing methodologies into three distinct groups, providing a comparative analysis of their similarities, differences, and respective advantages and disadvantages. Additionally, we offer an empirical study contrasting various state-of-the-art methods to highlight concerns regarding fairness in comparisons. The source code to reproduce these evaluations is available at: https://github.com/sun-hailong/LAMDA-PILOT

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoning

    cs.AI 2025-07 conditional novelty 7.0 of 10

    LTLZinc generates image-based temporal reasoning and continual learning benchmarks from LTLf formulas over MiniZinc constraints, and experiments show existing methods often fail.

  2. Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning

    cs.LG 2025-06 conditional novelty 7.0 of 10

    A lifelong robot learning method that mixes a growing library of LoRA-style experts with a context router and replays router coefficients to achieve forward transfer with near-zero forgetting.

  3. Forward-Only Continual Learning

    cs.LG 2025-09 conditional novelty 6.0 of 10

    FoRo achieves strong continual learning accuracy and low forgetting on CIFAR-100, ImageNet-R, and CUB-200 using only forward updates, via CMA-ES prompt tuning and a recursive knowledge encoding matrix.

  4. LifelongPR: Lifelong point cloud place recognition based on sample replay and prompt learning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LifelongPR combines information-amount-based replay selection with prompt learning to reduce catastrophic forgetting in lifelong point cloud place recognition.

  5. CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    CL-LoRA adds a fixed random-orthogonal shared LoRA branch for cross-task knowledge and task-specific LoRA branches with block-wise weights, improving rehearsal-free class-incremental learning accuracy at low parameter cost.

  6. Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

    cs.CV 2026-07 conditional novelty 5.0 of 10

    MILES adds per-task lightweight adapters and sub-networks to a frozen pre-trained ViT, with distance regularization, and reports top accuracy on six class-incremental benchmarks.

  7. OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance

    cs.CV 2026-03 unverdicted novelty 5.0 of 10

    Generating synchronized four-view orthogonal foreground videos with geometry-enhanced attention, then using them as rigid guidance, improves physical realism in video generation over direct 2D methods.

  8. Continual Speech Learning with Fused Speech Features

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Gated fusion of frozen Whisper layers improves continual learning on six speech tasks, with the double-stage variant best overall.

  9. SplitLoRA: Balancing Stability and Plasticity in Continual Learning Through Gradient Space Splitting

    cs.LG 2025-05 reject novelty 5.0 of 10

    SplitLoRA picks the LoRA update subspace size from previous-task gradient singular values using a hyperparameter alpha, and freezes the projection to keep updates in that subspace.

  10. Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models

    cs.IR 2026-06 unverdicted novelty 4.0 of 10

    Token Factory transforms traditional signals into soft tokens for efficient integration and compression into Large Recommendation Models, avoiding prompt length explosion while enhancing performance.

Pith tools