Pith. sign in

REVIEW 10 cited by

New Insights on Reducing Abrupt Representation Change in Online Continual Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.05025 v3 pith:UA3X2RP2 submitted 2021-04-11 cs.LG

Lucas Caccia , Rahaf Aljundi , Nader Asadi , Tinne Tuytelaars , Joelle Pineau , Eugene Belilovsky This is my paper
classification cs.LG
keywords classesdatalearningcontinualrepresentationschangeeffectiveempirical
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. Experience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes' representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning

    cs.LG 2026-04 unverdicted novelty 7.0 of 10

    TeLAPA preserves behaviorally diverse policy neighborhoods in a shared latent space, improving MiniGrid continual RL transfer, revisit recovery, and retention over single-model preservation.

  2. Replay Can Provably Increase Forgetting

    cs.LG 2025-06 conditional novelty 7.0 of 10

    In an over-parameterized linear regression setting, replaying old samples can provably increase forgetting, both in worst-case scenarios and on average, contradicting the assumption that replay is always benign.

  3. STAIL: Semantic Text-Anchored Incremental Learning for Medical Imaging via Large Language Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    STAIL anchors evolving visual features to frozen LLM text embeddings and rehearses a small image set plus many text descriptions, cutting storage and forgetting in medical class-incremental learning.

  4. SUM: Unified Geometric Surgery on Spatio-Temporal Adaptation Vectors for Federated Class Incremental Learning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    SUM projects client and task adaptation vectors to remove directional interference during server aggregation, improving federated class-incremental learning accuracy without client-side changes.

  5. Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay

    cs.LG 2025-08 conditional novelty 6.0 of 10

    A SOM-VAE generative replay method stores per-unit Gaussian statistics instead of raw data and reports competitive class-incremental accuracy on standard benchmarks.

  6. Grounding Multilingual Multimodal LLMs With Cultural Knowledge

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A Wikidata-derived multilingual multimodal dataset improves cultural understanding of a vision-language model, yielding state-of-the-art results on cultural benchmarks among open models.

  7. PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    PROL achieves state-of-the-art rehearsal-free online continual learning accuracy on CIFAR100, ImageNet-R, ImageNet-A, and CUB with a single lightweight prompt generator and 16 trainable numbers per class.

  8. Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Representation discrepancy, a new metric with theoretical bounds, shows continual learning forgets features faster in deeper layers and slower in wider networks.

  9. Continual Learning Beyond Experience Rehearsal and Full Model Surrogates

    cs.LG 2025-05 conditional novelty 5.0 of 10

    SPARC achieves strong continual learning accuracy with a fraction of the parameters of surrogate-based methods by combining task-specific depthwise filters with shared pointwise filters updated by exponential averaging.

  10. Data-Distill-Net: A Data Distillation Approach Tailored for Reply-based Continual Learning

    cs.LG 2025-05 conditional novelty 5.0 of 10

    A plug-in module that generates learned soft labels for memory buffer samples improves accuracy and reduces forgetting across several replay-based continual learning baselines.

Pith tools