Pith. sign in

REVIEW 2 major objections 4 minor 64 references

Memory-efficient Continual Learning with Neural Collapse Contrastive

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Memory-free contrastive learner beats replay baselines in continual learning.

desk verdict A plausible memory-free continual learning method whose headline results are undercut by the paper's own admission that hyperparameters were tuned on test accuracy. read the letter →

arxiv 2412.02865 v3 pith:ETTBNGY4 submitted 2024-12-03 cs.CV cs.AIcs.LG

Trung-Anh Dang , Vincent Nguyen , Ngoc-Son Vu , Christel Vrain This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords continuallearningcatastrophicforgettingneuralcollapsecontrastiveknowledgedistillationsimplexequiangulartightframememory-freeclass-incremental
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that continual learning fails when a method uses only one kind of feature relationship: soft inter-sample similarities shift across tasks and smear class clusters together, while hard sample-to-prototype alignments compress classes and drag old representations toward new ones. It introduces FNC$^2$, a plasticity loss that combines both relationships and up-weights hard samples, and HSD, a distillation loss that preserves instance-wise and sample-to-prototype relations while shifting weight between them over training. The central empirical claim is that a model trained with these losses needs no stored exemplars to rival rehearsal-based methods, and it exceeds the prior memory-free baseline on every dataset and scenario tested. This matters for continual learning under data-privacy constraints, where keeping a buffer of old samples is not allowed.

What carries the argument

The load-bearing object is a pre-defined simplex Equiangular Tight Frame (ETF): $K$ fixed, mutually equidistant unit vectors in $\mathbb{R}^d$, used as optimal class prototypes before training begins. FNC$^2$ is a focal supervised contrastive loss whose per-sample terms are weighted by $(1-c_{ij})^\gamma$ and $(1-r_i)^\gamma$, so hard positives—views far from the anchor or far from their prototype—dominate the gradient; old-task prototypes enter the denominators of $c_{ij}$ and $r_i$, which is how pseudo-replay is implemented. HSD is an interpolation between instance-wise relation distillation and sample-to-prototype relation distillation, with the mixing weight $\alpha = \max(0, e^{-e_0/E})$ ramping up after a warm-up, so the relative emphasis on hard stability grows as representations settle.

What would settle it

Run the method on a class-incremental stream whose total class count is not announced in advance (for instance, open-ended or with a class count that exceeds the pre-built $K$): if accuracy collapses once a new class appears, the central memory-free result is confined to closed class sets. A sharper check is to rerun the Seq-Cifar-10 memory-free setting with the focusing term set to $\gamma=0$ and the HSD mixing weight fixed at $\alpha=0$; the paper's ablation table indicates the gain would shrink, but the comparison would isolate how much of the benefit comes from the two new losses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that 'hardness' and 'softness' are complementary in neural-collapse-based continual learning, and that both are needed in both the plasticity and stability phases. The plasticity loss FNC$^2$ pulls each anchor toward its positive views and toward a fixed simplex ETF prototype for its class, while using all previous-task prototypes as negative references, so old classes act as pseudo-replay without stored images. The stability loss HSD preserves the teacher's instance-wise similarity distribution early in each task and the sample-to-prototype distribution later, with an epoch-dependent weight $\alpha$ controlling the transition. The measured consequence is that the method exceeds Co2L in every memory-free setting reported, for example 69.26% versus 58.89% on Seq-Cifar-10 Class-IL, and that its buffer-size-200 results beat or approach the best replay-based methods.

Load-bearing premise

The whole pipeline depends on knowing the total number of classes $K$ before training starts, since the fixed prototype frame is built once and never updated; with an unknown or growing class count, the memory-free claim does not apply.

Editorial extensions

If this is right

  • If the reported numbers hold, a continual classifier can be trained with no stored exemplars at all, removing the data-retention and privacy obstacle that motivates most rehearsal buffers.
  • The pseudo-replay mechanism turns previously seen class prototypes into negatives for current-task learning, so forgetting is reduced without access to old images.
  • The method remains competitive when a buffer is available, with improvements most visible at small buffer sizes, where representation drift is worst.
  • Because the classifier is trained separately on top of the frozen encoder, representation quality and classifier accuracy are decoupled, and later tasks inherit more stable features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the prototype frame were made learnable or expandable, the same hard-soft balance would likely apply to open-ended class streams; the authors name this exact extension as future work.
  • A finer ablation isolating the focal exponent $\gamma$ from the pseudo-replay prototype negatives would show which mechanism carries the memory-free gain, since the reported ablations do not vary $\gamma$ independently.
  • Because the gains are smallest on Tiny-ImageNet, where the evaluation classifier uses few old samples, stronger classifier adaptation may widen the memory-free advantage.
  • The two losses do not require task identity at test time, so the same design could be dropped into online or fully unsupervised continual learning settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes two loss functions for continual representation learning: FNC2, a focal contrastive loss that combines sample-sample ('softness') and sample-prototype ('hardness') terms using fixed simplex ETF prototypes and cross-task prototype negatives, and HSD, a distillation loss that mixes instance-relation distillation with sample-prototype relation distillation under a time-varying coefficient. The method is evaluated in class-incremental and task-incremental settings on Seq-Cifar-10, Seq-Cifar-100, and Seq-Tiny-ImageNet with buffer sizes 0, 200, and 500, reporting state-of-the-art average accuracy in the memory-free and small-buffer regimes, with ablations attributing gains to the plasticity loss, the distillation loss, and the pseudo-replay prototypes.

Significance. If the empirical claims hold, the paper makes a useful and timely contribution: it demonstrates that fixed ETF prototypes can serve as pseudo-replay in memory-free continual learning, and the decomposition into plasticity and stability losses is clean and well ablated. The manuscript includes extensive comparisons, standard deviations over five trials, and memory-free evaluations that are rare among recent contrastive continual learning methods. The main caveat is that the reported superiority is currently weakened by the test-set hyperparameter selection protocol in Appendix A, which may inflate the headline margins over baselines.

major comments (2)
  1. [Appendix A, Tables 6-7] The sentence 'The selections of these hyperparameters are based on the average test accuracy over five independent trials' directly admits model selection on the test set, contradicting the preceding claim that a randomly drawn 10% validation split was used. Since the headline memory-free gains in Table 2 (e.g., 69.26 vs. 58.89 on Seq-Cifar-10 Class-IL) compare against baselines with their published hyperparameters, the margins may be inflated by test-set selection bias. Please re-run the comparison with all methods selected on the same validation protocol, or provide a corrected description showing that no test information was used; report validation-based results and the selection curves.
  2. [Eq. (10) and Sec. 4.4] As written, the HSD weighting is defined as α = max(0, e^{-e0/E}), which is independent of the current epoch index e and therefore cannot implement the described warm-up schedule of gradually decreasing LIRD and increasing LS-PRD after e0 epochs. Please state the exact schedule used in the experiments as a function of e, e0, and E; if the formula is a typesetting artifact, correct it. This is load-bearing for the reproducibility of the central stability loss.
minor comments (4)
  1. [Sec. 5.1 vs. Appendix Table 7] The main text says the backbone is trained for 100 epochs on all non-initial tasks 'for all datasets', but Appendix Table 7 lists Et≥2 = 50 for Seq-Tiny-ImageNet; please reconcile this discrepancy and confirm that Co2L uses the same epoch schedule for that dataset.
  2. [Table 3] The statement 'The test Lasym SupCon with LS-PRD is omitted due to incompatibility' is unexplained; please specify why the asymmetric SupCon loss cannot be combined with sample-prototype relation distillation.
  3. [Conclusion] The acknowledged limitation that the number of prototypes must be predefined is welcome and appropriate; for clarity, please state this assumption earlier in the paper, since it is central to the method's applicability.
  4. [Sec. 5.1] There is a typo in the first sentence of Sec. 5.1: 'thee datasets' should read 'three datasets'.

Circularity Check

1 steps flagged · score 6.0 of 10

Test-set hyperparameter selection in Appendix A makes the reported memory-free SOTA margins partly an artifact of the selection criterion; the loss design itself is not circular.

  1. fitted input called prediction [Appendix A, Hyperparameter selection, paragraph following Table 6]
    "The selections of these hyperparameters are based on the average test accuracy over five independent trials, and the final chosen values are detailed in Tab. 7."

    The paper's central claim is that the model 'significantly outperforms recent replay-based methods' and, notably, 'without using a buffer, our method outperforms Co2L in all settings and datasets' (Section 5.2). These claims are supported by Table 2, which reports average test accuracy over five trials. Appendix A states that the hyperparameters (gamma, temperatures, warm-up epochs, learning rate, etc.) were selected 'based on the average test accuracy over five independent trials.' Thus the reported test accuracies are the very objective used to choose the model configuration, while the Co2L baseline uses hyperparameters from its original paper, not re-tuned on this test set.

full rationale

The core method is an empirical loss design, not a derivation: FNC2 is built from external components (SupCon, focal loss, fixed simplex ETF prototypes) and HSD combines IRD with a new sample-prototype distillation term. No equation in Section 4 is derived from the target outcome, and the losses are defined independently of the measured accuracies, so there is no self-definitional circularity in the loss construction. The known-K limitation is explicitly acknowledged by the authors and is a scope limitation, not circularity. References [37] and [24] include co-author Ngoc-Son Vu, but they are related-work citations and are not load-bearing; the NC premise is independently supported by external works [38, 55, 56]. No uniqueness theorem is imported by self-citation, and no ansatz is smuggled in via citation. The only significant circularity is the Appendix A test-set hyperparameter selection: hyperparameters were chosen to maximize average test accuracy over five trials, and the same average test accuracy is then presented as evidence of state-of-the-art performance. This biases the comparison against Co2L, whose hyperparameters were taken from the original publication, and it directly affects the headline memory-free SOTA claim. The loss design itself remains independent, so the circularity is partial rather than total, but it is central to the empirical superiority claim.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new entities postulated; the fixed prototypes are taken from the neural collapse literature (ETF), and the method only recombines existing loss components. The free parameters are hyperparameters tuned per dataset, with gamma and e0 being the most load-bearing.

free parameters (2)
  • gamma (focusing hyperparameter) = gamma=1 (Seq-Cifar-10), gamma=4 (Seq-Cifar-100, Seq-Tiny-ImageNet)
    Grid-searched over {0,1,2,4,7,10}; chosen based on test accuracy (Figure 4, Appendix A), which is a post-hoc selection.
  • e0 (warm-up epochs in HSD) = e0=30 (CIFAR-10, CIFAR-100), e0=20 (Tiny-ImageNet)
    Grid-searched over {10,20,30}; chosen based on validation/test accuracy, affecting the alpha schedule in Eq. (10).
assumptions (3)
  • domain assumption Fixed simplex ETF of K prototypes remains optimal for continual learning across all tasks
    Invoked in Section 4.1 and 4.2 to justify pulling representations toward fixed prototypes; no proof of optimality in the CL setting; the paper acknowledges the limitation when K is unknown.
  • standard math Standard supervised contrastive learning assumptions: two augmented views per sample, cosine similarity, temperature-scaled softmax
    Used in Eq. (3), (7)-(9); standard in the cited literature.
  • domain assumption Evaluation protocol of Co2L (classifier trained on final task data and memory) is a fair measure of representations
    Section 5.1; the paper follows this protocol without questioning whether memory-free linear probing can classify unseen old classes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memory-efficient Continual Learning with Neural Collapse Contrastive." pith.science (2026). https://pith.science/paper/ETTBNGY4

@misc{pith2026241202865,
  author       = {Pith},
  title        = {Pith review of: Memory-efficient Continual Learning with Neural Collapse Contrastive},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ETTBNGY4}},
  note         = {Machine review of arXiv:2412.02865}
}
read the original abstract

Contrastive learning has significantly improved representation quality, enhancing knowledge transfer across tasks in continual learning (CL). However, catastrophic forgetting remains a key challenge, as contrastive based methods primarily focus on "soft relationships" or "softness" between samples, which shift with changing data distributions and lead to representation overlap across tasks. Recently, the newly identified Neural Collapse phenomenon has shown promise in CL by focusing on "hard relationships" or "hardness" between samples and fixed prototypes. However, this approach overlooks "softness", crucial for capturing intra-class variability, and this rigid focus can also pull old class representations toward current ones, increasing forgetting. Building on these insights, we propose Focal Neural Collapse Contrastive (FNC^2), a novel representation learning loss that effectively balances both soft and hard relationships. Additionally, we introduce the Hardness-Softness Distillation (HSD) loss to progressively preserve the knowledge gained from these relationships across tasks. Our method outperforms state-of-the-art approaches, particularly in minimizing memory reliance. Remarkably, even without the use of memory, our approach rivals rehearsal-based methods, offering a compelling solution for data privacy concerns.

Figures

Figures reproduced from arXiv: 2412.02865 by the authors.

Figure 1
Figure 1. Our method mitigates the drawbacks of both “soft” and “hard” learning by using fixed, equidistant prototypes to minimize [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Shifts in representations of current samples at task [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overall architecture of our method. Augmented samples from each batch are fed into the current model f t to learn new knowledge via LF NC2 and the frozen previous model f t−1 for distillation using LHSD. The buffer is optional, and NC-based prototypes are directly involved in both loss functions during training. When using memory, we employ an asymmetric version of this loss, where only current samples serve as anch… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Test accuracy over different values of γ. Buffer Dataset Seq-Cifar-10 Seq-Cifar-100 Seq-Tiny-ImageNet Scenario Class-IL Task-IL Class-IL Task-IL Class-IL Task-IL 500 ER [43] 57.74±0.27 93.61±0.27 27.66±0.61 66.23±1.52 9.99±0.29 48.64±0.46 iCaRL [42] 47.55±3.95 88.22±2.…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 51 canonical work pages

  1. [1]

    Gradient based sample selection for online continual learning

    Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Ben- gio. Gradient based sample selection for online continual learning. In NeurIPS, 2019. 7, 8, 12

    work page 2019

  2. [2]

    Mudur, Rahaf Aljundi, and Eugene Belilovsky

    Nader Asadi, Mohammad Davar, Sudhir P. Mudur, Rahaf Aljundi, and Eugene Belilovsky. Prototype-sample rela- tion distillation: Towards replay-free continual learning. In ICML, 2023. 2

    work page 2023

  3. [3]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In NeurIPS,

  4. [4]

    New insights on reducing abrupt representation change in online continual learning

    Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuyte- laars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In ICLR, 2021. 1

    work page 2021

  5. [5]

    Co$ˆ2$L: Con- trastive Continual Learning

    Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co$ˆ2$L: Con- trastive Continual Learning. In ICCV, 2021. 1, 2, 4, 6, 7, 8, 11, 12

    work page 2021

  6. [6]

    Regu- larizing with pseudo-negatives for continual self-supervised learning

    Sungmin Cha, Kyunghyun Cho, and Taesup Moon. Regu- larizing with pseudo-negatives for continual self-supervised learning. In ICML, 2024. 2

    work page 2024

  7. [7]

    Dokania, Thalaiyasingam Ajan- than, and Philip H

    Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajan- than, and Philip H. S. Torr. Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence. In ECCV, 2018. 7, 11

    work page 2018

  8. [8]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020. 2, 6

    work page 2020

Show all 64 references
  1. [9]

    Girshick, and Kaiming He

    Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He. Improved baselines with momentum contrastive learn- ing. arXiv preprint 2003.04297, 2020. 2

    2003 arXiv

  2. [10]

    Driscoll, Lea Duncker, and Christopher D

    Laura N. Driscoll, Lea Duncker, and Christopher D. Har- vey. Representational drift: Emerging theories for continual learning and experimental future directions. Current Opin- ion in Neurobiology, page 102609, 2022. 1

    2022

  3. [11]

    Explor- ing deep neural networks via layer-peeled model: Minority collapse in imbalanced training

    Cong Fang, Hangfeng He, Qi Long, and Weijie J Su. Explor- ing deep neural networks via layer-peeled model: Minority collapse in imbalanced training. PNAS, 2021. 1, 3

    2021

  4. [12]

    Turrisi da Costa, Xavier Alameda- Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal

    Enrico Fini, Victor G. Turrisi da Costa, Xavier Alameda- Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal. Self- supervised models are continual learners. In CVPR, 2022. 1, 2, 6, 7

    2022

  5. [13]

    On the role of neural collapse in transfer learning

    Tomer Galanti, Andr’as Gyorgy, and Marcus Hutter. On the role of neural collapse in transfer learning. In ICLR, 2022. 3

    2022

  6. [14]

    Hofer, Marc Niethammer, and Roland Kwitt

    Florian Graf, Christoph D. Hofer, Marc Niethammer, and Roland Kwitt. Dissecting supervised constrastive learning. In ICML, 2021. 3

    2021

  7. [15]

    X. Y . Han, Vardan Papyan, and David L. Donoho. Neural collapse under mse loss: Proximity to and dynamics on the central path. In ICLR, 2022. 1, 3

    2022

  8. [16]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In CVPR, 2020. 2

    2020

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR,

  10. [18]

    Hinton, Oriol Vinyals, and Jeffrey Dean

    Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distill- ing the knowledge in a neural network. In NIPS Workshops,

  11. [19]

    Learning a unified classifier incrementally via rebalancing

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, 2019. 2

    2019

  12. [20]

    Wenlong Ji, Yiping Lu, Yiliang Zhang, Zhun Deng, and Wei- jie J. Su. An unconstrained layer-peeled perspective on neu- ral collapse. In ICLR, 2022. 1, 3

    2022

  13. [21]

    Continual learning with node-importance based adap- tive group sparse regularization

    Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with node-importance based adap- tive group sparse regularization. In NeurIPS, 2020. 2

    2020

  14. [22]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. InNeurIPS,

  15. [23]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Uni- versity of Toronto, Toronto, 2009. 6

    2009

  16. [24]

    Seeable: Soft discrepancies and bounded contrastive learning for exposing deepfakes

    Nicolas Larue, Ngoc-Son Vu, Vitomir Struc, Peter Peer, and Vassilis Christophides. Seeable: Soft discrepancies and bounded contrastive learning for exposing deepfakes. In ICCV, 2022. 3

    2022

  17. [25]

    Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge, 2015. 6

    2015

  18. [26]

    Contrastive continual learning with importance sampling and prototype-instance relation distillation

    Jiyong Li, Dilshod Azizov, Yang Li, and Shangsong Liang. Contrastive continual learning with importance sampling and prototype-instance relation distillation. In AAAI, 2024. 2, 7

    2024

  19. [27]

    Tcp: Triplet contrastive-relationship preserving for class-incremental learning

    Shiyao Li, Xuefei Ning, Shanghang Zhang, Lidong Guo, Tianchen Zhao, Huazhong Yang, and Yu Wang. Tcp: Triplet contrastive-relationship preserving for class-incremental learning. In WACV, 2024. 1

    2024

  20. [28]

    Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting.arXiv preprint 1904.00310, 2019

    Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting.arXiv preprint 1904.00310, 2019. 2

    1904 arXiv

  21. [29]

    Loss decoupling for task- agnostic continual learning

    Yan-Shuo Liang and Wu-Jun Li. Loss decoupling for task- agnostic continual learning. In NeurIPS, 2023. 2, 4, 5, 7

    2023

  22. [30]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In ICCV,

  23. [31]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. In NeurIPS, 2017. 7, 8, 12

    2017

  24. [32]

    Learning with mixture of prototypes for out-of-distribution detection

    Haodong Lu, Dong Gong, Shuo Wang, Jason Xue, Lina Yao, and Kristen Moore. Learning with mixture of prototypes for out-of-distribution detection. In ICLR, 2024. 8

    2024

  25. [33]

    Neural collapse with cross-entropy loss

    Jianfeng Lu and Stefan Steinerberger. Neural collapse with cross-entropy loss. arXiv preprint 2012.08465, 2020. 1, 3

    2012 arXiv

  26. [34]

    Representational continuity for unsu- pervised continual learning

    Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Representational continuity for unsu- pervised continual learning. In ICLR, 2022. 1, 2

    2022

  27. [35]

    Z. Mai, R. Li, H. Kim, and S. Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In CVPRW, 2021. 2

    2021

  28. [36]

    Michael McCloskey and Neal J. Cohen. Catastrophic Inter- ference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation, 1989. 1 9

    1989

  29. [37]

    Neu- ral collapse inspired contrastive continual learning

    Antoine Montmaur, Nicolas Larue, and Ngoc-Son Vu. Neu- ral collapse inspired contrastive continual learning. In BMVC, 2024. 1

    2024

  30. [38]

    Vardan Papyan, X. Y . Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. PNAS, 2020. 3, 5

    2020

  31. [39]

    Continual learning by asymmetric loss approximation with single-side overestimation

    Dongmin Park, Seokil Hong, Bohyung Han, and Kyoung Mu Lee. Continual learning by asymmetric loss approximation with single-side overestimation. In ICCV, 2019. 2

    2019

  32. [40]

    Explicit regularization and implicit bias in deep network classifiers trained with the square loss

    Tomaso Poggio and Qianli Liao. Explicit regularization and implicit bias in deep network classifiers trained with the square loss. arXiv preprint 2101.00072, 2020. 3

    2020 arXiv

  33. [41]

    Julien Pourcel, Ngoc-Son Vu, and Robert M. French. Online task-free continual learning with dynamic sparse distributed memory. In ECCV, 2022. 1

    2022

  34. [42]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental clas- sifier and representation learning. In CVPR, 2017. 7, 8, 12

    2017

  35. [43]

    Learning to learn without forgetting by maximizing transfer and minimizing interference

    Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In ICLR, 2019. 7, 8, 12

    2019

  36. [44]

    Anthony V . Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connect. Sci., pages 123–146, 1995. 1

    1995

  37. [45]

    Rusu, Neil C

    Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Raz- van Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1606.04671, 2016. 1, 2

    2016 arXiv

  38. [46]

    Learning equi-angular rep- resentations for online continual learning

    Minhyuk Seo, Hyun woo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyun- woo Kim, and Jonghyun Choi. Learning equi-angular rep- resentations for online continual learning. In CVPR, 2024. 1

    2024

  39. [47]

    Szatkowski, M

    F. Szatkowski, M. Pyla, M. Przewiezlikowski, S. Cygert, B. Twardowski, and T. Trzcinski. Adapt your teacher: Improv- ing knowledge distillation for exemplar-free continual learn- ing. In ICCVW, 2023. 2

    2023

  40. [48]

    C. Tang, L. Qendro, D. Spathis, F. Kawsar, C. Mascolo, and A. Mathur. Kaizen: Practical self-supervised continual learn- ing with continual fine-tuning. In WACV, 2024. 1, 2

    2024

  41. [49]

    Extended unconstrained features model for exploring deep neural collapse

    Tom Tirer and Joan Bruna. Extended unconstrained features model for exploring deep neural collapse. In ICML, 2022. 1, 3

    2022

  42. [50]

    Gcr: Gradient coreset based replay buffer selection for continual learning

    Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Gcr: Gradient coreset based replay buffer selection for continual learning. In CVPR, 2022. 1, 7, 8, 11, 12

    2022

  43. [51]

    Repre- sentation learning with contrastive predictive coding

    A ¨aron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint 1807.03748, 2018. 2

    2018 arXiv

  44. [52]

    Jeffrey S. Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software, pages 37–57, 1985. 5

    1985

  45. [53]

    Provable contrastive continual learning

    Yichen Wen, Zhiquan Tan, Kaipeng Zheng, Chuanlong Xie, and Weiran Huang. Provable contrastive continual learning. In ICML, 2024. 1, 2, 7, 8, 12

    2024

  46. [54]

    Inducing neural collapse in imbalanced learning: Do we really need a learnable clas- sifier at the end of deep neural network? In NeurIPS, 2022

    Yibo Yang, Shixiang Chen, Xiangtai Li, Liangru Xie, Zhouchen Lin, and Dacheng Tao. Inducing neural collapse in imbalanced learning: Do we really need a learnable clas- sifier at the end of deep neural network? In NeurIPS, 2022. 3

    2022

  47. [55]

    Neural collapse inspired feature- classifier alignment for few-shot class-incremental learning

    Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse inspired feature- classifier alignment for few-shot class-incremental learning. In ICLR, 2023. 1, 3, 4

    2023

  48. [56]

    Neural collapse terminus: A unified so- lution for class incremental learning and its variants, 2023

    Yibo Yang, Haobo Yuan, Xiangtai Li, Jianlong Wu, Lefei Zhang, Zhouchen Lin, Philip Torr, Dacheng Tao, and Bernard Ghanem. Neural collapse terminus: A unified so- lution for class incremental learning and its variants, 2023. 1, 3, 4

    2023

  49. [57]

    Lifelong learning with dynamically expandable net- works

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. arXiv preprint 1708.01547, 2018. 1, 2

    2018 arXiv

  50. [58]

    L. Yu, B. Twardowski, X. Liu, L. Herranz, K. Wang, Y . Cheng, S. Jui, and J. van de Weijer. Semantic drift com- pensation for class-incremental learning. In CVPR, 2020. 1

    2020

  51. [59]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In ICML, 2021. 6

    2021

  52. [60]

    Contin- ual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Contin- ual learning through synaptic intelligence. In ICML, 2017. 2

    2017

  53. [61]

    Integrating present and past in unsupervised continual learning

    Yipeng Zhang, Laurent Charlin, Richard Zemel, and Mengye Ren. Integrating present and past in unsupervised continual learning. In CoLLAs, 2024. 2, 4, 5

    2024

  54. [62]

    Unleashing the power of contrastive self-supervised visual models via contrast-regularized fine-tuning

    Yifan Zhang, Bryan Hooi, Dapeng Hu, Jian Liang, and Jiashi Feng. Unleashing the power of contrastive self-supervised visual models via contrast-regularized fine-tuning. In NeurIPS, 2021. 4

    2021

  55. [63]

    On the optimization landscape of neural col- lapse under mse loss: Global optimality with unconstrained features

    Jinxin Zhou, Xiao Li, Tianyu Ding, Chong You, Qing Qu, and Zhihui Zhu. On the optimization landscape of neural col- lapse under mse loss: Global optimality with unconstrained features. In ICML, 2022. 1, 3

    2022

  56. [64]

    A geometric analysis of neu- ral collapse with unconstrained features

    Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A geometric analysis of neu- ral collapse with unconstrained features. In NeurIPS, 2021. 3 10 APPENDIX A. Hyperparameter selection To select hyperparameters, we employ a grid search strategy...

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.