REVIEW 7 cited by
Gradient Episodic Memory for Continual Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.
Forward citations
Cited by 7 Pith papers
-
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
SFT lessons — reason-based training, on-model replay, and wash-out robustness — transfer across toy models, model organisms, and alignment SFT, improving the capability–safety tradeoff.
-
Fine-Tuning Regimes Define Distinct Continual Learning Problems
The relative rankings of continual learning methods are not preserved across different fine-tuning regimes defined by trainable parameter depth.
-
Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay
A SOM-VAE generative replay method stores per-unit Gaussian statistics instead of raw data and reports competitive class-incremental accuracy on standard benchmarks.
-
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
Quantized (INT8/INT4) LLMs can outperform FP16 in later-task forward accuracy and retention during continual learning, though single-seed runs leave the effect unquantified.
-
SWE-Bench-CL: Continual Learning for Coding Agents
SWE-Bench-CL reorganizes SWE-Bench Verified into 8 time-ordered sequences of 273 total tasks to measure continual learning in coding agents, adding CL-specific metrics and a semantic memory agent design.
-
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Continual learning should pivot from weight-update-based methods to continual compositionality and orchestration of foundation models and agents.
-
Class Incremental Learning for Algorithm Selection
In an algorithm-selection stream where new solver classes appear, rehearsal-based class-incremental learning retains 82.3% accuracy across four solver classes, about 7% below an oracle that sees all data.
Discussion (0). Sign in to comment.