REVIEW 4 cited by
Toward Understanding Catastrophic Forgetting in Continual Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure that makes use of recent developments in task space modeling as well as correlation analysis to specify and analyze the properties we are interested in. As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity. We show that error rates are strongly and positively correlated to a task sequence's total complexity for some state-of-the-art algorithms. We also show that, surprisingly, the error rates have no or even negative correlations in some cases to sequential heterogeneity. Our findings suggest directions for improving continual learning benchmarks and methods.
Forward citations
Cited by 4 Pith papers
-
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
A mixture-of-experts LLM trained with reinforcement learning to perform retrieval from its own parametric memory can replace external retrieval in some settings, at lower latency.
-
DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree Search
A multi-agent, retrieval-augmented search framework lets a 7B language model outperform larger baselines on drug-target interaction prediction, but its headline recall relies on knowing the ground-truth output count.
-
DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers
DA-VPT guides visual prompts with a proxy-anchor metric loss and dynamic class-to-prompt clustering, reporting consistent gains over VPT baselines across classification and segmentation.
-
UrbanMind: Towards Urban General Intelligence via Tool-Enhanced Retrieval-Augmented Generation and Multilevel Optimization
The paper introduces UrbanMind, a tool-enhanced RAG framework with a multilevel optimization formulation for continual adaptation in urban AI, but offers only qualitative prototype results.
Discussion (0). Continue with ORCID to comment.