FOREVER aligns replay intervals in LLM continual learning with a model-centric time based on optimizer update magnitudes and an Ebbinghaus-inspired forgetting curve to reduce catastrophic forgetting.
arXiv preprint arXiv:2301.12314 , year=
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
citation-role summary
background 1
dataset 1
citation-polarity summary
representative citing papers
Muon-OGD introduces a spectral-norm constrained orthogonal projection method solved via dual iterations and Newton-Schulz approximations to improve stability-plasticity trade-off in sequential LLM adaptation.
SETA decomposes parameters into task-specific and shared sparse experts with adaptive anchoring and routing regularization to improve retention and backward transfer in LLM continual learning.
A survey paper providing an overview of Large Language Models, their background, and recent advances in the field.
citing papers explorer
-
FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
FOREVER aligns replay intervals in LLM continual learning with a model-centric time based on optimizer update magnitudes and an Ebbinghaus-inspired forgetting curve to reduce catastrophic forgetting.
-
Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
Muon-OGD introduces a spectral-norm constrained orthogonal projection method solved via dual iterations and Newton-Schulz approximations to improve stability-plasticity trade-off in sequential LLM adaptation.
-
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
SETA decomposes parameters into task-specific and shared sparse experts with adaptive anchoring and routing regularization to improve retention and backward transfer in LLM continual learning.
-
A Comprehensive Overview of Large Language Models
A survey paper providing an overview of Large Language Models, their background, and recent advances in the field.
- Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts