REVIEW 5 cited by
LAMOL: LAnguage MOdeling for Lifelong Language Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Most research on lifelong learning applies to images or games, but not language. We present LAMOL, a simple yet effective method for lifelong language learning (LLL) based on language modeling. LAMOL replays pseudo-samples of previous tasks while requiring no extra memory or model capacity. Specifically, LAMOL is a language model that simultaneously learns to solve the tasks and generate training samples. When the model is trained for a new task, it generates pseudo-samples of previous tasks for training alongside data for the new task. The results show that LAMOL prevents catastrophic forgetting without any sign of intransigence and can perform five very different language tasks sequentially with only one model. Overall, LAMOL outperforms previous methods by a considerable margin and is only 2-3% worse than multitasking, which is usually considered the LLL upper bound. The source code is available at https://github.com/jojotenya/LAMOL.
Forward citations
Cited by 5 Pith papers
-
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
Spectrum-initialized LoRA with elbow ranks and recursive SVD consolidation of the effective weight beats rank-swept PEFT baselines on three of four 7–8B models in continual GLUE fine-tuning.
-
GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning
GRID combines output-space constrained decoding with gradient-guided prompt compression for task-agnostic, bounded-memory continual prompt tuning.
-
Attribution-Guided Continual Learning for Large Language Models
LRP-derived element-wise parameter importance scores gate gradients so parameters critical to earlier tasks receive smaller updates during continual LLM fine-tuning.
-
OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance
Generating synchronized four-view orthogonal foreground videos with geometry-enhanced attention, then using them as rigid guidance, improves physical realism in video generation over direct 2D methods.
-
Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.
Discussion (0). Continue with ORCID to comment.