Preconditioned delta-rule models with a diagonal curvature approximation improve upon standard DeltaNet, GDN, and KDA by better approximating the test-time regression objective.
Variational Continual Learning
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.
verdicts
UNVERDICTED 5representative citing papers
Janus-LoRA uses gradient rectification via online subspace estimation and a decoupled margin loss to enforce parameter orthogonality and feature separation in LoRA-based continual learning, reporting new SOTA results.
BRPC is an online Bayesian calibration framework that decouples parameter tracking from discrepancy modeling for gradual nonstationarity and adds restart mechanisms to handle abrupt regime shifts.
PAPA directly optimizes diffusion models via real-time user feedback for personalized preference alignment, drawing from variational inference, with an efficiency-enhanced variant EPAPA.
DLC inserts lightweight classifier-proximal plugins into distillation-based continual learning to achieve 8% accuracy gains on large benchmarks with only 4% extra backbone parameters.
citing papers explorer
-
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Preconditioned delta-rule models with a diagonal curvature approximation improve upon standard DeltaNet, GDN, and KDA by better approximating the test-time regression objective.
-
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
Janus-LoRA uses gradient rectification via online subspace estimation and a decoupled margin loss to enforce parameter orthogonality and feature separation in LoRA-based continual learning, reporting new SOTA results.
-
Online Bayesian Calibration under Gradual and Abrupt System Changes
BRPC is an online Bayesian calibration framework that decouples parameter tracking from discrepancy modeling for gradual nonstationarity and adds restart mechanisms to handle abrupt regime shifts.
-
PAPA: Online Personalized Active Preference Alignment
PAPA directly optimizes diffusion models via real-time user feedback for personalized preference alignment, drawing from variational inference, with an efficiency-enhanced variant EPAPA.
-
Pushing the Limits of Distillation-Based Continual Learning via Classifier-Proximal Lightweight Plugins
DLC inserts lightweight classifier-proximal plugins into distillation-based continual learning to achieve 8% accuracy gains on large benchmarks with only 4% extra backbone parameters.