Dead-Direction Conditioners provide gauge-equivariant preconditioning by conditioning optimizer state on symmetry orbits, yielding improved resistance to over-training collapse and higher detection of dead directions compared to AdamW and Muon.
Riemannian Adaptive Optimization Methods
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
Several first order stochastic optimization methods commonly used in the Euclidean domain such as stochastic gradient descent (SGD), accelerated gradient descent or variance reduced methods have already been adapted to certain Riemannian settings. However, some of the most popular of these optimization tools - namely Adam , Adagrad and the more recent Amsgrad - remain to be generalized to Riemannian manifolds. We discuss the difficulty of generalizing such adaptive schemes to the most agnostic Riemannian setting, and then provide algorithms and convergence proofs for geodesically convex objectives in the particular case of a product of Riemannian manifolds, in which adaptivity is implemented across manifolds in the cartesian product. Our generalization is tight in the sense that choosing the Euclidean space as Riemannian manifold yields the same algorithms and regret bounds as those that were already known for the standard algorithms. Experimentally, we show faster convergence and to a lower train loss value for Riemannian adaptive methods over their corresponding baselines on the realistic task of embedding the WordNet taxonomy in the Poincare ball.
citation-role summary
method 2
citation-polarity summary
years
2026 8roles
method 2polarities
use method 2representative citing papers
SMAVE recasts MAVE for SDR as Riemannian optimization on the Stiefel manifold, yielding a stochastic algorithm with almost-sure convergence and improved runtime over OPG and RMAVE.
VarLenRec learns variable-length semantic IDs for generative recommendation by allocating longer codes to tail items via popularity-weighted information budget allocation, hyperbolic residual quantization, and a differentiable soft length controller.
LA-Sign achieves state-of-the-art skeleton-based sign language recognition on WLASL and MSASL by using recurrent looped transformers with adaptive hyperbolic geometry alignment.
Hyperbolic RNN and GRU neural quantum states outperform Euclidean versions on Heisenberg J1J2 and J1J2J3 models with 100 spins.
DFSOS computes all sparse discriminant vectors at once with global orthogonality via Bregman iteration and augmented Lagrangian, achieving classification accuracy comparable to or better than deflation-based sparse optimal scoring on synthetic and real time series data.
Pion is an optimizer that preserves the singular values of weight matrices in LLM training by applying orthogonal equivalence transformations.
citing papers explorer
-
Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks
Dead-Direction Conditioners provide gauge-equivariant preconditioning by conditioning optimizer state on symmetry orbits, yielding improved resistance to over-training collapse and higher detection of dead directions compared to AdamW and Muon.
-
Riemannian Stochastic Optimization for Sufficient Dimension Reduction
SMAVE recasts MAVE for SDR as Riemannian optimization on the Stiefel manifold, yielding a stochastic algorithm with almost-sure convergence and improved runtime over OPG and RMAVE.
-
Learning Variable-Length Tokenization for Generative Recommendation
VarLenRec learns variable-length semantic IDs for generative recommendation by allocating longer codes to tail items via popularity-weighted information budget allocation, hyperbolic residual quantization, and a differentiable soft length controller.
-
LA-Sign: Looped Transformers with Geometry-aware Alignment for Skeleton-based Sign Language Recognition
LA-Sign achieves state-of-the-art skeleton-based sign language recognition on WLASL and MSASL by using recurrent looped transformers with adaptive hyperbolic geometry alignment.
-
New non-Euclidean neural quantum states from additional types of hyperbolic recurrent neural networks
Hyperbolic RNN and GRU neural quantum states outperform Euclidean versions on Heisenberg J1J2 and J1J2J3 models with 100 spins.
-
Deflation-Free Optimal Scoring
DFSOS computes all sparse discriminant vectors at once with global orthogonality via Bregman iteration and augmented Lagrangian, achieving classification accuracy comparable to or better than deflation-based sparse optimal scoring on synthetic and real time series data.
-
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
Pion is an optimizer that preserves the singular values of weight matrices in LLM training by applying orthogonal equivalence transformations.
- Inversion-Free Natural Gradient Descent on Riemannian Manifolds