Recognition: unknown
Towards Riemannian Accelerated Gradient Methods
read the original abstract
We propose a Riemannian version of Nesterov's Accelerated Gradient algorithm (RAGD), and show that for geodesically smooth and strongly convex problems, within a neighborhood of the minimizer whose radius depends on the condition number as well as the sectional curvature of the manifold, RAGD converges to the minimizer with acceleration. Unlike the algorithm in (Liu et al., 2017) that requires the exact solution to a nonlinear equation which in turn may be intractable, our algorithm is constructive and computationally tractable. Our proof exploits a new estimate sequence and a novel bound on the nonlinear metric distortion, both ideas may be of independent interest.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
A second-order method landing on the Stiefel manifold via Newton$\unicode{x2013}$Schulz iteration
A second-order method achieves local quadratic convergence on the Stiefel manifold without retractions by combining a modified Newton tangent step with Newton-Schulz normal steps for constraint satisfaction.
-
Improved Penalty Function Approaches for Optimization Problems with General Orthogonality
A penalty function for generalized orthogonality constraints matches first- and second-order stationary points of the original problem and runs faster than manifold optimization.
-
Natural gradient descent with momentum
Introduces natural-gradient versions of Heavy-Ball and Nesterov momentum methods for function approximation on differentiable nonlinear manifolds.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.