Pith. sign in

REVIEW 4 cited by

Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2002.01113 v1 pith:NQNC2HU6 submitted 2020-02-04 cs.LG stat.ML

Jun Li , Li Fuxin , Sinisa Todorovic This is my paper · ORCID
classification cs.LGstat.ML
keywords cayleymanifoldoptimizationstiefeladamalgorithmstransformconvergence
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Strictly enforcing orthonormality constraints on parameter matrices has been shown advantageous in deep learning. This amounts to Riemannian optimization on the Stiefel manifold, which, however, is computationally expensive. To address this challenge, we present two main contributions: (1) A new efficient retraction map based on an iterative Cayley transform for optimization updates, and (2) An implicit vector transport mechanism based on the combination of a projection of the momentum and the Cayley transform on the Stiefel manifold. We specify two new optimization algorithms: Cayley SGD with momentum, and Cayley ADAM on the Stiefel manifold. Convergence of Cayley SGD is theoretically analyzed. Our experiments for CNN training demonstrate that both algorithms: (a) Use less running time per iteration relative to existing approaches that enforce orthonormality of CNN parameters; and (b) Achieve faster convergence rates than the baseline SGD and ADAM algorithms without compromising the performance of the CNN. Cayley SGD and Cayley ADAM are also shown to reduce the training time for optimizing the unitary transition matrices in RNNs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Quasi-SVD: Learning a Lie-constrained matrix factorisation for real-time imaging

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Quasi-SVD learns a Lie-constrained approximate SVD whose one-sided orthogonal factor enables GPU-parallel medical imaging decompositions above 25 FPS with SSIM 0.89–0.94.

  2. Spectral-factorized Positive-definite Curvature Learning for NN Training

    stat.ML 2025-02 conditional novelty 6.0 of 10

    The paper derives a Riemannian update rule for the spectral factors of a positive-definite preconditioner, making arbitrary matrix roots fast and numerically stable for low-precision NN training.

  3. Optimizing LOCC Protocols on Product Stiefel Manifold

    quant-ph 2025-10 conditional novelty 5.0 of 10

    Fixed-round LOCC protocols are parameterized by a product Stiefel manifold and optimized with Riemannian gradient methods, yielding achievable distillation and state-merging fidelities that sometimes match PPT upper bounds.

  4. Quantum Solvers: Predictive Aeroacoustic & Aerodynamic modeling

    quant-ph 2025-07 conditional novelty 4.0 of 10

    The paper archives a winning Airbus/BMW challenge solution that compresses CFD operators into matrix product states and quantum circuits, reporting 0.1%-accurate cylinder flow at compression greater than 10.

Pith tools