REVIEW 1 major objections 5 minor 22 references
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
T0 review · 1 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The Stiefel Muon linear minimization oracle has an exact closed-form solution, given by the SVD of a skew-symmetric matrix.
desk verdict Solid exact closed-form result for the Stiefel Muon LMO, with a clean equivalence proof; the convergence theorem doesn't cover the implemented NS/momentum algorithm, but that gap is fixable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing construction is the direct sum decomposition of the space of skew-symmetric matrices $A_n = L_X \oplus K_X$, where $L_X$ is isomorphic to the Stiefel tangent space (infinitesimal rotations that move the frame) and $K_X$ consists of rotations in the complementary subspace that leave $X$ fixed. This decomposition lets the authors lift a tangent-space candidate $B$ to a skew-symmetric $Y$ with $YX = B$ and control the spectral norm via a norm-preserving dilation lemma. The SVD of $N = \mathrm{skew}(M X^\top)$ then supplies the closed-form optimum of the lifted problem, and the equivalence theorem shows no gap is introduced.
What would settle it
Choose $n > p$, a random $X \in \mathrm{St}(n,p)$ and $M \in \mathbb{R}^{n \times p}$; compute $N = \mathrm{skew}(M X^\top)$, its SVD, and the closed-form $B = Y_* X$ from Proposition 1. Verify that $B \in T_X \mathrm{St}(n,p)$, $\|B\|_2 = 1$ unless $P_X M = 0$, and $\langle M, B\rangle = -\|N\|_*$. If a high-accuracy convex solver finds a feasible tangent vector with strictly smaller objective value, the claimed equivalence is false; likewise, the characterization in the paper's Remark 4 would be falsified by any $B$ with $\|B\|_2 \le 1$ that cannot be written as $YX$ for skew $Y$ with $\|Y\|_2 \le 1$.
Extended reading notes
Core claim
The central claim is that the Stiefel Muon problem $\min \langle M, B\rangle$ subject to $\|B\|_2 \le 1$ and $B \in T_X \mathrm{St}(n,p)$ is equivalent to the Skewon problem $\min \langle \mathrm{skew}(M X^\top), Y \rangle$ subject to $\|Y\|_2 \le 1$ and $Y \in A_n$. Proposition 1 gives the closed form $Y_* = -U_1 V_1^\top - U_2 W V_2^\top$ from an SVD of $N = \mathrm{skew}(M X^\top)$, with optimal value $-\|N\|_*$. Theorem 2 proves that for any optimal $B_*$ of the SMP there exists an optimal $Y_*$ of the Skewon problem with $Y_* X = B_*$, $\|Y_*\|_2 = \|B_*\|_2 = 1$, so the Stiefel Muon update is exactly $B = Y_* X$.
Load-bearing premise
The stated first-order convergence guarantee holds only when the update direction is the exact Euclidean gradient and the polar factor is computed by an exact thin SVD; the practical Skewon algorithms use an iterative polar-factor computation and momentum updates, so the $O(1/\sqrt{T})$ rate does not directly cover the algorithm as implemented.
Editorial extensions
If this is right
- The Stiefel Muon update becomes a direct SVD-based computation instead of an iterative fixed-point or search procedure, eliminating per-step approximation overhead.
- Skewon has the same asymptotic complexity as unconstrained Muon ($O(n^3)$ in general, $O(np^2)$ when $p \ll n$), so orthogonality-constrained layers can be trained with Muon-style updates at comparable cost.
- The equivalence gives a clean characterization of the spectral-norm ball of the Stiefel tangent space as the image of the spectral-norm ball of skew-symmetric matrices under $Y \mapsto YX$.
- A first-order convergence rate of $O(\sqrt{pL/T})$ holds for smooth non-convex objectives under the standard Riemannian Lipschitz-gradient assumption, when the polar factor is computed exactly.
- Prior heuristic or relaxed SMP solvers can be replaced by Skewon in Riemannian optimizers, with accuracy limited only by the chosen SVD or iterative polar-factor implementation.
Reading between the lines
- The same lift-and-solve strategy likely extends to other matrix manifolds whose tangent spaces are parameterized by a Lie algebra, such as Grassmannian or symplectic Stiefel manifolds, giving closed-form spectral linear minimization oracles there as well.
- Because the convergence theorem assumes an exact thin SVD and the Euclidean gradient, the practical iterative version should be paired with a convergence check or an adaptive step-size rule; a direct empirical test would be to track the Riemannian gradient norm under the algorithm as written.
- The singular-value structure proved in the appendix for the skew-symmetrized polar factor suggests that rank-deficient cases can be handled robustly by thresholding singular values below machine precision, which may be important for numerical stability in low-rank settings.
- The equivalence's characterization of the tangent-space ball could inform analyses of Muon-style optimizers beyond the Stiefel case, such as bounds on the norm of the update direction in terms of $\mathrm{skew}(M X^\top)$.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the Muon linear minimization oracle on the Stiefel manifold. It introduces an auxiliary optimization problem over skew-symmetric matrices, called the Skewon problem, proves that it is equivalent to the Stiefel Muon problem (SMP), and derives a closed-form solution from the SVD of skew(MX^T). Based on this solution, it proposes two practical algorithms: a general one using Newton-Schulz iterations and a low-rank QR-based variant for p << n. The paper also states first-order convergence guarantees for smooth non-convex objectives and reports numerical comparisons with recent heuristic solvers. The central mathematical claims are the equivalence theorem and the closed-form update; the convergence section is where the claims and the implemented algorithms diverge.
Significance. If the results are taken as stated, the paper resolves a previously open subproblem in Muon-type optimization: the Stiefel tangent-space spectral LMO has an exact closed-form solution, and the solution can be evaluated with the same asymptotic complexity as Muon itself. The equivalence proof via the Davis-Kahan-Weinberger theorem is elegant and self-contained, with no fitted constants or normalization conventions forcing the conclusion. The numerical experiments are a useful sanity check against existing iterative heuristics. The main caveat is that the advertised first-order convergence guarantees apply only to an exact-SVD, Euclidean-gradient variant of the algorithm, not to the Newton-Schulz-based implementation with momentum that the paper actually presents as Skewon.
major comments (1)
- [Section 5, Theorem 3; Appendix A.2; Algorithms 1 and 2] The convergence guarantee is proved only for the variant in which M_t = ∇f(X_t) and the polar factor is computed by an exact thin SVD. The descent proof in Appendix A.2 uses the identity ⟨∇f(X_t), B_t⟩ = -‖skew(∇f(X_t)X_t^T)‖_*, which holds for the exact solution of the Skewon problem with direction equal to the Euclidean gradient. Algorithms 1 and 2 as stated in Section 4 compute Y_t = -skew(NS(N_t)) with a Newton-Schulz approximation and allow M_t to be a momentum direction. For an approximate polar factor, the descent inequality becomes f(X_{t+1}) ≤ f(X_t) - η_t(‖N_t‖_* - δ_t) + η_t^2 p L/2 with no control on the approximation gap δ_t, and for a momentum accumulator ⟨M_t, B_t⟩ is no longer the directional derivative of f. Thus the abstract's claim of 'first-order convergence guarantees for Skewon' is broader than the formal result. I recommend either proving a convergence theorem for the exact-SVD gradient variant and explicitly labeling the Newton-Schulz/momentum version as heuristic, or extending the analysis to control the approximation error and the momentum direction.
minor comments (5)
- [Section 2, after Eq. (6)] The sentence 'skew-symmetry does not increase the spectral norm' is used repeatedly; making explicit that this refers to the inequality ‖(Y - Y^T)/2‖₂ ≤ ‖Y‖₂ would improve readability.
- [Theorem 2, proof] The assertion that the map C ↦ X_⊥^T C X_⊥ is surjective onto A_{n-p} is stated without proof; the construction C = X_⊥ D X_⊥^T for D ∈ A_{n-p} makes this immediate and should be included.
- [Section 5, first paragraph] The phrase 'for any realization of Algorithm 1 and Algorithm 2, in which the polar factor is evaluated using a thin SVD' is ambiguous, because Algorithms 1 and 2 as written in Section 4 use Newton-Schulz iterations. I suggest naming the analyzed variant explicitly, e.g. 'Skewon-SVD', and distinguishing it from the default Newton-Schulz implementation.
- [Appendix A.3, Proposition 4] The multiset notation σ(S) = {1, ..., 1, μ_1, ..., μ_{n-2k}} is clear, but it would be helpful to state explicitly that the μ_j are allowed to be zero and that the 2k singular values equal to 1 come in pairs from the J blocks.
- [Section 6, Figures 1-3] The figure captions list ten methods; checking that the legend order matches the plotting order in each figure would remove potential ambiguity.
Circularity Check
No significant circularity: the exact closed-form and equivalence proofs are self-contained and do not reduce to their inputs.
full rationale
The central derivation chain is self-contained. Proposition 1 solves the Skewon LMO directly from the SVD of N = skew(MX^T) by a unitary-invariance argument; no fitted constant or prior result of the authors is used. Theorem 1 is a direct block decomposition of A_n, and Theorem 2 combines it with the external Davis–Kahan–Weinberger theorem to map optimal tangent vectors to optimal skew-symmetric matrices; the equivalence is proved in both directions (every Skewon optimum maps to a feasible SMP point and vice versa), so the equality of optimal values is not assumed by defining one problem in terms of the other. The self-citations in the related-work section (Refs. 14, 15, 16) and the mention of CANS [16] as an alternative polar-factor solver do not carry any load in the proofs. The only notable caveat is that Theorem 3 is proved under exact thin-SVD polar factors and M_t = ∇f(X_t), while Algorithms 1–2 use Newton–Schulz and permit momentum directions; that is a mismatch between theorem assumptions and the practical implementation, i.e., a correctness/scope gap, not a circular reduction. The convergence argument itself uses the identity ⟨∇f(X_t), B_t⟩ = −‖skew(∇f(X_t)X_t^T)‖_*, which follows from Proposition 1 and is not an input restated as an output.
Assumptions & free parameters
assumptions (5)
- standard math Davis-Kahan-Weinberger norm-preserving dilation theorem
- standard math Real Schur decomposition for skew-symmetric matrices
- domain assumption Riemannian L-smoothness of f with respect to the retraction (Assumption 1)
- standard math Compactness of the Stiefel manifold
- standard math SVD existence and unitary invariance of spectral and Frobenius norms
Cite this review
Pith. "Pith review of Muon on the Stiefel Manifold Admits an Exact Closed-Form Update." pith.science (2026). https://pith.science/paper/LO76PM2G
@misc{pith2026260806218,
author = {Pith},
title = {Pith review of: Muon on the Stiefel Manifold Admits an Exact Closed-Form Update},
year = {2026},
howpublished = {\url{https://pith.science/paper/LO76PM2G}},
note = {Machine review of arXiv:2608.06218}
}
read the original abstract
We study Muon, a recently proposed matrix-aware optimization method, in the context of the Stiefel manifold. This manifold consists of matrices with orthonormal columns and is ubiquitous in machine learning and scientific computing. Existing extensions of Muon to this manifold rely on heuristic, approximate, or iterative updates with varying computational efficiency. We show that the corresponding Stiefel Muon update admits an exact closed-form solution and use this result to develop Skewon, a practical algorithm for orthogonality-constrained optimization with an efficient implementation. We further establish first-order convergence guarantees for Skewon in the smooth non-convex setting.
Reference graph
Works this paper leans on
-
[1]
https:// kellerjordan.github.io/posts/muon/
Jordan, K., Jin, Y., Boza, V., You, J., Cesista, F., Newhouse, L., Bernstein, 13 J.: Muon: An optimizer for hidden layers in neural networks (2024). https:// kellerjordan.github.io/posts/muon/
work page 2024
-
[2]
Transactions on Machine Learning Research (2026)
Shen, W., Huang, R., Huang, M., Shen, C., Zhang, J.: On the convergence analysis of muon. Transactions on Machine Learning Research (2026). J2C Certification
work page 2026
-
[3]
https://thinkingmachines.ai/blog/modular- manifolds (2025)
Bernstein, J.: Modular Manifolds. https://thinkingmachines.ai/blog/modular- manifolds (2025)
work page 2025
-
[4]
https://leloykun.github.io/ponder/steepest-descent-stiefel/
Cesista, F.L.: Heuristic Solutions for Steepest Descent on the Stiefel manifold (2025). https://leloykun.github.io/ponder/steepest-descent-stiefel/
work page 2025
-
[5]
Su, J.: Steepest Descent on Manifolds: 3. Muon + Stiefel (2025). https://www. kexue.fm/archives/11221
work page 2025
-
[6]
Preprint at https:// arxiv.org/abs/2601.21487v1 (2026)
Yang, K., Lai, L.: Manifold constrained steepest descent. Preprint at https:// arxiv.org/abs/2601.21487v1 (2026)
arXiv 2026
-
[7]
Preprint at https://arxiv.org/abs/2605.09238v1 (2026)
Li, Y., Pandey, B.L., Sah, R., Han, A., Mostajeran, C., Jawanpuria, P., Mishra, B.: Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds. Preprint at https://arxiv.org/abs/2605.09238v1 (2026)
arXiv 2026
-
[8]
Cambridge University Press, Cambridge (2023)
Boumal, N.: An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, Cambridge (2023)
2023
Show all 22 references
-
[9]
Princeton University Press, Princeton, NJ (2008)
Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008)
2008
-
[10]
SIAM journal on Matrix Analysis and Applications20(2), 303–353 (1998)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthog- onality constraints. SIAM journal on Matrix Analysis and Applications20(2), 303–353 (1998)
1998
-
[11]
Preprint at https: //arxiv.org/abs/2604.05183 (2026)
Aliev, A., Garifullin, K., Yudin, N., Soboleva, V., Molozhavenko, A., Oseledets, I., Alanov, A., Rakhuba, M.: OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models. Preprint at https: //arxiv.org/abs/2604.05183 (2026)
2026 arXiv
-
[12]
Bansal, N., Chen, X., Wang, Z.: Can we gain more from orthogonality regular- izations in training deep networks? Advances in Neural Information Processing Systems31(2018)
2018
-
[13]
Preprint at https://arxiv.org/abs/2506.03133v2 (2025)
Lion, K., Zhang, L., Li, B., He, N.: PoLAR: Polar-Decomposed Low-Rank Adapter Representation. Preprint at https://arxiv.org/abs/2506.03133v2 (2025)
2025
-
[14]
Preprint at https://arxiv.org/abs/2606.25975 (2026) 14
Bogachev, V., Aletov, V., Molozhavenko, A., Kudriashov, S., Rakhuba, M.: Tensorion: A Tensor-Aware Generalization of the Muon Optimizer. Preprint at https://arxiv.org/abs/2606.25975 (2026) 14
2026 arXiv
-
[15]
Preprint at https://arxiv.org/abs/2507.12142 (2025)
Bogachev, V., Aletov, V., Molozhavenko, A., Bobkov, D., Soboleva, V., Alanov, A., Rakhuba, M.: LoRA meets Riemannion: Muon Optimizer for Parametrization- independent Low-Rank Adapters. Preprint at https://arxiv.org/abs/2507.12142 (2025)
2025
-
[16]
Preprint at https://arxiv
Grishina, E., Smirnov, M., Rakhuba, M.: Accelerating Newton-Schulz Iteration for Orthogonalization via Chebyshev-type Polynomials. Preprint at https://arxiv. org/abs/2506.10935 (2025)
2025
-
[17]
Preprint at https://arxiv.org/abs/2505.16932 (2025)
Amsel, N., Persson, D., Musco, C., Gower, R.M.: The polar express: Optimal matrix sign methods and their application to the muon algorithm. Preprint at https://arxiv.org/abs/2505.16932 (2025)
2025 arXiv
-
[18]
SIAM Journal on Numerical Analysis19(3), 445–469 (1982)
Davis, C., Kahan, W., Weinberger, H.: Norm-preserving dilations and their appli- cations to optimal error bounds. SIAM Journal on Numerical Analysis19(3), 445–469 (1982)
1982
-
[19]
Preprint at https://arxiv.org/abs/2602.11948v1 (2026)
Gonon, A., Mu¸ sat, A.-A., Boumal, N.: Insights on muon from simple quadratics. Preprint at https://arxiv.org/abs/2602.11948v1 (2026)
2026
-
[20]
SIAM Journal on Matrix Analysis and Applications34(3), 1171–1191 (2013)
Barlow, J.L.: Reorthogonalized block classical gram–schmidt. SIAM Journal on Matrix Analysis and Applications34(3), 1171–1191 (2013)
2013
-
[21]
Journal of Optimization Theory and Applications169(3), 1042–1068 (2016)
O’Donoghue, B., Chu, E., Parikh, N., Boyd, S.: Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applications169(3), 1042–1068 (2016)
2016
-
[22]
Diamond, S., Boyd, S.: CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research17(83), 1–5 (2016) Appendix A Additional proofs A.1 Proof of Proposition 2 Every optimal solution of the SMP (2) with nonzeroPX Mis attained on the bo...
2016
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.