Pith. sign in

REVIEW 5 minor 40 references

The smoothed polar map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2) makes Muon-style spectral gradient flows globally well-posed and convergent, with O(1/t), exponential, and O(1/T) rates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 18:25 UTC pith:HMR6H52G

load-bearing objection A clean, honest continuous-time analysis of a smoothed polar spectral flow; the core theorems hold, the local advantage criterion is a real contribution, and the limitations are properly scoped — worth refereeing.

arxiv 2608.01911 v1 pith:HMR6H52G submitted 2026-08-03 math.OC

A Continuous-Time Analysis of Smoothed Matrix-Polar Spectral Gradient Flows for Muon-Type Optimization

Jinlin Liu , Song Chen , Jiaxu Liu , Chao Xu This is my paper
classification math.OC MSC 34D2390C30
keywords smoothed matrix-polar feedbackspectral gradient flowMuon optimizercontinuous-time optimizationmatrix optimizationLyapunov analysisPolyak–Lojasiewicz conditionconvergence rates
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper studies matrix-valued optimization where the update direction is a spectrally normalized gradient, as in Muon-type optimizers. Because the exact polar factor Z(Z^T Z)^(-1/2) is nonsmooth at rank-deficient matrices, the authors replace it with the smoothed map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2), which preserves singular vectors but remains smooth. They prove that the resulting continuous-time flows are globally well-posed, dissipative, and converge to the stationary set, with explicit rates in nonconvex, convex, and Polyak–Lojasiewicz settings. They also compare the smoothed spectral direction with the Frobenius direction locally, deriving a simple numerical criterion for when the spectral direction descends faster. The work provides a continuous-time foundation for spectral normalization, including a finite-horizon O(epsilon) approximation to the ideal matrix-polar flow.

Core claim

The central claim is that the smoothed spectral feedback law h_epsilon(G)=G(G^T G+epsilon I)^(-1/2) is a regular, monotone, bounded, dissipative model of matrix-polar, Muon-type updates. For the direct flow dot W = -h_epsilon(grad f(W)), the paper establishes global existence and uniqueness, boundedness, monotone decrease of f, vanishing of the gradient, and convergence to the stationary set. In convex and PL landscapes it obtains O(1/t) and exponential objective-gap bounds, and a min-gradient O(1/T) stationarity certificate in the nonconvex case. For the momentum-augmented system dot W = -h_epsilon(M), dot M = a grad f(W) - b M, it proves a Lyapunov identity and convergence to the stationar

What carries the argument

The central object is the smoothed matrix-polar map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2), defined for epsilon > 0. In a singular value decomposition it acts as sigma -> sigma/sqrt(sigma^2+epsilon), so it preserves singular vectors, is smooth and globally Lipschitz, and is exactly the gradient of the convex spectral potential Phi_epsilon(Z)=Tr[(Z^T Z+epsilon I)^(1/2)-sqrt(epsilon)I]. This gradient structure yields the dissipation identity d/dt f(W) = -sum_i sigma_i^2/sqrt(sigma_i^2+epsilon), which drives all the rate bounds. The local comparison uses the ratio Gamma_epsilon = R_epsilon/R_F of calibrated descent rates, which for the quadratic layerwise loss simplifies to r_eff(G) sim_HS(rho_

Load-bearing premise

The load-bearing premise is that the relevant gradient or momentum singular values stay uniformly bounded below by a positive constant on the time horizon; near zero singular values the smoothed map and the polar map behave differently, so the O(epsilon) approximation and the Muon connection break down.

What would settle it

Run the smoothed direct flow and the ideal polar flow on f(W)=||W||_F^2/2 from the same full-rank W0, and record sup_{t<=T} ||W_epsilon(t)-W_P(t)||_F as epsilon -> 0: for T long enough that the minimum singular value of grad f(W_P) drops below sqrt(epsilon), the error should saturate at order 1 rather than O(epsilon), directly testing the rank-safe assumption behind Proposition 2.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Every solution of the smoothed direct flow converges to the stationary set {W : grad f(W)=0}, with min-norm squared gradient O(1/T) in the nonconvex case.
  • Under convexity the objective gap decays as O(1/t); under a Polyak–Lojasiewicz inequality it decays exponentially, with constants governed by the sector coefficient c_epsilon = 1/sqrt(K_G^2 + epsilon).
  • On any finite time interval where gradient or momentum singular values stay bounded below by 2 sigma, the smoothed flow shadows the ideal matrix-polar flow uniformly with O(epsilon) error.
  • For the quadratic layerwise loss, the matrix-polar direction has a larger local descent rate than the Frobenius direction exactly when r_eff(G) sim_HS(rho_G, rho_A) > 1.
  • The momentum-augmented system is dissipative with Lyapunov function V = a(f - f_inf) + Phi_epsilon(M), and every limit point lies in S times {0}.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the local criterion is predictive in practice, an optimizer could switch between Frobenius and spectral directions pointwise based on measured r_eff and sim_HS, avoiding a global hyperparameter search.
  • The failure of uniform convergence near zero singular values suggests Muon's benefits may be confined to regimes where gradient singular values stay away from rank transitions; modeling rank-crossing dynamics is a natural next step.
  • The smoothness and Lipschitz constants of h_epsilon could enable convergence proofs for discrete Muon-type methods by connecting them to Euler discretizations of the smoothed flow.
  • The effective-rank factor in the criterion yields a testable prediction for deep networks: spectral normalization should help most in layers whose gradient spectrum is relatively flat and whose activation covariance aligns with the gradient covariance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper introduces a smoothed matrix-polar feedback law h_epsilon(G)=G(G^T G+epsilon I)^{-1/2} for matrix-valued optimization, motivated by Muon-type spectral normalization. It proves basic analytic properties of this map (Lipschitz regularity, monotonicity, boundedness, sector dissipation), then studies the direct flow dot W=-h_epsilon(grad f(W)) and a momentum-augmented system dot W=-h_epsilon(M), dot M=a grad f(W)-b M. For both systems it establishes global well-posedness, boundedness, dissipation identities, and convergence to the stationary set. It derives quantitative rates: O(1/T) minimum-gradient nonconvex stationarity, O(1/t) convex objective-gap decay, exponential decay under a Polyak-Lojasiewicz condition for the direct flow, and an O(T^{-1/2}) averaged convex rate for the momentum system. Proposition 2 gives an O(epsilon) finite-horizon approximation of the ideal matrix-polar flow under a uniform lower singular-value bound, and Remark 2 explicitly scopes this result away from rank degeneracy. The final part develops a local descent-rate comparison between the smoothed spectral direction and the Frobenius direction under a calibrated quadratic model, yielding the criterion r_eff(G) sim_HS(rho_G,rho_A)>1 for the limiting matrix-polar advantage in the quadratic-layer case. Numerical experiments illustrate the approximation, the local threshold, and the convergence bounds.

Significance. If the result stands, this is a useful rigorous continuous-time foundation for spectral normalization methods of Muon type. The paper's main theorems are not empirically fitted: the rates and the local criterion are derived from the definitions and standard ODE/Lyapunov arguments, and the numerical section is a controlled verification of the algebra rather than a benchmark. The authors are unusually careful about the scope of their results: Proposition 2 is explicitly presented as a finite-horizon, rank-safe approximation statement, and Remark 2 demonstrates the nonuniformity near rank deficiency with a scalar example. The local advantage criterion is a concrete, falsifiable prediction that connects spectral uniformity of the gradient, covariance alignment, and Hessian curvature. The paper is modest in scope but internally consistent and technically sound.

minor comments (5)
  1. [Section II, Lemma 1] The Lipschitz estimate (5) is stated with constant 1/sqrt(epsilon). The proof via the symmetric dilation is correct, but it relies on an implicit Frobenius-norm Lipschitz property for scalar functions applied to Hermitian matrices. A one-sentence justification or reference would improve readability, especially because this constant is used for well-posedness.
  2. [Section V-A, Eqs. (23)-(24)] The calibrated rate R_W(D) is introduced as a comparison device, but the actual flows in Theorem 1 do not carry the factor alpha_D^*. It would help to state explicitly that alpha_D^* is the optimal step of the local quadratic model and that the comparison is intrinsic to the direction, not to the uncalibrated trajectories.
  3. [Section V-B, Eq. (36)] The quantity r_eff(G)=||G||_*^2/||G||_F^2 is called the 'effective rank'. This is nonstandard terminology; the usual effective rank is (sum sigma_i^2)^2/(sum sigma_i^4). Since r_eff(G) is always at least 1, a footnote clarifying the definition would avoid confusion.
  4. [Section VI-A, Eq. (45)-(46)] The discrete momentum recursion uses the coefficient (1-beta)nabla f(W_k), while the continuous model uses lambda(nabla f(W)-M). The matching beta=e^{-lambda eta} is only first-order in eta. Stating this approximation order explicitly would make the comparison between discrete and continuous trajectories more precise.
  5. [Remark 2] The scalar example contains a rendering typo: h_epsilon(z_epsilon)=epsilon/sqrt(epsilon^2+epsilon) should equal sqrt(epsilon/(1+epsilon)), not 'r epsilon/(1+epsilon)'. The text as displayed is missing a square-root sign.

Circularity Check

0 steps flagged

No significant circularity: the derivations are self-contained from the stated feedback law and standard ODE/Lyapunov arguments.

full rationale

I walked the derivation chain from the definition of the smoothed spectral feedback law h_epsilon(Z) = Z(Z^T Z + eps I)^(-1/2) through the flow theorems, the momentum system, the finite-horizon approximation, and the local descent-rate comparison. Equations (1)-(4) define the object; Lemma 1 proves its Lipschitz, monotone, bounded, and sector properties directly from the spectral decomposition and convexity of the spectral potential. The dissipation identities (11) and (14) are exact consequences of grad Phi_epsilon = h_epsilon, not imposed or fitted. The convergence theorems (Theorems 1-7) follow from these identities plus standard assumptions (Assumptions 1-2) and classical tools (Barbalat, LaSalle, Jensen, Gronwall). The convergence-rate constants c_epsilon, K_G, K_M are computed from sublevel-set bounds of the given objective and the fixed smoothing parameter; epsilon is a user-chosen hyperparameter and is not tuned to match a predicted outcome. The local comparison Theorem 8 and Corollary 1 are algebraic consequences of the quadratic model q_D(alpha), and the numerical sweeps in Section VI-B construct controlled examples that verify the threshold reff(G) simHS(rho_G, rho_A) = 1; they do not fit any parameter and call a fit a prediction. The only place where a connection to the ideal matrix-polar flow is claimed, Proposition 2, is explicitly conditioned on a uniform lower singular-value lower bound (18)/(20). Remark 2 itself demonstrates the non-uniformity near zero singular values with the scalar example h_epsilon(epsilon) -> 0 while Polar(epsilon) = 1, and Section VI-A confirms the expected breakdown near rank deficiency. This is an honest, explicit scoping of the Muon motivation, not a circular reduction: the core convergence and rate theorems do not depend on the approximation statement. The self-citations [19] and [21] appear only in the introduction as motivational context ('Control-inspired optimization frameworks, including PIDAO models [21], motivate a natural question'), and no load-bearing lemma or theorem imports a conclusion from those papers. I find no step where the claimed result is equivalent to its input by definition, no fitted parameter renamed as a prediction, and no load-bearing self-citation chain. The derivation is therefore self-contained, and the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

2 free parameters · 7 axioms · 0 invented entities

The central analysis rests on standard smoothness and coercivity assumptions on f, standard ODE theorems, and pointwise curvature or full-rank conditions for the local and approximation statements. The only design hyperparameters are epsilon and the momentum gains a,b. No physical entity or new conserved quantity is introduced.

free parameters (2)
  • epsilon = hyperparameter, not fitted (e.g., 1e-2, 1e-3 in experiments)
    Introduced in (1) to regularize the polar map; all constants and rates depend on it; no fitting procedure determines it.
  • a, b = arbitrary positive constants chosen by hand (e.g., b=2 in experiments)
    Define the momentum-augmented system (12); the rates depend on these gains; they are not fitted to data.
axioms (7)
  • domain assumption f is C^1 with locally Lipschitz gradient (Assumption 1)
    Used to apply Picard-Lindelof and to ensure grad f is uniformly continuous on the bounded sublevel set; invoked in Prop. 1, Thm. 1, and all rate theorems.
  • domain assumption f is bounded below and coercive (Assumption 2)
    Guarantees bounded trajectories and a finite dissipation budget; used in Theorems 1, 2, 4, 5, 6, 7.
  • domain assumption Convexity, PL inequality, or strong convexity when rates are claimed
    Theorems 4, 5, and 7 require convexity or PL-type structure; these are standard and stated in each theorem.
  • domain assumption Positive directional curvature <D,H_W[D]> > 0 for D=G and D=h_epsilon(G) (Assumption 3)
    Needed for the alpha* calibration and the descent-rate ratio; if curvature is nonpositive the local comparison is undefined.
  • domain assumption Full-rank or non-singular G and uniform lower singular-value bounds sigma_min >= 2 sigma on the ideal trajectory (Prop. 2; Section V-B)
    Needed for the O(epsilon) approximation to the ideal polar flow and for the limiting formula r_eff times simHS; the paper shows the bound is essential.
  • standard math Classical ODE facts: Picard-Lindelof, continuation, Barbalat, LaSalle invariance, Gronwall
    Used throughout Appendices II-IV for global existence, boundedness, convergence, and rate estimates.
  • standard math Convexity of the spectral potential Phi_epsilon and the Frobenius-Lipschitz estimate for matrix functions
    Lemma 1 relies on these matrix-analysis facts; they are asserted with standard references (Bhatia, Higham) but not proved in full.

pith-pipeline@v1.3.0-daily-deepseek · 20815 in / 23741 out tokens · 274802 ms · 2026-08-04T18:25:22.503997+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of A Continuous-Time Analysis of Smoothed Matrix-Polar Spectral Gradient Flows for Muon-Type Optimization." pith.science (2026). https://pith.science/paper/HMR6H52G

@misc{pith2026260801911,
  author       = {Pith},
  title        = {Pith review of: A Continuous-Time Analysis of Smoothed Matrix-Polar Spectral Gradient Flows for Muon-Type Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HMR6H52G}},
  note         = {Machine review of arXiv:2608.01911}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper studies smoothed matrix-polar spectral gradient flows for unconstrained matrix-valued optimization.The canonical polar-factor map loses smoothness at rank-deficient matrices and becomes ill-conditioned as singular values approach zero, creating analytical difficulties.We therefore introduce a spectral feedback law generated by a smooth spectral potential and establish the regularity, monotonicity, boundedness, and dissipation properties of the feedback.Based on this feedback law, we propose a smoothed spectral gradient flow and prove well-posedness and global convergence of the flow.We derive convergence-rate results for the spectral gradient flow in nonconvex, convex, and Polyak--Lojasiewicz (PL) settings and analyze the Lyapunov structure and convergence of a momentum-augmented system under the same spectral feedback law. Furthermore, we provide a local descent-rate comparison between the smoothed spectral-gradient direction and the standard Frobenius-gradient direction using a general Hessian-based quadratic model. This analysis yields a verifiable normalized descent-rate advantage condition, showing that the local benefit of the spectral direction depends on both first-order alignment with the gradient matrix and the directional curvature induced by the Hessian.

Figures

Figures reproduced from arXiv: 2608.01911 by Chao Xu, Jiaxu Liu, Jinlin Liu, Song Chen.

Figure 1
Figure 1. Figure 1: A common approach is to vectorize these parameters [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: A selected neural-network weight block can be represented in tensor, matrix, or vector form, leading to a training problem minX∈X f(X) on the chosen parameter space X . From this problem, one may design a discrete optimization algorithm Xk+1 = Xk − η U(Xk), where f is the loss function and U is an update operator related to ∇f(X). [11], [12]. Related continuous-time gradient and consensus dynamics, as well… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the Hessian action and directional curva￾ture. At the point W, the direction D determines the slice s 7→ f(W + sD) shown by the red curve. The quantity ⟨D, HW [D]⟩ = d 2 ds2 f(W + sD) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cumulative trajectory discrepancies between the discrete exact-polar algorithms and the corresponding smoothed continuous-time models. Left: direct spectral-gradient method. Right: momentum-augmented Muon-type method [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Measured initial descent rates under controlled variations of covariance similarity and gradient effective rank. The crossings occur near the theoretical thresholds determined by Γ(W0) = 1. To visualize this comparison, we select two representative covariance-similarity settings. They satisfy Γ(W0) = 1.376 > 1 and Γ(W0) = 0.8 < 1, respectively. The results illustrate that the local matrix-polar advantage i… view at source ↗
Figure 5
Figure 5. Figure 5: Representative loss trajectories and initial tangent lines. The matrix-polar trajectory has the steeper initial decrease when Γ(W0) > 1, whereas the Frobenius-gradient trajectory is steeper when Γ(W0) < 1 [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ratio between the observed objective gap and the theoretical trajectory bound. Left: strongly convex quadratic loss. Right: general convex loss. The dashed line denotes Q(t) = 1. VII. CONCLUSION This paper studied a class of spectral gradient flows induced by smoothed matrix-polar feedback. The flow uses hϵ(G) = G(G ⊤G + ϵI) −1/2 as the gradient-feedback direction. This map preserves the directional struct… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 1 canonical work pages

  1. [1]

    Modular duality in deep learning,

    J. Bernstein and L. Newhouse, “Modular duality in deep learning,” inProceedings of the 42nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 12 IEEE TRANSACTIONS AND JOURNALS TEMPLATE

  2. [2]

    Muon is scalable for LLM training,

    J. Liu, J. Su, X. Yao, Z. Jiang, G. Lai, Y . Du, Y . Qin, W. Xu, E. Lu, J. Yan, Y . Chen, H. Zheng, Y . Liu, S. Liu, B. Yin, W. He, H. Zhu, Y . Wang, J. Wang, M. Dong, Z. Zhang, Y . Kang, H. Zhang, X. Xu, Y . Zhang, Y . Wu, X. Zhou, and Z. Yang, “Muon is scalable for LLM training,” 2025. [Online]. Available: https://arxiv.org/abs/2502.16982

    Pith/arXiv arXiv 2025

  3. [3]

    An iterative algorithm for computing the best estimate of an orthogonal matrix,

    A. Bj ¨orck and C. Bowie, “An iterative algorithm for computing the best estimate of an orthogonal matrix,”SIAM Journal on Numerical Analysis, vol. 8, no. 2, pp. 358–364, 1971

    1971

  4. [4]

    Convergence of Muon with Newton–Schulz,

    G. Y . Kim and M. Oh, “Convergence of Muon with Newton–Schulz,”

  5. [5]

    A note on the convergence of Muon and further,

    J. Li and M. Hong, “A note on the convergence of Muon and further,”

  6. [6]

    On the convergence analysis of Muon,

    W. Shen, R. Huang, M. Huang, C. Shen, and J. Zhang, “On the convergence analysis of Muon,” 2025. [Online]. Available: https://arxiv.org/abs/2505.23737

    Pith/arXiv arXiv 2025

  7. [7]

    Some methods of speeding up the convergence of iter- ation methods,

    B. T. Polyak, “Some methods of speeding up the convergence of iter- ation methods,”USSR Computational Mathematics and Mathematical Physics, vol. 4, no. 5, pp. 1–17, 1964

    1964

  8. [8]

    A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights,

    W. Su, S. Boyd, and E. J. Cand `es, “A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights,”Journal of Machine Learning Research, vol. 17, no. 153, pp. 1–43, 2016

    2016

  9. [9]

    A variational perspective on accelerated methods in optimization,

    A. Wibisono, A. C. Wilson, and M. I. Jordan, “A variational perspective on accelerated methods in optimization,”Proceedings of the National Academy of Sciences, vol. 113, no. 47, pp. E7351–E7358, 2016

    2016

  10. [10]

    A Lyapunov analysis of accelerated methods in optimization,

    A. C. Wilson, B. Recht, and M. I. Jordan, “A Lyapunov analysis of accelerated methods in optimization,”Journal of Machine Learning Research, vol. 22, no. 113, pp. 1–34, 2021

    2021

  11. [11]

    Dissipative dynamical systems part I: General theory,

    J. C. Willems, “Dissipative dynamical systems part I: General theory,” Archive for Rational Mechanics and Analysis, vol. 45, no. 5, pp. 321– 351, 1972

    1972

  12. [12]

    Uniting two control Lyapunov functions for affine systems,

    V . Andrieu and C. Prieur, “Uniting two control Lyapunov functions for affine systems,”IEEE Transactions on Automatic Control, vol. 55, no. 8, pp. 1923–1927, 2010

    1923

  13. [13]

    Distributed continuous-time convex opti- mization on weight-balanced digraphs,

    B. Gharesifard and J. Cort ´es, “Distributed continuous-time convex opti- mization on weight-balanced digraphs,”IEEE Transactions on Automatic Control, vol. 59, no. 3, pp. 781–786, 2014

    2014

  14. [14]

    Zero-gradient-sum algorithms for distributed convex optimization: The continuous-time case,

    J. Lu and C. Y . Tang, “Zero-gradient-sum algorithms for distributed convex optimization: The continuous-time case,”IEEE Transactions on Automatic Control, vol. 57, no. 9, pp. 2348–2354, 2012

    2012

  15. [15]

    Quantization enabled privacy protection in decentralized stochastic optimization,

    Y . Wang and T. Bas ¸ar, “Quantization enabled privacy protection in decentralized stochastic optimization,”IEEE Transactions on Automatic Control, vol. 68, no. 7, pp. 4038–4052, 2023

    2023

  16. [16]

    Stability of extremum seeking feedback for general nonlinear dynamic systems,

    M. Krsti ´c and H.-H. Wang, “Stability of extremum seeking feedback for general nonlinear dynamic systems,”Automatica, vol. 36, no. 4, pp. 595–601, 2000

    2000

  17. [17]

    Adaptive extremum seeking control of nonlinear dynamic systems with parametric uncertainties,

    M. Guay and T. Zhang, “Adaptive extremum seeking control of nonlinear dynamic systems with parametric uncertainties,”Automatica, vol. 39, no. 7, pp. 1283–1293, 2003

    2003

  18. [18]

    Continuous-time zeroth-order dynam- ics with projection maps: Model-free feedback optimization with safety guarantees,

    X. Chen, J. I. Poveda, and N. Li, “Continuous-time zeroth-order dynam- ics with projection maps: Model-free feedback optimization with safety guarantees,”IEEE Transactions on Automatic Control, vol. 70, no. 8, pp. 5005–5020, 2025

    2025

  19. [19]

    A backstepping framework for unconstrained accelerated optimization algorithms,

    S. Chen, J. Liu, and C. Xu, “A backstepping framework for unconstrained accelerated optimization algorithms,” 2026. [Online]. Available: https://arxiv.org/abs/2606.27722

    Pith/arXiv arXiv 2026

  20. [20]

    Complexity of derivative-free policy optimization for structuredH ∞ control,

    X. Guo, D. Keivan, G. E. Dullerud, P. J. Seiler, and B. Hu, “Complexity of derivative-free policy optimization for structuredH ∞ control,” inAdvances in Neural Information Processing Systems, vol. 36, 2023, pp. 5050–5078. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/2023/hash/ 1052b823a161aa2c808dd51c0f58dc37-Abstract-Conference.html

    2023

  21. [21]

    Accelerated optimization in deep learning with a proportional-integral-derivative controller,

    S. Chen, J. Liu, P. Wanget al., “Accelerated optimization in deep learning with a proportional-integral-derivative controller,”Nature Communications, vol. 15, p. 10263, 2024. [Online]. Available: https://doi.org/10.1038/s41467-024-54451-3

    work page doi:10.1038/s41467-024-54451-3 2024

  22. [22]

    Bhatia,Matrix Analysis

    R. Bhatia,Matrix Analysis. New York, NY , USA: Springer, 1997

    1997

  23. [23]

    N. J. Higham,Functions of Matrices: Theory and Computation. Philadelphia, PA, USA: SIAM, 2008

    2008

  24. [24]

    Fixed-time stable gradient flows: Applications to continuous-time optimization,

    K. Garg and D. Panagou, “Fixed-time stable gradient flows: Applications to continuous-time optimization,”IEEE Transactions on Automatic Control, vol. 66, no. 5, pp. 2002–2015, 2021

    2002

  25. [25]

    Distributed continuous-time optimiza- tion: Nonuniform gradient gains, finite-time convergence, and convex constraint set,

    P. Lin, W. Ren, and J. A. Farrell, “Distributed continuous-time optimiza- tion: Nonuniform gradient gains, finite-time convergence, and convex constraint set,”IEEE Transactions on Automatic Control, vol. 62, no. 5, pp. 2239–2253, 2017

    2017

  26. [26]

    Reaching an optimal consensus: Dynamical systems that compute intersections of convex sets,

    G. Shi, K. H. Johansson, and Y . Hong, “Reaching an optimal consensus: Dynamical systems that compute intersections of convex sets,”IEEE Transactions on Automatic Control, vol. 58, no. 3, pp. 610–622, 2013

    2013

  27. [27]

    Distributed continuous-time convex optimization with time-varying cost functions,

    S. Rahili and W. Ren, “Distributed continuous-time convex optimization with time-varying cost functions,”IEEE Transactions on Automatic Control, vol. 62, no. 4, pp. 1590–1605, 2017

    2017

  28. [28]

    Distributed continuous-time algorithm for constrained convex optimizations via nonsmooth analysis approach,

    X. Zeng, P. Yi, and Y . Hong, “Distributed continuous-time algorithm for constrained convex optimizations via nonsmooth analysis approach,” IEEE Transactions on Automatic Control, vol. 62, no. 10, pp. 5227– 5233, 2017

    2017

  29. [29]

    Ex- tremum problems with total variation distance and their applications,

    C. D. Charalambous, I. Tzortzis, S. Loyka, and T. Charalambous, “Ex- tremum problems with total variation distance and their applications,” IEEE Transactions on Automatic Control, vol. 59, no. 9, pp. 2353–2368, 2014

    2014

  30. [30]

    Convex analysis on the Hermitian matrices,

    A. S. Lewis, “Convex analysis on the Hermitian matrices,”SIAM Journal on Optimization, vol. 6, no. 1, pp. 164–177, 1996

    1996

  31. [31]

    Nonsmooth analysis of singular values. part I: Theory,

    A. S. Lewis and H. S. Sendov, “Nonsmooth analysis of singular values. part I: Theory,”Set-Valued Analysis, vol. 13, no. 3, pp. 213–241, 2005

    2005

  32. [32]

    Understanding gradient orthogonalization for deep learning via Non-Euclidean trust-region optimization,

    D. Kovalev, “Understanding gradient orthogonalization for deep learning via Non-Euclidean trust-region optimization,” 2025. [Online]. Available: https://arxiv.org/abs/2503.12645

    Pith/arXiv arXiv 2025

  33. [33]

    Spectral conditions for feature learning,

    D. Davis, M. Jiang, B. Jiang, and R. M. Jiang, “Spectral conditions for feature learning,” 2025. [Online]. Available: https: //arxiv.org/abs/2502.06749

    Pith/arXiv arXiv 2025

  34. [34]

    J. P. LaSalle and S. Lefschetz,Stability by Liapunov’s Direct Method with Applications. New York, NY , USA: Academic Press, 1961

    1961

  35. [35]

    H. K. Khalil,Nonlinear Systems, 3rd ed. Upper Saddle River, NJ, USA: Prentice Hall, 2002. APPENDIXI PROOFS OFSPECTRALFEEDBACKPROPERTIES Proof:Let A(M) =M ⊤M+ϵI. Sinceϵ >0,A(M)is positive definite. For any perturbation H∈R m×n, the differential formula for a spectral trace function gives dTr(A 1/2) = 1 2 Tr(A−1/2dA). Consequently, dΦϵ(M)[H] = 1 2 Tr A(M) −...

    2002

  36. [39]

    Continuity of∇fgives∇f( ¯W) = 0, and thereforeω(W 0)⊆ S

    If ¯W∈ω(W 0), then there exists a sequencet k → ∞such thatW(t k)→ ¯W. Continuity of∇fgives∇f( ¯W) = 0, and thereforeω(W 0)⊆ S. APPENDIXIII PROOFS FOR THEMOMENTUM-AUGMENTEDSYSTEM A. Global Convergence Proof:The vector field in (12) is locally Lipschitz under Assumption 1. Hence a unique maximal solution exists. On every finite interval,∥ ˙W(t)∥ F ≤ √ d, so...

    2025

  37. [40]

    His research interests cover distributed op- timization, convex optimization, robust control, federated learning theory, and their practical ap- plications in robotic systems

    He is currently an associate professor in the School of Science at Huzhou Normal Uni- versity. His research interests cover distributed op- timization, convex optimization, robust control, federated learning theory, and their practical ap- plications in robotic systems. Chao Xu(Senior Member, IEEE) received the Ph.D. degree in mechanical engineering from ...

    2010

  38. [267]

    3920–3930

    PMLR, 2025, pp. 3920–3930. [Online]. Available: https: //proceedings.mlr.press/v267/bernstein25a.html

    2025

  39. [2025]

    Available: https://arxiv.org/abs/2502.02900

    [Online]. Available: https://arxiv.org/abs/2502.02900

    Pith/arXiv arXiv

  40. [2026]

    Available: https://arxiv.org/abs/2601.19156

    [Online]. Available: https://arxiv.org/abs/2601.19156

    arXiv