pith. sign in

arxiv: 2606.25311 · v1 · pith:YJYVFO5Enew · submitted 2026-06-24 · 🧮 math.OC

Structured Spectral Step-Sizes and a Hanoi Ordering Principle for Gradient Methods

Yijia Zhou , Ran Gu This is my paper

Pith reviewed 2026-06-25 21:35 UTC · model grok-4.3

classification 🧮 math.OC
keywords spectral gradient methodsstep-size selectionHanoi ordering principlerebound phenomenonR-linear convergencequadratic optimizationill-conditioned problemsadaptive gradient methods
0
0 comments X

The pith

A Hanoi ordering principle for spectral step sizes in gradient methods yields global R-linear convergence on strictly convex quadratics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a unified structural framework that relates four families of spectral step sizes and spells out their inclusion and equivalence relations. It then isolates a rebound phenomenon in which low-frequency targeting steps amplify high-frequency gradient components. Analysis of that phenomenon produces the Hanoi ordering principle, which requires recursive suppression of high-frequency components before each low-frequency step and extends to memory-m settings through phase-wise recomputation. An adaptive method built on this ordering, using weighted component energy and a settlement-continuation rule, is shown to converge globally with R-linear rate on strictly convex quadratics and to outperform standard spectral methods and L-BFGS on ill-conditioned test cases.

Core claim

By characterizing the rebound phenomenon and deriving the Hanoi ordering principle from it, the authors construct an adaptive gradient method that selects and reuses spectral steps according to weighted component energy and a settlement-continuation rule; this method provably attains global R-linear convergence for strictly convex quadratic objectives.

What carries the argument

The Hanoi ordering principle, which requires recursive suppression of high-frequency components prior to each low-frequency step, obtained by analyzing the rebound phenomenon in gradient iterations.

If this is right

  • The adaptive Hanoi-like method converges globally with an R-linear rate for any strictly convex quadratic objective.
  • The ordering principle extends to memory-m gradient methods via a practical phase-wise recomputation strategy.
  • Numerical performance on ill-conditioned problems exceeds that of representative spectral gradient methods and limited-memory BFGS.
  • The structural framework unifies the general Huang class, its rank-one subclass, the narrow Huang class, and the Gu-Du class with explicit inclusion and equivalence relations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the rebound analysis generalizes beyond quadratics, the same ordering could improve step-size robustness for non-quadratic convex problems.
  • Analogous frequency-suppression orderings might be tested in conjugate-gradient or limited-memory quasi-Newton methods to control component amplification.
  • The phase-wise reuse strategy could be examined for reducing per-iteration cost in very large-scale or distributed optimization settings.

Load-bearing premise

The rebound phenomenon analysis holds and the Hanoi ordering principle derived from it correctly suppresses high-frequency components without introducing new instabilities.

What would settle it

A strictly convex quadratic on which the proposed method fails to converge globally with an R-linear rate, or a set of ill-conditioned test problems on which it does not outperform the compared spectral gradient and L-BFGS methods.

read the original abstract

Gradient methods are widely used for large-scale optimization, yet their practical convergence performance depends critically on step-size selection. However, a unified structural understanding of reliable spectral step sizes and a theoretically grounded mechanism for ordering step sequences remain underdeveloped. This paper addresses both gaps. We first establish a unified structural framework encompassing four families of spectral step sizes: the general Huang class, its rank-one subclass, the narrow Huang class, and the Gu-Du class. We rigorously characterize their inclusion relations and equivalence conditions. We then identify and analyze the rebound phenomenon in gradient iterations, whereby steps targeting low-frequency eigenvalues inadvertently amplify high-frequency gradient components. From this analysis, we derive the Hanoi ordering principle, which mandates recursive suppression of high-frequency components prior to each low-frequency step. This principle is further generalized to memory--m settings via a practical phase-wise recomputation strategy. Building on these foundations, we propose an adaptive Hanoi-like gradient method incorporating weighted component energy to select candidates and a settlement-continuation rule to compute the phase length for step reuse. We prove its global R-linear convergence for strictly convex quadratic objectives. Numerical experiments on ill-conditioned test problems demonstrate that the proposed method outperforms representative spectral gradient methods and the limited-memory BFGS method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript develops a unified structural framework for four families of spectral step sizes (general Huang class, rank-one subclass, narrow Huang class, Gu-Du class), characterizes their inclusion relations and equivalence conditions, analyzes the rebound phenomenon in which low-frequency steps amplify high-frequency gradient components, derives the Hanoi ordering principle for recursive high-frequency suppression, generalizes it to memory-m via phase-wise recomputation, proposes an adaptive Hanoi-like gradient method using weighted component energy and a settlement-continuation rule, proves global R-linear convergence on strictly convex quadratics, and reports numerical outperformance versus spectral gradient methods and L-BFGS on ill-conditioned problems.

Significance. If the central derivation and proof hold, the work supplies a principled mechanism for ordering spectral steps that directly addresses a known practical instability, together with a clean unification of existing step-size families. The R-linear convergence result for quadratics would be a concrete theoretical advance for adaptive gradient methods on ill-conditioned problems.

major comments (1)
  1. [Proof of R-linear convergence (abstract and the section containing the rebound-to-Hanoi derivation)] The rebound-phenomenon analysis to Hanoi-ordering derivation is load-bearing for the claimed global R-linear convergence. The manuscript must exhibit the explicit recursive suppression rule (or the phase-wise recomputation and settlement-continuation rule) together with the spectral-radius or contraction-factor bound showing that high-frequency components are suppressed without introducing new growth that would destroy the R-linear rate on the quadratic.
minor comments (2)
  1. Clarify the precise definition of 'weighted component energy' used for candidate selection and the exact formula for the phase length in the settlement-continuation rule; both appear only descriptively in the abstract.
  2. The numerical section should report the precise condition numbers of the test problems and the number of runs or statistical measures supporting the outperformance claim versus BB, other spectral methods, and L-BFGS.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the thorough reading and for highlighting the centrality of the rebound-to-Hanoi derivation to the R-linear convergence claim. We address the single major comment below.

read point-by-point responses

  1. Referee: [Proof of R-linear convergence (abstract and the section containing the rebound-to-Hanoi derivation)] The rebound-phenomenon analysis to Hanoi-ordering derivation is load-bearing for the claimed global R-linear convergence. The manuscript must exhibit the explicit recursive suppression rule (or the phase-wise recomputation and settlement-continuation rule) together with the spectral-radius or contraction-factor bound showing that high-frequency components are suppressed without introducing new growth that would destroy the R-linear rate on the quadratic.

    Authors: The manuscript derives the explicit recursive suppression rule in Section 3 as the core of the Hanoi ordering principle and presents its memory-m generalization via phase-wise recomputation together with the settlement-continuation rule in Section 4. The R-linear convergence proof in Section 5 then constructs the iteration matrix under this ordering and bounds its spectral radius by a factor strictly less than one, with the high-frequency suppression step ensuring that no additional growth terms arise that would violate the linear rate. To address the request for explicit linkage, we will add a short clarifying paragraph immediately following the derivation that restates the suppression rule and directly substitutes it into the contraction-factor argument. revision: partial

Circularity Check

0 steps flagged

No circularity; derivation presented as independent analysis-to-proof chain

full rationale

Abstract and description claim: (1) unified framework for spectral step-size families with inclusion/equivalence characterizations; (2) identification of rebound phenomenon; (3) derivation of Hanoi ordering principle from that analysis; (4) generalization to memory-m; (5) proposal of adaptive method with energy weighting and settlement rule; (6) separate proof of global R-linear convergence on strictly convex quadratics. No equations, no fitted parameters renamed as predictions, no self-citations invoked as load-bearing uniqueness theorems, and no self-definitional loops are visible. The rebound-to-Hanoi step is described as an analysis-derived principle, not a renaming or tautology. Per hard rules, absence of any quotable reduction (Eq. X = input by construction) requires score 0. The mathematical proof claim is treated as independent content unless specific text exhibits circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only abstract available; no free parameters, axioms, or invented entities are specified or can be extracted.

pith-pipeline@v0.9.1-grok · 5741 in / 1033 out tokens · 16843 ms · 2026-06-25T21:35:37.800658+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 20 canonical work pages

  1. [1]

    IMA Journal of Numerical Analysis8(1), 141–148 (1988)

    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA Journal of Numerical Analysis8(1), 141–148 (1988). URLhttp://dx.doi.org/10.1093/imanum/8.1.141

    work page doi:10.1093/imanum/8.1.141 1988

  2. [2]

    Journal of Com- putational Mathematics37(6), 916–936 (2019)

    Burdakov, O., Dai, Y., Huang, N.: Stabilized barzilai-borwein method. Journal of Com- putational Mathematics37(6), 916–936 (2019). URL http://dx.doi.org/10.4208/jcm. 1911-m2019-0171

    work page doi:10.4208/jcm 2019

  3. [3]

    Cauchy, A., et al.: M´ ethode g´ en´ erale pour la r´ esolution des systemes d’´ equations simultan´ ees. Comp. Rend. Sci. Paris25(1847), 536–538 (1847)

  4. [4]

    IMA Journal of Numerical Analysis38(2), 720–742 (2017)

    Curtis, F.E., Guo, W.: R-linear convergence of limited memory steepest descent. IMA Journal of Numerical Analysis38(2), 720–742 (2017). URL http://dx.doi.org/10.1093/ imanum/drx016

    2017

  5. [5]

    Journal of Industrial and Management Optimization1(2), 181–192 (2005)

    Dai, Y., Yuan, Y.X.: Analysis of monotone gradient methods. Journal of Industrial and Management Optimization1(2), 181–192 (2005). URL http://dx.doi.org/10.3934/jimo. 2005.1.181

    work page doi:10.3934/jimo 2005

  6. [6]

    Mathematical Programming103(3), 541–559 (2005)

    Dai, Y.H., Fletcher, R.: On the asymptotic behaviour of some new gradient methods. Mathematical Programming103(3), 541–559 (2005). URL http://dx.doi.org/10.1007/ s10107-004-0516-9

    2005

  7. [7]

    Computational Optimization and Applications74(1), 43–65 (2019)

    Dai, Y.H., Huang, Y., Liu, X.W.: A family of spectral gradient methods for optimization. Computational Optimization and Applications74(1), 43–65 (2019). URL http://dx.doi. org/10.1007/s10589-019-00107-8

    work page doi:10.1007/s10589-019-00107-8 2019

  8. [8]

    IMA Journal of numerical analysis22(1), 1–10 (2002)

    Dai, Y.H., Liao, L.Z.: R-linear convergence of the barzilai and borwein gradient method. IMA Journal of numerical analysis22(1), 1–10 (2002). URL http://dx.doi.org/10.1093/ imanum/22.1.1

    2002

  9. [9]

    IMA Journal of Numerical Analysis23(3), 377–393 (2003)

    Dai, Y.H., Yuan, Y.X.: Alternate minimization gradient method. IMA Journal of Numerical Analysis23(3), 377–393 (2003). URLhttp://dx.doi.org/10.1093/imanum/23.3.377

    work page doi:10.1093/imanum/23.3.377 2003

  10. [10]

    Computational Optimization and Applications 59(3), 541–563 (2014)

    De Asmundis, R., Di Serafino, D., Hager, W.W., Toraldo, G., Zhang, H.: An efficient gradient method using the yuan steplength. Computational Optimization and Applications 59(3), 541–563 (2014). URLhttp://dx.doi.org/10.1007/s10589-014-9669-5

    work page doi:10.1007/s10589-014-9669-5 2014

  11. [11]

    Applied Mathematics and Computation318, 176–195 (2018)

    Di Serafino, D., Ruggiero, V., Toraldo, G., Zanni, L.: On the steplength selection in gradient methods for unconstrained optimization. Applied Mathematics and Computation318, 176–195 (2018). URLhttp://dx.doi.org/10.1016/j.amc.2017.07.037 37

    work page doi:10.1016/j.amc.2017.07.037 2018

  12. [12]

    Mathematical programming91(2), 201–213 (2002)

    Dolan, E.D., Mor´ e, J.J.: Benchmarking optimization software with performance profiles. Mathematical programming91(2), 201–213 (2002). URL http://dx.doi.org/10.1007/ s101070100263

    2002

  13. [13]

    Mathematical Programming135, 413–436 (2012)

    Fletcher, R.: A limited memory steepest descent method. Mathematical Programming135, 413–436 (2012). URLhttp://dx.doi.org/10.1007/s10107-011-0479-6

    work page doi:10.1007/s10107-011-0479-6 2012

  14. [14]

    Journal of Industrial and Management Optimization4(2), 299–312 (2008)

    Frassoldati, G., Zanni, L., Zanghirati, G.: New adaptive stepsize selections in gradient methods. Journal of Industrial and Management Optimization4(2), 299–312 (2008). URL http://dx.doi.org/10.3934/jimo.2008.4.299

    work page doi:10.3934/jimo.2008.4.299 2008

  15. [15]

    SIAM Journal on Numerical Analysis36(1), 275–289 (1998)

    Friedlander, A., Mart´ ınez, J.M., Molina, B., Raydan, M.: Gradient method with retards and generalizations. SIAM Journal on Numerical Analysis36(1), 275–289 (1998). URL http://dx.doi.org/10.1137/s003614299427315x

    work page doi:10.1137/s003614299427315x 1998

  16. [16]

    Computational Optimization and Applications63(2), 523–542 (2015)

    Gonzaga, C.C., Schneider, R.M.: On the steepest descent algorithm for quadratic functions. Computational Optimization and Applications63(2), 523–542 (2015). URL http://dx. doi.org/10.1007/s10589-015-9775-z

    work page doi:10.1007/s10589-015-9775-z 2015

  17. [17]

    IMA Journal of Numerical Analysis41(1), 247–270 (2021)

    Gu, R., Du, Q.: A modified limited memory steepest descent method motivated by an inexact super-linear convergence rate analysis. IMA Journal of Numerical Analysis41(1), 247–270 (2021). URLhttp://dx.doi.org/10.1093/imanum/drz059

    work page doi:10.1093/imanum/drz059 2021

  18. [18]

    Computational Optimization and Applications81(3), 717–740 (2022)

    Huang, Y., Dai, Y.H., Liu, X.W., Zhang, H.: On the acceleration of the barzilai–borwein method. Computational Optimization and Applications81(3), 717–740 (2022). URL http://dx.doi.org/10.1007/s10589-022-00349-z

    work page doi:10.1007/s10589-022-00349-z 2022

  19. [19]

    Journal of Scientific Computing90(1), 7 (2022)

    Huang, Y., Dai, Y.H., Liu, X.W., Zhang, H.: On the asymptotic convergence and acceleration of gradient methods. Journal of Scientific Computing90(1), 7 (2022). URL http://dx. doi.org/10.1007/s10915-021-01685-8

    work page doi:10.1007/s10915-021-01685-8 2022

  20. [20]

    SIAM Journal on Optimization31(4), 3068–3096 (2021)

    Huang, Y.K., Dai, Y.H., Liu, X.W.: Equipping the barzilai–borwein method with the two dimensional quadratic termination property. SIAM Journal on Optimization31(4), 3068–3096 (2021). URLhttp://dx.doi.org/10.1137/21m1390785

    work page doi:10.1137/21m1390785 2021

  21. [21]

    Mathematics of the USSR-Sbornik1(4), 457–483 (1967)

    Marˇ cenko, V.A., Pastur, L.A.: Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik1(4), 457–483 (1967). URL http://dx.doi.org/10. 1070/sm1967v001n04abeh001994

    1967

  22. [22]

    SIAM (1998)

    Parlett, B.N.: The symmetric eigenvalue problem. SIAM (1998). URL http://dx.doi. org/10.1137/1.9781611971163

    work page doi:10.1137/1.9781611971163 1998

  23. [23]

    Optimization Letters14(7), 1943–1955 (2020)

    Sun, C., Liu, J.P.: New stepsizes for the gradient method. Optimization Letters14(7), 1943–1955 (2020). URLhttp://dx.doi.org/10.1007/s11590-019-01512-y

    work page doi:10.1007/s11590-019-01512-y 1943

  24. [24]

    arXiv preprint arXiv:2602.13141 (2026)

    Xie, Y., Liu, J.P., Sun, C., Yuan, Y.X.: New gradient methods with 3 dimensional quadratic termination. arXiv preprint arXiv:2602.13141 (2026). URL https://arxiv.org/abs/2602. 13141

    arXiv 2026

  25. [25]

    Computational Optimization and Applications92(1), 301–325 (2025)

    Xie, Y., Sun, C., Yuan, Y.X.: Adaptive cyclic gradient methods with interpolation. Computational Optimization and Applications92(1), 301–325 (2025). URL http: //dx.doi.org/10.1007/s10589-025-00691-y

    work page doi:10.1007/s10589-025-00691-y 2025

  26. [26]

    Journal of Computational Mathematics pp

    Yuan, Y.x.: A new stepsize for the steepest descent method. Journal of Computational Mathematics pp. 149–156 (2006) 38

    2006

  27. [27]

    Journal of the Operations Research Society of China12(3), 809–828 (2024)

    Zhang, Y., Sun, C.: Cyclic gradient methods for unconstrained optimization. Journal of the Operations Research Society of China12(3), 809–828 (2024). URL http://dx.doi. org/10.1007/s40305-022-00432-6

    work page doi:10.1007/s40305-022-00432-6 2024

  28. [28]

    Structured Spectral Step-Sizes and a Hanoi Ordering Principle for Gradient Methods

    Zhigljavsky, A., Pronzato, L., Bukina, E.: An asymptotically optimal gradient algorithm for quadratic optimization with low computational cost. Optimization Letters7(6), 1047–1059 (2013). URLhttp://dx.doi.org/10.1007/s11590-012-0491-7 39 Supplementary Information for “Structured Spectral Step-Sizes and a Hanoi Ordering Principle for Gradient Methods” S1 P...

    work page doi:10.1007/s11590-012-0491-7 2013