Pith. sign in

REVIEW 2 major objections 6 minor 37 references

A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read A speed-restart rule makes a Hessian-damped inertial system converge linearly in objective value for any convex, smooth function satisfying the Polyak-Lojasiewicz inequality, with no strong convexity assumption.

desk verdict The PL-extension is real and the proof structure is sound, but Proposition 3.8 contains a load-bearing algebraic inversion that breaks the stated theorem as written. read the letter →

arxiv 2412.06691 v1 pith:Y7N7MFVE submitted 2024-12-09 math.OC

classification math.OC MSC 37N4090C2565K1034A12
keywords convexoptimizationinertialmethodsHessian-drivendampingspeedrestartPolyak-Lojasiewiczinequalitylinearconvergencecontinuous-timedynamicalsystemheavyball
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies a continuous-time inertial optimization dynamics with Hessian-driven damping, $x'' + \alpha x' + \beta \nabla^2 f(x) x' + \gamma \nabla f(x) = 0$, under a speed-restart rule: once the squared velocity stops increasing, the trajectory is cut and restarted from its current position with zero velocity. The authors' goal is to prove that this restarted dynamics has linear convergence of objective values, $f(x(t))-f^* \leq C e^{-Kt} (f(z)-f^*)$, for every convex $C^2$ function with Lipschitz gradient that satisfies the Polyak-Lojasiewicz inequality $2\mu(f(z)-f^*) \leq \|\nabla f(z)\|^2$. This matters because the linear rate is obtained without assuming strong convexity, which is the usual hypothesis in the prior speed-restart analyses. The proof gives uniform bounds on the time between restarts and a uniform decrease factor per restart, and the numerical experiments show the rule also improves the observed decay rates for the discrete inertial algorithm they test.

What carries the argument

The argument is carried by three quantitative objects: the restart time $T(z)=\inf\{t>0: \frac{d}{dt}\|x'(t)\|^2 \le 0\}$; an auxiliary function $H(t)$, defined in (3.5) as $H(t)=1+\frac{2L\gamma}{\alpha^2}-\frac{L\beta}{\alpha}-\frac{Lt(\frac{\gamma}{\alpha}e^{\alpha t}+\frac{\gamma}{\alpha}-\beta)}{e^{\alpha t}-1}$, which decreases from 1 to $-\infty$ and whose zeros $\tau_1$, $\tau_2$ mark the time scale on which the frozen-gradient bounds are valid; and a companion function $G(t)$ whose zero $\tau_3$ gives the uniform lower bound on $T(z)$. The proof also uses the supremum $M_z(t)=\sup_{u\in(0,t]} \|x'(u)\|/(1-e^{-\alpha u})$ to bound the integral terms $I_z(t)$ and $J_z(t)$ defined in (3.3). Lemma 3.1 bounds those integrals by $L$ times $M_z(t)$ times explicit functions of $t$; Lemma 3.5 shows $\langle x'(t),x''(t)\rangle$ is at least $\gamma^2(e^{\alpha t}-1)\|\nabla f(z)\|^2/(\alpha e^{2\alpha t} H(t)^2) G(t)$, so while $H,G$ stay positive the speed is still increasing and $T(z)$ has not been reached. Proposition 3.8 uses the PL inequality to bound $T(z)$ from above, and Proposition 3.9 integrates a lower bound on the speed over $[0,\tau]$ to get the per-restart factor $Q$. In short, $H$ and $G$ control the kinematics of the restart, and the PL inequality converts those kinematics into progress on the objective.

What would settle it

Integrate the system for $f(x)=\frac{1}{2}\|x\|^2$, $\alpha=3$, $\beta=0$, $\gamma=1$, from $z=(1,0,\ldots,0)$, and locate the first time $T(z)$ at which $\frac{d}{dt}\|x'(t)\|^2$ changes sign; Theorem 2.2 requires $T(z)$ to lie between the paper's $\tau_*$ and $\tau^*$ and the first restart to shrink the objective gap by at most the factor $Q$ from Proposition 3.9. A violation of either check, or an analogous check on any convex $C^2$, $L$-smooth PL function, would refute the claimed uniform linear convergence.

Watch

Extended reading notes

Core claim

The central discovery is that the speed-restart mechanism, previously known to give linear convergence for strongly convex objectives, works under the strictly weaker Polyak-Lojasiewicz condition. For the system (2.2) with any coefficients $\alpha>0$, $\beta\ge0$, $\gamma>0$, the paper constructs explicit positive constants $\tau_*$ and $\tau^*$ such that every restart interval has length between them, independent of the starting point, and proves that each restart reduces the function-value gap by a uniform factor $Q\in(0,1)$. Iterating the decrease factor and using the uniform time bounds yields the exponential estimate of Theorem 2.2. The technical route rewrites the ODE as an integral equation, controls the gradient and Hessian terms through a supremum of the normalized speed, and then converts a lower bound on $\langle x', x''\rangle$ into the restart-time bounds; the Polyak-Lojasiewicz inequality is what turns a lower speed bound into a quantified gap decrease. A corollary is that the same linear-rate statement covers the Heavy Ball case $\beta=0$, so the Hessian term is not needed for the rate, only for damping oscillations.

Load-bearing premise

The argument rests on the Polyak-Lojasiewicz inequality $2\mu(f(z)-f^*) \le \|\nabla f(z)\|^2$ holding with a known positive $\mu$; if that inequality fails or $\mu$ is unknown, the upper bound on the restart time and the fixed per-restart decrease both disappear, and the linear-rate proof collapses.

Editorial extensions

If this is right

  • For every convex $C^2$, $L$-smooth $f$ satisfying the PL inequality with some $\mu>0$, the speed-restarted trajectory of (WIN) satisfies $f(x(t))-f^* \le C e^{-Kt}(f(z)-f^*)$ with constants $C,K$ independent of the starting point $z$.
  • Because $\beta$ is allowed to be zero, the result applies to the classical Heavy Ball dynamics, giving linear convergence under PL rather than only under strong convexity.
  • The uniform bounds $\tau_* \le T(z) \le \tau^*$ imply the number of restarts up to time $t$ grows linearly in $t$, so the exponential decay rate is governed by the worst-case per-restart decrease factor $Q$.
  • Since strong convexity implies the PL inequality, the theorem covers every strongly convex smooth objective and extends the earlier speed-restart analyses to non-strongly-convex PL objectives.
  • In the numerical experiments, the speed-restart rule improves the fitted exponential decay rate $B$ for both the continuous system (with and without Hessian damping) and for a discretized inertial gradient algorithm; the warm-start variant does not improve the slope.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the PL constant $\mu$ is unknown, the explicit upper bound on the restart time cannot be computed; an adaptive scheme that estimates $\mu$ online while running the dynamics is a natural extension that the paper does not analyze.
  • The continuous-time theorem suggests that any inertial algorithm whose continuum limit is (WIN) should inherit linear convergence under PL when restarted at speed-stall events; the paper tests only one discretization, so whether other standard accelerated-gradient discretizations obey the same discrete rate remains open.
  • A testable prediction of the proof is that conditioning enters the rate only through $L$ and $\mu$, not through the starting point; one could verify this by fitting $C$ and $K$ on a family of quadratic PL functions with increasing condition numbers and comparing them with the theorem's constants.
  • The discrete restart condition used in Algorithm 1, $\|x_{k+1}-x_k\| < \|x_k-x_{k-1}\|$, is a natural proxy for the continuous speed-stall condition; the continuous analysis provides a heuristic justification for it, not a proof for the discrete method.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper analyzes the weighted inertial Newton-like system (2.2) with constant coefficients and proposes a speed-restart scheme in which the dynamics are re-initialized whenever the speed stops increasing. The main claim, Theorem 2.2, is a linear convergence rate for the objective values along the restarted trajectory under convexity, C^2 regularity, L-smoothness, and the Polyak-Łojasiewicz inequality (2.1), with constants independent of the starting point. The proof is organized through estimates on the restart time and on the per-restart function decrease (Lemmas 3.1–3.5, Propositions 3.6–3.9), followed by numerical illustrations.

Significance. If the main theorem is correct, the paper extends speed-restart linear-convergence results, previously known for strongly convex objectives in the setting of Su-Boyd-Candès and for Hessian-driven damping variants, to the wider Polyak-Łojasiewicz class. The paper is self-contained, gives explicit constants, and covers both β=0 and β>0. The numerical experiments support the qualitative benefit of restarting, although they are not the main contribution. The main issues are in the technical proof of the restart-time lower bound and in the uniform definition of the restart-time upper bound.

major comments (2)
  1. [Lemma 3.5 (Section 3.2)] The lower bound for ⟨x',x''⟩ is not valid for all t∈(0,τ1). In the expansion of ⟨x',A(t)⟩, the cross term is γ(2−e^{αt})⟨I_z(t)+J_z(t),∇f(z)⟩/e^{2αt}. When e^{αt}<2 the coefficient 2−e^{αt} is positive, so the correct lower bound is −γ(2−e^{αt})‖I_z(t)+J_z(t)‖‖∇f(z)‖/e^{2αt}, which is negative, whereas the manuscript replaces it by the positive quantity −γ²(e^{αt}−2)(e^{αt}−1)(1−H)/H/(αe^{2αt})‖∇f(z)‖². A direct check with f(x)=x²/2, α=3, β=0, γ=1, z=1, and t=0.1 gives ⟨x'(t),x''(t)⟩≈0.06353, while the right-hand side of Lemma 3.5 evaluates to approximately 0.06366, so the lemma is false as stated. Since Corollary 3.7 and the lower restart-time bound in (2.4) depend on Lemma 3.5, the proof of Theorem 2.2 is incomplete as written.
  2. [Section 2 and Proposition 3.8] The quantity used as an upper restart-time bound in the proof of Theorem 2.2 is not defined independently of the starting point. After Proposition 3.8 the text defines τ* as an argmin over (0,τ2)∩(0,T(z)], a set that depends on z, while the proof of Theorem 2.2 counts restarts using a single τ* valid for all z. The authors should define τ* = argmin over (0,τ2) of φ(τ)=τ+α/(2µγ(1−e^{−ατ})²Ψ(τ)) and then show uniform T(z)≤τ* by the two-case argument: if τ*≤T(z) use Proposition 3.8, and if τ*>T(z) the inequality is trivial. The same issue affects the use of Proposition 3.9: the per-restart factor Q must be chosen uniformly and shown to lie in (0,1), for instance by taking a fixed sufficiently small τ in (0,min(τ2,τ3)).
minor comments (6)
  1. [Proposition 3.8] The displayed bound T(z) ≤ τ + α/(2µγ(1−e^{−ατ})²Ψ(τ)) is ambiguous in the supplied text; the proof shows that (1−e^{−ατ})²Ψ(τ) should be in the denominator. Please ensure the typeset formula is unambiguous.
  2. [Section 2] The lower and upper restart-time bounds are both denoted by symbols that look identical in the plain text (τ∗ and τ∗). Please distinguish τ_* and τ^* clearly.
  3. [Table II] Table II has no caption, and one entry reads “3.39-e2”, which appears to be a typo for 3.39e−2.
  4. [Section 4] In Table III the regression is presented as f(x_k)∼Ae^{−Bt}, but the data are discrete in k; the notation should be f(x_k)∼Ae^{−Bk}.
  5. [Algorithm 1] In the speed-restart branch the pseudocode sets xk = xk−1 when the norm of the step decreases. Please clarify whether this is intentional or whether the intended reset is to the preceding velocity/iterate state.
  6. [Section 2] There is a typo “Propositoin” in the sentence before Theorem 2.2, and the phrase “Polyak-/suppress Lojasiewicz” should read “Polyak–Łojasiewicz”.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the linear-rate theorem is derived from stated assumptions on f, and prior-work citations are contextual, not load-bearing.

full rationale

The derivation of Theorem 2.2 is self-contained from Section 3. The proof requires (i) the lower restart-time bound Corollary 3.7, (ii) the upper restart-time bound Proposition 3.8, and (iii) the per-restart function decrease Proposition 3.9; each is proved from the dynamics (2.2), the Polyak-Lojasiewicz inequality (2.1), and the smoothness assumptions, with no parameter fitted to data and no external result imported. The constants C and K are explicit functions of α, β, γ, μ, and L through these estimates. Citations to the authors' prior work appear only as context, comparisons of numerical behavior, and earlier restart analyses; none is needed to justify a step of the proof. The numerical A and B fits in Section 4 report observed behavior and are not fed back into the theorem, so they do not constitute a fitted input called a prediction. The only substantive concern raised by a skeptical reader, the algebraic inversion in Proposition 3.8's upper-bound expression, would if sustained be a mathematical error affecting the validity of the uniform upper restart-time bound and hence Theorem 2.2; however, an erroneous algebraic manipulation is not a circularity, since it is not an equivalence-by-construction, a fitted-parameter renaming, or a self-citation chain. No step in the paper defines its conclusion into its hypotheses.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claim rests entirely on the stated regularity and PL assumptions and on standard ODE theory; no new physical entities or fitted parameters are introduced. The numerical section fits exponential rates to reported trajectories, but those fits do not enter the theorem.

assumptions (2)
  • domain assumption f is convex, C^2, attains its minimum f*, has L-Lipschitz gradient, and satisfies 2mu(f(z)-f*) <= ||f'(z)||^2 for all z and some mu>0.
    This is the class of functions for which the linear rate is proved. The PL inequality replaces strong convexity and is used in Proposition 3.8 and Proposition 3.9 to bound restart times and per-restart decrease.
  • standard math The second-order ODE (2.2) has a unique global solution for each initial condition (z,0), and the gluing of restarted arcs produces a continuous piecewise differentiable trajectory.
    C^2 convexity with Lipschitz gradient ensures local existence and uniqueness; the paper assumes global continuation and the well-definedness of the restart procedure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients." pith.science (2026). https://pith.science/paper/Y7N7MFVE

@misc{pith2026241206691,
  author       = {Pith},
  title        = {Pith review of: A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7N7MFVE}},
  note         = {Machine review of arXiv:2412.06691}
}
read the original abstract

In this paper, we study a speed restart scheme for an inertial system with Hessian-driven damping. We establish a linear convergence rate for the function values along the restarted trajectories without assuming the strong convexity of the objective function. Our numerical experiments show improvements in the convergence rates, both for the continuous-time dynamics, and when applied to inertial algorithms as a heuristic

Figures

Figures reproduced from arXiv: 2412.06691 by the authors.

Figure 1
Figure 1. Depiction of the function values on the interval [1,5] for [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of function values of (WIN) and (DIN-AVD) for α = 3, β = 6, γ as in (1.2) with ε = 1. will form an acute angle with the gradient at the current point. In both cases, the objective function values decrease along the iterations, and both correspond to 0 < d dt [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Evolution of the objective function values along the traje [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sequence of objective function values for Example [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 35 canonical work pages

  1. [1]

    Alamo, P

    T. Alamo, P. Krupa, and D. Limon. Gradient based restart FISTA . In 58th Conference on Decision and Control (CDC) , pages 3936–3941, 2019

  2. [2]

    Alamo, P

    T. Alamo, P. Krupa, and D. Limon. Restart FISTA with global linear convergence. In 18th European Control Conference (ECC), pages 1969–1974, 2019

  3. [3]

    Alamo, P

    T. Alamo, P. Krupa, and D. Limon. Restart of accelerated first o rder methods with linear convergence under a quadratic functional growth condition. IEEE Transactions on Automatic Control, 68(1), 2023

  4. [4]

    ´Alvarez

    F. ´Alvarez. On the minimizing property of a second-order dissipative sy stem in Hilbert spaces. SIAM Journal on Control and Optimization , 38(4):1102–1119, 2000

  5. [5]

    ´Alvarez, H

    F. ´Alvarez, H. Attouch, J. Bolte, and P. Redont. A second-order gr adient-like dissipative dynamical system with Hessian-driven damping.: Application to optimiz ation and mechanics. Journal de Math´ ematiques Pures et Appliqu´ ees, 81:747–779, 2002

  6. [6]

    ´Alvarez and J

    F. ´Alvarez and J. P´ erez. A dynamical system associated with Newton’s method for parametric approximations of convex minimization problems. Applied Mathematics and Optimization , 38:193–217, 1998

  7. [7]

    Applegate, O

    D. Applegate, O. Hinder, H. Lu, and M. Lubin. Faster first-order primal-dual methods for linear programming using restarts and sharpness. Mathematical Programming, 201(1):133–184, 2023

  8. [8]

    Attouch, A

    H. Attouch, A. Balhag, Z. Chbani, and H. Riahi. Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling. Evolution Equations and Control Theroy, 11(2):487–514, 2022

Show all 37 references
  1. [9]

    Attouch, Z

    H. Attouch, Z. Chbani, J. Fadili, and H. Riahi. First-order optimiza tion algorithms via inertial systems with Hessian driven damping. Mathematical Programming, 193:113–155, 2022

  2. [10]

    Attouch, Z

    H. Attouch, Z. Chbani, J. Peypouquet, and P. Redont. Fast c onvergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Mathematical Programming, 168:123–175, 2018

  3. [11]

    Attouch, Z

    H. Attouch, Z. Chbani, and H. Riahi. Fast convex optimization via time scaling of damped inertial gradient dynamics. Pure and Applied Functional Analysis , 2019

  4. [12]

    Attouch, Z

    H. Attouch, Z. Chbani, and H. Riahi. Fast proximal methods via t ime scaling of damped inertial dynamics. SIAM Journal on Optimization , 29:2227–2256, 2019

  5. [13]

    Attouch and J

    H. Attouch and J. Peypouquet. The rate of convergence of n esterov’s accelerated forward- backward method is actually faster than 1/kˆ2. SIAM Journal on Optimization , 26(3):1824– 1834, 2016. 16

  6. [14]

    Attouch, J

    H. Attouch, J. Peypouquet, and P. Redont. Fast convex opt imization via inertial dynamics with Hessian driven damping. Journal of Differential Equations , 261(10):5734–5783, 2016

  7. [15]

    Aujol, C

    J.-F. Aujol, C. H. Dossal, H. Labarri` ere, and A. Rondepierre. FISTA restart using an automatic estimation of the growth parameter. hal-03153525v4, 2022

  8. [16]

    Drori and M

    Y. Drori and M. Teboulle. Performance of first-order method s for smooth convex minimization: a novel approach. Mathematical Programming, 145:451–482, 2014

  9. [17]

    Fercoq and Z

    O. Fercoq and Z. Qu. Adaptive restart of accelerated gradien t methods under local quadratic growth condition. IMA Journal of Numerical Analysis , 39(4):2069–2095, 2019

  10. [18]

    Giselsson and S

    P. Giselsson and S. Boyd. Monotonicity and restart in fast grad ient methods. In 53rd IEEE Conference on Decision and Control , page 5058–5063, 2014

  11. [19]

    Kim and J

    D. Kim and J. Fessler. Optimized first-order methods for smoot h convex minimization. Math- ematical Programming, 159:81–107, 2016

  12. [20]

    Kim and J

    D. Kim and J. Fessler. On the convergence analysis of the optimiz ed gradient method. Journal of Optimization Theory and Applications , 172:187–205, 2017

  13. [21]

    Li and Z

    H. Li and Z. Lin. Restarted nonconvex accelerated gradient d escent: No more polylogarithmic factor in the in the o (εˆ(-7/4)) complexity. Journal of Machine Learning Research, 24(157):1– 37, 2023

  14. [22]

    Lin and L

    Q. Lin and L. Xiao. An adaptive accelerated proximal gradient me thod and its homotopy continuation for sparse optimization. In Proceedings of the 31st International Conference on Machine Learning, volume 23, pages 73–81, 2014

  15. [23]

    J. J. Maul´ en and J. Peypouquet. A speed restart scheme fo r a dynamics with Hessian-driven damping. Journal of Optimization Theory and Applications , 199:831–855, 2023

  16. [24]

    Necoara, Y

    I. Necoara, Y. Nesterov, and F. Glineur. Linear convergence of first order methods for non- strongly convex optimization. Mathematical Programming, 175(1):69–107, 2019

  17. [25]

    Nemirovskii and Y

    A. Nemirovskii and Y. Nesterov. Optimal methods of smooth co nvex minimization. USSR Computational Mathematics and Mathematical Physics , 25(2):21–30, 1985

  18. [26]

    Nesterov

    Y. Nesterov. A method for solving the convex programming pro blem with convergence rate O(1/k 2). Soviet Mathematics Doklady , 27:372–376, 1983

  19. [27]

    Nesterov

    Y. Nesterov. Gradient methods for minimizing composite functio ns. Mathematical Program- ming, 140(1):125–161, 22–24 Jun 2013

  20. [28]

    Nesterov

    Y. Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013

  21. [29]

    O’Donoghue and E

    B. O’Donoghue and E. Cand` es. Adaptive restart for accelerated gradient schemes. Foundations of Computational Mathematics , 15(3):715–732, 2015. 17

  22. [30]

    C. Park, J. Park, and E. Ryu. Factor- √ 2 acceleration of accelerated gradient methods. Applied Mathematics & Optimization , 88(77), 2023

  23. [31]

    J. Park. Accelerated additive Schwarz methods for convex op timization with adaptive restart. Journal of Scientific Computing , 89(3):58, 2021

  24. [32]

    B. Polyak. Some methods of speeding up the convergence of ite ration methods. USSR compu- tational mathematics and mathematical physics , 4(5):1–17, 1964

  25. [33]

    B. Polyak. Introduction to optimization. Optimization Software, 1987

  26. [34]

    Renegar and B

    J. Renegar and B. Grimmer. A simple nearly optimal restart sche me for speeding up first-order methods. Foundations of computational mathematics , 22(1):211–256, 2022

  27. [35]

    Roulet and A

    V. Roulet and A. d’Aspremont. Sharpness, restart, and acce leration. SIAM Journal on Opti- mization, 30(1):262–289, 2020

  28. [36]

    W. Su, S. Boyd, and E. J. Cand` es. A differential equation for m odeling Nesterov’s accelerat- edgradient method: theory and insights. Journal of Machine Learning Research, 17(153):1–43, 2016

  29. [37]

    B. Wang, T. Nguyen, T. Sun, A. Bertozzi, R. Baraniuk, and S. O sher. Scheduled restart momentum for accelerated stochastic gradient descent. SIAM Journal on Imaging Sciences , 15(2):738–761, 2022. 18

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.