Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One Lyapunov inequality unifies momentum method convergence

desk verdict A useful IGDm unification with genuinely new local results, but the uniform PLK lemma has a constant-factor gap and the Nesterov/FISTA coverage is overclaimed; both are fixable. read the letter →

arxiv 2505.03050 v1 pith:DICK4HXK submitted 2025-05-05 math.OC

Pham Duy Khanh , Boris Mordukhovich , Dat Ba Tran This is my paper · ORCID
classification math.OC MSC 90C5290C5649J5390C2590C26
keywords inexactgradientdescentmomentumPLKconditionLyapunovfunctionextragradientmethodsharpness-awareminimizationproximalpointconvergencerates
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single algorithm—an inexact gradient descent method with momentum (IGDm)—is a common convergence engine for several widely used first-order methods with momentum: the extragradient method, sharpness-aware minimization, and the inexact proximal point method. The argument rests on one Lyapunov descent inequality that holds under a uniform momentum bound and a coupled step-size/inexactness condition. From that inequality, stationarity of accumulation points, full sequence convergence under the basic Polyak–Łojasiewicz–Kurdyka (PLK) condition, and constructive linear or sublinear convergence rates under the exponential PLK condition all follow. If the framework is correct, it converts separate convergence analyses for these momentum methods into corollaries of a single theorem, including new local convergence results for prox-regular functions.

What carries the argument

The central object is the Lyapunov function $H_\alpha(x,y)=f(x)+\alpha\|x-y\|^2$, first used in heavy-ball analysis, together with the relative inexact-gradient condition (3.3) and the coupled parameter condition (3.4): $\max\{L\tau, 2L\tau\bar\delta+(L\tau+1)\bar\beta^2\}<1-\nu$, where $\bar\beta=\sup_k\beta_k$ and $\bar\delta=\sup_k|\beta_k-\gamma_k|$. These three ingredients make the descent constant $C_1$ in (3.5) positive, yielding the two inequalities (3.6) and (3.7) used throughout. The PLK condition (Definition 2.2) is the regularity hypothesis that turns the descent inequality into full convergence and explicit rates; Proposition 2.9 shows that the PLK property transfers from $f$ to the whole family of Lyapunov functions $\{H_\alpha\}_{\alpha\ge\varepsilon}$ uniformly, which is what makes the local analysis work.

What would settle it

Run Algorithm 1 on a strongly convex quadratic with constant momentum $\beta=0.9$, $\gamma=0.9$, $\nu=0.1$, and step size $\tau$ satisfying (3.4); Theorem 3.5(iii) predicts linear convergence of $x_k$ to the minimizer and of $f(x_k)$ to the minimum value. If the observed convergence is sublinear, the rate claim is false. More directly, a single function in $C_L^{1,1}$ with $\inf_k f(x_k)>-\infty$, parameters satisfying (3.4), and an accumulation point that is not stationary would refute Theorem 3.5(i).

Watch

Extended reading notes

Core claim

The paper introduces Algorithm 1, IGDm, whose update reads $x_{k+1}=x_k+\beta_k(x_k-x_{k-1})-\tau g_k$, with the inertial point $x_k+\beta_k(x_k-x_{k-1})$ used as the next iterate and the extrapolated point $x_k+\gamma_k(x_k-x_{k-1})$ used for gradient evaluation; the gradient estimate $g_k$ is only required to satisfy the relative error bound $\|g_k-\nabla f(x_k+\gamma_k(x_k-x_{k-1}))\|\le\nu\|g_k\|$. Its central discovery is Proposition 3.2: under the parameter condition $\max\{L\tau, 2L\tau\bar\delta+(L\tau+1)\bar\beta^2\}<1-\nu$, the Lyapunov function $H_\alpha(x,y)=f(x)+\alpha\|x-y\|^2$ decreases along the lifted sequence $z_k=(x_k,x_{k-1})$ according to $C_1\|z_{k+1}-z_k\|^2\le H_\alpha(z_k)-H_\alpha(z_{k+1})$, and $\nabla H_\alpha(z_k)$ is bounded by a constant multiple of $\|z_{k+1}-z_k\|$. Feeding these two inequalities into known PLK theory yields Theorem 3.5 (global: $\nabla f(x_k)\to 0$, full convergence under the basic PLK condition, linear rates for PLK exponent $q\le 1/2$ and $O(k^{-(1-q)/(2q-1)})$ rates for $q\in(1/2,1)$) and Theorem 3.6 (the same conclusions locally around a local minimizer). The paper then shows that the extragradient method and sharpness-aware minimization become instances of IGDm when their inner step $\tau_2\le\nu/(L(\nu+1))$, and that the inexact proximal point method reduces to IGDm applied to the Moreau envelope $e_{\lambda h}$, so all three inherit the global and local convergence theorems.

Load-bearing premise

The load-bearing premise is that the momentum coefficients satisfy $\bar\beta<1$ uniformly and the coupled step-size/inexactness condition (3.4) holds, so that $C_1>0$ in the Lyapunov descent inequality; standard Nesterov acceleration and FISTA schedules, whose $\beta_k$ tend to 1, are outside this theory, as are the paper's own experiments with $\beta_k=k/(k+3)$.

Editorial extensions

If this is right

  • The extragradient method with momentum (EGm) converges globally for general $C_L^{1,1}$ functions and locally around local minimizers when $f$ is locally smooth and satisfies PLK (Theorems 4.1 and 4.2).
  • Sharpness-aware minimization with momentum (SAMm) has analogous global and local convergence guarantees (Theorems 4.3 and 4.4).
  • The inexact proximal point method with momentum (IPPm) converges globally for weakly convex functions and locally for prox-regular semialgebraic functions, with explicit rates for the residual $\|x_k-\mathrm{Prox}_{\lambda h}(x_k)\|$ (Theorems 4.11 and 4.12).
  • The convergence rates are constructive: linear for PLK exponent $q\le 1/2$ and $O(k^{-(1-q)/(2q-1)})$ for $q\in(1/2,1)$, applying to both the iterates and the gradients.
  • Derivative-free implementations using forward or central finite differences inherit the convergence guarantees, and the paper's experiments show momentum speeds up these methods by about a factor of two.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because condition (3.4) forces $\bar\beta<1$, the framework does not cover Nesterov's standard acceleration schedule $\beta_k=k/(k+3)$ or FISTA's $\beta_k\to 1$, despite those being the paper's own test settings; a time-varying Lyapunov function or a weighted norm would be needed to close that gap.
  • The reduction of IPPm to IGDm on the Moreau envelope suggests a recipe: any smooth-world momentum result for IGDm automatically becomes a nonsmooth prox-regular result, so future refinements of the Lyapunov argument would immediately upgrade proximal-type algorithms.
  • The relative error condition $\|g_k-\nabla f(x_k^{\mathrm{ex}})\le\nu\|g_k\|$ is satisfied by finite-difference and Gupal-type estimators, so the framework is a candidate for stochastic or zeroth-order momentum methods whenever the estimator satisfies the same relative bound with high probability.
  • Keeping $\beta_k$ and $\gamma_k$ close matters: the constant $C_1$ degrades with $\bar\delta=\sup_k|\beta_k-\gamma_k|$, so the theory suggests a practical design rule that the inertial and extrapolation momentum coefficients should track each other even if both are allowed to grow.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes an inexact gradient descent method with momentum (IGDm) and analyzes it through the Lyapunov function H_alpha(x,y)=f(x)+alpha||x-y||^2. It proves a descent inequality and a gradient bound for the iterates, then derives global convergence and Polyak-Lojasiewicz-Kurdyka (PLK) convergence rates (Theorem 3.5) and local convergence results (Theorem 3.6). The framework is then specialized to an extragradient method with momentum (EGm), sharpness-aware minimization with momentum (SAMm), and an inexact proximal point method with momentum (IPPm), with global and local convergence theorems in Section 4. Section 5 reports derivative-free numerical experiments for IGDm variants with finite-difference gradients.

Significance. The framework is broad and the reductions of EGm, SAMm, and IPPm to IGDm are conceptually appealing. The paper's global analysis for uniformly bounded momentum coefficients is detailed and largely checkable, and the numerical study for derivative-free problems is relevant. If the technical issues are repaired, the local convergence results for momentum versions of these methods, especially for prox-regular functions, would be a useful contribution. The paper does not introduce fitted parameters, and the main claims are not defined into existence. However, the advertised coverage of Nesterov and FISTA schedules is currently outside the proved regime, and the local convergence rates rest on a faulty uniform PLK proof and on external unrefereed results.

major comments (3)
  1. [Section 2, Proposition 2.9] The proof of Proposition 2.9 contains a constant-factor error that invalidates the displayed derivation of the uniform PLK inequality (2.15). Multiplying (2.13), namely phi'(t)=sqrt(2/epsilon) phi_0'(t/2), by the lower bound (2.16d), namely ||nabla H_alpha|| >= sqrt(epsilon)/2 (||nabla f(x)|| + sqrt(alpha)||x-y||), gives (1/sqrt(2)) phi_0'(S)(||nabla f(x)|| + sqrt(alpha)||x-y||), not phi_0'(S)(||nabla f(x)|| + sqrt(alpha)||x-y||) as written. Consequently the proof only establishes the lower bound 1/sqrt(2), so inequality (2.15) is not proved. Since Theorem 3.6 uses Proposition 2.9 to obtain the uniform PLK inequality (3.21), and Theorems 4.2, 4.4, and 4.12 inherit this step, the local convergence claims are currently unsupported. The defect appears repairable by enlarging the constant in (2.8) from sqrt(8/epsilon) to sqrt(16/epsilon) and adjusting the constants in (3.23), but as written the proof fails.
  2. [Section 3.1, Remark 3.1 and Section 5] Algorithm 1 imposes the standing condition \bar{beta} = sup_k beta_k < 1, and Proposition 3.2 and Theorems 3.5 and 3.6 rely on it. However, Remark 3.1 advertises Nesterov's acceleration and FISTA schedules, for which beta_k tends to 1, and the experiments in Section 5 use beta_k = k/(k+3), also with supremum 1. These settings violate the paper's own theoretical conditions, so the claims that IGDm encompasses those schedules and that the numerical results confirm the theory are not supported by the proved statements. The authors should either extend the analysis to the case \bar{beta}=1 or explicitly reposition the Nesterov/FISTA discussion and the experiments as heuristic or outside the proved regime.
  3. [Section 3.2, Proposition 2.6 and Proposition 2.10] The constructive convergence rates in Theorems 3.5 and 3.6 are not proved in the manuscript but are imported from [6, Theorem 4] and [17, Proposition B.5]. Proposition 2.6 is taken from [6], an optimization-online preprint by the same research group, and Proposition 2.10 is taken from the authors' own NeurIPS paper. These external results carry the rate conclusions advertised in the abstract, so the paper should either supply full proofs or state precisely which portions of the rates are new and self-contained. As it stands, the rate claims are conditional on the correctness and accessibility of those sources.
minor comments (3)
  1. [Abstract] There is a typo in the abstract: 'convergence properies' should be 'convergence properties'.
  2. [Section 5] The list of derivative-free methods contains 'DF-fordif' twice; the second item should presumably be 'DFn-fordif' for Nesterov momentum with forward differences. In the same paragraph, the 'noiseless setting' is described as having additive noise and should be called the 'noisy setting'.
  3. [Theorem 4.12] The displayed condition on \bar{beta} in Theorem 4.12 appears as \bar{beta} in [0,1-nu), while Theorem 3.6 and its proof require \bar{beta} in [0,sqrt(1-nu)). If the weaker condition is intended, this is a harmless strengthening, but the formulas should be harmonized to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the IGDm Lyapunov descent is proved from the algorithm's own iterate estimates, and the PLK-based rates are applied as prior theorems rather than being defined into existence.

full rationale

The central claim is the Lyapunov estimate (3.6)-(3.7) of Proposition 3.2, and this is derived directly from Algorithm 1: the proof combines the Lipschitz smoothness inequalities, the relative inexact condition (3.3), and the parameter condition (3.4); the constants alpha, C1, C2 in (3.5) are chosen from algorithm parameters, not from the desired limit. No parameter is fitted to data and later renamed a prediction. The PLK arguments are applications, not assumptions packed into the algorithm: Theorem 3.5 verifies the two hypotheses (2.6) for the sequence z_k and the smooth function H_alpha, then invokes Proposition 2.6 (from [6]) and Proposition 2.8 (from [47]); Theorem 3.6 uses Proposition 2.9's uniform desingularizer to verify the PLK inequality (3.21). The reductions in Section 4 are genuine inclusions: Algorithms 2 and 3 are shown to satisfy the relative inexact condition (4.4)-(4.5), and Algorithm 4 is shown via (4.12) to reduce to Algorithm 1 on the Moreau envelope. The paper does cite [6] and [17], which share authors with this paper, and those citations are load-bearing for rate transfer; however, they are cited as parameter-free theorems whose assumptions do not include the convergence claims being proved, so under the rules of this review they are independent support rather than circularity. A separate non-circular correctness concern: the proof of Proposition 2.9 appears to omit a factor 1/sqrt(2) when multiplying (2.13) and (2.16d), which would affect the displayed verification of (2.15); this is an algebraic proof error, not a circular definition or fitted prediction.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The proofs introduce no fitted parameters; the algorithmic quantities tau, nu, beta_k, and gamma_k are constrained by inequalities, not estimated from data. The main hypotheses are the PLK regularity of f, the relative inexactness (3.3), the uniform momentum bound, and, for IPPm, weak convexity or prox-regularity plus semialgebraicity. Several load-bearing rate results are imported from the authors' own recent preprint [6] and other prior work.

assumptions (6)
  • domain assumption The objective f satisfies the basic PLK condition at relevant accumulation points or local minimizers (Definition 2.2).
    Without PLK, only stationarity of accumulation points is obtained in Theorem 3.5(i); all convergence conclusions and rates in Theorems 3.5, 3.6, 4.1, 4.2, 4.3, 4.4, 4.11, and 4.12 rely on it.
  • domain assumption The relative inexact gradient condition ||g_k - grad f(x_ex^k)|| <= nu ||g_k|| with nu in (0,1), plus nonstationarity Assumption 1 (grad f(x_ex^k) != 0).
    This is the algorithmic error model in (3.3); the proof uses it to derive (3.10), to show z_{k+1} != z_k, and to transfer gradient convergence. It is not derived from first principles.
  • domain assumption Uniform momentum bound beta_bar = sup_k beta_k < 1 and the coupled parameter condition (3.4): max{L tau, 2 L tau delta_bar + (L tau + 1) beta_bar^2} < 1 - nu.
    This guarantees C1 > 0 and the Lyapunov descent estimate (3.6). It also excludes the Nesterov and FISTA schedules with beta_k tending to 1 that the paper claims to cover in Remark 3.1.
  • standard math The external rate and PLK-inheritance results used as black boxes: Proposition 2.6 from [6], Proposition 2.8 from [47], and Proposition 2.10 from [17].
    These cited results provide the rate conclusions and the Lyapunov function PLK inheritance. They are not proved in this paper, and [6] is an unpublished preprint with overlapping authorship.
  • standard math For the IPPm results, the Moreau envelope properties in Propositions 4.7, 4.8, and 4.9, and the weak convexity or prox-regularity of h.
    These allow Algorithm 4 to be reduced to Algorithm 1 on the Moreau envelope e_lambda h with a known Lipschitz constant. They are quoted from [8,43,16].
  • domain assumption In Theorem 4.12, h is prox-bounded, prox-regular at a local minimizer, and semialgebraic.
    Semialgebraicity is used to ensure the Moreau envelope e_lambda h satisfies the PLK condition, which is the entry point for the local IGDm analysis. This restriction defines the function class for the local proximal point result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method." pith.science (2026). https://pith.science/paper/DICK4HXK

@misc{pith2026250503050,
  author       = {Pith},
  title        = {Pith review of: Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DICK4HXK}},
  note         = {Machine review of arXiv:2505.03050}
}
read the original abstract

This paper introduces a novel inexact gradient descent method with momentum (IGDm) considered as a general framework for various first-order methods with momentum. This includes, in particular, the inexact proximal point method (IPPm), extragradient method (EGm), and sharpness-aware minimization (SAMm). Asymptotic convergence properies of IGDm are established under both global and local assumptions on objective functions with providing constructive convergence rates depending on the Polyak-\L ojasiewicz-Kurdyka (PLK) conditions for the objective function. Global convergence of EGm and SAMm for general smooth functions and of IPPM for weakly convex functions is derived in this way. Moreover, local convergence properties of EGm and SAMm for locally smooth functions as well as of IPPm for prox-regular functions are established. Numerical experiments for derivative-free optimization problems are conducted to confirm the efficiency of the momentum effects of the developed methods under inexactness of gradient computations

Figures

Figures reproduced from arXiv: 2505.03050 by the authors.

Figure 1
Figure 1. Inexact Gradient Descent Algorithm with Momentum [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Derivative-free least-square convex and nonconvex problems [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Worst-case convergence analysis of relatively inexact gradient descent on smooth convex functions

    math.OC 2025-06 conditional novelty 6.0 of 10

    For smooth convex functions, relatively inexact gradient descent has a three-regime worst-case one-step rate, and the largest provably safe stepsize is 2/(1+delta).

Reference graph

Works this paper leans on

49 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [6]

    Bento, B

    G. Bento, B. S. Mordukhovich, T. Mota, and Yu. Nesterov. Convergence of descent optimization al- gorithms under Polyak-Łojasiewicz-Kurdyka conditions.https://optimization-online.org/?p=29340, 2025

    work page 2025

  2. [1]

    Absil, R

    P.-A. Absil, R. Mahony, and B. Andrews. Convergence of the iterates of descent methods for analytic cost functions. SIAM J. Optim., 16:531–547, 2005

    work page 2005

  3. [2]

    Andriuschenko and N

    M. Andriuschenko and N. Flammarion. Towards understanding sharpness-aware minimization. ICML, pages 639–668, 2022

    work page 2022

  4. [3]

    Attouch, J

    H. Attouch, J. Bolté, P. Redont, and A. Soubeyran. Proximal alternating minimization and pro- jection methods for nonconvex problems. an approach based on the Kurdyka-Łojasiewicz property. Math. Oper. Res., 35:438–457, 2010

    work page 2010

  5. [4]

    Attouch, J

    H. Attouch, J. Bolté, and B. F. Svaiter. Convergence of descent methods for definable and tame problems: Proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program., 137:91–129, 2013

    work page 2013

  6. [5]

    Beck and M

    A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse prob- lems. SIAM J. Imaging Sci., 2:183–202, 2009

    work page 2009

  7. [7]

    A. Cauchy. Method général pour la résolution des systemes d’équations simulanées.Comptes Rendus, 25:536, 1847

    work page

  8. [8]

    Davis and D

    D. Davis and D. Drusviatskiy. Proximal methods avoid active strict saddles of weakly convex func- tions. Found. Comput. Math., 22:561–606, 2022

    work page 2022

Show all 49 references
  1. [9]

    Foret, A

    P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur. Sharpness-aware minimization for efficiently improving generalization. ICLR, 2021

    2021

  2. [10]

    Gilmore and C

    P. Gilmore and C. T. Kelley. An implicit filtering algorithm for optimization of functions with many local minima. SIAM J. Optim., 5:269–285, 1995

    1995

  3. [11]

    O. Guller. New proximal point algorithms for convex minimization.SIAM J. Optim., 2:649–664, 1992

    1992

  4. [12]

    A. D. Ioffe. An invitation to tame optimization.SIAM J. Optim., 19:1894–1917, 2009

    1917

  5. [13]

    A. F. Izmailov and M. V. Solodov.Newton-Type Methods for Optimization and Variational Problems. Springer, New York, 2014

    2014

  6. [14]

    C. Josz. Global convergence of the gradient method for functions definable in o-minimal structures. Math. Program., 2023

    2023

  7. [15]

    Linearconvergenceofgradientandproximal-gradientmethods under the polyak-łojasiewicz condition.Machine Learning and Knowledge Discovery in Databases, Part 1, pages 795–811, 2016

    H.Karimi, J.Nutini, andM.Schmidt. Linearconvergenceofgradientandproximal-gradientmethods under the polyak-łojasiewicz condition.Machine Learning and Knowledge Discovery in Databases, Part 1, pages 795–811, 2016

    2016

  8. [16]

    P. D. Khanh, V. V. H. Khoa, B. S. Mordukhovich, and V. T. Phat. Second-order subdifferential optimality conditions in nonsmooth optimization.SIAM J. Optim., 35(2), 2025

    2025

  9. [17]

    P. D. Khanh, H.-C. Luong, B. S. Mordukhovich, and D. B. Tran. Fundamental convergence analysis of sharpness-aware minimization.Proceedings of NeurIPS 2024, 2024

    2024

  10. [18]

    P. D. Khanh, B. S. Mordukhovich, and V. T. Phat. Variational convexity of functions and variational sufficiency in optimization.SIAM J. Optim, 33:1211–1158, 2023

    2023

  11. [19]

    P. D. Khanh, B. S. Mordukhovich, V. T. Phat, and D. B. Tran. Inexact proximal methods for weakly convex functions.J. Global Optim. DOI: 10.1007/s10898-024-01460-7, 2023

    2023 doi

  12. [20]

    P. D. Khanh, B. S. Mordukhovich, and D. B. Tran. Globally convergent derivative-free methods in nonconvex optimization with and without noise.https://optimization-online.org/?p=26889, 2024. 24

    2024

  13. [21]

    P. D. Khanh, B. S. Mordukhovich, and D. B. Tran. Inexact reduced gradient methods in smooth nonconvex optimization.J. Optim. Theory Appl., 203:2138–2178, 2024

    2024

  14. [22]

    P. D. Khanh, B. S. Mordukhovich, and D. B. Tran. A new inexact gradient de- scent method with applications to nonsmooth convex optimization. Optim. Methods Softw. DOI:10.1080/10556788.2024.2322700, 2024

    2024

  15. [23]

    D. Kim. Accelerated proximal point method for maximally monotone operators.Math. Program., 190:57–87, 2021

    2021

  16. [24]

    G. M. Korpelevich. An extragradient method for finding saddle points and for other problems.Ekon. Mat. Metod., 12:747–756, 1976

    1976

  17. [25]

    J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright. Convergence properties of the nelder– mead simplex method in low dimensions.SIAM J. Optim., 9:112–147, 1998

    1998

  18. [26]

    S. C. Lásló. Convergence rates for an inertial algorithm of gradient type associated to a smooth non-convex minimization.Math. Program., 190:285–329, 2021

    2021

  19. [27]

    Li and T

    G. Li and T. K. Pong. Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applica- tions to linear convergence of first-order methods.Found. Comput. Math., 18:1199–1232, 2018

    2018

  20. [28]

    T. Lin, L. Kong, S. U. Stich, and M. Jaggi. Extrapolation for large-batch training in deep learning. ICML, 2020

    2020

  21. [29]

    B. S. Mordukhovich. Variational Analysis and Generalized Differentiation, I: Basic Theory, II: Applications. Springer, Berlin, 2006

    2006

  22. [30]

    B. S. Mordukhovich.Variational Analysis and Applications. Springer, Cham, Switzerland, 2018

    2018

  23. [31]

    Nesterov

    Yu. Nesterov. A method of solving a convex programming problem with convergence rateO(1/k2). Soviet Math. Dokl, 1983

    1983

  24. [32]

    Nesterov

    Yu. Nesterov. Gradient methods for minimizing composite functions.Math. Program., 140:125–161, 2013

    2013

  25. [33]

    Nesterov.Lectures on Convex Optimization

    Yu. Nesterov.Lectures on Convex Optimization. Springer, Cham, Switzerland, 2018

    2018

  26. [34]

    Nesterov and V

    Yu. Nesterov and V. Spokoiny. Random gradient-free minimization of convex functions.Found. Comput. Math., 17:527–566, 2017

    2017

  27. [35]

    T. P. Nguyen, E. Pauwels, E. Richard, and B. W. Sutter. Extragradient method in optimization: convergence and complexity.J. Optim. Theory Appl., 176:137–162, 2018

    2018

  28. [36]

    B. T. Polyak. Some methods of speeding up the convergence of iteration methods.USSR Comput. Math. Math. Phys., 4:1–17, 1964

    1964

  29. [37]

    B. T. Polyak.Introduction to Optimization. Optimization Software, New York, 1987

    1987

  30. [38]

    L. M. Rios and N. V. Sahinidis. Derivative-free optimization: a review of algorithms and comparison of software implementations.J. Global Optim., 56:1247–1293, 2013

    2013

  31. [39]

    R. T. Rockafellar. Augmented Lagrangians and applications of the proximal point algorithm in convex programming.Math. Oper. Res., 1:97–116, 1976

    1976

  32. [40]

    R. T. Rockafellar. Monotone operators and the proximal point algorithm.SIAM J. Control. Optim., 14:877–898, 1976

    1976

  33. [41]

    R. T. Rockafellar. Advances in convergence and scope of the proximal point algorithm.J. Nonlinear Convex Anal., 22:2347–2375, 2021

    2021

  34. [42]

    R. T. Rockafellar. Convergence of augmented Lagrangian methods in extensions beyond nonlinear programming. Math. Program., 199:375–420, 2023

    2023

  35. [43]

    R. T. Rockafellar and R. J.-B. Wets.Variational Analysis. Springer, Berlin, 1998

    1998

  36. [44]

    B. Shi, S. S. Du, M. I. Jordan, and W. J. Su. Understanding the acceleration phenomenon via high-resolution differential equations.Math. Program., 195:79–148, 2022. 25

    2022

  37. [45]

    H. M. Shi, M. Q. Xuan, F. Oztoprak, and J. Nocedal. On the numerical performance of finite- difference-based methods for derivative-free optimization.Optim. Methods Softw., 38:289–311, 2023

    2023

  38. [46]

    Themelis, L

    A. Themelis, L. Stella, and P. Patrinos. Forward–backward quasi-Newton methods for nonsmooth optimization problems. Comput. Optim. Appl., 67:443–487, 2017

    2017

  39. [47]

    Wang and Z

    X. Wang and Z. Wang. Calculus rules of the generalized concave Kurdyka–Łojasiewicz property.J. Optim. Theory Appl., 2023

    2023

  40. [48]

    B. Wen, X. Chen, and T. K. Pong. Linear convergence of proximal gradient algorithm with extrap- olation for a class of nonconvex nonsmooth minimization problems.SIAM J. Optim., 27:124–145, 2017

    2017

  41. [49]

    Zavriev and F

    S. Zavriev and F. Kostyuk. Heavy ball method in nonconvex optimization problems.Comput. Math. Model., 4:336–341, 1993. 26

    1993

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.