Pith. sign in

REVIEW 2 major objections 3 minor 2 cited by

Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization

T0 review · 2 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Accelerated GRAAL achieves near-optimal accelerated iteration complexity for convex L-smooth and (L0,L1)-smooth functions without line search or hyperparameter tuning, by letting the stepsize grow geometrically with local curvature…

desk verdict A genuinely new adaptive acceleration mechanism with a load-bearing but fixable overclaim about no tuning. read the letter →

arxiv 2507.09823 v2 pith:BT3Q3BY7 submitted 2025-07-13 math.OC cs.LG

Ekaterina Borodich , Dmitry Kovalev This is my paper
classification math.OCcs.LG MSC 90C2590C6065K05
keywords adaptivegradientmethodNesterovaccelerationconvexoptimizationlocalcurvatureestimationGRAAL(L0L1)-smoothnessiterationcomplexitystepsize
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to settle a question left open by prior adaptive methods: can a gradient method that chooses its stepsize from local curvature estimates, with no line search and no tuned parameters, also be accelerated to the optimal rate of accelerated gradient descent? It proposes Accelerated GRAAL, whose stepsize rule permits geometric growth, unlike earlier accelerated adaptive methods whose stepsize growth is restricted to a sublinear rate. The paper proves that for convex $L$-smooth functions with $\eta_0 L \leq 1$, the method reaches accuracy $\epsilon$ in $O(1 + \sqrt{L\|x_0-x^*\|^2/\epsilon} + \ln(1/(\eta_0 L)))$ iterations, which is the optimal accelerated complexity up to an additive logarithmic factor. For $(L_0,L_1)$-smooth convex functions with $\eta_0 L_0 \exp(L_1\|x_0-x^*\|) \leq 1$, it obtains the first near-optimal adaptive guarantee, with the same square-root dependence on the precision plus additive constant terms. A sympathetic reader would care because this is the first demonstration that full adaptivity to local curvature and accelerated momentum can coexist, which is what practitioners would need to run accelerated methods without knowing the smoothness constants.

What carries the argument

The load-bearing object is the adaptive momentum-coupling step: at iteration $k$ the algorithm forms $\tilde{x}_k = \alpha_k \hat{x}_k + (1-\alpha_k)x_k$, takes the gradient step $x_{k+1} = x_k - \eta_k \nabla f(\tilde{x}_k)$, then forms $x_{k+1} = \beta_k \tilde{x}_k + (1-\beta_k)x_k$ with $\beta_k = \eta_k/(\alpha_k H_k)$, where $H_k$ is the cumulative sum of stepsizes. Choosing $\alpha_k = (1+\gamma)\eta_{k-1}/(H_{k-1} + (1+\gamma)\eta_{k-1})$ keeps $\beta_k \leq 1$ and satisfies $\eta_k/(\alpha_k\beta_k) = H_k$. This avoids the constraint $\eta_k/\alpha_k \leq \eta_{k-1}/\alpha_{k-1} + \eta_k$ that forced prior methods (AC-FGM, AdaNAG) to predefine the momentum parameter and restrict stepsize growth. The curvature estimator $\lambda_{k+1} = \min\{\Lambda(x_{k+1}; \tilde{x}_k), \Lambda(x_{k+1}; \tilde{x}_{k+1})\}$ supplies the local Lipschitz information, and the Lyapunov function $\Psi_k(x) = \frac{1}{2}\|x_k - x\|^2 + H_{k-1}(f(x_k) - f(x)) + (\theta\eta_k\eta_{k-1}/\lambda_k)D_f(x_{k-1}; \tilde{x}_{k-1}) + \frac{\gamma\theta}{2}\|x_k - x_{k-1}\|^2$ telescopes across iterations, turning descent into a lower bound on $H_k$ that becomes the iteration complexity.

What would settle it

Run Algorithm 1 on the one-dimensional quadratic $f(x) = x^2/2$, with $x_0 = 10$, $x^* = 0$, initial stepsizes $\eta_0 = 10^{-4}$ and $\eta_0 = 10^{-10}$, and a sequence of accuracies $\epsilon = 10^{-4}, 10^{-6}, 10^{-8}$, using any fixed constants satisfying eq. (19). If the number of iterations grows roughly like $1/\epsilon$ instead of $\sqrt{1/\epsilon}$ up to a logarithmic term, or if the stepsize $\eta_k$ fails to grow geometrically toward a value of order 1, then the central complexity and geometric-growth claims for $L$-smooth functions are wrong.

Watch

Extended reading notes

Core claim

Algorithm 1, called Accelerated GRAAL, combines accelerated momentum with the GRAAL idea of estimating the local inverse gradient Lipschitz constant. The curvature estimator is $\Lambda(x; z) = 2D_f(x; z)/\|\nabla f(x) - \nabla f(z)\|^2$, and the stepsize update $\eta_{k+1} = \min\{(1+\gamma)\eta_k, \nu H_{k-1}\lambda_{k+1}/\eta_{k-1}\}$ allows the stepsize to grow at a geometric rate. The central mechanism is an additional coupling step $x_{k+1} = \beta_k \tilde{x}_k + (1 - \beta_k)x_k$ with $\beta_k = \eta_k/(\alpha_k H_k)$, which decouples the momentum parameter from future curvature estimates and lets $\alpha_k$ itself adapt to the stepsize history. The proofs establish a Lyapunov function $\Psi_k$ and show that the cumulative stepsize $H_k$ grows at least as $c(k - m)/\sqrt{L}$ for $L$-smooth functions, and at least as $c(k - |T_2(k)| - |T_4(k)| - 1)/\sqrt{L_0}$ for $(L_0,L_1)$-smooth functions. These lower bounds, combined with the Lyapunov inequality, yield the near-optimal iteration complexities in Corollaries 2 and 3.

Load-bearing premise

The convergence theorems require the initial stepsize $\eta_0$ to be small enough relative to how curved the function is and, in the generalized-smooth case, how far the starting point is from the solution; the paper suggests picking a tiny value but gives no way to verify the needed inequality without already knowing those quantities.

Editorial extensions

If this is right

  • For convex $L$-smooth functions, choosing any initial stepsize $\eta_0$ with $\eta_0 L \leq 1$ gives the optimal accelerated complexity up to a logarithmic additive term: $K = O(1 + \sqrt{L\|x_0 - x^*\|^2/\epsilon} + \ln(1/(\eta_0 L)))$ .
  • For $(L_0,L_1)$-smooth convex functions, the same algorithm is the first adaptive method with near-optimal complexity, reaching $K = O(1 + \sqrt{L_0 D^2/\epsilon} + L_1^3 D^3 + (1 + L_1^2 D^2)\ln(1/(\eta_0 L_0)))$ where $D = O(\|x_0 - x^*\|)$ .
  • The only user input is the initial stepsize; the constants $\theta$, $\gamma$, $\nu$ are universal and fixed by eq. (19), so the method needs no line search and no hyperparameter tuning.
  • Because the stepsize can grow geometrically rather than at the sublinear rate $(1 + 1/k)$, a deliberately small initial stepsize costs only a logarithmic additive factor, whereas the paper shows AC-FGM and AdaNAG pay factors of $1/\sqrt{\eta_0 L}$ or $\eta_0 L$.
  • Under $(L_0,L_1)$-smoothness the geometric stepsize growth is the feature that avoids exponential factors from drifting local Lipschitz constants; the paper conjectures that the restricted-growth methods cannot reach near-optimal complexity in this class.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The practical recipe 'take $\eta_0 = 10^{-10}$' is not certified by the theorems: Corollaries 2 and 3 condition on $\eta_0$ being small relative to $L$, $L_0$, $L_1$, and $\|x_0 - x^*\|$, and the paper gives no way to check that condition from data alone.
  • The additive $(L_1 D)^3$ term in Corollary 3 is worse than the $(L_1 D)^{5/3}$ term of the non-adaptive near-optimal baseline, suggesting that the adaptive framework may still have room to tighten its additive dependence.
  • The same coupling-step construction might extend to stochastic gradients, composite objectives, or non-convex generalized-smooth problems, since it decouples the momentum schedule from future curvature information; these extensions are natural next steps that the paper does not take.
  • A computational check on ill-conditioned quadratics and on neural-network losses with $(L_0,L_1)$-smoothness could test whether geometric stepsize growth actually occurs in practice and whether the logarithmic penalty for a tiny initial stepsize is benign.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes an accelerated version of GRAAL (Algorithm 1) that combines a local-curvature-based adaptive stepsize rule with Nesterov momentum via an additional coupling step. It establishes a Lyapunov descent inequality (Theorem 1) that holds for any convex differentiable function. For L-smooth objectives, it proves a lower bound on the cumulative stepsize (Theorem 2) that with eta0 L <= 1 yields iteration complexity O(sqrt(L ||x0-x*||^2/epsilon) + log(1/(eta0 L))) (Corollary 2). For (L0,L1)-smooth objectives, it proves a similar lower bound (Theorem 3), giving complexity O(sqrt(L0 D^2/epsilon) + (L1 D)^3 + (1+L1^2 D^2) log(1/(eta0 L0))) under eta0 L0 exp(L1 ||x0-x*||) <= 1 (Corollary 3). The paper argues this is the first adaptive accelerated method with geometric stepsize growth and near-optimal guarantee under generalized smoothness.

Significance. The conceptual contribution is substantial: it resolves the open question of combining Nesterov acceleration with the GRAAL-type stepsize update without line search or a predefined stepsize sequence. The proof of the Lyapunov descent (Theorem 1) is carried out in detail, and the cumulative-stepsize lower bounds (Theorems 2 and 3) are nontrivial and mostly self-contained. The paper is explicit about the price of adaptation (logarithmic/additive terms), and the comparison with AC-FGM and AdaNAG is informative. The method is constructive and accompanied by a full proof apparatus, including machine-checkable-style lemmas and explicit statements of all constant conditions. However, the advertised 'no hyperparameter tuning' property is undermined by the initial-stepsize conditions in Corollaries 2 and 3, whose verification requires knowledge of problem-dependent constants. The result remains conditional and the claim should be revised accordingly.

major comments (2)
  1. [§3.1, Corollary 2 and the paragraph after it; §4.1, Corollary 3] The paper claims that choosing eta0 = 10^{-10} 'will only result in a small logarithmic additive factor' and that no hyperparameter tuning is needed. However, the theorem's hypothesis eta0 L <= 1 cannot be verified from observable quantities, and for every fixed eta0 > 0 there exist admissible instances with eta0 L > 1. In that regime the logarithmic term ln(1/(eta0 L)) in eq. (26) is negative and the displayed O(·) bound is vacuous, while the proof gives no fallback control (the first gradient step may overshoot by a factor eta0 L). The same issue appears in Corollary 3 with the condition eta0 L0 exp(L1 ||x0-x*||) <= 1. The theorem statements themselves are sound, but the central no-tuning claim goes beyond what is proven. Please either (a) restate the results as requiring a user-supplied upper bound on L, or on L0 exp(L1 ||x0-x*||), or (b) add a verification/certification mechanism (e.g., a doubling procedure for eta0) that makes the condition checkable without knowing the constants.
  2. [§C.6, proof of Corollary 3 (derivation of D = O(||x0-x*||))] In the chain of inequalities for D, step (c) is justified by 'uses eq. (31)'. Equation (31) alone gives ||grad f(x0)|| <= (L0 + L1 ||grad f(x0)||)(exp(L1 ||x0-x*||)-1)/L1, which cannot be rearranged to ||grad f(x0)|| <= (L0/L1)(exp(L1 ||x0-x*||)-1) without a Gronwall-type argument (or the equivalent lemma of Vankov et al., 2024). The paper cites the equivalence of eqs. (30) and (31) to Vankov et al., Lemma 2.5, but the proof here does not invoke the full lemma. Since the claim D = O(||x0-x*||) drives the additive-term order in Corollary 3, this step should be made explicit or replaced by a direct citation.
minor comments (3)
  1. [§2.1, Algorithm 1, lines 6-7] The variable x_{k+1} is assigned twice; line 6 sets it to the gradient step, then line 7 overwrites it. This notational collision is confusing; please use a temporary variable (e.g., gstep) for the gradient step to avoid ambiguity.
  2. [§2.2, eq. (19)] It would be helpful to provide one explicit admissible triple (theta, gamma, nu) instead of leaving existence to the reader. The paper says 'easy to verify', but a concrete value would make the method directly reproducible.
  3. [Abstract and §1.3] The abstract and contributions advertise `without hyperparameter tuning'; since eta0 is still a user-chosen parameter and the theorems require a condition on it, consider replacing `no hyperparameter tuning' by, for example, `no line search and no tuning of L-dependent parameters' to match the theorems.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence proofs are self-contained and do not reduce to fitted inputs or self-citations.

full rationale

The paper's main derivation chain is self-contained. Algorithm 1 is defined by explicit update rules, and the convergence results follow from stated assumptions (convexity, continuous differentiability, L-smoothness or (L0,L1)-smoothness) through the algebraic inequalities in Theorem 1, Theorems 2-3, and the corollaries. The adaptive stepsize (17) is designed to make the Lyapunov decrease inequality (20) hold; this is a standard constructive analysis, not a circular one. The lower bounds on H_k in Theorems 2 and 3 are derived from the stepsize rule and from lower bounds on the curvature estimator λ_k, which in turn follow from Lipschitz/smoothness inequalities (Lemmas 3, 6, 7). Corollaries 2 and 3 then combine these bounds with the general inequality in Corollary 1. No parameter is fitted to data and then renamed as a prediction; the only user-provided quantity is the initial step size η0, which appears as an explicit assumption (η0 L ≤ 1 or η0 L0 exp(L1‖x0−x*‖) ≤ 1) rather than a fitted quantity. The suggestion to take η0 very small is a heuristic for trying to satisfy that assumption, not a statistical or definitional shortcut. The self-citations to Kovalev and Borodich (2024) and Kovalev (2025) are used only for background interpretation of Nesterov acceleration and for inequality (14), which the paper explicitly avoids via the additional coupling step; the proof of Theorem 1 does not depend on those works. The unverifiability of the initial-stepsize condition is a practical assumption gap, not a circularity, and would be a correctness or robustness concern at most. Thus the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard convexity and smoothness assumptions, two external lemmas from the prior literature, the initial stepsize choice, and the existence of three universal constants. No new entities or fitted quantities are introduced.

free parameters (2)
  • eta0 (initial stepsize) = user-chosen, e.g. 1e-10
    Required to be small enough (eta0 <= 1/L or eta0 L0 exp(L1D) <= 1); complexity depends on ln(1/(eta0 L)).
  • theta, gamma, nu (algorithm constants) = not specified numerically
    Universal constants satisfying eq. (19); existence is stated, no explicit values given.
assumptions (5)
  • domain assumption f is convex and continuously differentiable; a solution x* exists
    Stated in Section 1.
  • domain assumption f is L-smooth (gradient L-Lipschitz) in Section 3
    Eq. (23).
  • domain assumption f is twice continuously differentiable with ||nabla^2 f(x)|| <= L0 + L1||nabla f(x)|| in Section 4
    Eq. (30).
  • standard math Vankov et al. (2024) Corollary 2.8 and Lemma 2.5 hold and are applicable
    Used in Lemmas 6 and 7 to relate Bregman divergences and gradient differences under (L0,L1)-smoothness.
  • standard math Parameters theta, gamma, nu exist satisfying eq. (19)
    Used throughout Theorem 1; the paper asserts existence but does not prove it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization." pith.science (2026). https://pith.science/paper/BT3Q3BY7

@misc{pith2026250709823,
  author       = {Pith},
  title        = {Pith review of: Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BT3Q3BY7}},
  note         = {Machine review of arXiv:2507.09823}
}
abstract

In this paper, we focus on the problem of minimizing a continuously differentiable convex objective function, $\min_x f(x)$. Recently, Malitsky (2020); Alacaoglu et al.(2023) developed an adaptive first-order method, GRAAL. This algorithm computes stepsizes by estimating the local curvature of the objective function without any line search procedures or hyperparameter tuning, and attains the standard iteration complexity $\mathcal{O}(L\lVert x_0-x^*\rVert^2/\epsilon)$ of fixed-stepsize gradient descent for $L$-smooth functions. However, a natural question arises: is it possible to accelerate the convergence of GRAAL to match the optimal complexity $\mathcal{O}(\sqrt{L\lVert x_0-x^*\rVert^2/\epsilon})$ of the accelerated gradient descent of Nesterov (1983)? Although some attempts have been made by Li and Lan (2025); Suh and Ma (2025), the ability of existing accelerated algorithms to adapt to the local curvature of the objective function is highly limited. We resolve this issue and develop GRAAL with Nesterov acceleration, which can adapt its stepsize to the local curvature at a geometric, or linear, rate just like non-accelerated GRAAL. We demonstrate the adaptive capabilities of our algorithm by proving that it achieves near-optimal iteration complexities for $L$-smooth functions, as well as under a more general $(L_0,L_1)$-smoothness assumption (Zhang et al., 2019).

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Optimal Parameter-Free First-Order Methods for Convex Optimization with Unknown Growth and Smoothness

    math.OC 2026-07 accept novelty 7.5 of 10

    Affine W-certificate bundle-level methods (BLW/A-BLW) attain optimal parameter-free rates under unknown Hölder smoothness and growth for convex first-order optimization.

  2. Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates

    math.OC 2025-11 conditional novelty 5.0 of 10

    Non-Euclidean SGD variants (SignSGD, Muon) provably match adaptive optimizers' convergence rates under structured smoothness and noise assumptions.

Reference graph

Works this paper leans on

41 extracted references · 27 canonical work pages · cited by 2 Pith papers

  1. [1]

    Alacaoglu, A., B \"o hm, A., and Malitsky, Y. (2023). Beyond the golden ratio for variational inequality algorithms. Journal of machine learning research , 24(172):1--33

    work page 2023

  2. [2]

    Armijo, L. (1966). Minimization of functions having lipschitz continuous first partial derivatives. Pacific Journal of mathematics , 16(1):1--3

    work page 1966

  3. [3]

    and Borwein, J

    Barzilai, J. and Borwein, J. M. (1988). Two-point step size gradient methods. IMA journal of numerical analysis , 8(1):141--148

    work page 1988

  4. [4]

    Burdakov, O., Dai, Y., and Huang, N. (2019). Stabilized barzilai-borwein method. Journal of Computational Mathematics , pages 916--936

    work page 2019

  5. [5]

    Cutkosky, A. (2019). Anytime online-to-batch, optimism and acceleration. In International conference on machine learning , pages 1446--1454. PMLR

    work page 2019

  6. [6]

    and Orabona, F

    Cutkosky, A. and Orabona, F. (2018). Black-box reductions for parameter-free online learning in banach spaces. In Conference On Learning Theory , pages 1493--1529. PMLR

    2018

  7. [7]

    and Liao, L.-Z

    Dai, Y.-H. and Liao, L.-Z. (2002). R-linear convergence of the barzilai and borwein gradient method. IMA Journal of numerical analysis , 22(1):1--10

    work page 2002

  8. [8]

    Daskalakis, C., Ilyas, A., Syrgkanis, V., and Zeng, H. (2017). Training gans with optimism. arXiv preprint arXiv:1711.00141

    arXiv 2017

Show all 41 references
  1. [9]

    and Mishchenko, K

    Defazio, A. and Mishchenko, K. (2023). Learning-rate-free learning by d-adaptation. In International Conference on Machine Learning , pages 7449--7479. PMLR

    2023

  2. [10]

    Defazio, A., Zhou, B., and Xiao, L. (2022). Grad-gradagrad? a non-monotone adaptive stochastic gradient method. arXiv preprint arXiv:2206.06900

    2022 arXiv

  3. [11]

    and Teboulle, M

    Drori, Y. and Teboulle, M. (2014). Performance of first-order methods for smooth convex minimization: a novel approach. Mathematical Programming , 145(1):451--482

    2014

  4. [12]

    Duchi, J., Hazan, E., and Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research , 12(7)

    2011

  5. [13]

    and Nesterov, Y

    Gasnikov, A. and Nesterov, Y. (2016). Universal fast gradient method for stochastic composit optimization problems. arXiv preprint arXiv:1604.05275

    2016 arXiv

  6. [14]

    Goldstein, A. A. (1962). Cauchy's method of minimization. Numerische Mathematik , 4(1):146--150

    1962

  7. [15]

    and Kakade, S

    Hazan, E. and Kakade, S. (2019). Revisiting the polyak step size. arXiv preprint arXiv:1905.00313

    2019 arXiv

  8. [16]

    Ivgi, M., Hinder, O., and Carmon, Y. (2023). Dog is sgd’s best friend: A parameter-free dynamic step size schedule. In International Conference on Machine Learning , pages 14465--14499. PMLR

    2023

  9. [17]

    Y., Bach, F., and Cevher, V

    Kavis, A., Levy, K. Y., Bach, F., and Cevher, V. (2019). Unixgrad: A universal, adaptive algorithm with optimal guarantees for constrained optimization. Advances in neural information processing systems , 32

    2019

  10. [18]

    Khaled, A., Mishchenko, K., and Jin, C. (2023). Dowg unleashed: An efficient universal parameter-free gradient descent method. Advances in Neural Information Processing Systems , 36:6748--6769

    2023

  11. [19]

    Korpelevich, G. M. (1976). The extragradient method for finding saddle points and other problems. Matecon , 12:747--756

    1976

  12. [20]

    Kovalev, D. (2025). Sgd with adaptive preconditioning: Unified analysis and momentum acceleration. arXiv preprint arXiv:2506.23803

    2025 arXiv

  13. [21]

    and Borodich, E

    Kovalev, D. and Borodich, E. (2024). On linear convergence in smooth convex-concave bilinearly-coupled saddle-point optimization: Lower bounds and optimal algorithms. arXiv preprint arXiv:2411.14601

    2024 arXiv

  14. [22]

    Kovalev, D., Gasnikov, A., and Richt \'a rik, P. (2022). Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling. Advances in Neural Information Processing Systems , 35:21725--21737

    2022

  15. [23]

    Kreisler, I., Ivgi, M., Hinder, O., and Carmon, Y. (2024). Accelerated parameter-free stochastic optimization. In The Thirty Seventh Annual Conference on Learning Theory , pages 3257--3324. PMLR

    2024

  16. [24]

    Y., Yurtsever, A., and Cevher, V

    Levy, K. Y., Yurtsever, A., and Cevher, V. (2018). Online adaptive methods, universality and acceleration. Advances in neural information processing systems , 31

    2018

  17. [25]

    and Lan, G

    Li, T. and Lan, G. (2023). A simple uniformly optimal method without line search for convex optimization. arXiv preprint arXiv:2310.10082

    2023 arXiv

  18. [26]

    and Orabona, F

    Li, X. and Orabona, F. (2019). On the convergence of stochastic gradient descent with adaptive stepsizes. In The 22nd international conference on artificial intelligence and statistics , pages 983--992. PMLR

    2019

  19. [27]

    Malitsky, Y. (2020). Golden ratio algorithms for variational inequalities. Mathematical Programming , 184(1):383--410

    2020

  20. [28]

    and Mishchenko, K

    Malitsky, Y. and Mishchenko, K. (2020). Adaptive gradient descent without descent. In International Conference on Machine Learning , pages 6702--6712. PMLR

    2020

  21. [29]

    and Tam, M

    Malitsky, Y. and Tam, M. K. (2020). A forward-backward splitting method for monotone inclusions without cocoercivity. SIAM Journal on Optimization , 30(2):1451--1472

    2020

  22. [30]

    McMahan, H. B. and Streeter, M. (2010). Adaptive bound optimization for online convex optimization. arXiv preprint arXiv:1002.4908

    2010 arXiv

  23. [31]

    and Defazio, A

    Mishchenko, K. and Defazio, A. (2023). Prodigy: An expeditiously adaptive parameter-free learner. arXiv preprint arXiv:2306.06101

    2023 arXiv

  24. [32]

    Mishchenko, K., Kovalev, D., Shulgin, E., Richt \'a rik, P., and Malitsky, Y. (2020). Revisiting stochastic extragradient. In International Conference on Artificial Intelligence and Statistics , pages 4573--4582. PMLR

    2020

  25. [33]

    Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence O (1/k^2) . Dokl. Akad. Nauk. SSSR , 269(3):543

    1983

  26. [34]

    Nesterov, Y. (2013). Gradient methods for minimizing composite functions. Mathematical programming , 140(1):125--161

    2013

  27. [35]

    Nesterov, Y. et al. (2018). Lectures on convex optimization , volume 137. Springer

    2018

  28. [36]

    Orabona, F. (2023). Normalized gradients for all. arXiv preprint arXiv:2308.05621

    2023 arXiv

  29. [37]

    and P \'a l, D

    Orabona, F. and P \'a l, D. (2021). Parameter-free stochastic optimization of variationally coherent functions. arXiv preprint arXiv:2102.00236

    2021 arXiv

  30. [38]

    Polyak, B. T. (1963). Gradient methods for minimizing functionals. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki , 3(4):643--653

    1963

  31. [39]

    Polyak, B. T. (1969). Minimization of unsmooth functionals. USSR Computational Mathematics and Mathematical Physics , 9(3):14--29

    1969

  32. [40]

    Raydan, M. (1993). On the barzilai and borwein choice of steplength for the gradient method. IMA Journal of Numerical Analysis , 13(3):321--326

    1993

  33. [41]

    Rodomanov, A., Jiang, X., and Stich, S. U. (2024). Universality of adagrad stepsizes for stochastic optimization: Inexact oracle, acceleration and variance reduction. Advances in Neural Information Processing Systems , 37:26770--26813

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.