Pith. sign in

REVIEW 5 minor 26 references

Projected Adam can keep average regret bounded away from zero for every choice of moment decay rates, including those used in practice.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 01:54 UTC pith:EYG4GOU6

load-bearing objection Clean, elementary closure of the last parameter gap in Adam non-convergence, with uniform bounded slopes and systematic extensions to the usual variants.

arxiv 2607.03519 v1 pith:EYG4GOU6 submitted 2026-07-03 cs.LG math.OCstat.ML

On the Convergence of Adam, Revisited

Steven Heilman , Sampad Mohanty This is my paper
classification cs.LG math.OCstat.ML MSC 68W2765K1068Q32
keywords Adamonline optimizationregretadaptive methodsmoment decayRMSPropAdamWnonconvergence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Adam is the default optimizer for large-scale training, yet its theoretical convergence was previously known to fail only when the first moment decays faster than the square root of the second. This paper removes that restriction: for every pair of decay rates in [0,1), every epsilon, and every step-size sequence that goes to zero slowly enough, there is a simple three-periodic sequence of linear losses on the interval [-1,1] on which projected Adam drives the iterate to the wrong endpoint and leaves average regret stuck at a positive constant. The same construction works for AdamW, RMSProp, NAdam, Adan, AdaMax and Muon, and even when the three slopes are drawn i.i.d. rather than periodically. The result shows that the short-memory adaptive denominator, not merely a special regime of the hyperparameters, is what permits permanent drift away from the minimizer.

Core claim

For every beta_1, beta_2 in [0,1), epsilon >= 0 and every step-size sequence alpha_t -> 0 with alpha_{t+1}/alpha_t -> 1 and sum alpha_t = infinity, there exists delta > 0 such that projected Adam run on the three-periodic linear losses with slopes 2+delta, -1, -1 satisfies x_t -> 1 while the best fixed comparator is -1, so lim R_T/T = 2 delta/3 > 0.

What carries the argument

A contractive three-step return map for the moment pairs (m,v) that converges exponentially to an explicit steady-state triple (M_i(a), V_i(a)); the signed sum S(a) of the resulting normalized directions is strictly negative at a=2 and remains negative for a slightly larger than 2, producing a uniform positive net drift that the projection lemma turns into convergence of x_t to the wrong endpoint.

Load-bearing premise

The step sizes must satisfy alpha_{t+1}/alpha_t tends to 1; without that regularity the three-step net drift need not stay uniformly positive and the projection argument no longer forces the iterates to the wrong endpoint.

What would settle it

Exhibit any beta_1, beta_2, epsilon and a step-size sequence obeying the paper's limits for which the three-periodic linear sequence with some fixed a near 2 produces average regret tending to zero (or x_t not tending to 1).

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • No fixed choice of beta_1 and beta_2, no matter how close to 1, guarantees vanishing average regret for projected Adam on all bounded-gradient online problems.
  • The same permanent-regret construction applies verbatim to AdamW, RMSProp, NAdam, Adan, AdaMax and the scalar case of Muon.
  • Even an i.i.d. mixture of the same two slopes (probability 1/3 for the large positive slope) produces almost-sure nonzero average regret under the usual 1/sqrt(t) schedule.
  • Methods that keep a longer memory of squared gradients (AMSGrad, AdaGrad, NosAdam) escape the construction, confirming that short-memory second-moment estimation is the essential failure mode.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Practitioners who already run beta_1 = 0.9 and beta_2 = 0.95 or 0.999 are operating inside the regime the paper shows can permanently fail; the counter-example is therefore not merely theoretical.
  • Because the gradients stay uniformly bounded between 1 and 3, the failure cannot be dismissed as an artifact of unbounded or discontinuous losses.
  • A natural next test is whether adding a slowly growing lower bound on the second-moment estimate (as AMSGrad does) is the minimal change that restores vanishing regret for every beta pair.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper proves that projected Adam on the interval [-1,1], for arbitrary fixed moment parameters β₁,β₂∈[0,1) and ε≥0, can have average regret bounded away from zero. The counterexample is a three-periodic sequence of linear losses with slopes 2+δ,-1,-1 (δ>0 small), under standard step-size conditions α_t o0, α_{t+1}/α_t o1 and ∑α_t=∞. The argument first obtains exponential convergence of the first and second moments to unique period-3 fixed points via the contractive mapping theorem, shows that the resulting three-step drift S(2) is strictly negative by elementary comparison of square roots, and then uses continuity, a three-step projection lemma, and a regret identity to conclude lim x_t=1 and lim R_T/T=2δ/3>0. The same skeleton is adapted, with explicit algebra, to AdamW, NAdam, Adan, AdaMax and Muon, and an appendix establishes an almost-sure i.i.d. analogue for Adam via a stationary negative mean obtained from the Harris inequality.

Significance. The result closes a genuine gap left by Reddi-Kale-Kumar (2018), who required β₁<√β₂ and allowed arbitrarily large slopes. Here the slopes remain uniformly bounded (between 1 and 3) for every β₁,β₂∈[0,1), and the same construction covers several widely used Adam variants. The proofs are elementary, fully written out, and free of hidden parameter restrictions; the i.i.d. appendix further shows that strict periodicity is unnecessary. These are clean, falsifiable negative results that clarify the theoretical status of Adam and its relatives in the online linear setting.

minor comments (5)
  1. In the outline of the proof (Section 1.2) the phrase "negative mean drift S(a) … is negative when a=2" is slightly confusing; S(a) is already defined as a signed sum, so "S(a)<0" would be clearer.
  2. Lemma 4.1 is stated for three increments with signs (-,+,+). A one-sentence remark that the same argument works for any fixed period length with one negative and the rest positive increments would make the later Muon and Adan adaptations more transparent.
  3. In the AdaMax section the fixed-point argument for V_1=a is correct, but the claim that the maps for V_2 and V_3 are contractions is only sketched; a short verification that the Lipschitz constant remains q^3 would remove any residual doubt.
  4. The i.i.d. appendix (Theorem A.1) assumes α_t=α/√t. It would be helpful to note whether the same almost-sure conclusion holds under the more general step-size hypotheses of Theorem 5.1, or whether the weighted strong-law argument requires the concrete 1/√t decay.
  5. Typographical: "Drift A way from the Minimizer" (Section 3 heading) should be "Away"; a few other minor spacing issues appear around the displayed equations for M_i and V_i.

Circularity Check

0 steps flagged

No circularity: self-contained existence proof via contraction and continuity on an explicit period-3 counterexample.

full rationale

The paper constructs an explicit three-periodic linear counterexample (slopes a=2+δ, −1, −1) and proves nonzero average regret for projected Adam (and listed variants) for every β₁, β₂ ∈ [0,1) and ε ≥ 0. The derivation is elementary and self-contained: Lemma 2.1 obtains unique period-3 fixed points of the moment maps by the contractive-mapping theorem and solves them in closed form (5)–(6); Lemma 3.1 evaluates the three-step drift S(2) by direct algebra (M₁+M₂+M₃=0 together with V₁>V₂,V₃) and obtains S(2+δ)<0 by continuity; Lemma 4.1 controls the projection; Theorem 5.1 assembles the pieces into xt o1 and RT/T o2δ/3. The same skeleton is reused for AdamW/NAdam/Adan/AdaMax/Muon with only local algebraic changes, and the i.i.d. appendix replaces the deterministic drift by a strictly negative stationary mean obtained from the Harris inequality. No quantity is defined in terms of the claimed limit, no parameter is fitted to data and then re-presented as a prediction, and there are no load-bearing self-citations or uniqueness theorems imported from the authors. The only external inputs are classical fixed-point and correlation inequalities. This is an ordinary existence argument, not a circular reduction.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

The paper is a pure existence proof inside online convex optimization. The only free parameter is the slope offset δ, chosen small enough that the continuous function S(a) remains negative; all other ingredients are standard fixed-point and comparison arguments. No new physical or statistical entities are postulated.

free parameters (1)
  • δ (slope offset) = sufficiently small positive number
    Chosen sufficiently small (depending only on β1, β2, ε) so that the three-step drift S(2+δ) stays strictly negative; existence is guaranteed by continuity of S at a=2, but the concrete value is not computed.
axioms (3)
  • standard math Contractive mapping theorem on R (unique fixed point of an affine map with Lipschitz constant <1)
    Invoked in Lemma 2.1 to obtain unique steady-state triples (M1,M2,M3) and (V1,V2,V3).
  • domain assumption Definition of projected Adam (and listed variants) with the usual exponential moving averages
    Taken as the object of study; the paper does not re-derive the algorithm.
  • domain assumption Step-size conditions lim αt=0, lim αt+1/αt=1, ∑αt=∞
    Standard vanishing-step-size regime for online regret analysis; used after the drift calculation to force xt→1.

pith-pipeline@v1.1.0-grok45 · 27716 in / 2421 out tokens · 21525 ms · 2026-07-12T01:54:57.180393+00:00 · methodology

0 comments
read the original abstract

We show that projected Adam for online optimization with arbitrary moment decay parameters $\beta_1,\beta_2\in[0,1)$ can have average regret bounded away from zero. A similar result of Reddi-Kale-Kumar from 2018 required $\beta_1<\sqrt{\beta_2}$. Similar to their result, we use a three-periodic sequence of linear functions on $[-1,1]$ with slopes $c,-1,-1$, though we use $c$ slightly larger than $2$. This nonzero average regret result extends to Adam variants such as AdamW, RMSProp, NAdam, Adan, AdaMax, Muon, and to an i.i.d. variant of the three-periodic sequence of slopes for Adam.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 7 linked inside Pith

  1. [1]

    A new regret analysis for Adam-type algorithms

    Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos, and Volkan Cevher. A new regret analysis for Adam-type algorithms. International Conference on Machine Learning (2020), 119, pp. 202--210

    2020

  2. [2]

    Understanding Adam optimizer via online learning of updates: Adam is FTRL in disguise

    Kwangjun Ahn, Zhiyu Zhang, Yunbum Kook, and Yan Dai. Understanding Adam optimizer via online learning of updates: Adam is FTRL in disguise. International Conference on Machine Learning (2024)

    2024

  3. [3]

    A general system of differential equations to model first order adaptive algorithms

    Andr\'e Belotto da Silva and Maxime Gazeau. A general system of differential equations to model first order adaptive algorithms. Journal of Machine Learning Research (2020), 21 (129), pp. 1--42

    2020

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    2020

  5. [5]

    Non-convergence and limit cycles in the Adam optimizer

    Sebastian Bock and Martin Georg Wei . Non-convergence and limit cycles in the Adam optimizer. International Conference on Artificial Neural Networks (2019), vol 11728

    2019

  6. [6]

    Towards understanding Adam convergence on highly degenerate polynomials

    Zhiwei Bai, Jiajie Zhao, Zhangchen Zhou, Zhi-Qin John Xu, and Yaoyu Zhang. Towards understanding Adam convergence on highly degenerate polynomials. International Conference on Machine Learning (2026), to appear

    2026

  7. [7]

    DeepSeek-V3 Technical Report

    DeepSeek-AI. DeepSeek-V3 Technical Report. (2024), Preprint, arXiv:2412.19437 https://arxiv.org/abs/2412.19437v1

    Pith/arXiv arXiv 2024

  8. [8]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Chapter of the Association for Computational Linguistics (2019), pp. 4171--4186

    2019

  9. [9]

    Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer

    Steffen Dereich, Thang Do, Arnulf Jentzen, and Philippe von Wurstemberger. Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer. (2025), Preprint, arXiv:2511.06675 https://arxiv.org/abs/2511.06675

    Pith/arXiv arXiv 2025

  10. [10]

    Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates

    Steffen Dereich, Robin Graeber, and Arnulf Jentzen. Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates. (2024), Preprint, arXiv:2407.08100 https://arxiv.org/abs/2407.08100

    Pith/arXiv arXiv 2024

  11. [11]

    Thang Do, Sonja Hannibal, and Arnulf Jentzen. Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation. Journal of Mathematical Analysis and Applications (2026), 130724

    2026

  12. [12]

    Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks

    Thang Do, Arnulf Jentzen, and Adrian Riekert. Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks. (2025), Preprint, arXiv:2503.01660 https://arxiv.org/abs/2503.01660

    Pith/arXiv arXiv 2025

  13. [13]

    Nostalgic Adam: Weighting More of the Past Gradients When Designing the Adaptive Learning Rate

    Haiwen Huang, Chang Wang, and Bin Dong. Nostalgic Adam: Weighting More of the Past Gradients When Designing the Adaptive Learning Rate. International Joint Conference on Artificial Intelligence (2019), pp. 2556--2562

    2019

  14. [14]

    Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks

    Arnulf Jentzen and Adrian Riekert. Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks. SIAM/ASA Journal on Uncertainty Quantification (2025), 13 (3), pp. 1294--1333

    2025

  15. [15]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (2015). (Poster)

    2015

  16. [16]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. International Conference on Learning Representations (2019) (Poster)

    2019

  17. [17]

    Adaptive gradient methods with dynamic bound of learning rate

    Liangchen Luo, Yuanhao Xiong, Yan Liu, and Xu Sun. Adaptive gradient methods with dynamic bound of learning rate. International Conference on Learning Representations (2019)

    2019

  18. [18]

    Tetiana Parshakova, Ahmed Khaled, Michael Crawshaw, Guillaume Garrigos, and Robert M. Gower. Muon Does Not Converge on Convex Lipschitz Functions. (2026), Preprint, arXiv:2605.08980 https://arxiv.org/abs/2605.08980

    Pith/arXiv arXiv 2026

  19. [19]

    Reddi, Satyen Kale, and Sanjiv Kumar

    Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of Adam and beyond. International Conference on Learning Representations (2018)

    2018

  20. [20]

    Philippe L. Toint. Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example. (2023), Preprint, arXiv:2308.00720 https://arxiv.org/abs/2308.00720

    Pith/arXiv arXiv 2023

  21. [21]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Harts...

    Pith/arXiv arXiv 2023

  22. [22]

    Divergence results and convergence of a variance reduced version of Adam

    Ruiqi Wang and Diego Klabjan. Divergence results and convergence of a variance reduced version of Adam. (2022), Preprint, arXiv:2210.05607 https://arxiv.org/abs/2210.05607

    arXiv 2022

  23. [23]

    Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

    Xingyu Xie, Pan Zhou, Huan Li, Zhouchen Lin, and Shuicheng Yan. Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024), 46 (12), pp. 9508--9520

    2024

  24. [24]

    Adam can converge without any modification on update rules

    Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, and Zhi-Quan Luo. Adam can converge without any modification on update rules. Advances in Neural Information Processing Systems (2022)

    2022

  25. [25]

    Adam converges without any modification on update rules

    Yushun Zhang, Bingran Li, Congliang Chen, Zhi-Quan Luo, and Ruoyu Sun. Adam converges without any modification on update rules. (2026), Preprint, arXiv:2603.02092 https://arxiv.org/abs/2603.02092

    arXiv 2026

  26. [26]

    Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar

    Manzil Zaheer, Sashank J. Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. Adaptive methods for nonconvex optimization. Advances in Neural Information Processing Systems (2018)

    2018