{"id":"bfa4c160-2fe8-4953-bb12-c6b91a474f91","arxiv_id":"2511.06675","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"For quadratic losses with two-point random data, stochastic Adam converges to the true minimizer if and only if the data is symmetrically distributed; for non-symmetric data it provably converges to a wrong point.","lead":"This paper proves a complete 'if and only if' for when the Adam optimizer, the standard training algorithm in modern AI, provably fails: for the simplest quadratic problem with two-point random data, Adam converges to the true minimizer exactly when the data distribution is symmetric. It also derives Adam error rates in step count, mini-batch size, and second-moment parameter, revealing an extra error term that cannot be removed.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Only-if direction of the symmetry theorem rests on external Theorem 3.2 ([14, Cor. 1.10]); unverified here and load-bearing for non-convergence.","rationale":"The reader's weakest assumption is exactly the load-bearing concern we identify: Theorem 3.2, imported from the companion paper [14], is the only bridge connecting convergence of the stochastic Adam iterates to zeros of the deterministic vector field. The symmetry theorem's non-convergence half is a contrapositive of this bridge. Our independent reading confirms that the rest of the proof structure is coherent: Proposition 3.5 gives a correct computation of f(0)<0 for non-symmetric centered two-point data; the contrapositive logic in Theorem 4.12 is valid; and the symmetric-data direction (iii)⇒(ii) is self-contained via Proposition 4.10. No internal inconsistency was found. However, the external theorem is unproved in this manuscript and posted in a companion paper the same week, so the central claim is conditional on that theorem's correctness. This does not require changing the reader's CONDITIONAL verdict: the concern is real, but it is a missing verification rather than a demonstrated error. If [14, Cor. 1.10] is correct, the theorem stands; if it fails in this regime, only the non-convergence direction is affected, while the rate results and symmetric-data convergence remain. We therefore leave the verdict unchanged and recommend the concrete test of an independent re-derivation of Theorem 3.2 in the two-point quadratic setting.","tokens_in":73539,"tokens_out":6776,"duration_ms":66977,"concrete_test":"Independently re-derive Theorem 3.2 for the special case M=1, d=1, L(θ,x)=|θ−x|^2 with two-point centered data, without invoking [14, Cor. 1.10]. In particular, prove that if limsup_n E[min{1,|Θ_n−ϑ|}]=0, then f(ϑ)=0 a.s. by checking the two approximation steps in [14]: (a) the interpolated Adam process converges to the ODE θ̇=f(θ) in probability/expectation; (b) any limit point of the ODE with decreasing step sizes must satisfy f(θ)=0. If step (a) requires an additional condition (e.g., sup_n E[|Θ_n|^2]<∞ or f∈C^1 with bounded derivative), then the only-if direction of Theorem 1.1 must be restated with that condition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is Theorem 3.2 in Section 3.1, imported without proof from the companion paper [14, Corollary 1.10]: if the Adam iterates (Θ_n^{0,M}) converge in probability to a random variable ϑ, then P(f(ϑ)=0)=1, where f is the Adam vector field. This is the only bridge from stochastic-process convergence to an algebraic condition on the vector field. The 'only if' direction of the Adam symmetry theorem (items (i),(ii) ⇒ (iii) in Theorem 1.1 / Theorem 4.12) is precisely the contrapositive: for non-symmetric two-point data, Proposition 3.5 shows f(0)<0 at the minimizer 0 (after centering), so convergence to the minimizer would contradict Theorem 3.2. If Theorem 3.2 fails in this two-point quadratic regime—for instance, if the ODE-interpolation argument in [14] needs additional boundedness or regularity assumptions not stated here—then the non-convergence claim for non-symmetric data collapses. The symmetric-data direction (iii)⇒(ii) and the rate theorems (Theorem 1.2, Corollary 2.11) are independent of this bridge and would survive. The paper itself flags the dependency: Theorem 3.2 is 'a slightly generalized variant of Corollary 1.10 in [14]', and its proof is a one-line citation. Since [14] is posted the same week and not independently verified in this manuscript, this is the weakest link in the central characterization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes Adam on strongly convex stochastic optimization problems (SOPs). The first main result, the “Adam symmetry theorem” (Theorem 1.1 / Theorem 4.12), considers quadratic losses L(θ,x)=‖θ−x‖² with i.i.d. two-point data X∈{v,w}, under standard Adam hyperparameter conditions and learning-rate assumptions. It claims equivalence between (i) an L^p error bound of order √γ_n, (ii) convergence in probability of Adam to the unique minimizer, and (iii) symmetric distribution of the data. The second main result (Theorem 1.2 / Theorem 2.10) gives strong error bounds for strongly convex SOPs under an explicit boundedness assumption, including a non-learning-rate term of order M^{-1}(1−β₂). The proof framework splits into a rate-analysis part built on the authors' earlier work [13] and a non-convergence part built on a limit-point characterization imported from the companion paper [14].","tokens_in":73817,"tokens_out":8242,"duration_ms":79753,"significance":"If the symmetry theorem is correct, it is a striking and nontrivial result: Adam provably fails to converge to the unique minimizer for non-symmetric two-point data even in a prototypical strongly convex quadratic problem, and the failure is governed by a spurious fixed point of the Adam vector field at distance O(M^{-1}(1−β₂)). The rate bound with the explicit M^{-1}(1−β₂) term is also a useful new contribution, and the paper gives detailed proofs for the self-contained parts. The numerical simulations support the sharpness of the claimed rates and the predicted spurious fixed point. The main caveat is that the non-convergence direction — the most surprising half of the symmetry theorem — rests on an external theorem that is not proved in this manuscript; the rate results and the symmetric-data direction are independent of that bridge and are much more firmly established.","major_comments":[{"comment":"Theorem 3.2 is the load-bearing bridge for the “only if” direction of the symmetry theorem. It states that if Adam converges in probability to a random variable ϑ, then f(ϑ)=0 P-a.s., where f is the Adam vector field. The proof is a one-line citation to [14, Corollary 1.10]. Proposition 3.5 and Corollary 3.6 then use f(0)<0 to conclude non-convergence for non-symmetric two-point data. Thus the central claim that Adam fails to converge to the minimizer is not independently established in this manuscript: if [14, Cor. 1.10] fails in this two-point quadratic regime, the non-convergence claim collapses. The symmetric-data direction and the rate theorems would survive. I recommend either proving the needed variant of the limit-point characterization in this paper, or explicitly stating it as an assumption and clearly marking the non-convergence claim as conditional on it.","section":"§3.1, Theorem 3.2 (Eq. (234))"},{"comment":"The announced convergence rates for “strongly convex SOPs” are conditional on the assumption inf_{N,m} P(‖Θ^{0,N,β}_m‖≤p)=1 for a fixed p. This boundedness assumption is not derived in the paper for the general setting; the paper notes that it is satisfied in the quadratic case by a companion result [12]. Since the theorem is stated with this assumption, the claim is formally correct. Still, the abstract and introduction tend to state the rate result more broadly, and the reader could be misled. Please make the conditional nature prominent in the abstract and in the introduction.","section":"§1.2 / Theorem 1.2"}],"minor_comments":[{"comment":"In the proof of Lemma 2.14, the text says “the fact that (γ_n) is increasing” for γ_n=n^{-r}, which is decreasing. Also in Eq. (361), the telescoping sum contains an index typo: the summand should be (γ_k−γ_{k+1}), not (γ_n−γ_{n−1}). The conclusion is correct, but the typos should be fixed.","section":"§2.5, Lemma 2.14 proof"},{"comment":"Theorem 1.2 states p∈N, while the abstract advertises strong L^p rates for p∈[1,∞). State the precise p-range in the abstract or adjust the theorem statement. The condition β₂∈[p^{-1},1) is empty for p=1, which also deserves a comment.","section":"§1.2 / Theorem 1.2 statement"},{"comment":"The equivalence in Theorem 4.12 is stated for all p∈(0,∞), but the proof of (iii)⇒(ii) invokes Proposition 4.10, which is stated for integer p. A short interpolation or monotonicity argument is needed to cover non-integer p; otherwise the theorem should be restricted to integer p.","section":"§4.4, Theorem 4.12"},{"comment":"The right panel of Figure 4 uses P(X=−1)=w/(1+w) as the x-axis, but this expression is not defined in the caption text or in Setting 5.1. Please add a one-line explanation.","section":"§5, Figure 4 caption"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the dependence of the central non-convergence theorem on [14, Cor. 1.10]. If the editor has access to [14], it would be useful to verify that the cited result indeed covers the exact two-point quadratic setting used in Theorem 3.2. If it does, the gap can be closed by a more explicit statement; if not, the authors should prove the needed variant. The rate-theorem part is solid and could be published even if the external bridge is weakened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper gives, for the simplest nontrivial strongly convex test problem (two-point data, squared loss), a complete iff characterization of Adam convergence to the minimizer: the data must be symmetric. That is new — Reddi's examples and Bock & Weiss's limit cycles don't give a full characterization, and the additive bias term M^{-1}(1-beta_2) is new too. If correct, this is the cleanest explanation yet of why Adam misses the optimum in the asymmetric case.\n\nWhat it does well: the proof structure is coherent. I spot-checked the heavy lemmas (2.4, 2.6, 2.7, 2.13, 2.14) and the contrapositive logic of Theorem 1.1; I found no internal contradiction. The computation f(0) != 0 for asymmetric data is a genuine derivation, not a fitting artifact. The paper also flags its own dependency, which is honest.\n\nThe soft spots, in order:\n\n1. The 'only if' direction (non-symmetry => no convergence) rests on Theorem 3.2, imported from the companion paper [14, Cor. 1.10] posted the same week. The proof here is a one-line citation. That bridge — convergence in probability implies the limit is a zero of the vector field — is load-bearing. If it needs extra boundedness/regularity beyond what is stated, the non-convergence claim collapses. The symmetric-data direction and the rate theorems do not depend on it. This is not a demonstrated error, but it is exactly where a referee should push.\n\n2. The abstract claims convergence rates for strongly convex SOPs without the boundedness assumptions that Theorem 1.2 actually states (bounded data, and boundedness of the Adam process). The advertised statement is stronger than the theorem. That should be corrected.\n\n3. The numerics are figures only — no code, seeds, or error bars. They illustrate, but they don't verify.\n\nNone of this undermines the core idea: the symmetry phenomenon is real, and the rate results are a solid advance. The paper deserves a serious referee, but the referee needs to verify [14] carefully, and the authors should be asked to either prove the needed theorem here or cite a fully checked version. Also fix the abstract/theorem mismatch.\n\nWho it's for: anyone working on the theory of adaptive optimizers; it gives a concrete, low-dimensional example that will be useful for teaching and as a testbed. I'd take it to a reading group for the discussion of external dependencies alone.","headline":"A genuinely new Adam non-convergence characterization, conditional on an unproved companion-paper bridge that a referee must verify.","tokens_in":74441,"tokens_out":2635,"would_cite":true,"duration_ms":28524,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C15","62L20","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"In a prototypical convex quadratic problem, Adam converges to the minimizer if and only if the data distribution is symmetric; asymmetric two-point data provably stalls at a spurious fixed point.","keywords":["Adam optimizer","stochastic gradient descent","strongly convex optimization","convergence rates","non-convergence","symmetry theorem","Adam vector field","mini-batch size"],"falsifier":"Fix d=1, β_1=0.9, β_2=0.999, ε=1e-8, γ_n=n^{-0.7}, and let X take values -1 and 0.1 with probabilities 1/11 and 10/11, so the minimizer is 0. Run Adam with M=1 from θ_0=0 for about 10^6 steps and estimate E[min{1,|θ_n|}] as well as E[θ_n]. The theorem predicts these do not go to 0; the limiting value should match the zero of the Adam vector field, located roughly at distance 0.001 from 0. Observing convergence to 0 would refute the characterization; observing a plateau at the predicted nonzero root would confirm it.","tokens_in":73320,"feed_emoji":"🎯","tokens_out":9379,"duration_ms":81096,"temperature":0.7,"pith_summary":"The paper aims to give a complete convergence characterization for the Adam optimizer on the simplest nontrivial strongly convex stochastic problem: minimize E[(θ-X)^2] over θ, where X comes from a two-point distribution. It proves that Adam converges to the unique minimizer—in the strong L^p sense and at the standard rate sqrt(γ_n)—if and only if the data distribution is symmetric around the minimizer. For every asymmetric two-point distribution, Adam provably does not converge to the minimizer; instead it approaches a spurious fixed point of the 'Adam vector field', an expected normalized-update field governing Adam's large-time behavior. The paper also proves quantitative error bounds for strongly convex stochastic optimization, showing an additional unavoidable error term of order M^{-1}(1-β_2) alongside the standard sqrt(γ_n) term. If these theorems are right, Adam has a fundamental bias on convex quadratics that no number of training steps removes—only larger mini-batches or β_2 closer to 1 shrink it.","feed_headline":"Adam provably misses the minimum for asymmetric data","feed_subtitle":"On a simple quadratic problem, the optimizer stalls at a spurious point of order (1-beta2)/batch size instead of reaching the optimum.","key_machinery":"The central object is the Adam vector field, defined for a frozen θ as f_i^M(θ) = (1-α) E[ Σ_{n≥0} α^n X_{n,i}^M(θ) / (ε + sqrt((1-β) Σ_{n≥0} β^n |X_{n,i}^M(θ)|^2)) ], where X_{n,i}^M is the negative mini-batch averaged gradient component and α, β are the momentum parameters. This field is what the Adam recursion tracks in the large-time limit, and the paper's central mechanism is the bridge 'limit points of Adam = zeros of this field', imported from a companion article. For symmetric two-point data the field is strictly monotone and vanishes exactly at the minimizer, so convergence rates follow from a local-monotonicity framework. For asymmetric data the field is nonzero at the minimizer an","core_discovery":"Stated on the paper's own terms, the Adam symmetry theorem says: for the loss L(θ,x)=||θ-x||^2 with i.i.d. two-point data taking values v and w, any β_1∈(0,1), β_2∈(β_1^2,1), and any decreasing learning rates satisfying a standard regularity condition, the following are equivalent: (i) the p-th moment of the Adam error from the minimizer ϑ is bounded by c sqrt(γ_n); (ii) for every mini-batch size M, the iterates converge to ϑ in probability; and (iii) the data X_{1,1} is symmetrically distributed. The heart of the proof is the Adam vector field: the expected value of Adam's normalized update evaluated at a frozen point. The paper invokes a companion result, quoted as Theorem 3.2, that any li","pith_inferences":["My inference: the load-bearing structural condition exposed by the proof is symmetry of the stochastic gradient noise at the minimizer, so for a general strongly convex loss the analogous non-convergence should be triggered by asymmetric gradient noise, not necessarily by asymmetric raw data; this extension is not in the paper.","My inference: since the bias is of order M^{-1}(1-β_2), a practical remedy suggested by the theorem—increasing batch size or annealing β_2 toward 1—should remove the plateau; the paper treats fixed β_2, so this remedy is my extrapolation, not its claim.","My inference: the characterization supplies a cheap falsification probe for practitioners: if a convex quadratic training run stalls, checking whether the plateau location moves linearly with 1-β_2 and 1/M can distinguish an Adam spurious fixed point from a learning-rate effect.","My inference: replacing the two-point distribution by continuous asymmetric noise and checking whether the same spurious zero persists at the same order is a natural stress test of the mechanism beyond the theorem's hypotheses."],"forward_implications":["In the quadratic two-point test problem, Adam with any hyperparameters satisfying β_1^2 < β_2 fails to converge to the minimizer for every asymmetric two-point data distribution, while the standard SGD method converges in the same setting.","The strong-convexity error bound for Adam contains an additive M^{-1}(1-β_2) term that the symmetry theorem shows cannot be removed: the bias persists for all n and is governed by a spurious zero of the Adam vector field.","When the data is symmetric around the minimizer, Adam attains the standard sqrt(γ_n) rate, so the extra bias is not a general defect of Adam but specifically a symmetry effect.","The deterministic case v=w is automatically symmetric, so Adam's non-convergence to the minimizer does not occur when stochasticity is absent.","The paper's numerical experiments indicate that the rates sqrt(γ_n), M^{-1}, and 1-β_2 are sharp, with the plateau appearing exactly at the predicted spurious root."],"fun_headline_variants":["Adam only converges on symmetric data, new proof","Adam provably diverges for asymmetric data","New theorem: Adam needs symmetric data to converge","Why Adam stalls on asymmetric data: a proof","Adam's convergence hinges on data symmetry"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The non-convergence half of the Adam symmetry theorem depends on the companion-paper result (quoted here as Theorem 3.2) that every limit point of Adam must be a zero of the Adam vector field; this bridge is not proved in this manuscript, and if it breaks, Adam could converge to the minimizer even for asymmetric two-point data.","fun_headline_variants_meta":{"raw":{"variants":["Adam only converges on symmetric data, new proof","Adam provably diverges for asymmetric data","New theorem: Adam needs symmetric data to converge","Why Adam stalls on asymmetric data: a proof","Adam's convergence hinges on data symmetry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1288,"prompt_tokens":955,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":699,"completion_tokens_details":{"reasoning_tokens":265}},"tokens_in":699,"tokens_out":333,"duration_ms":3765,"temperature":1.0,"reasoning_tokens":265,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T23:13:59.009262+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fix d=1, β_1=0.9, β_2=0.999, ε=1e-8, γ_n=n^{-0.7}, and let X take values -1 and 0.1 with probabilities 1/11 and 10/11, so the minimizer is 0. Run Adam with M=1 from θ_0=0 for about 10^6 steps and estimate E[min{1,|θ_n|}] as well as E[θ_n]. The theorem predicts these do not go to 0; the limiting value should match the zero of the Adam vector field, located roughly at distance 0.001 from 0. Observing convergence to 0 would refute the characterization; observing a plateau at the predicted nonzero root would confirm it.","supporting_citations":[],"review_version":1}