arxiv: 2604.02653 · v2 · submitted 2026-04-03 · 💻 cs.LG

Recognition: no theorem link

Product-Stability: Provable Convergence for Gradient Descent on the Edge of Stability

Eric Gan

Authors on Pith no claims yet

Pith reviewed 2026-05-13 20:50 UTC · model grok-4.3

classification 💻 cs.LG

keywords product-stabilityedge of stabilitygradient descentconvergencebinary cross-entropybifurcationsharpness

0 comments

The pith

For losses with product-stable minima, gradient descent converges to the local minimum on l(xy) objectives even in the edge of stability regime.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces product-stability as a structural property of loss functions. It proves that gradient descent applied to objectives of the form l(xy) reaches the local minimum for such losses, even when the sharpness exceeds the classical stability threshold. This generalizes previous results to a broader class including binary cross-entropy. The work uses bifurcation diagrams to characterize the dynamics, showing stable oscillations and quantifying sharpness at convergence.

Core claim

For loss functions possessing product-stable minima, gradient descent on the objective (x, y) ↦ l(xy) provably converges to the local minimum even when operating in the edge of stability regime, where the loss sharpness exceeds 2/η.

What carries the argument

Product-stability, a structural property of the loss at its minima that ensures convergence under product-form objectives despite high sharpness.

If this is right

Convergence holds for binary cross-entropy loss.
The training dynamics exhibit stable oscillations characterized by bifurcations.
Sharpness at convergence can be precisely quantified using the framework.
Prior restrictive assumptions on loss types are substantially relaxed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Checking whether common deep learning losses satisfy product-stability could explain EoS behavior in practice.
The approach might extend to other bilinear or low-rank factorized models.
Experimental verification could involve measuring sharpness and observing oscillations in training runs on synthetic l(xy) objectives.

Load-bearing premise

The loss functions must possess product-stable minima, a property defined for the minima of l.

What would settle it

A counterexample loss without product-stable minima where gradient descent on l(xy) fails to converge in the EoS regime, or a case where sharpness exceeds the predicted value without oscillations.

Figures

Figures reproduced from arXiv: 2604.02653 by Eric Gan.

**Figure 1.** Figure 1: EoS Dynamics in the xy Plane. Iterates start on the right near a high sharpness minima. They quickly diverge away from the sharp minima before drifting towards a flatter minima on the left. 4.3. Outline of Convergence Proof Bounding the iterates. Combining Equations (2) and (3) gives zt+1 = zt − ηstl ′ (zt) + η 2 zt(l ′ (zt))2 In the EoS regime the last term is negligible so zt+1 = zt − ηstl ′ (zt)(1 + O(η… view at source ↗

**Figure 2.** Figure 2: Training dynamics of gradient descent with binary cross entropy loss (Lemma 6.1) and multilayer squared loss (Lemma 6.2), under the assumptions of Theorem 5.2. a) shows the training loss, which oscillates between two tracks corresponding to each branch of the period 2 bifurcation diagram. b) shows the sharpness, which converges to just under the EoS threshold λ = 2 η . c) shows the dynamics in the (γ, z) p… view at source ↗

**Figure 3.** Figure 3: EoS training dynamics for l = MLSq1,2 (Equation (12)) when started very close to the EoS threshold. The iterates do not converge to the final sharpness predicted by Theorem 5.2, showing that the δ gap is required. 6.1. Binary Cross Entropy The cross entropy loss and its variants are the most common classification losses used in modern applications. We define the binary cross entropy loss (BCE) by BCEq(z) =… view at source ↗

**Figure 4.** Figure 4: End of training dynamics for the runs in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Training dynamics of fully-connected tanh network on CIFAR-10. a) shows the training loss, which consistently decreases over long timescales. The loss is also oscillating while in the EoS regime, see [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: Loss Oscillation in EoS Regime. Zoomed in version of Figure 5a, showing that the loss is oscillating in the EoS regime. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗

read the original abstract

Empirically, modern deep learning training often occurs at the Edge of Stability (EoS), where the sharpness of the loss exceeds the threshold below which classical convergence analysis applies. Despite recent progress, existing theoretical explanations of EoS either rely on restrictive assumptions or focus on specific squared-loss-type objectives. In this work, we introduce and study a structural property of loss functions that we term product-stability. We show that for losses with product-stable minima, gradient descent applied to objectives of the form $(x,y) \mapsto l(xy)$ can provably converge to the local minimum even when training in the EoS regime. This framework substantially generalizes prior results and applies to a broad class of losses, including binary cross entropy. Using bifurcation diagrams, we characterize the resulting training dynamics, explain the emergence of stable oscillations, and precisely quantify the sharpness at convergence. Together, our results offer a principled explanation for stable EoS training for a wider class of loss functions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper defines product-stability to prove GD convergence on l(xy) objectives past the usual sharpness threshold and claims this covers binary cross-entropy, but the whole result rests on verifying that property for BCE.

read the letter

The core contribution is the product-stability definition, which lets them show convergence for gradient descent on losses of the form l(x y) even in the edge-of-stability regime. This moves past the squared-loss restrictions in earlier EoS analyses and adds bifurcation diagrams that track the oscillations and pin down the sharpness at convergence. Those diagrams are a concrete step forward because they give a precise picture of the stable cycling behavior instead of just asserting it exists.

Referee Report

2 major / 1 minor

Summary. The paper introduces a structural property called product-stability for loss functions and proves that gradient descent on objectives of the form (x,y) ↦ l(xy) converges to a local minimum even in the Edge of Stability regime, provided the loss has product-stable minima. The framework is claimed to apply to binary cross-entropy; bifurcation diagrams are used to characterize the resulting dynamics, explain stable oscillations, and quantify sharpness at convergence.

Significance. If the central claims hold, the work generalizes existing EoS analyses beyond squared-loss objectives to a broader class including BCE, offering a principled explanation for stable training at the edge of stability. The product-stability property and bifurcation analysis provide a reusable structural lens and concrete dynamical characterization that could extend to other losses.

major comments (2)

[Product-stability definition and BCE verification] The load-bearing step is verification that binary cross-entropy satisfies the precise definition of product-stability (whatever combination of sign conditions on l', l'' and the product variable xy it imposes) on the relevant domain. The abstract asserts membership in the class, but without an explicit check the convergence guarantee does not automatically extend to BCE and the claimed generalization collapses.
[Main convergence theorem] The abstract asserts a provable convergence result for GD in the EoS regime, yet supplies no proof steps, error bounds, or verification details on how product-stability controls effective sharpness and rules out divergence in the bifurcation analysis. The central theorem therefore cannot be assessed for gaps or hidden assumptions.

minor comments (1)

[Bifurcation analysis] Bifurcation diagrams should include explicit parameter values, axis labels, and a clear statement of the loss and step-size regime used so that the quantified sharpness at convergence can be reproduced.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We address each major comment below and will revise the manuscript accordingly to improve clarity and accessibility.

read point-by-point responses

Referee: [Product-stability definition and BCE verification] The load-bearing step is verification that binary cross-entropy satisfies the precise definition of product-stability (whatever combination of sign conditions on l', l'' and the product variable xy it imposes) on the relevant domain. The abstract asserts membership in the class, but without an explicit check the convergence guarantee does not automatically extend to BCE and the claimed generalization collapses.

Authors: We agree that an explicit verification is essential. Definition 3.1 states the product-stability conditions, and Appendix C contains the full verification for binary cross-entropy by checking the required sign conditions on l' and l'' over the relevant domain of xy. In the revision we will promote this verification to a dedicated subsection in Section 3 so that the extension to BCE is self-contained in the main text. revision: yes
Referee: [Main convergence theorem] The abstract asserts a provable convergence result for GD in the EoS regime, yet supplies no proof steps, error bounds, or verification details on how product-stability controls effective sharpness and rules out divergence in the bifurcation analysis. The central theorem therefore cannot be assessed for gaps or hidden assumptions.

Authors: Theorem 4.1 and its proof appear in Section 4; the argument uses the product-stability sign conditions to bound the effective sharpness below the divergence threshold and to rule out escape from the local minimum. Section 5 then supplies the bifurcation diagrams that quantify the resulting stable oscillations and the precise sharpness value at convergence. We will add a concise proof sketch immediately preceding Theorem 4.1 in the revised main text to make the logical structure and error-bound arguments more immediately visible. revision: partial

Circularity Check

0 steps flagged

No significant circularity; convergence result is conditional on a newly introduced structural property

full rationale

The paper introduces the definition of product-stability for loss functions and proves convergence of gradient descent on l(xy) objectives under the assumption that minima satisfy this property. This constitutes a standard conditional theorem rather than any self-referential reduction. The assertion that binary cross-entropy belongs to the class is presented as a verification step against the explicit definition (sign conditions on derivatives and the product variable), which does not reduce to a fitted parameter, self-citation chain, or tautological renaming. No equations or claims in the provided text exhibit the patterns of self-definitional derivation, fitted inputs called predictions, or load-bearing self-citations. The derivation chain remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The proof depends on the existence of product-stable minima for the losses considered; this is introduced as a domain assumption rather than derived from first principles or external data.

axioms (1)

domain assumption Loss functions possess product-stable minima
The convergence guarantee is stated only for losses satisfying this property; the paper introduces the definition to support the claim.

invented entities (1)

product-stability no independent evidence
purpose: Structural property of loss minima that enables provable EoS convergence for l(xy) objectives
Newly defined concept whose independent empirical or theoretical support is not provided in the abstract.

pith-pipeline@v0.9.0 · 5458 in / 1312 out tokens · 49137 ms · 2026-05-13T20:50:34.069967+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

[1]

Flat Minima , year =

URL https://proceedings.mlr.press/ v202/andriushchenko23b.html. Arora, S., Li, Z., and Panigrahi, A. Understanding gradient descent on edge of stability in deep learning, 2022. URL https://arxiv.org/abs/2205.09745. Cai, Y ., Wu, J., Mei, S., Lindsey, M., and Bartlett, P. L. Large stepsize gradient descent for non-homogeneous two-layer networks: Margin imp...

work page doi:10.1162/neco.1997.9.1.1 2022
[2]

Then |zt −z ∗| ≥Z ±( 2 l′′(z0 + ϵ

work page
[3]

It follows that Equation (19) holds at timeT=O(log 1 η )

so ˆZ ′(ζt)γtl′′(θt)gt is lower bounded by some constantc. It follows that Equation (19) holds at timeT=O(log 1 η ). Now thatγ t+2 −γ t =O(η 2), so γT ≥γ 0 − X t |γt+2 −γ t| = 2 l′′(z0) +δ−O(η 2 log 1 η ) ≥ 2 l′′(z0) + δ 2 Case 3:|g t| ≤M η 2. Since we are in the EoS regime, the update Equation (9) has a linearly unstable critical point atz∗. Hence the it...

work page