{"id":"f37a6eca-cbaf-41b2-94d9-4389f81cb292","arxiv_id":"2412.08025","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Under specific conditions, gradient descent converges in the unstable edge-of-stability regime for a quadratic loss on a depth-2 diagonal linear network, with a bias bound depending on step size and initialization.","lead":"Gradient descent with unusually large step sizes is supposed to diverge according to classical theory, but this paper shows it can still converge for a simple regression model with a quadratic loss when the predictor is parameterized as the difference of two squared vectors. The significance is that 'edge of stability' behavior is not solely about loss curvature: parameterization matters, and this refines prior claims that only subquadratic losses produce it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorems 1 and 2 are proved for a reduced two-variable system after setting a'_t=b'_t=0, but Lemma 7 controls these auxiliary variables only up to the end of the initial phase; the return to the original 4-variable GD dynamics is never justified.","rationale":"The central claim is that EoS occurs for quadratic loss under the diagonal linear parameterization, with convergence and bias bounds. The strongest evidence is the exact reduction in Lemmas 4–6 and the empirical figures, which support the phenomenon in tested cases. The load-bearing gap is the unjustified passage from the exact 4-variable recurrence to the a′=b′=0 reduced system for all t ≥ t0. Because every convergence-rate statement in Theorems 1 and 2 is proved after this reduction, a failure of the reduction would sever the proof from the model. This is not a disagreement with consensus; it is an internal completeness issue: Lemma 7 bounds auxiliary variables only at one time, and the recurrence allows later growth during positive-residual phases. Lemma 14's appeal to 'negligible' measure is a separate but related weakness; both are under the same umbrella of missing control of the full dynamics. Since the reader already identified exactly this reduction gap and issued CONDITIONAL, my pass does not change the verdict. The concrete test would either reveal a counterexample to the reduction or confirm empirically that the neglected terms stay small; in the latter case the theorem would still need a proof, so the manuscript should not be accepted without an error estimate for a′_t,b′_t over the whole trajectory.","tokens_in":33771,"tokens_out":4782,"duration_ms":46256,"concrete_test":"Fix µ=1, η=1.1, x=0.5, α=0.01 (within Theorem 2) and run both the exact 4-variable GD (Lemma 5) and the reduced 2-variable system with a′=b′=0 for t≥t0 for T=10^6. Record sup_{t≥t0} max(|a′_t|, |b′_t|, |r_t − r_t^{red}|). If this sup exceeds Θ(α^c) for a small c, or the two trajectories diverge, the reduction step is invalid; if it stays small, the theorem still lacks a proof but the gap is empirically benign in this regime. Repeat for ηµ = 0.8 (Theorem 1) and for α=10^{-3} to test the α-scaling.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Appendix A.2 (after Eq. (10)) the proof 'regards a′_t ≡ b′_t ≡ 0 for any t ≥ t0' and all later lemmas (Lemmas 8–17) analyze the reduced (r_t,s_t) system. Lemma 7 bounds a′_τ and b′_τ only at the single time τ at the end of the initial phase, not for t > τ. Under the sign-changing oscillation that defines EoS, the factors (1+η r_t)^2 and (1+x η r_t)^2 in Lemma 5 exceed 1 whenever r_t > 0, so a′_t and b′_t can in principle grow later; no monotonicity or uniform bound is proved. Consequently the linear convergence and bias bounds of Theorems 1 and 2 are established only for the reduced system, not for the GD iterates of the stated model. Lemma 14 also dismisses chaotic or divergent orbits as 'negligible' without a measure-theoretic proof, and Proposition 1 explicitly assumes non-divergence, so the ηµ > 1 regime has a second unresolved gap. (A minor sign: the Lemma 7 line 'a′_τ ≤ Θ(µ/α^4)' is inconsistent with its own product bound a′_τ a_τ ≤ Θ(α^4), a_τ ≥ µ/8, which would give α^4/µ; neither bound extends past τ.)","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies constant-step-size gradient descent on the one-sample linear regression loss L(β)=⟨x,β⟩²/4 under the quadratic parameterization β=w_+²−w_-² (a depth-2 diagonal linear network). It claims that the Edge of Stability can occur even though l is quadratic, contrary to earlier claims that subquadratic growth is necessary. After an empirical study of regimes, the paper restricts to d=2 with x=(1,x), y=µ, β*=(µ,0), initialization w0,±=α1, and states two theorems: for ηµ∈(0,1) and assuming sign-changing residuals for t≥t0, GD converges linearly to an interpolator with ‖β∞−β*‖=O(α^C); for ηµ∈(1,min{(3√2−2)/2,1+1/(4C)}) and |x|<1/(ηµ), the same conclusion holds with ‖β∞−β*‖≤C(ηµ−1). The proof reduces the four-dimensional GD recursion exactly to a quadruplet, then approximates two of the four variables by zero and analyzes a two-dimensional map (r_t,s_t).","tokens_in":34018,"tokens_out":10197,"duration_ms":98167,"significance":"If the proof gaps were closed, the paper would be a significant contribution: it gives a concrete example where a quadratic loss exhibits EoS, an exact algebraic reduction of GD on a diagonal linear network to a low-dimensional map, and falsifiable predictions for the bias (α^C versus ηµ−1 scaling) that differ sharply between the two EoS subregimes. The empirical section is careful and identifies plausible necessary conditions (d≥2, nondegenerate x, y≠0). However, the rigorous bridge from the reduced system to the original GD trajectory is missing, and the existence of the sign-changing regime is assumed rather than proved, so the advertised theoretical result is not yet established.","major_comments":[{"comment":"The reduction from the four-variable GD dynamics to the two-variable system is not justified for t>τ. Lemma 7 bounds a'_τ and b'_τ only at the end of the initial phase, and the proof then sets a'_t ≡ b'_t ≡ 0 for all t≥t0. In the EoS sign-changing regime r_t is positive on alternating steps, so the factors (1+ηr_t)^2 and (1+xηr_t)^2 in Lemma 5 exceed 1 and a'_t,b'_t can in principle grow after τ. No uniform bound on these variables, or on their contribution to the γ_t term in Eq. (9) and to r_t in Lemma 6, is given. Therefore the linear convergence and bias bounds in Theorems 1 and 2 are established only for the reduced (r_t,s_t) system, not for the original GD iterates.","section":"Appendix A.2, Eq. (10)"},{"comment":"Lemma 14 is the load-bearing step for the ηµ>1 regime: it asserts that the (r_t,s_t) system eventually crosses α_t from negative to positive. The proof rules out divergence and chaos by saying their total measure is negligible, but no measure-theoretic or dynamical-systems argument is supplied. Since Proposition 1 explicitly assumes non-divergence and non-chaos, the convergence claim for ηµ>1 remains conditional on exactly the outcome that Lemma 14 is supposed to prove. A rigorous exclusion of chaotic/divergent orbits, or an explicit invariant set with a Lyapunov argument, is needed.","section":"Appendix A.4, Lemma 14"},{"comment":"The contraction computation in Lemma 10 gives |r_{t+2}| ≤ (2−2ηµ)^2 |r_t|, which is a contraction only when ηµ>1/2. The proof then states |r_t| ≤ exp(Θ(2ηµ−1)(t−t0))|r_{t0}|; the exponent changes sign at ηµ=1/2 and is not derived from the preceding factor. If the sign-change assumption excludes ηµ≤1/2 in the EoS regime, that exclusion must be stated and proved. As written, Theorem 1's linear-rate claim for the full stated interval ηµ∈(0,1) is not supported by the proof.","section":"Appendix A.3, Lemma 10 and Theorem 1"},{"comment":"There is an internal inconsistency in Lemma 7: the statement bounds a'_τ by Θ(α^4/µ), while the proof concludes a'_τ ≤ Θ(µ/α^4); the product bound a'_τ a_τ ≤ Θ(α^4) with a_τ≥µ/8 gives a'_τ ≤ O(α^4/µ). Moreover, none of these bounds extends to t>τ. This matters because the smallness of a'_τ is part of the justification for the reduction criticized above; the lemma, as written, does not establish even its stated bound.","section":"Appendix A.2, Lemma 7"},{"comment":"The paper's central claim that 'EoS occurs even when l is quadratic' is not established by the theorems. Theorem 1 assumes r_t r_{t+1}<0 for all t≥t0, which is the defining signature of the EoS trajectory; the proof therefore establishes convergence and bias for trajectories already in EoS, but not that such sign-changing trajectories exist for the stated parameter ranges. The occurrence claim remains an empirical observation (Claim 1). Please separate the conditional convergence result from the existence claim, or prove that the sign-change condition holds for some open set of parameters.","section":"Theorem 1 and abstract"}],"minor_comments":[{"comment":"The initialization line 'a0 = a′0 = b0 = b′0 = µ/(2(1+x²))' should read 'α²/(2(1+x²))'; Lemma 7's proof uses the α² form.","section":"Appendix A.2"},{"comment":"The proof relies on 'Mathematica Symbolic Calculation' for the roots of the 2-periodic system without displaying the polynomial system or verifiable output; please include the algebra or a computer-verifiable derivation.","section":"Appendix A.4, Lemma 13"},{"comment":"The displayed update has an extra factor: it should be 'b_{t+1} = (1−x·η r_t)^2 · b_t > b_t' rather than 'b_{t+1} = (1−x·η r_t)^2 · b_t · b′_t > b_t'.","section":"Lemma 8"},{"comment":"The symbol t used for the start of the convergence phase is visually close to t0 and to the subscript of r_t; please use a distinct symbol such as t_1 to avoid confusion.","section":"Theorem 2"}],"recommendation":"major_revision","confidential_remarks":"The main gaps are substantive but likely addressable: one needs a uniform smallness estimate for a'_t,b'_t, a rigorous exclusion of non-convergent orbits for Lemma 14, and a corrected contraction-rate argument in Lemma 10. Given the novelty of the quadratic-loss EoS claim, I would not reject if these can be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"X, here's my take on 2412.08025. The headline claim—that you can get Edge of Stability with a purely quadratic loss when beta = w_+^2 - w_-^2—is new and, based on the experiments, probably true. The paper does a real service by showing that the subquadratic-loss necessary condition from Ahn et al. and Song & Yun does not apply to parameterized regression. The exact reduction to the four-variable system (Lemma 5) is clean, and the phase-transition argument for ημ>1 is a genuine novelty.\n\nThe softness is in the proofs. The shift from the exact dynamics to the reduced (r,s) system is done by declaring a'_t = b'_t = 0 for t ≥ t0 (Appendix A.2, after Eq. 10). Lemma 7 only bounds those variables at the single time τ at the end of the initial phase. Once r_t changes sign, (1+ηr_t)^2 > 1 during positive r_t segments, so a'_t and b'_t can grow; no uniform bound is given. So Theorems 1 and 2 prove linear convergence for the reduced system, not for the GD iterates of the stated model. That is a real gap, not a stylistic quibble.\n\nLemma 14 is similar: the statement that chaotic/divergent orbits have 'negligible total measure' is asserted, not proved, and Proposition 1 explicitly assumes them away. So the ημ>1 branch has a second unresolved issue. There's also a small inconsistency in Lemma 7: the statement says a'_τ ≤ Θ(α^4/μ), the proof line gives Θ(μ/α^4); one of those is wrong. The Mathematica root computations behind Lemma 13 are not included, which makes verification harder.\n\nNone of this convinces me the central claim is false. The experiments look consistent, and the reduction gap may be fixable with a more careful error bound on (a'_t,b'_t). But as it stands, the paper's main theorems are conditional on an unproven simplification.\n\nWho should read this? People working on EoS theory and implicit bias of diagonal linear networks. It would be a useful contribution once the gap is closed. I'd send it to serious referees and ask them to focus on the dimensional reduction and the chaotic-orbit exclusion.","headline":"Plausible and potentially important claim that quadratic loss can produce EoS, but the main theorems are proved only for a reduced system; the gap is real.","tokens_in":34615,"tokens_out":4154,"would_cite":true,"duration_ms":39449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C30","37N30","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Quadratic losses can trigger Edge of Stability: with $\\beta=w_+^2-w_-^2$, gradient descent at step-sizes above the classical threshold converges linearly to an interpolator, and the bias depends on which side of $\\eta\\mu=1$ the step-size…","keywords":["edge of stability","gradient descent","diagonal linear network","quadratic loss","implicit bias","phase transition","bifurcation","overparameterized regression"],"falsifier":"Run the exact GD iterates for $d=2$, $x=(1,0.5)$, $\\mu=1$, $\\eta=1.1$, $\\alpha=10^{-3}$ and record $r_t=\\langle\\beta_{w_t},x\\rangle-\\mu$ together with $a'_t,b'_t$; if $|r_t|$ does not eventually obey the claimed $\\exp(-\\Theta(\\eta\\mu-1)(t-\\bar t))$ contraction, or if $a'_t+b'_t$ is ever comparable to the envelope of $r_t$ after the initial phase, the reduction behind the theorems fails. A second check: sweep $\\eta\\mu$ upward from $1$ and record the first step-size at which the orbit has period larger than $2$; if that period arises inside the interval below $(3\\sqrt{2}-2)/2$ for $|x|<1/(\\eta\\mu)$, the phase-transition lemma is false.","tokens_in":33505,"feed_emoji":"⚡","tokens_out":17909,"duration_ms":153864,"temperature":0.7,"pith_summary":"Contrary to the belief that a quadratic loss cannot produce the Edge of Stability, this paper shows that it can: when the regression vector is parameterized as $\\beta = w_+^2 - w_-^2$, gradient descent with a step-size above the classical $2/L$ threshold converges to a linear interpolator under explicit conditions. The formal proof works for one-sample regression in $d=2$, with loss $l(s)=s^2/4$, and the experiments indicate the phenomenon requires $d\\ge2$, a nonzero target, and a non-degenerate input. The paper also proves linear convergence rates and bounds the distance $\\|\\beta_\\infty-\\beta^*\\|$ in two regimes, showing that the limit changes character when $\\eta\\mu$ crosses $1$. If correct, the right question for EoS is not whether the loss grows subquadratically, but what the parameterization does to the landscape.","feed_headline":"Quadratic losses can trigger edge-of-stability convergence","feed_subtitle":"A quadratic loss still crosses the 2/step-size threshold, yet GD converges linearly—and the limit depends on step-size.","key_machinery":"The carrying object is the residual iteration obtained from the quadratic parameterization. GD on $\\beta=w_+^2-w_-^2$ is converted exactly into a recurrence for $(a_t,a'_t,b_t,b'_t)$, with $a_{t+1}=(1-\\eta r_t)^2a_t$, $b_{t+1}=(1-x\\eta r_t)^2b_t$, and symmetric updates for the primed variables, where $r_t=(1+x^2)(a_t-a'_t+x(b_t-b'_t))-\\mu$. After an initial phase the primed variables are treated as negligible, reducing the residual update to $r_{t+1}=-(1-\\alpha_t+\\beta_tr_t)r_t$, matching the paper's toy model $r_{t+1}=-(1-\\alpha_t)r_t-\\beta_tr_t^2$. The sign of $\\alpha_t=2-2\\eta(\\mu-c_x b_t)$ controls everything: $\\alpha_t>0$ gives damped oscillation and contraction, while $\\alpha_t<0$ gives envelope expansion. The hard part is proving that for $|x|<1/(\\eta\\mu)$ the map eventually crosses from $\\alpha_t<0$ to $\\alpha_t>0$ and stays there; this is established through a bifurcation analysis of the two-dimensional map $(g,h)$, including a no-2-periodic-orbits lemma that rules out the non-convergent alternatives.","core_discovery":"Under Assumption 1 ($d=2$, $x=(1,x)$, $y=\\mu$, $\\beta^*=(\\mu,0)$, $w_{0,\\pm}=\\alpha\\mathbf{1}$, $l(a)=a^2/4$), the paper proves two theorems. For $\\eta\\mu\\in(0,1)$, once the residual changes sign after some time $t_0$, GD converges linearly, $|\\langle\\beta_{w_t}-\\beta_\\infty,x\\rangle|\\le C_1e^{-\\Theta(\\eta\\mu)(t-t_0)}|\\langle\\beta_{w_{t_0}}-\\beta_\\infty,x\\rangle|$, and $\\|\\beta_\\infty-\\beta^*\\|\\le O(\\alpha^{C_2})$; the sparse prior is recovered as $\\alpha\\to0$. For $\\eta\\mu\\in(1,\\min\\{(3\\sqrt{2}-2)/2,\\,1+1/(4C)\\})$ with $\\alpha^2\\ll\\eta\\mu-1$ and $|x|<1/(\\eta\\mu)$, GD again converges linearly, $|\\langle\\beta_{w_t}-\\beta_\\infty,x\\rangle|\\le C_3e^{-\\Theta(\\eta\\mu-1)(t-\\bar t)}|\\langle\\beta_{w_{\\bar t}}-\\beta_\\infty,x\\rangle|$, and $\\|\\beta_\\infty-\\beta^*\\|\\le C(\\eta\\mu-1)$; here the sparse prior is not recovered even as $\\alpha\\to0$. These results are a formal demonstration that a quadratic loss can sit in the EoS regime.","pith_inferences":["A testable extension of the same phase-transition argument is the $n$-sample diagonal network with $d>n$: one would predict a sharp change in the distance to the sparse prior as the step-size crosses the analogue of $\\eta\\mu=1$, measurable by sweeping $\\eta$ at fixed $\\alpha$.","The $\\eta\\mu>1$ error bound hints that large step-sizes act as an implicit regularizer pushing the solution away from the sparse prior; this could be checked by probing whether test error follows the $\\eta\\mu-1$ scale rather than the $\\alpha$ scale.","The upper limit $(3\\sqrt{2}-2)/2$ in Theorem 2 is a proof artifact, so the real bifurcation boundary between 2-cycles and longer orbits in the reduced $(r,s)$ map should be located by numerical continuation; doing so would yield a sharper conjecture for the maximal convergent step-size.","If one can bound the reduction error from freezing $a'_t,b'_t$ instead of assuming it away, the same scheme would likely extend the convergence proof beyond the stated $\\eta\\mu$ interval and to general $d$, because the hard part is the scalar phase transition rather than the linear algebra."],"forward_implications":["Subquadratic growth of the loss is not required for EoS; a quadratic loss combined with the quadratic parameterization is sufficient, so EoS analyses should track the full parameterized landscape.","For $\\eta\\mu<1$, $\\|\\beta_\\infty-\\beta^*\\|\\le O(\\alpha^{C})$ means the sparse prior is recovered in the $\\alpha\\to0$ limit, extending the small-step-size rich-regime bias into the EoS regime.","For $\\eta\\mu>1$, $\\|\\beta_\\infty-\\beta^*\\|\\le C(\\eta\\mu-1)$ means the sparse prior is not recovered even with vanishing initialization; the step-size itself controls the bias.","In both regimes the residual contracts at a linear rate, so the EoS trajectory converges to a linear interpolator at a quantified speed rather than merely oscillating."],"supporting_citations":[{"why":"defines the EoS phenomenon and the sharpness threshold $2/\\eta$ that the paper studies.","marker":"Cohen et al. (2021)"},{"why":"establishes the subquadratic-loss condition for EoS in the $d=1$ model, the claim this paper challenges and extends.","marker":"Ahn et al. (2022a)"},{"why":"introduces the parameterized linear regression setting and the bifurcation-theory viewpoint that this paper recovers and pushes into $\\eta\\mu>1$.","marker":"Song & Yun (2023)"},{"why":"provides the rich/kernel regime and sparse-recovery baseline for diagonal linear networks against which the bias bounds are compared.","marker":"Woodworth et al. (2020)"},{"why":"analyzes GD dynamics in quadratic regression as a fixed-parameter dynamical system, the contrast that motivates the paper's varying-parameter phase-transition proof.","marker":"Chen et al. (2023)"},{"why":"supplies the third-order-derivative self-stabilization mechanism invoked to explain why a quadratic loss can still show EoS under reparameterization.","marker":"Damian et al. (2022)"}],"fun_headline_variants":["Quadratic loss enters EoS regime, step-size decides sparsity","EoS without subquadratic growth: quadratic loss suffices","Step-size controls sparse recovery in quadratic EoS","Quadratic diagonal nets show edge-of-stability convergence","Sparse prior not always recovered in quadratic EoS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that after a short initial phase the two auxiliary variables $a'_t$ and $b'_t$ can be frozen at zero, so the original GD trajectory is exactly described by the reduced two-variable map; if that reduction leaks error into the residual, or if a chaotic orbit is reachable before contraction, the convergence rates in the theorems do not apply to the actual iterates.","fun_headline_variants_meta":{"raw":{"variants":["Quadratic loss enters EoS regime, step-size decides sparsity","EoS without subquadratic growth: quadratic loss suffices","Step-size controls sparse recovery in quadratic EoS","Quadratic diagonal nets show edge-of-stability convergence","Sparse prior not always recovered in quadratic EoS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1656,"prompt_tokens":1127,"completion_tokens":529,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":743,"completion_tokens_details":{"reasoning_tokens":449}},"tokens_in":743,"tokens_out":529,"duration_ms":5231,"temperature":1.0,"reasoning_tokens":449,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:18:30.390612+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact GD iterates for $d=2$, $x=(1,0.5)$, $\\mu=1$, $\\eta=1.1$, $\\alpha=10^{-3}$ and record $r_t=\\langle\\beta_{w_t},x\\rangle-\\mu$ together with $a'_t,b'_t$; if $|r_t|$ does not eventually obey the claimed $\\exp(-\\Theta(\\eta\\mu-1)(t-\\bar t))$ contraction, or if $a'_t+b'_t$ is ever comparable to the envelope of $r_t$ after the initial phase, the reduction behind the theorems fails. A second check: sweep $\\eta\\mu$ upward from $1$ and record the first step-size at which the orbit has period larger than $2$; if that period arises inside the interval below $(3\\sqrt{2}-2)/2$ for $|x|<1/(\\eta\\mu)$, the phase-transition lemma is false.","supporting_citations":[{"cited_title":"Trajectory alignment: Understanding the edge of stability phenomenon via bifurcation theory","cited_arxiv_id":null,"evidence_quote":"introduces the parameterized linear regression setting and the bifurcation-theory viewpoint that this paper recovers and pushes into $\\eta\\mu>1$."}],"review_version":1}