{"id":"038bf2a3-51e7-4543-a45b-63b4b227f501","arxiv_id":"2607.08380","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Large-step GD near a flat-minima manifold of overparametrised least squares has a normal form that yields subcritical, critical, and supercritical convergence theorems, including for deep matrix factorisation.","lead":"This paper proves that large-step gradient descent near a manifold of flat minima in overparametrised least squares behaves like Riemannian gradient descent on sharpness, with three precise convergence regimes. It generalises prior codimension-1 theory to vector outputs and manifolds, and shows deep matrix factorisation fits the framework.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"Critical-regime Θ(t^{-1/2}) rate for general F still rests on unproven Conjecture D.7 (plus D.12) except when ∇²_M λ1|νF is a scalar multiple of the identity.","rationale":"The reader correctly isolates the only place where the paper’s strongest claim is not fully proved: the general critical-regime rate. The normal form, the singular-PDE construction, the sub- and super-critical theorems, and the matrix-factorisation geometry (fibre-bundle structure of F, Morse-Bott property of λ1) are all derived under the stated geometric hypotheses and are supported by the detailed appendices and the numerical checks for L=2,3. Because the paper itself flags the dependence on Conjecture D.7 / Assumption D.12, the appropriate verdict remains CONDITIONAL rather than unconditional ACCEPT; no stronger objection (e.g., a flaw in the PDE solution or a failure of the normal-form hypotheses on matrix factorisation) is visible. The concrete numerical test above would settle whether the remaining gap is merely technical or actually obstructs the rate.","tokens_in":63250,"tokens_out":743,"duration_ms":16474,"concrete_test":"Take the abstract map T of (D.206–D.214) with non-constant A(z) (e.g., A(z)=diag(α+εz_1,β) for small ε>0, α>a) on a low-dimensional domain (dz=1,dv=1,du=1). Numerically integrate many orbits starting near W_ϕ and test whether the v-coordinate decays strictly faster than the (u,y1) coordinates at the predicted rate while remaining tangent to a Lipschitz foliation; if the observed decay fails to factor through a Lipschitz leaf structure, Conjecture D.7 is false in the setting needed by Thm D.13 and the general critical claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim packages three regimes under Assumptions 3.1–3.5. Subcritical (Thm 5.1/D.5) and supercritical (Thm 5.3/D.15) are fully proved from the C¹ normal form (Thm 4.1/B.7) after the centre-manifold reduction and singular-PDE solution (Thm C.4). The critical claim (Thm 5.2/D.13) is not: after reducing via Fermi coordinates and the bottom-eigenspace frame, the system is put into the abstract form (D.206–D.214); the Θ(t^{-1/2}) rate then requires either that ∇²_M λ1|ν_M F is a constant multiple of the identity, or both Assumption D.12 (constant smallest eigenvalue of constant multiplicity, local invariance of the bottom-eigenspace–ν1 submanifold, and z-independence of the (u,y1) updates) and Conjecture D.7 (existence of a Lipschitz strong-stable foliation for the normally parabolic system on V_ρ ∩ {y_{2:q}=0}). The paper states this reduction explicitly and verifies D.12 only for two-layer matrix factorisation (Prop. E.7); the foliation conjecture is left open. Without it the general critical rate is conditional, exactly as the reader notes. No other gap undermines the normal form or the other two regimes.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper extends the large-step-size GD analysis of MacDonald et al. (2025) from codimension-1 least squares with an isolated flat minimum to arbitrary-codimension overparametrised least squares near a manifold F of flat minima. Under geometric Assumptions 3.1–3.5 it constructs a C^{1} normal form (Theorem 4.1 / B.7) in which the dynamics split into Riemannian GD on sharpness along the solution manifold M (step size controlled by y_{1}^{2}), a flip bifurcation in the top Hessian eigendirection y_{1}, and linear contraction in the remaining normal directions y_{2:q}. From this form it proves three local convergence regimes: subcritical exponential convergence to a suboptimal flat minimiser (Theorem 5.1), critical Θ(t^{-1/2}) convergence to F (Theorem 5.2, under extra spectral/invariance hypotheses or Conjecture D.7), and supercritical exponential convergence to a period-2 orbit of amplitude Θ((ηλ_{1}|F−2)^{1/2}) along span(ν_{1}|F) (Theorem 5.3). The framework is verified for deep matrix factorisation, where F is shown to be a fibre bundle over a product of spheres and λ_{1} is Morse-Bott along F (Appendix E).","tokens_in":63636,"tokens_out":1097,"duration_ms":9294,"significance":"The work supplies a rigorous dynamical-systems account of large-step GD near manifolds of flat minima for vector-valued least squares, substantially enlarging the scope of the earlier codimension-1 theory. The normal form, the centre-manifold reduction, and especially the novel solution of the singular PDE (Theorem C.4) are technically substantial and may be of independent interest. The matrix-factorisation structural results (fibre-bundle description of F, Morse-Bott property of sharpness) are new and concrete. Subcritical and supercritical theorems are fully proved from the normal form; numerical experiments on 2\times2 three-layer factorisation are consistent with all three regimes. The main limitation is that the general critical-rate claim remains conditional on an unproven parabolic foliation conjecture (or a restrictive spectral hypothesis).","major_comments":[{"comment":"Theorem 5.2 / D.13: the claimed Θ(t^{-1/2}) rate for general F is unconditional only when ∇^{2}_M λ_{1}|ν_M F is a constant multiple of the identity. Otherwise the proof reduces the system to the abstract form (D.206)–(D.214) and then invokes both Assumption D.12 (constant smallest eigenvalue of constant multiplicity, local invariance of the bottom-eigenspace–ν_{1} submanifold, z-independent (u,y_{1}) updates) and the unproven Conjecture D.7 (Lipschitz strong-stable foliation for the normally parabolic system). Assumption D.12 is verified only for two-layer matrix factorisation (Proposition E.7); the foliation conjecture is left open. The manuscript should either prove Conjecture D.7, restrict the critical theorem to the cases already covered, or state the critical claim more carefully as conditional.","section":null},{"comment":"Section 6 and the abstract present the three convergence theorems as parallel generalisations of [39]. Because the critical regime is not fully proved for general manifolds of flat minima, the packaging overstates the unconditional reach of the theory. A clearer separation between the fully rigorous subcritical/supercritical results and the conditional critical result would better match the proofs.","section":null}],"minor_comments":[{"comment":"Assumption 3.5 (positivity and vanishing derivative of α_η on F) is stated abstractly; a short geometric interpretation of why these conditions are natural for least-squares models would help the reader.","section":null},{"comment":"Figures 3–5 show only 2\times2 three-layer factorisation. A brief remark on whether the same qualitative behaviour is expected for larger dimensions or deeper factorisations would strengthen the experimental section.","section":null},{"comment":"The centre-manifold preprint [38] is cited as arXiv:2604.18202; ensure the reference remains accessible or supply the needed statement inline if the preprint is not yet public.","section":null},{"comment":"Notation for the normal-form coordinates (x,y_{1},y_{2:q}) and the successive maps Φ_{1}–Φ_{4} is dense; a short schematic of the coordinate changes would improve readability of Section 4 / Appendix B.","section":null}],"recommendation":"major_revision","confidential_remarks":"The technical core (normal form, singular PDE, subcritical/supercritical theorems, matrix-factorisation geometry) is solid and of genuine interest for a theory-oriented ML or dynamical-systems venue. The only load-bearing gap is the conditional status of the general critical theorem; once that is clarified or restricted, the paper is close to acceptance. The self-citation of the authors’ earlier codimension-1 work and centre-manifold preprint is appropriate and not circular."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a careful, technical extension of the authors’ earlier codimension-1 edge-of-stability normal form to arbitrary-codimension least squares near a manifold of flat minima. The main new pieces are the C1 normal form (Thm 4.1/B.7) after centre-manifold reduction, a genuinely new method for the singular PDE when the singularity sits along a manifold (Sec. C / Thm C.4), and the three local convergence regimes. Subcritical and supercritical are fully proved from the normal form. The matrix-factorisation application is also new and clean: flat minima form a fibre bundle over a product of spheres, and sharpness is Morse-Bott along it (App. E).\n\nWhat works: the geometric hypotheses are stated clearly, the coordinate changes are written out, and the numerics for 3-layer 2\times2 factorisation match the three predicted regimes. Self-citation of the prior paper and the centre-manifold preprint is foundational, not circular. The local character of the results is acknowledged; they do not claim to explain progressive sharpening.\n\nThe soft spot is exactly the one the stress-test flags and the paper itself flags. The general critical Θ(t^{-1/2}) rate (Thm 5.2/D.13) needs either that the Hessian of λ1 on the normal bundle is a constant multiple of the identity, or both Assumption D.12 and the unproven Conjecture D.7 (Lipschitz strong-stable foliation for a normally parabolic system). D.12 is verified only for two-layer factorisation. Everything else stands without that conjecture. That is a real but limited gap, not a collapse of the contribution.\n\nWho it is for: people working on dynamical-systems analyses of large-step GD, edge-of-stability, and the geometry of flat minima in overparametrised least squares / matrix factorisation. A serious referee should see it; the proofs are detailed enough to check, and the conditional status of the critical case is already transparent. I would engage with the normal form and the matrix-factorisation geometry; I would treat the general critical rate as conditional until the foliation is settled.","headline":"Solid local normal-form extension of large-step GD theory to vector outputs and flat-minima manifolds; critical rate is conditional on a stated parabolic-foliation conjecture except in special spectral cases.","tokens_in":64236,"tokens_out":539,"would_cite":true,"duration_ms":8872,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Near a manifold of flat minima, large-step gradient descent splits into sharpness descent along the solution set plus a flip bifurcation and contraction in the normal directions, with three distinct convergence regimes.","keywords":["gradient descent","edge of stability","sharpness","flat minima","normal form","overparametrised least squares","matrix factorisation","Morse-Bott"],"falsifier":"Run large-step gradient descent on a multi-output least-squares problem whose flat minima form a manifold that violates the constant-multiplicity or invariance hypotheses of Assumption D.12 and check whether the critical-regime distance to the manifold still decays as Theta(t^{-1/2}); a clear power different from -1/2 would refute the general claim.","tokens_in":64073,"feed_emoji":"📉","tokens_out":965,"duration_ms":9408,"temperature":0.7,"pith_summary":"Classical gradient-descent theory demands a step size smaller than twice the reciprocal of the sharpness, yet deep networks routinely violate that bound and still train. This paper shows that, for overparametrised least-squares problems of any output dimension, the same large-step dynamics remain well-behaved once one is near a manifold of flat minima rather than an isolated flat point. In suitably chosen coordinates the iteration becomes Riemannian gradient descent on the sharpness along the solution manifold, a flip bifurcation in the top Hessian eigendirection, and linear contraction in the remaining normal directions. The relative size of the step size to the flat-manifold sharpness then produces three regimes: exponential approach to a suboptimally flat minimiser, polynomial approach to the flat manifold itself, or exponential approach to a stable period-two orbit. The same geometric hypotheses hold for deep matrix factorisation, where the flat minima form a fibre bundle over a product of spheres and the sharpness is Morse-Bott. The result therefore supplies a local dynamical picture that covers the multi-output, multi-observation setting actually used in practice.","feed_headline":"Large-step GD near flat minima follows a clean three-regime law","feed_subtitle":"Normal form splits dynamics into sharpness descent, flip bifurcation and contraction for multi-output least squares","key_machinery":"The C1 normal form of Theorem 4.1 (and its precise version Theorem B.7), obtained by successive tubular, centre-manifold, singular-PDE and strong-stable-foliation coordinate changes; it makes the three-regime analysis possible.","core_discovery":"Under four geometric assumptions on an overparametrised least-squares loss, large-step gradient descent near a manifold of flat minima admits a C1 normal form that decouples into Riemannian gradient descent on sharpness along the solution manifold, a cubic flip bifurcation in the top eigendirection, and exponential contraction in the remaining normal directions; the three classical step-size regimes relative to twice the reciprocal of the flat-manifold sharpness then yield the three corresponding convergence theorems.","pith_inferences":["Because the normal form is local to the flat manifold, progressive sharpening and the bulk of the edge-of-stability phase remain outside its present scope and will need a global geometric theory.","The fibre-bundle description of flat minima in matrix factorisation suggests that not all flat points are equivalent for generalisation; distinguished sub-bundles may be statistically preferred.","The novel singular-PDE technique used to straighten the centre manifold may be reusable for other resonant bifurcations that occur along submanifolds rather than isolated points."],"forward_implications":["Local large-step training dynamics near flat minima are governed by sharpness descent plus a one-dimensional flip bifurcation, independent of output dimension.","Deep matrix-factorisation landscapes possess an explicit fibre-bundle structure of flat minima over products of spheres, with Morse-Bott sharpness.","Subcritical, critical and supercritical step-size choices produce, respectively, exponential suboptimal-flat convergence, t^{-1/2} approach to the flat manifold, and exponential approach to a period-two orbit.","The same normal-form programme can in principle be applied to other overparametrised models once the four geometric assumptions are verified."],"fun_headline_variants":["Large-step GD near flat-minima manifold admits three-regime normal form","Normal form splits large-step GD into sharpness flow flip and contraction","Large-step GD dynamics near flat minima follow three classical regimes","GD with large steps near flat manifold: Riemannian descent then flip","Three-regime law for large-step GD on multi-output flat-minima manifolds"],"cache_read_input_tokens":49280,"weakest_assumption_plain":"The general critical-regime rate proof needs either a constant multiple of the identity for the Hessian of sharpness normal to the flat manifold, or an invariance assumption plus an unproven parabolic stable-foliation conjecture.","fun_headline_variants_meta":{"raw":{"variants":["Large-step GD near flat-minima manifold admits three-regime normal form","Normal form splits large-step GD into sharpness flow flip and contraction","Large-step GD dynamics near flat minima follow three classical regimes","GD with large steps near flat manifold: Riemannian descent then flip","Three-regime law for large-step GD on multi-output flat-minima manifolds"]},"model":"grok-4.5","effort":"low","cost_usd":0.005152,"raw_usage":{"total_tokens":1480,"prompt_tokens":836,"num_sources_used":0,"completion_tokens":99,"cost_in_usd_ticks":51520000,"prompt_tokens_details":{"text_tokens":836,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":545,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":836,"tokens_out":99,"duration_ms":5741,"temperature":1.0,"reasoning_tokens":545,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T08:23:32.586260+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run large-step gradient descent on a multi-output least-squares problem whose flat minima form a manifold that violates the constant-multiplicity or invariance hypotheses of Assumption D.12 and check whether the critical-regime distance to the manifold still decays as Theta(t^{-1/2}); a clear power different from -1/2 would refute the general claim.","supporting_citations":[],"review_version":1}