{"id":"ecf84abc-ed7d-4453-9ad7-8d9e7def3846","arxiv_id":"2607.07623","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"RNC-LM extends geodesic-accelerated Levenberg-Marquardt to arbitrary-order Riemann normal coordinate corrections, reusing the LM matrix factorization for all orders and achieving large speedups on PINN and potential-fitting benchmarks.","lead":"The paper introduces a Levenberg-Marquardt optimizer variant that follows curved paths in parameter space using Riemann normal coordinates, extending geodesic acceleration to arbitrary order. It matters because optimization of nonlinear least-squares problems in scientific ML could become substantially faster and more robust, with a demonstrated 34x wall-clock speedup on a potential-energy-surface fitting task.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The PINN result — the most striking claim — confounds higher-order RNC corrections with the line-search-along-curve step acceptance mechanism; no ablation isolates which factor drives the improvement.","rationale":"The mathematical construction is sound: the recursion in Eq. (25) correctly reuses the damped metric factorization, and the linearization argument (Eq. 24) is exact at the base point. The residual-trajectory interpretation (Appendix A.1) correctly shows that higher-order corrections enforce normality in the moving tangent frame, not just at the expansion point. The classical benchmarks (Rosenbrock, MGH10) provide controlled evidence that both higher order and line search contribute, with clean separation of failure modes. The 34x PES speedup is plausible given that per-iteration costs are dominated by Jacobian evaluation and factorization, which are shared.\n\nHowever, the PINN result is the paper's most impactful empirical claim, and it rests on a comparison that changes two variables simultaneously. The paper itself acknowledges (§III.B) that LM-GA's acceleration-ratio rule keeps λ much larger than RNC-LM's, suggesting that the step acceptance mechanism plays a significant role. The MGH10 results (Table II) already show that the acceptance mechanism accounts for a 3x improvement at 2nd order (887 → 286 iterations). On the PINN benchmark, the improvement is far more dramatic (relative L2 error from ~1e-0 to ~1e-3), making it even more important to isolate the contribution of the RNC corrections from that of the line search.\n\nThe reader's identified weakest assumption (G as a reliable local metric) is reasonable but is mitigated by the trust-region mechanism and empirically tested on rank-deficient problems. The reader also correctly flags single-seed experiments and missing code/data, which compound the concern: without error bars or reproducibility, the dramatic PINN improvement could be a trajectory-specific artifact. The supplementary information (damping/backtracking details) being absent from the arXiv submission further limits assessment of whether the line-search hyperparameters were tuned specifically for RNC-LM.\n\nThe verdict should remain CONDITIONAL. The core algorithmic contribution (recursive shared-matrix RNC construction) is sound and novel. But the experimental support for the most striking claim (PINN recovery) has a confound that a single ablation would resolve. If the 2nd-order ablation shows that line search alone is insufficient, the claim is substantially strengthened and the verdict could move toward ACCEPT. If not, the PINN result needs to be reattributed and the headline claim adjusted.","tokens_in":18963,"tokens_out":5378,"duration_ms":439967,"concrete_test":"Run 2nd-order RNC-LM (i.e., geodesic acceleration with line-search-along-curve acceptance instead of acceleration-ratio control) on the reaction-diffusion PINN benchmark (ν=5, ρ=5) with the same network architecture, collocation points, and initialization as in Table III. If 2nd-order RNC-LM with line search achieves relative L2 error comparable to ~1e-3, then the improvement is driven by the step acceptance mechanism, not the higher-order RNC corrections, and the central claim about arbitrary-order extensions needs reframing. If it fails (error ~1e-1 like LM-GA), then the 3rd-order corrections are essential and the claim is strengthened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's most surprising result is that RNC-LM reduces the PINN relative L2 error from ~1e-0 (LM, LM-GA) to ~1e-3 (Table III), which it attributes to the higher-order RNC geometric corrections. However, the comparison confounds two algorithmic changes: (1) the RNC corrections themselves (3rd-order curve construction) and (2) the line-search-along-curve step acceptance mechanism (trust-region ratio ρ(t) with backtracking, §II.C). LM-GA, the baseline, uses a fundamentally different acceptance criterion — the acceleration-ratio rule (∥a∥/∥v∥ < α) — not line search along its geodesic curve. Figure 2 shows that LM-GA maintains damping λ several orders of magnitude larger than RNC-LM, which the paper attributes to the acceleration-ratio rule being overly conservative. But this means the difference in PINN outcome could be driven by the step acceptance strategy (line search vs. acceleration-ratio) rather than by the higher-order geometric corrections. The paper already demonstrates on MGH10 (Table II) that switching from acceleration-ratio control to line search along a 2nd-order curve reduces iterations from 887 to 286 — a 3x improvement attributable to the acceptance mechanism alone, not to higher-order corrections. Yet on the PINN benchmark, no 2nd-order RNC-LM result is reported. Without this ablation, the attribution of the PINN improvement to 3rd-order RNC corrections (as opposed to the line-search-along-curve mechanism) is unsupported. The reader's concern about G being a reliable local metric is valid but is empirically addressed by the trust-region mechanism and the MGH10 rank-deficient experiments. The more load-bearing gap is the missing ablation on the PINN benchmark.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper introduces RNC-LM, a Levenberg-Marquardt variant that replaces the coordinate-straight update with a finite-order Riemann normal coordinate (RNC) expansion. The central algebraic contribution is a recursive construction (Eq. 25: G c_n = -b_n) where all higher-order correction coefficients reuse the same damped Gauss-Newton factorization, making the additional cost primarily that of extra right-hand-side triangular solves and directional derivatives via automatic differentiation. The method is evaluated on classical nonlinear least-squares benchmarks (generalized Rosenbrock, NIST StRD MGH10), a reaction-diffusion PINN failure-mode benchmark, and a large-scale potential-energy-surface fitting task, reporting substantial improvements over LM and LM-GA including a 34x wall-clock speedup on the PES task.","tokens_in":19238,"tokens_out":1723,"duration_ms":191965,"significance":"The recursive reformulation of the geodesic equation via the first-kind geodesic residual (Eq. 20) is a genuine algorithmic contribution: it makes arbitrary-order geometric corrections practical by reusing a single matrix factorization, avoiding explicit Hessian or Christoffel-symbol construction. The parameter-free derivation (no fitted constants in the recursion) and the residual-trajectory analysis in Appendix A (showing that RNC corrections enforce normality of residual acceleration in the moving tangent frame, not just at the base point) provide clear geometric intuition. The PES fitting result (Table IV) demonstrates practical scalability near the memory limit of full-batch LM on GPU. The PINN result (Table III) is the most striking claim but also the most confounded, as discussed below.","major_comments":[{"comment":"The PINN result — the paper's most surprising claim — confounds higher-order RNC corrections with the line-search-along-curve step acceptance mechanism. On MGH10 (Table II), the paper itself shows that switching from acceleration-ratio control to line search along a 2nd-order curve reduces iterations from 887 to 286 — a 3x improvement attributable to the acceptance mechanism alone, not to higher-order corrections. Yet on the PINN benchmark, no 2nd-order RNC-LM result is reported; only 3rd-order RNC-LM with line search is compared against LM-GA (which uses the acceleration-ratio rule). Figure 2 shows LM-GA maintains damping lambda several orders of magnitude larger than RNC-LM, which the paper attributes to the acceleration-ratio rule being overly conservative. But this means the PINN improvement (relative L2 error from ~1e-0 to ~1e-3) could be driven by the step acceptance strategy (line","section":"§III.B, Table III, Figure 2"},{"comment":"continuation: search vs. acceleration-ratio) rather than by the 3rd-order geometric corrections. Without a 2nd-order RNC-LM-with-line-search ablation on the PINN benchmark, the attribution of the improvement to higher-order RNC corrections is unsupported. This is load-bearing for the central claim that higher-order geometric updates drive the PINN improvement. Adding a single row to Table III for 2nd-order RNC-LM with line search would resolve this.","section":"§III.B, Table III"},{"comment":"No convergence proof or error bound for the finite-order RNC expansion is provided. The trust-region ratio and line search are heuristic mechanisms that detect failure post-hoc but do not guarantee that the higher-order coefficients c_n improve the update when G is ill-conditioned or when the model manifold has high intrinsic curvature. The paper acknowledges this informally but does not provide even a local convergence rate or a bound on the RNC truncation error as a function of step size t and order K. While the experimental evidence is suggestive, the absence of any theoretical guarantee for the finite-order expansion — even a simple bound showing when the K-th order RNC curve is a better approximation to the geodesic than the (K-1)-th order — leaves the method's reliability unclear in regimes not covered by the benchmarks. At minimum, the conditions under which the recursion G c_n = ","section":"§II.C and §III"},{"comment":"continuation: -b_n is well-posed (e.g., when G is invertible and the b_n remain bounded) should be stated explicitly, along with a discussion of what happens when J is nearly rank-deficient so that the higher-order coefficients may amplify noise.","section":"§II.B, Eq. 25"}],"minor_comments":[{"comment":"The abstract states 'reduces the relative L2 error to the order of 1e-3' for the PINN benchmark, but Table III shows values ranging from 4.57e-3 to 7.87e-3. 'Order of 1e-3' is acceptable but borderline; consider stating 'to the low 10^-3 range' for precision.","section":"Abstract"},{"comment":"For nu=20, the LM-GA relative L2 error is reported as 9.97e-2, while all other methods report ~9.98e-1. This appears to be a typo (likely 9.97e-1). Please verify.","section":"§III.B, Table III"},{"comment":"The table caption states 'Since LM and LM-GA do not use this strategy, they are therefore shown only in the full-step column.' The word 'therefore' is redundant. Also, the dash entries for LM and LM-GA under reduced t_min columns could be clarified as 'N/A' rather than em-dashes for readability.","section":"§III.A, Table I"},{"comment":"The trust-region ratio formula for rho(t) is given inline without an equation number. Assigning it an equation number would aid referencing.","section":"§II.C"},{"comment":"The y-axis label for the lambda panel in the top row appears to show values around 9e1 to 9.9e1, which seems inconsistent with the text's claim that LM-GA maintains lambda 'several orders of magnitude larger' than RNC-LM. Please verify axis labels and scaling, or clarify whether the displayed range is a subset of the full optimization trajectory.","section":"Figure 2"},{"comment":"The legend labels read '4rd RNC-LM' and '3rd RNC-LM'; '4rd' should be '4th'.","section":"§III.C, Figure 3"},{"comment":"The sentence beginning 'This whitening step should be viewed as the linear part of the RNC construction. And also clarifies...' contains a sentence fragment starting with 'And also'. Please revise for grammatical correctness.","section":"Appendix A"},{"comment":"References [29] and [32] appear to duplicate [12] and [30] respectively (both are Transtrum & Sethna 2012 and More 1978). Please remove duplicates.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core methodological contribution (the recursive reuse of the damped metric factorization for arbitrary-order RNC corrections) is sound and novel. The major revision is warranted primarily by the missing ablation on the PINN benchmark, which is a localized fix (one additional table row) but is load-bearing for the paper's most prominent claim. The lack of convergence theory is a secondary concern; I would not block publication on that alone given the experimental evidence, but the authors should at least state the well-posedness conditions for the recursion. The self-citation pattern appears normal for this topic area; the cited works by Transtrum, Sethna, and others are the appropriate prior art."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The referee identifies two major issues: (1) the PINN result confounds higher-order RNC corrections with the line-search step acceptance mechanism, and (2) no convergence proof or error bound for the finite-order RNC expansion is provided. On the first point, we agree that a 2nd-order RNC-LM-with-line-search ablation on the PINN benchmark is needed and will add it to Table III. On the second, we agree that explicit well-posedness conditions and a truncation error discussion should be added, but a full convergence proof is beyond the scope of this paper and is left as future work.","responses":[{"response":"The referee is correct that the PINN comparison as currently presented confounds two factors: the step acceptance mechanism (line search vs. acceleration-ratio) and the RNC order (3rd vs. 2nd). We agree that a 2nd-order RNC-LM-with-line-search row in Table III is needed to disentangle these effects. We will run this ablation and add the result to the revised Table III. We note that the MGH10 data already shows that both factors contribute: switching from acceleration-ratio to line search at 2nd order reduces iterations from 887 to 286, and then increasing the RNC order from 2nd to 3rd further reduces from 286 to 142. The PINN ablation will show whether a similar decomposition holds on that benchmark. If 2nd-order RNC-LM with line search already achieves the ~1e-3 relative L2 error, we will revise the text to attribute the PINN improvement primarily to the acceptance mechanism rather than to higher-order corrections. If it does not, the current attribution is supported. Either way, the revised manuscript will present the evidence honestly.","revision_made":"yes","referee_comment":"The PINN result confounds higher-order RNC corrections with the line-search-along-curve step acceptance mechanism. On MGH10, switching from acceleration-ratio control to line search along a 2nd-order curve reduces iterations from 887 to 286 — a 3x improvement attributable to the acceptance mechanism alone. Yet on the PINN benchmark, no 2nd-order RNC-LM result is reported; only 3rd-order RNC-LM with line search is compared against LM-GA (which uses the acceleration-ratio rule). Without a 2nd-order RNC-LM-with-line-search ablation on the PINN benchmark, the attribution of the improvement to higher-order RNC corrections is unsupported."},{"response":"We agree that the manuscript should state explicitly the conditions under which the recursion G c_n = -b_n is well-posed and discuss the truncation error behavior. We will add a paragraph to Section II.B specifying that the recursion requires G to be invertible (guaranteed by the damping term lambda > 0) and that the b_n must remain bounded, which depends on the boundedness of higher-order directional derivatives of the residual map along the trial curve. We will also discuss the near-rank-deficient case: when J is nearly rank-deficient, the damping parameter lambda regularizes G, but the higher-order coefficients c_n may amplify noise in directions where G has small eigenvalues; the trust-region ratio and line search serve as practical safeguards, but we will state this limitation explicitly. Regarding a formal convergence proof or a bound showing that the K-th order RNC curve is a better geodesic approximation than the (K-1)-th order: we agree this would strengthen the paper, but such a result requires careful analysis involving the curvature of the model manifold and the spectral properties of G along the curve. We can provide an informal argument based on the residual-trajectory analysis in Appendix A — specifically, each additional RNC order eliminates the tangential component of residual acceleration in the moving tangent frame at one higher order in t — but a rigorous bound on the truncation error as a function of step size t and order K is beyond what we can establish in this revision. We will add the well-posedness conditions, the noise-amplification discussion, and the informal truncation argument, and we will explicitly acknowledge the absence of a formal convergence guarantee as a limitation.","revision_made":"partial","referee_comment":"No convergence proof or error bound for the finite-order RNC expansion is provided. The trust-region ratio and line search are heuristic mechanisms that detect failure post-hoc but do not guarantee that the higher-order coefficients c_n improve the update when G is ill-conditioned or when the model manifold has high intrinsic curvature. The absence of any theoretical guarantee for the finite-order expansion — even a simple bound showing when the K-th order RNC curve is a better approximation to the geodesic than the (K-1)-th order — leaves the method's reliability unclear in regimes not covered by the benchmarks."}],"tokens_in":19024,"tokens_out":1078,"duration_ms":112797,"standing_objections":["A full convergence proof or rigorous truncation error bound for the finite-order RNC expansion as a function of step size t and order K is not available. We can state well-posedness conditions and provide informal geometric arguments, but a formal guarantee is left as future work."]},"desk_editor":{"model":"glm-5.2","letter":"The core contribution is the recursive construction in Eqs. 20-25: reformulate the geodesic equation as a first-kind residual condition along the residual trajectory, then match Taylor coefficients order by order. Each higher-order coefficient c_n requires solving G c_n = -b_n, where G is the same damped metric already factorized for the LM step. This is a clean, parameter-free derivation — no fitted constants, no ad hoc corrections. The fact that all orders reuse the same Cholesky factorization is the practical payoff: higher-order corrections cost additional triangular solves and JVP evaluations, not new matrix factorizations. Appendix A's analysis of the moving tangent frame gives a genuine geometric justification for why higher-order terms matter beyond the second-order geodesic acceleration of Transtrum & Sethna — the tangential residual acceleration that's normal at the base point acquires tangential components as the tangent space rotates along the step. This is not just adding Taylor terms; it's changing the path in parameter space. That distinction is real and well-explained.","headline":"RNC-LM extends geodesic acceleration to arbitrary order via a clean recursive construction that reuses the LM factorization. The derivation is solid and the algorithmic idea is genuinely new. The main experimental gap is a missing ablation on the PINN benchmark that would isolate the effect of higher-order corrections from the line-search-along-curve acceptance mechanism.","tokens_in":19796,"tokens_out":1087,"would_cite":true,"duration_ms":96119,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"34x faster least-squares fitting via curved optimizer steps","keywords":["Levenberg-Marquardt","Riemann normal coordinates","geodesic acceleration","nonlinear least squares","parameter-effect curvature","physics-informed neural networks","potential energy surface fitting","trust region"],"falsifier":"On a problem where the model manifold has high intrinsic (not parameter-effect) curvature, RNC-LM corrections beyond second order should provide diminishing or no improvement, because the RNC construction eliminates parameter-effect curvature but cannot remove embedding curvature of the manifold itself. If higher-order corrections consistently help even when intrinsic curvature dominates, the mechanism is different from what the paper describes.","tokens_in":19025,"feed_emoji":"📐","tokens_out":1281,"duration_ms":192895,"temperature":0.7,"pith_summary":"The paper introduces RNC-LM, a modification of the Levenberg-Marquardt optimizer for nonlinear least-squares problems. Standard LM computes a descent direction using local geometry but then applies it as a straight-line update in parameter coordinates, which introduces parameterization-dependent curvature errors at finite step sizes. Geodesic acceleration partially fixes this with a second-order correction, but only exactly at the expansion point. RNC-LM extends this correction to arbitrary order by recursively solving a sequence of linear systems that all share the same damped metric matrix as the original LM step, meaning the expensive matrix factorization is computed once and reused for every higher-order correction. The resulting curved update path eliminates parameterization-induced tangential residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with LM's linear prediction. A line search along the constructed curve controls how far the optimizer travels, separating step-length overshoot from curve-approximation failure. The method is tested on classical benchmarks, a reaction-diffusion physics-informed neural network where it recovers physically meaningful solutions that other optimizers miss, and a large-scale potential-energy-surface fitting task where fourth-order RNC-LM achieves a 34-fold wall-clock speedup over standard LM.","feed_headline":"Curved optimizer steps cut least-squares fitting time 34-fold","feed_subtitle":"Replacing straight-line parameter updates with recursive geodesic corrections lets Levenberg-Marquardt follow model geometry, recovering PIN","key_machinery":"The first-kind geodesic residual A(t) = J^T R''(t) + lambda * theta''(t) reformulates the geodesic equation into a form computable via JVP/VJP automatic differentiation without explicit Christoffel symbols or higher-order derivative tensors. The trust-region ratio rho(t) along the RNC curve separates lambda (which controls curve shape) from t (which controls distance traveled), and because higher-order corrections are scaled by t^q, backtracking naturally suppresses unreliable high-order terms faster than the first-order LM displacement.","core_discovery":"The central algebraic structure is Equation 25: G c_n = -b_n, a recursion where each higher-order correction coefficient c_n is found by solving a linear system with the same left-hand matrix G (the damped Gauss-Newton metric already factorized for the standard LM step), and the right-hand side b_n is computed via automatic differentiation along the already-constructed lower-order trial curve. This makes arbitrary-order geodesic corrections computationally cheap — additional triangular solves plus directional derivatives — while the resulting update curve enforces the geodesic condition not just at the base point but progressively along the step, eliminating tangential residual acceleration,","pith_inferences":["The separation of lambda (curve shape) from t (step length) implies an adaptive order selection strategy: if the line search consistently accepts very small t values, the curve order K is too low and should be increased; if t near 1 is accepted but convergence stalls, the metric itself is unreliable and lambda should change. The paper hints at this but does not formalize an automatic order-selecti","The observation that RNC-LM follows qualitatively different trajectories than LM-GA on the PINN benchmark — avoiding collocation overfitting with less damping — suggests the curved path may implicitly regularize by biasing the optimizer toward solutions whose residual trajectory is geometrically consistent, which is a form of inductive bias distinct from explicit regularization but not analyzed as","If parameter-effect curvature dominates intrinsic curvature by orders of magnitude in sloppy models, and RNC-LM systematically removes parameter-effect curvature order by order, then there should exist a problem-dependent crossover order beyond which further RNC corrections yield diminishing returns because only intrinsic curvature remains. Characterizing this crossover could provide a principled "],"forward_implications":["The recursive structure G c_n = -b_n is agnostic to how G is represented, so it could be paired with approximate metric factorizations (e.g., Kronecker-factored or Nyström-preconditioned) to extend higher-order geometric corrections to models with millions or billions of parameters where exact LM is infeasible.","If the first-kind geodesic residual formulation transfers from pullback metrics of residual maps to Fisher information metrics, the same recursion could yield higher-order finite-step natural-gradient or stochastic-reconfiguration methods for variational optimization.","The PINN result — where RNC-LM avoids collocation overfitting that traps standard LM and LM-GA — suggests that the failure mode is partly an optimizer-pathology rather than purely a loss-landscape or expressivity problem, and that changing the finite-step trajectory through parameter space can steer training away from spurious minima.","For potential-energy-surface fitting, the 34x speedup directly impacts computational chemistry workflows where neural network potentials are trained on ab initio data; reducing wall-clock time from ~387 hours to ~11 hours could make higher-accuracy or larger-molecule fits practical."],"fun_headline_variants":["Higher-order geodesic updates speed up Levenberg-Marquardt 34-fold","RNC-LM improves curved-valley convergence and speeds up fitting 34-fold","Geometric LM updates cut machine-learning potential fitting time 34-fold","Recursive geodesic corrections recover PINN solutions, speed up LM 34-fold","Finite-step geodesic LM cuts large-scale least-squares time 34-fold"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The entire recursive construction assumes that the damped Gauss-Newton metric G = J^T J + lambda I, computed at the current iterate, is a sufficiently accurate local geometry for building higher-order corrections along a finite step. If the model manifold has high intrinsic curvature or the Jacobian is nearly rank-deficient so that G is ill-conditioned, the higher-order coefficients may amplify noise rather than improve the update, and the paper provides no formal convergence","fun_headline_variants_meta":{"raw":{"variants":["Higher-order geodesic updates speed up Levenberg-Marquardt 34-fold","RNC-LM improves curved-valley convergence and speeds up fitting 34-fold","Geometric LM updates cut machine-learning potential fitting time 34-fold","Recursive geodesic corrections recover PINN solutions, speed up LM 34-fold","Finite-step geodesic LM cuts large-scale least-squares time 34-fold","Riemann-normal coordinates accelerate nonlinear least-squares 34-fold"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1321,"prompt_tokens":626,"completion_tokens":695,"prompt_tokens_details":null},"tokens_in":626,"tokens_out":695,"duration_ms":48916,"temperature":1.0,"reasoning_tokens":560,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T04:44:13.273410+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"On a problem where the model manifold has high intrinsic (not parameter-effect) curvature, RNC-LM corrections beyond second order should provide diminishing or no improvement, because the RNC construction eliminates parameter-effect curvature but cannot remove embedding curvature of the manifold itself. If higher-order corrections consistently help even when intrinsic curvature dominates, the mechanism is different from what the paper describes.","supporting_citations":[],"review_version":1}