{"id":"a33c278e-0b0c-4786-bd02-5aeef67cd4ce","arxiv_id":"2505.03552","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PeN-ODE training is reformulated as a direct collocation NLP, giving fast, stable training of small neural surrogates on two benchmark systems.","lead":"This paper trains physics-enhanced neural ODEs by converting the training problem into a large nonlinear optimization solved with implicit collocation and the Ipopt solver. On a quarter-vehicle model and a Van-der-Pol oscillator it reports laptop-level training in minutes or seconds with small neural components.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed pre-specified collocation grid without adaptive refinement is the load-bearing assumption; the two smooth, hand-gridded benchmarks do not test the claimed general stability and step-size advantage over ODE-solver training.","rationale":"The paper is a clear, honest engineering contribution. The formulation in Sections 2.3 and 2.4 is standard, the implementation is open source, and the two demonstrations are real evidence that on smooth, small-scale PeN-ODEs the method trains quickly. I agree with the reader that the weakest assumption is the adequacy of the fixed collocation grid and collocation order. Section 2.5.1 states the grid must be given a-priori and that adaptive mesh refinement is future work, so the method has no built-in error control at training time. The QVM grid is explicitly hand-chosen at 2500 intervals because of fast dynamics, and the Van-der-Pol problem is smooth. The method is therefore not tested on the stiff or localized-event problems where ODE-solver training is most painful and where the allowable step size claim would matter. The Appendix high-noise failure modes reinforce that NLP robustness is not automatic. I do not see an internal inconsistency; the concern is a scope and robustness limitation. The missing direct comparison to [12] on identical hardware is a real but secondary issue: it affects how strongly the speed and accuracy claim can be stated, whereas the grid issue affects whether the proposed method works in the regime the paper claims to address. The paper also flags grid selection, initialization, batch-wise training, and larger networks as key challenges, and those self-reported limitations corroborate the conditional verdict. For these reasons the verdict should remain CONDITIONAL, which means no change from the reader's assessment.","tokens_in":16405,"tokens_out":7473,"duration_ms":72665,"concrete_test":"Train a Van der Pol NODE with mu=100 (stiff) or a problem with a localized fast transient using the same 500-interval, 5-node fLGR grid from Section 4.2, and compare against training with an adaptive hp-collocation solver (for example GPOPS-II or CasADi) or an adaptive implicit ODE-solver trainer on identical data and hardware. If the fixed-grid run misses the fast region unless the grid is manually refined, or requires far more intervals than the adaptive run to reach the same trajectory error, then the claimed general efficiency and stability advantage is conditional on smooth problems with favorable grids.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Sections 2.3-2.4 and Conclusion) is that simultaneously optimizing states and parameters with high-order Radau collocation overcomes ODE-solver-based training in order, stability, accuracy, and allowable step size. For that claim to hold generally, the fixed discretization must resolve the true dynamics. Section 2.5.1 acknowledges this directly: the collocation scheme and grid are embedded into the NLP and must be given a-priori, with mesh refinement left to future work. The empirical evidence is restricted to two smooth problems with hand-selected grids: a 2500-interval equidistant grid chosen because the QVM contains very fast dynamics (Section 4.1.2), and 500 intervals for the Van-der-Pol oscillator (Section 4.2). This does not test the regime in which ODE-solver training is hardest, namely stiff dynamics or localized fast transients. If the grid misses such a region, the trained surrogate is wrong; if the user makes the uniform grid fine enough everywhere, the claimed runtime advantage can disappear. The Appendix sensitivity analysis further shows that under high noise several runs converge to poor local optima or fail to converge, so NLP robustness on a fixed grid is not guaranteed even for the benchmark problems. This is a scope limitation, not an internal inconsistency, but it makes the central claim conditional.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes training Physics-enhanced Neural ODEs (PeN-ODEs) by expressing the training problem as a dynamic optimization problem and transcribing it into a large-scale NLP via direct collocation at flipped Legendre-Gauss-Radau points. The state trajectories and neural-network parameters are optimized simultaneously with Ipopt, using a custom, parallelized open-source implementation (an extension of GDOPT). The method is demonstrated on a quarter-vehicle model with two neural force surrogates and on a Van-der-Pol oscillator learned as a pure NODE, reporting training times of minutes or seconds on a laptop, good surrogate accuracy, generalization to an unseen road profile, and a 100-run sensitivity analysis to measurement noise. The paper also outlines an intended integration into OpenModelica for Neural DAE training. The central claim is that this collocation-NLP formulation overcomes key limitations of ODE-solver-based training in terms of order, stability, accuracy, and allowable step size.","tokens_in":16696,"tokens_out":5501,"duration_ms":54976,"significance":"If the claims hold, this is a practical contribution to the training of small PeN-ODEs: it gives an alternative to ODE-solver backpropagation that is fast, stable on smooth problems, supports hard physical constraints such as zero crossings, and is accompanied by a publicly available, parallelized implementation. The mathematical transcription from DOP to NLP is standard and appears correctly presented, and the open-source code and reproducible experiments are concrete strengths. However, the demonstrated advantages are conditional on fixed, hand-selected collocation grids that resolve the dynamics, and the speed comparison against ODE-solver-based training is not a controlled one. The paper is transparent about the grid-selection limitation in Section 2.5.1, but the abstract and conclusion state the advantage in more general terms than the evidence supports.","major_comments":[{"comment":"The conclusion states that the approach \"overcomes key limitations of ODE solver-based training in terms of order, stability, accuracy, and allowable step size,\" but the experiments use only fixed, hand-chosen equidistant grids: 2500 intervals for the quarter-vehicle model (Section 4.1.2) and 500 intervals for the Van-der-Pol oscillator (Section 4.2). Section 2.5.1 explicitly says that the collocation scheme and grid are embedded into the NLP and must be given a-priori, with adaptive mesh refinement left to future work. Consequently, the step-size and stability advantage is conditional on the grid resolving the true dynamics; for stiff dynamics or localized fast transients the fixed discretization can miss behavior, and a uniformly fine grid may erase the runtime advantage. Please qualify the central claim to the resolved-grid, smooth regime, or add a stiff/localized-dynamics experiment or an adaptive mesh refinement study.","section":"Section 2.5.1 and Conclusion"},{"comment":"The speed comparison against ODE-solver-based training is not controlled. The reported 4.5 hours from [2] were obtained on different hardware, with different network sizes, and in a different paper; Table 1 gives only absolute training times on a laptop. The claim of \"superior accuracy, speed, generalization\" compared with other training techniques would be substantially strengthened by a head-to-head baseline using ODE-solver-based backpropagation on the same machine, network architecture, data, and initialization. Without such a baseline, the runtime advantage is indicative but not directly demonstrated.","section":"Section 4.1.3 and Table 1"},{"comment":"The robustness claim \"even under severe noise\" rests on a single high-noise run in Section 4.2.1, while the Appendix's 100-run sensitivity analysis shows that at sigma = 0.5 several runs converge to poor local optima or fail to converge, producing period mismatches or trajectory collapse. The main text should report the failure rate and the median/quantile bands from Figure 10, and should temper the statement that the method is robust under high noise, since the single displayed run is not representative of the distribution of outcomes.","section":"Section 4.2.1 and Appendix (Figure 10)"}],"minor_comments":[{"comment":"The relation between the interval-local Lagrange polynomials l_j(t) in Eq. (7) and the reference polynomials l_tilde_k(tau) in Eq. (9) is implicit; explicitly stating l_j(t) = l_tilde_j((t - t_i)/Delta t_i) would remove ambiguity.","section":"Section 2.3, Eq. (8)-(9)"},{"comment":"The table heading \"Total Ipopt Callbacks #Epochs\" is ambiguous because it is unclear which columns are times and which are counts; adding units such as \"Total [s], Ipopt [s], Callbacks [s], #Epochs\" would improve readability.","section":"Section 4.2, Table 3"},{"comment":"The phrase \"the optimization terminates prematurely, since the optimality tolerance is fulfilled\" is contradictory; \"prematurely\" should be replaced by wording such as \"terminates early\" or \"terminates at a local optimum\" to reflect that termination is due to optimality, not an error.","section":"Section 4.2.1"},{"comment":"The statement that OpenModelica's index reduction and BLT transformations \"restructure DAEs into semi-explicit ODE form with index 1\" is imprecise; the standard result is a transformation into a semi-explicit index-1 DAE form, not generally into an ODE form.","section":"Section 5.3"},{"comment":"The road input derivative dot z_r = u appears in the model equations, but the link between the ISO 8608 road profile generation and the input u is not formally specified in the data-generation paragraph; a brief statement would clarify the setup.","section":"Section 4.1.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the core algorithmic idea is sound and standard. The main concern is that the abstract and conclusion make broader claims than the evidence supports, particularly about stability and allowable step size, given the fixed a-priori grids and the absence of a controlled ODE-solver baseline. The paper's own limitations section is honest, but the claims should be aligned with the scope of the experiments. A direct baseline and a stiff or localized-dynamics test would resolve most of my concerns. No concerns about citation practice; the related work is credited appropriately."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: this is a competent and honest paper. It extends Shapovalova and Tsay's global Chebyshev collocation idea for NODEs to physics-enhanced NODEs, using local Radau IIA collocation, hard physical constraints, a high-order quadrature loss, and an open-source parallel implementation. The math is standard and correctly transcribed. The code is public, and the appendix sensitivity analysis (100 runs per noise level) is a genuine robustness check. That is real evidence, and the authors deserve credit for it.\n\nWhat's new: the constraint handling (zero-crossing constraints on neural force elements), the rational-function surrogate experiment, and the parallel callback implementation. The VdP comparison to [12]'s global method is actually included on the same hardware, which the reader's report misses—though it is only for the VdP case and is presented as an aside. The QVM results are the more interesting application, but there the comparison to [2] is mostly qualitative: one unseen road profile, no error bars, no repeated runs. That is the softest spot.\n\nThe bigger issue is scope. The paper claims to overcome ODE-solver training in order, stability, and allowable step size, but the evidence is restricted to two smooth problems with hand-picked grids (2500 intervals for the QVM because of fast dynamics, 500 for VdP). The fixed grid is explicitly acknowledged in Section 2.5.1 as a limitation, and mesh refinement is deferred to future work, so the authors are not hiding it. But the central claim remains conditional: stiff or event-rich dynamics could defeat a fixed uniform grid, and the runtime advantage could disappear if the grid has to be fine everywhere. The stress-test note is fair, and the conditional verdict is about right.\n\nWho this is for: people working on NODE/PeN-ODE training who want an alternative to adjoint backprop, and Modelica folks interested in the planned OpenModelica integration. It deserves a serious referee, but a revision should add at least one stiff or multi-scale benchmark and quantitative repeated-run reporting for the QVM.\n\nRecommendation: send it to peer review. I would not desk-reject this.\n\nBest.","headline":"A solid, honest extension of direct-collocation NODE training to physics-enhanced models; the central stability claim is plausible but only tested on smooth, hand-gridded benchmarks.","tokens_in":17216,"tokens_out":1831,"would_cite":true,"duration_ms":18860,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65L60","65L06","49M37","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that training physics-enhanced neural ODEs can be recast as one large nonlinear program solved by direct collocation, cutting training time from hours to minutes while matching reference dynamics.","keywords":["physics-enhanced neural ODEs","direct collocation","nonlinear programming","Radau IIA collocation","simultaneous optimization","neural ordinary differential equations","dynamic optimization","trajectory optimization"],"falsifier":"Train the same architecture on a stiff or multi-timescale system, e.g. the Van-der-Pol oscillator at large $\\mu$ or a suspension with a fast tire mode, using the paper's fixed 500-interval, five-point Radau grid; if the learned surrogate fails to reproduce a validation trajectory through the fast region, or works only after hand-refining the grid, the claimed step-size advantage over solver-based training would not hold outside smooth benchmarks.","tokens_in":16189,"feed_emoji":"🧮","tokens_out":7526,"duration_ms":70434,"temperature":0.7,"pith_summary":"The paper claims that training physics-enhanced neural ODEs can be cast as a single dynamic optimization problem and solved by direct collocation: discretize both the known physical dynamics and the embedded neural network on a high-order implicit Runge-Kutta grid, then let a nonlinear optimizer adjust state trajectories and network parameters simultaneously. This avoids repeated forward simulation and backpropagation through an ODE solver, which the paper argues is the main bottleneck of current training. If the claim is right, small hybrid surrogates that combine first-principles models with learned force terms can be trained on a laptop in minutes while matching reference dynamics, as demonstrated on a quarter-vehicle suspension model and a Van-der-Pol oscillator. It also makes physical knowledge, such as a learned force vanishing at zero displacement, a hard constraint rather than a penalty.","feed_headline":"Physics-enhanced neural ODEs train in minutes, not hours","feed_subtitle":"Collocation turns the learning problem into one nonlinear program optimizing states and weights together.","key_machinery":"The load-bearing object is the Radau IIA collocation scheme at flipped Legendre-Gauss-Radau points, embedded as equality constraints in a nonlinear program. On each interval the states are Lagrange polynomials that must satisfy the ODE at the collocation nodes, while the loss integral is evaluated with the matching Radau quadrature. The central mechanism is simultaneous optimization: the NLP variables include every discretized state value plus the network parameters, so a single interior-point solver updates trajectories and weights together using analytic first and second derivatives of the sparse constraint system, rather than propagating gradients through an ODE solver.","core_discovery":"On its own terms, the central discovery is that the PeN-ODE training problem—minimize an integral loss subject to the augmented differential equation—is an instance of a dynamic optimization problem, and that transcribing it with flipped Legendre-Gauss-Radau collocation (equivalently Radau IIA, an $A$-, $B$-, and $L$-stable method of order $2m-1$) yields a large, sparse nonlinear program in which the state values at all collocation nodes and the network weights are unknowns. Solving that program with an interior-point optimizer gives simultaneous access to first and second derivatives and lets the optimizer leave intermediate iterates infeasible, which the paper credits with faster, stabler convergence than solver-based training. The paper reports training times under seven minutes for the quarter-vehicle model, versus hours for the comparable baseline, and a few seconds for the Van-der-Pol oscillator, with learned vector fields that match the reference even under strong noise.","pith_inferences":["Editorial inference: if the speed advantage persists on stiffer problems, collocation-based training could become the default for small hybrid models, with ODE-solver backpropagation reserved for very large networks where exact Hessians are prohibitive.","Editorial inference: the fixed-grid requirement suggests the natural next test is adaptive mesh refinement; the paper itself flags this, and success there would extend the method to systems with localized fast events.","Editorial inference: because intermediate NLP iterates may violate the dynamics, the approach opens the door to adding physics-consistency penalties or regularization on the trajectory itself, which are awkward in forward-simulation training.","Editorial inference: the same transcription should apply directly to neural differential-algebraic equations once an equation-based environment provides index-reduced DAEs, a step the paper says is under development but does not yet demonstrate."],"forward_implications":["Training sessions for small neural surrogates drop from hours to minutes on a laptop; the quarter-vehicle benchmark completes in under seven minutes.","Because the loss is approximated with high-order Radau quadrature instead of a first-order sample mean, the discretization accuracy of the integrator is preserved in the objective.","Physical priors such as zero crossings of learned force elements become hard constraints in the NLP, not penalty terms that can distort the optimum.","The same machinery handles non-neural surrogates such as Chebyshev rational functions, matching the learned behavior with only 32 parameters in under 35 seconds.","The method is designed to extend to neural differential-algebraic equations inside an equation-based modeling environment, removing export and external-training steps."],"supporting_citations":[{"why":"Defines the quarter-vehicle PeN-ODE benchmark, its data-generation setup, and the reference training times and surrogate quality this paper compares against.","marker":"[2]"},{"why":"Supplies the Van-der-Pol collocation-training benchmark, the noise levels, and the comparison for vector-field recovery.","marker":"[11]"},{"why":"Introduces the fully discretized simultaneous-optimization approach for Neural ODEs that this paper extends to PeN-ODEs with physical constraints.","marker":"[12]"},{"why":"Describes the interior-point NLP solver whose derivative requirements motivate the sparse, second-order implementation.","marker":"[13]"},{"why":"Provides the Radau collocation and quadrature construction on which the transcription and the open-source implementation rest.","marker":"[20]"},{"why":"Defines Neural ODEs, the model class that appears as a special case of the PeN-ODE training formulation.","marker":"[1]"}],"fun_headline_variants":["Neural ODE training as one big nonlinear program","Collocation turns neural ODE training into one NLP","Train PeN-ODEs in minutes, not hours with collocation","One nonlinear program to train physics-enhanced neural ODEs","Faster neural ODE training via Radau collocation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a hand-chosen time grid and collocation order adequately resolve the dynamics; Section 2.5.1 concedes the grid must be fixed a priori, and the benchmarks use constant 500- or 2500-interval grids with five Radau points per interval and no mesh adaptation.","fun_headline_variants_meta":{"raw":{"variants":["Neural ODE training as one big nonlinear program","Collocation turns neural ODE training into one NLP","Train PeN-ODEs in minutes, not hours with collocation","One nonlinear program to train physics-enhanced neural ODEs","Faster neural ODE training via Radau collocation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1449,"prompt_tokens":920,"completion_tokens":529,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":447}},"tokens_in":536,"tokens_out":529,"duration_ms":5836,"temperature":1.0,"reasoning_tokens":447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:48:23.945523+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture on a stiff or multi-timescale system, e.g. the Van-der-Pol oscillator at large $\\mu$ or a suspension with a fast tire mode, using the paper's fixed 500-interval, five-point Radau grid; if the learned surrogate fails to reproduce a validation trajectory through the fast region, or works only after hand-refining the grid, the claimed step-size advantage over solver-based training would not hold outside smooth benchmarks.","supporting_citations":[{"cited_title":"Closing the sim-to-real gap with physics-enhanced neural odes","cited_arxiv_id":null,"evidence_quote":"Defines the quarter-vehicle PeN-ODE benchmark, its data-generation setup, and the reference training times and surrogate quality this paper compares against."},{"cited_title":"Collocation based training of neural ordinary differential equations","cited_arxiv_id":null,"evidence_quote":"Supplies the Van-der-Pol collocation-training benchmark, the noise levels, and the comparison for vector-field recovery."},{"cited_title":"Training neural odes using fully discretized simultaneous optimization,","cited_arxiv_id":null,"evidence_quote":"Introduces the fully discretized simultaneous-optimization approach for Neural ODEs that this paper extends to PeN-ODEs with physical constraints."},{"cited_title":"Adaptively refined mesh for collocation-based dynamic optimization, 12 2024","cited_arxiv_id":null,"evidence_quote":"Provides the Radau collocation and quadrature construction on which the transcription and the open-source implementation rest."}],"review_version":1}