Pith. sign in

REVIEW 4 major objections 6 minor 27 references

Overcoming error-in-variable problem in data-driven model discovery by orthogonal distance regression

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ODR-BINDy treats the model equation as a soft constraint and recovers governing equations from sparse, noisy data, including Lorenz63 with up to 30% noise.

desk verdict A solid within-subfield advance that pairs ODR with Bayesian evidence for SINDy, but the headline claim of consistent superiority over all existing methods is undercut by the paper's own limitations section and by a benchmark that is explicitly stacked in its favor. read the letter →

arxiv 2507.23426 v1 pith:IPGNMK3Y submitted 2025-07-31 stat.ME nlin.CDstat.ML

Lloyd Fung This is my paper · ORCID
classification stat.MEnlin.CDstat.ML MSC 62F1562J02
keywords SINDymodeldiscoveryerror-in-variabledataassimilationchaosBayesianevidenceorthogonaldistanceregressionstochasticrelaxation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that standard SINDy-style model discovery is limited not by the noise itself but by how the model equation is imposed during fitting. Existing nonlinear optimisation methods such as modified-SINDy and DySMHO enforce the discretised equation as a hard constraint, which prevents assimilation of chaotic data for longer than the Lyapunov time. ODR-BINDy treats the equation as a soft constraint by modelling the discretisation residual as Gaussian noise, so the denoised trajectory can stay close to the data over arbitrarily long records. It then uses Bayesian evidence to select the sparse model. In Lorenz63, Rössler, and Van der Pol benchmarks, the paper reports that ODR-BINDy consistently recovers the correct equation where comparable methods fail, reliably recovering Lorenz63 from data with up to 30% measurement noise.

What carries the argument

The central object is the soft-constrained loss of Eq. (6): $L(X,\Xi) = \frac{1}{2\sigma_x^2}\|\hat{X} - X\|_F^2 + \frac{1}{2\sigma_{\partial t}^2}\|L_{\partial t} X - L_I \Theta(X)\Xi\|_F^2 + \frac{1}{2\sigma_p^2}\|\Xi\|_F^2$. Each prefactor is the inverse of a noise or prior variance: $\sigma_x^2$ for measurement noise, $\sigma_{\partial t}^2$ for the equation residual, and $\sigma_p^2$ for the parameter prior. Optimising this loss is an orthogonal distance regression because both the state and the derivative residual are penalised in their own noise metrics. The same loss is equivalent to weak-constraint 4D-Var data assimilation and to a physics-informed neural network loss. Model selection is carried out by the Bayesian evidence computed with a Laplace approximation and a Gauss-Newton Hessian, which drives the greedy deletion of dictionary terms.

What would settle it

Run ODR-BINDy on a known damped cubic oscillator with 10% measurement noise and compute the residual $\eta = L_{\partial t} X - L_I \Theta(X)\Xi$ in sliding time windows; if its variance is not flat but decays and correlates with the state, the constant-$\sigma_{\partial t}^2$ assumption is violated, and the paper's own example predicts the method will return an oversimplified equation.

Watch

Extended reading notes

Core claim

The paper's central claim is that stochastic relaxation of the model equation is the key to overcoming the error-in-variable problem in data-driven model discovery. Where earlier methods enforce $L_{\partial t} X - L_I \Theta(X)\Xi = 0$ exactly, ODR-BINDy writes the residual as an independent Gaussian noise term $\eta \sim \mathcal{N}(0, \sigma_{\partial t}^2)$, so the equation becomes a soft target in the loss. This change allows the fitted trajectory to follow the observed data beyond the Lyapunov time, even for chaotic systems, and thereby lets the algorithm exploit the full information in long noisy time series. The same optimisation solves for the denoised states and the parameters, and a greedy Bayesian-evidence search provides the sparse model selection. The paper demonstrates on three benchmark systems that this combination recovers the true sparsity pattern more reliably and uses fewer data points than Ensemble-SINDy, modified-SINDy, DySMHO, and DAHSI.

Load-bearing premise

The load-bearing assumption is that the residual between the discretised model equation and the data is small, independent, identically distributed Gaussian noise with constant variance; when that residual grows or correlates with the signal, as the paper shows for decaying signals, the method selects an oversimplified model.

Editorial extensions

If this is right

  • A single optimisation can assimilate chaotic time series much longer than the Lyapunov time, so the previous practice of limiting the assimilation horizon to the Lyapunov time is no longer required.
  • Bayesian evidence can replace coefficient-magnitude thresholding in model discovery, reducing the risk of discarding small but physically significant terms.
  • The loss structure is independent of the time-discretisation scheme, so finite differences, weak-form collocation, or other linear derivative operators can be swapped in without changing the algorithm.
  • The algorithm's computational complexity is polynomial in the library size, and it converges in under ninety seconds on a laptop for the Lorenz63 example, making the approach practical for routine model discovery.
  • The documented failure on decaying signals shows that the next improvement is to model the equation residual as time-varying and signal-correlated rather than i.i.d. Gaussian.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete extension the paper does not explore is replacing the constant $\sigma_{\partial t}^2$ with a state-dependent variance; its own damped-oscillator example suggests that this should rescue the method for decaying signals.
  • Because the loss is built from an explicit noise model, the same framework should accommodate non-Gaussian measurement noise by substituting the corresponding energy function, a change the paper notes but does not implement.
  • The demonstrated soft-constraint assimilation is the same mechanism used in weak-constraint 4D-Var, so the method could plausibly serve as a parameter-estimation tool for operational data assimilation once the sparse model is identified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ODR-BINDy, a Bayesian sparse model-discovery method that treats the discretized governing equation as a soft constraint through orthogonal distance regression. The method simultaneously denoises the trajectory and estimates parameters, with Bayesian evidence used for sparse model selection. The authors compare ODR-BINDy with Ensemble-SINDy, modified-SINDy, DySMHO, DAHSI, and SIDDs on the Van der Pol, Lorenz63, and Rössler systems, reporting higher success rates in recovering the correct model at high noise levels and short data lengths, and they attribute the improvement to stochastic relaxation of the model equation. Section 4 acknowledges a significant limitation for decaying signals, where the i.i.d. Gaussian model-error assumption is violated.

Significance. If the central claims are supported, ODR-BINDy would be a valuable contribution: it addresses error-in-variable issues in SINDy, provides a principled Bayesian approach to sparsity, and enables assimilation of trajectories longer than the Lyapunov time by relaxing the model constraint. The paper is transparent about its limitations and provides code and data links. However, the headline claim of consistent superiority over all existing methods is not supported by the evidence as presented: the benchmark is explicitly not fair, DAHSI and SIDDs are not directly compared in the main experiments, and the method is conceded to fail on decaying signals. The core derivation is internally coherent, so the result is a defensible method for sustained oscillatory/chaotic signals, but the scope of the empirical claims must be narrowed or the missing evidence supplied.

major comments (4)
  1. [§4, Fig. 6] The manuscript itself concedes in Section 4 and Fig. 6 that for a damped cubic oscillator ODR-BINDy fails to recover the correct model because the time-invariant σ∂t model of the discretization error is violated. This directly contradicts the abstract's claim that ODR-BINDy 'consistently outperforms all existing methods in recovering the correct model from sparse and noisy datasets.' The success-rate experiments in Section 3 cover only sustained oscillatory/chaotic signals, so the general claim is not supported by the evidence. The authors should either restrict the claim to the tested regime or provide an extension that handles time-varying, signal-correlated model error.
  2. [§3, Table A1] The performance comparison is not a fair benchmark, as the authors acknowledge in Section 3. DySMHO is given a ten-times higher sampling rate (Δt=0.001 vs 0.01); Ensemble-SINDy and modified-SINDy are run without the constant term in the Lorenz63 library; DAHSI is excluded from the main results; SIDDs is not run but represented by a rough estimate from Fig. 5.4 of [18]; and the number of runs differs across methods (64, 20, 64, 64). Given these asymmetries, the conclusion that ODR-BINDy is 'superior... compared to all current SINDy variants' cannot be drawn from the presented evidence. A fair comparison with matched data, libraries, and run counts, or a clearly scoped claim, is required.
  3. [§2.4, Table A1] The central hyperparameter σ∂t, and to a lesser extent σp, is chosen per system in Table A1 (σ∂t = 1e-2, 1e-3, 5e-3; σp = 10, 100, 50) rather than independently measured or selected by the evidence. Section 2.4 itself advises optimizing σ∂t to maximize evidence. Because σ∂t controls the strength of the stochastic relaxation that underlies the method's advantage, the reported success rates may partly reflect favorable hyperparameter choices. The paper should report sensitivity to σ∂t or demonstrate that evidence-based tuning recovers the same performance.
  4. [§2.1, Eq. (6)] The i.i.d. Gaussian assumption for η is load-bearing for the stochastic relaxation argument. Section 4 shows it is violated for decaying signals, and for chaotic systems the discretization error is deterministic and correlated in time. The paper should more explicitly frame Eq. (6) as an approximate noise model, state the conditions under which it is adequate, and avoid presenting the method as robust in regimes where the assumption is known to fail.
minor comments (6)
  1. [General] There are numerous typos, including 'DySHMO' in §2.3.1, 'Orthongal' in §2.2, 'indepedently' in the Introduction, 'existance' in §3, 'Hyperparmeter' in §A.1, 'approxmation' in §A.5, and 'wtih' in §A.6.
  2. [Fig. A4] The SIDDs curve is described as 'a rough estimate from fig 5.4 of [18]'; this should be stated directly in the figure caption, and the curve should not be used for quantitative comparisons.
  3. [Table A1] Table A1 would benefit from reporting the number of runs per method and error bars; currently modified-SINDy has only 20 runs, which limits the reliability of the success-rate differences.
  4. [Eq. (2) and Eq. (6)] In Eq. (2), σx^2 is introduced as an Ñ×D matrix, but in Eq. (6) it is used as a scalar; please clarify the notational convention.
  5. [References] The reference list is incomplete: several entries lack article titles or page numbers (e.g., [1], [3], [17], [18]), which hinders reproducibility of the literature review.
  6. [Abstract and Conclusion] The phrase 'consistently outperforms all existing methods' appears in both the abstract and the conclusion, but it is not supported by the scope of the experiments; it should be replaced by a scope-limited statement consistent with Sections 3 and 4.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction found: the paper re-derives its evidence mathematics and benchmarks against ground truth; the noted 'decaying signals' failure and 'not a fair benchmark' caveat are scope/correctness problems, not circularity.

full rationale

The derivation chain is self-contained. The soft-constraint loss in Eq. 6 is acknowledged as weak-constraint 4D-Var and PINN loss, and the Bayesian evidence calculation is derived in Appendix A.2 with the Gauss-Newton approximation validated in A.5. Success-rate recovery is measured against known ground truth and direct runs of E-SINDy, modified-SINDy, and DySMHO, so the headline results are not equal by construction to any fitted input. Self-citation to reference [6] (BINDy) is repeated but non-load-bearing: the Gaussian-prior rationale cites external references [24,25], and the evidence formulas appear in this paper. The paper's own limitations do the main work: Section 4 concedes that the iid Gaussian assumption on eta fails for decaying signals, with Figure 6 showing the correct model is recovered only if the truncation-error trend is known a priori; Section 3 warns the comparison is 'not a fair benchmark'; and Appendix A.4 compares to SIDDs by reading their published figure. These statements contradict the abstract's 'consistently outperforms all existing methods', but they are empirical scope weaknesses, not circularity. Hyperparameters sigma_dt and sigma_p are set per system in Table A1 rather than measured, and Section 2.4 advises optimizing sigma_dt by evidence; this tuning is a legitimate concern for claimed advantage but it is not a fitted parameter being relabeled as a prediction for a closely related quantity. No specific equation-level reduction can be exhibited, so score is 2, the top of the honest non-finding band.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method's success depends on the Gaussian noise model and the hand-set hyperparameters; no new physical entities are introduced. The main load is the iid Gaussian assumption on model error, which the paper itself concedes is violated in decaying-signal cases.

free parameters (3)
  • σ∂t (model error standard deviation) = 1e-2 (Van Der Pol), 1e-3 (Lorenz63), 5e-3 (Rössler)
    Set per test system in Table A1; this controls the weight of the soft model constraint and directly affects the denoising and model selection. It is not independently measured, and the paper says hyperparameter optimization for σ∂t is advisable.
  • σp (prior standard deviation on parameters) = 10 (Van Der Pol), 100 (Lorenz63), 50 (Rössler)
    Weakly informative prior scale chosen per system in Table A1. The paper says results are insensitive to order of magnitude, but the specific values are hand-picked.
  • Finite difference order = 4th (Van Der Pol), 6th (Lorenz63/Rössler)
    Collocation discretization order chosen per example; affects the truncation error and therefore the effective σ∂t.
assumptions (6)
  • domain assumption The data are generated by an autonomous ODE of the form ẋ = Θ(x)Ξ with a dictionary that contains the true terms.
    Imposed in Eq. (1) and used throughout; if the dictionary is misspecified, model recovery is impossible.
  • domain assumption All states are fully observed with iid Gaussian measurement noise of known variance σx (Eq. 2).
    Establishes the Gaussian data-loss term; the paper explicitly assumes full and unbiased observation.
  • ad hoc to paper The discrepancy η between the discretized equation and data is iid Gaussian with constant variance σ∂t.
    Introduced in Section 2.1 as an empirical model of truncation/stochastic error; the paper's own Section 4 shows this is violated for decaying signals, where truncation error decays and correlates with the signal.
  • domain assumption Sampling is frequent enough relative to the Lyapunov time scale.
    Stated in Section 2; the method relies on local collocation to constrain the trajectory.
  • standard math Laplace approximation and Gauss-Newton Hessian are accurate for the Bayesian evidence.
    Used in Section 2.2 and Appendix A.2; validated empirically for model selection in Appendix A.5 but not proven generally.
  • domain assumption All candidate models are equally probable a priori (flat prior over models).
    Standard Bayesian model selection assumption from Section 2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Overcoming error-in-variable problem in data-driven model discovery by orthogonal distance regression." pith.science (2026). https://pith.science/paper/IPGNMK3Y

@misc{pith2026250723426,
  author       = {Pith},
  title        = {Pith review of: Overcoming error-in-variable problem in data-driven model discovery by orthogonal distance regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IPGNMK3Y}},
  note         = {Machine review of arXiv:2507.23426}
}
read the original abstract

Despite the recent proliferation of machine learning methods like SINDy that promise automatic discovery of governing equations from time-series data, there remain significant challenges to discovering models from noisy datasets. One reason is that the linear regression underlying these methods assumes that all noise resides in the training target (the regressand), which is the time derivative, whereas the measurement noise is in the states (the regressors). Recent methods like modified-SINDy and DySMHO address this error-in-variable problem by leveraging information from the model's temporal evolution, but they are also imposing the equation as a hard constraint, which effectively assumes no error in the regressand. Without relaxation, this hard constraint prevents assimilation of data longer than Lyapunov time. Instead, the fulfilment of the model equation should be treated as a soft constraint to account for the small yet critical error introduced by numerical truncation. The uncertainties in both the regressor and the regressand invite the use of orthogonal distance regression (ODR). By incorporating ODR with the Bayesian framework for model selection, we introduce a novel method for model discovery, termed ODR-BINDy, and assess its performance against current SINDy variants using the Lorenz63, Rossler, and Van Der Pol systems as case studies. Our findings indicate that ODR-BINDy consistently outperforms all existing methods in recovering the correct model from sparse and noisy datasets. For instance, our ODR-BINDy method reliably recovers the Lorenz63 equation from data with noise contamination levels of up to 30%.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 10 canonical work pages

  1. [18]

    Hokanson, J.M., Iaccarino, G., Doostan, A.: Simultaneous Ident ification and Denoising of Dynamical Systems https://doi.org/10.1137/22M1486303

    work page doi:10.1137/22m1486303

  2. [1]

    Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing eq uations from data by sparse identification of nonlinear dynamical systems 113(15), 3932–3937 https://doi.org/10.1073/pnas.1517384113

    doi:10.1073/pnas.1517384113

  3. [2]

    Zheng, P., Askham, T., Brunton, S.L., Kutz, J.N., Aravkin, A.Y.: A Un i- fied Framework for Sparse Relaxed Regularized Regression: SR3 7, 1404–1423 https://doi.org/10.1109/ACCESS.2018.2886528

    work page arXiv 2018

  4. [3]

    Rudy, S.H., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Data-driven d iscovery of par- tial differential equations 3(4), 1602614 https://doi.org/10.1126/sciadv.1602614

    doi:10.1126/sciadv.1602614

  5. [4]

    Kaheman, K., Kutz, J.N., Brunton, S.L.: SINDy-PI: A robust algor ithm for par- allel implicit sparse identification of nonlinear dynamics 476(2242), 20200279 https://doi.org/10.1098/rspa.2020.0279

    arXiv 2020

  6. [5]

    Fasel, U., Kutz, J.N., Brunton, B.W. Brunton, Brunton, S.L.: Ense mble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learn- ing and control 478(2260), 20210904 https://doi.org/10.1098/rspa.2021.0904

    arXiv

  7. [6]

    Fung, L., Fasel, U., Juniper, M.: Rapid Bayesian identification of spa rse nonlinear dynamics from scarce and noisy data 481(2307), 20240200 https://doi.org/10.1098/rspa.2024.0200

    arXiv 2024

  8. [7]

    Zhang, S., Lin, G.: Robust data-driven discovery of governing ph ysical laws with error bars 474(2217), 20180305 https://doi.org/10.1098/rspa.2018.0305

    arXiv 2018

Show all 27 references
  1. [8]

    Hirsh, S.M., Barajas-Solano, D.A., Kutz, J.N.: Sparsifying priors fo r Bayesian uncertainty quantification in model discovery 9(2), 211823 https://doi.org/10.1098/rsos.211823

    doi

  2. [9]

    Niven, R.K., Cordier, L., Mohammad-Djafari, A., Abel, M., Quade, M.: Dynam- ical system identification, model selection, and model uncertainty quantification by Bayesian inference 34(8) https://doi.org/10.1063/5.0200684 2401.16943

    arXiv

  3. [10]

    Cam- bridge University Press

    MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cam- bridge University Press

  4. [11]

    http://arxiv.org/abs/2301.12649 Accessed 2023-08-23

    Gao, L.M., Fasel, U., Brunton, S.L., Kutz, J.N.: Convergence of Un cer- tainty Estimates in Ensemble and Bayesian Sparse Model Discovery. http://arxiv.org/abs/2301.12649 Accessed 2023-08-23

    2023 arXiv

  5. [12]

    Reinbold, P.A.K., Gurevich, D.R., Grigoriev, R.O.: Using noisy or incom- plete data to discover models of spatiotemporal dynamics 101(1), 010203 https://doi.org/10.1103/PhysRevE.101.010203

    doi

  6. [13]

    Messenger, D.A., Bortz, D.M.: Weak SINDy for partial differentia l equations 443, 26 110525 https://doi.org/10.1016/j.jcp.2021.110525

    2021

  7. [14]

    Bulletin of Mathematical Biology 85(11), 110 (2023) https://doi.org/10.1007/s11538-023-01208-6

    Bortz, D.M., Messenger, D.A., Dukic, V.: Direct Estimation of Para m- eters in ODE Models Using WENDy: Weak-Form Estimation of Non- linear Dynamics. Bulletin of Mathematical Biology 85(11), 110 (2023) https://doi.org/10.1007/s11538-023-01208-6

    2023 doi

  8. [15]

    Kaheman, K., Brunton, S.L., Nathan Kutz, J.: Automatic differen tiation to simul- taneously identify nonlinear dynamics and extract noise probability d istributions from data 3(1), 015031 https://doi.org/10.1088/2632-2153/ac567a

    doi

  9. [16]

    Ribera, H., Shirman, S., Nguyen, A.V., Mangan, N.M.: Model selectio n of chaotic systems from data with hidden variables using sparse data assimilatio n 32(6), 063101 https://doi.org/10.1063/5.0066066

    doi

  10. [17]

    Lejarza, F., Baldea, M.: Data-driven discovery of the governin g equations of dynamical systems via moving horizon optimization 12(1), 11836 https://doi.org/10.1038/s41598-022-13644-w

    doi

  11. [19]

    Boggs, P.T., Byrd, R.H., Schnabel, R.B.: A Stable and Efficient Algo- rithm for Nonlinear Orthogonal Distance Regression 8(6), 1052–1078 https://doi.org/10.1137/0908085

    doi

  12. [20]

    Annual Revie ws in Control 34(1), 1–12 (2010) https://doi.org/10.1016/j.arcontrol.2009.12.001

    Ljung, L.: Perspectives on system identification. Annual Revie ws in Control 34(1), 1–12 (2010) https://doi.org/10.1016/j.arcontrol.2009.12.001

    2010 doi

  13. [21]

    Fundamentals of Algorithms, vol

    Asch, M., Bocquet, M., Nodet, M.: Data Assimilation: Methods, Alg o- rithms, and Applications. Fundamentals of Algorithms, vol. 11. SIAM . https://doi.org/10.1137/1.9781611974546

    doi

  14. [22]

    Academic Press, New York (1974)

    Bard, Y.: Nonlinear Parameter Estimation. Academic Press, New York (1974)

    1974

  15. [23]

    Ramsay, J.O., Hooker, G., Campbell, D., Cao, J.: Parameter estima tion for differential equations: A generalized smoothing approach 69(5), 741–796 https://doi.org/10.1111/j.1467-9868.2007.00610.x

    2007

  16. [24]

    In: Bishop, C.M., Frey, B.J

    Tipping, M.E., Faul, A.C.: Fast marginal likelihood maximisation for spa rse bayesian models. In: Bishop, C.M., Frey, B.J. (eds.) Proceedings of t he Ninth International Workshop on Artificial Intelligence and Statist ics. Pro- ceedings of Machine Learning Research, vol. R4, pp...

  17. [25]

    Faul, A.C.: A Concise Introduction to Machine Learning. CRC

  18. [26]

    MacKay, D.J.C.: Bayesian Interpolation 4(3), 415–447 27 https://doi.org/10.1162/neco.1992.4.3.415

    1992 doi

  19. [27]

    Kaptanoglu, A.A., Callaham, J.L., Aravkin, A., Hansen, C.J., Brunto n, S.L.: Promoting global stability in data-driven models of quadratic nonlinea r dynamics 6(9) https://doi.org/10.1103/PhysRevFluids.6.094401 28

    doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.