pith. machine review for the scientific record. sign in

arxiv: 2001.04385 · v4 · pith:QML2OPDTnew · submitted 2020-01-13 · 💻 cs.LG · math.DS· q-bio.QM· stat.ML

Universal Differential Equations for Scientific Machine Learning

Christopher Rackauckas , Yingbo Ma , Julius Martensen , Collin Warner , Kirill Zubov , Rohit Supekar , Dominic Skinner , Ali Ramadhan
show 1 more author
Alan Edelman
This is my paper

Pith reviewed 2026-05-18 00:20 UTC · model grok-4.3

classification 💻 cs.LG math.DSq-bio.QMstat.ML
keywords universal differential equationsscientific machine learningneural differential equationshybrid physics-ML modelssystem identificationstiff differential equationsparameter estimation
0
0 comments X

The pith

Universal differential equations combine known physical laws with neural networks to learn unknown dynamics from data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces universal differential equations as a framework for blending differential equations that capture known scientific laws with machine learning components that approximate missing parts. This setup lets models discover mechanisms in systems like biology while staying consistent with established physics. The authors show the approach applies to tasks such as identifying biological processes and tackling high-dimensional optimal control problems. Supporting software manages complexities including stochasticity, time delays, and stiff behavior through specialized training methods that support parallelism and hardware acceleration.

Core claim

We describe a mathematical object, which we denote universal differential equations (UDEs), as the unifying framework connecting the ecosystem. We show how a wide variety of applications, from automatically discovering biological mechanisms to solving high-dimensional Hamilton-Jacobi-Bellman equations, can be phrased and efficiently handled through the UDE formalism and its tooling.

What carries the argument

Universal differential equations (UDEs), which embed universal approximators such as neural networks into differential equation structures to represent unknown or partially known dynamics.

If this is right

  • Biological mechanism discovery can be automated by fitting UDEs to time-series data while respecting known reaction structures.
  • High-dimensional Hamilton-Jacobi-Bellman equations become solvable by expressing the value function via a UDE and training on sampled trajectories.
  • Models incorporating stochasticity or delays remain trainable without custom solvers for each variant.
  • Training scales across distributed systems and GPUs because the core mechanisms funnel into a shared set of optimized procedures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same structure could be applied to hybrid models in climate or materials science where some governing equations are known but parameters or sub-processes are not.
  • Long-term forecasting stability might improve when the known differential part enforces conservation laws that pure neural models often violate.
  • Parameter estimation in legacy scientific codes could be accelerated by replacing fixed subroutines with trainable UDE components.

Load-bearing premise

The assumption that the SciML tooling can efficiently train UDE models for stiff equations, stochasticity, delays, and implicit constraints while maintaining stability and accuracy across the claimed applications.

What would settle it

A concrete case where a UDE model for a stiff biological system with delays fails to converge or produces unstable solutions during training would indicate the formalism and tooling do not handle the claimed range of applications.

read the original abstract

In the context of science, the well-known adage "a picture is worth a thousand words" might well be "a model is worth a thousand datasets." In this manuscript we introduce the SciML software ecosystem as a tool for mixing the information of physical laws and scientific models with data-driven machine learning approaches. We describe a mathematical object, which we denote universal differential equations (UDEs), as the unifying framework connecting the ecosystem. We show how a wide variety of applications, from automatically discovering biological mechanisms to solving high-dimensional Hamilton-Jacobi-Bellman equations, can be phrased and efficiently handled through the UDE formalism and its tooling. We demonstrate the generality of the software tooling to handle stochasticity, delays, and implicit constraints. This funnels the wide variety of SciML applications into a core set of training mechanisms which are highly optimized, stabilized for stiff equations, and compatible with distributed parallelism and GPU accelerators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript introduces universal differential equations (UDEs) as a unifying mathematical framework within the SciML ecosystem that embeds mechanistic models and physical laws into data-driven neural network components. It demonstrates how applications such as automated discovery of biological mechanisms, solution of high-dimensional Hamilton-Jacobi-Bellman equations, and modeling with stochasticity, delays, and implicit constraints can be formulated and solved using the associated tooling, which leverages optimized differential equation solvers, adjoint sensitivities, and GPU/distributed support for stiff problems.

Significance. If the efficiency and stability claims hold, the work provides a practical bridge between scientific modeling and machine learning that could improve data efficiency and interpretability in domains like systems biology and optimal control. The reuse of mature, high-performance solvers for hybrid models is a concrete strength that avoids reimplementing core numerical infrastructure.

major comments (3)
  1. [§4] §4 (stiff and stochastic demonstrations): the reported training success on stiff and combined delay-stochastic examples lacks quantitative benchmarks such as wall-clock time, solver failure rates, or accuracy versus non-UDE baselines at increasing stiffness ratios or noise levels; without these, the claim that the ecosystem 'efficiently handled' these regimes cannot be assessed.
  2. [§5.2] §5.2 (HJB application): the high-dimensional example is presented as solved via UDEs, yet no scaling study or comparison to alternative methods (e.g., standard neural ODEs or PINNs) is given to show that the UDE formalism plus SciML tooling confers an advantage in dimensionality or constraint handling.
  3. [§3] §3 (UDE definition and training): the adjoint sensitivity method is asserted to remain stable for stiff UDEs, but the text provides no explicit tolerances, stiffness metrics, or convergence analysis for the hybrid neural-plus-mechanistic right-hand side, leaving the stability claim unverified for the general case.
minor comments (2)
  1. [Eq. (1)] Notation for the universal term (e.g., the neural component) is introduced inconsistently between the abstract, Eq. (1), and later application sections; a single definition with explicit dependence on parameters and time would improve clarity.
  2. [Figures 3-5] Several figures lack error bars or multiple random seeds, making it difficult to judge robustness of the reported trajectories.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback. We have revised the manuscript to strengthen the quantitative support for our claims on efficiency, to include comparisons for the HJB example, and to provide explicit details on tolerances and stability for the adjoint method.

read point-by-point responses

  1. Referee: [§4] §4 (stiff and stochastic demonstrations): the reported training success on stiff and combined delay-stochastic examples lacks quantitative benchmarks such as wall-clock time, solver failure rates, or accuracy versus non-UDE baselines at increasing stiffness ratios or noise levels; without these, the claim that the ecosystem 'efficiently handled' these regimes cannot be assessed.

    Authors: We agree that quantitative benchmarks are necessary to substantiate the efficiency claims. In the revised manuscript we have added wall-clock training times, solver failure rates across increasing stiffness ratios, and accuracy comparisons against non-UDE baselines for both the stiff and combined delay-stochastic cases. These metrics are reported in the updated §4. revision: yes

  2. Referee: [§5.2] §5.2 (HJB application): the high-dimensional example is presented as solved via UDEs, yet no scaling study or comparison to alternative methods (e.g., standard neural ODEs or PINNs) is given to show that the UDE formalism plus SciML tooling confers an advantage in dimensionality or constraint handling.

    Authors: The UDE formulation directly encodes the HJB structure and constraints, which is the core contribution, yet we acknowledge that explicit scaling and baseline comparisons would better demonstrate the practical advantage. The revised §5.2 now includes a scaling study with dimension and a targeted comparison to PINNs on constraint satisfaction. revision: partial

  3. Referee: [§3] §3 (UDE definition and training): the adjoint sensitivity method is asserted to remain stable for stiff UDEs, but the text provides no explicit tolerances, stiffness metrics, or convergence analysis for the hybrid neural-plus-mechanistic right-hand side, leaving the stability claim unverified for the general case.

    Authors: The adjoint implementations inherit the stiffness-handling capabilities of the underlying DifferentialEquations.jl solvers. To make this explicit we have added the tolerances used (reltol = 1e-6, abstol = 1e-8), stiffness-ratio diagnostics, and a short numerical convergence study for the hybrid right-hand side in the revised §3. revision: yes

Circularity Check

0 steps flagged

No circularity detected in UDE framework introduction or claims

full rationale

The paper introduces universal differential equations (UDEs) as a new unifying mathematical object and demonstrates its use across applications like biological mechanism discovery and high-dimensional HJB equations via the SciML ecosystem. No derivation steps reduce predictions or results to fitted inputs by construction, self-definitions, or load-bearing self-citations that make the central claims tautological. The tooling compatibility with stiff equations, stochasticity, delays, and implicit constraints is asserted through described demonstrations and ecosystem features rather than circular re-derivation of the same quantities. The framework remains self-contained against external benchmarks without requiring the target results as inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim depends on the existence and trainability of UDEs as a general modeling object and on the performance of the associated software tooling for stiff and constrained problems.

axioms (1)
  • domain assumption Hybrid differential-equation models with embedded universal approximators can be trained stably using standard optimization methods even for stiff systems.
    Invoked when the abstract states that training mechanisms are highly optimized and stabilized for stiff equations.
invented entities (1)
  • Universal Differential Equation (UDE) no independent evidence
    purpose: To serve as the unifying mathematical object that connects physical laws with machine-learning components.
    New object defined in the paper to organize the SciML applications.

pith-pipeline@v0.9.0 · 5719 in / 1264 out tokens · 34336 ms · 2026-05-18T00:20:03.437714+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • Cost Jcost_nonneg echoes
    ?
    echoes

    ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

    Training a UDE amounts to minimizing a cost function C(θ) defined on uθ(t), the current solution to the differential equation with respect to the choice of parameters θ.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. The Value of Mechanistic Priors in Sequential Decision Making

    cs.LG 2026-05 unverdicted novelty 7.0

    Mechanistic priors reduce Bayesian regret in sequential decisions by scaling with residual entropy H_mech, yielding a sample complexity reduction of H(μ)/H_mech asymptotically and lower bounds on penalties in the burn...

  2. Residual-Corrected Equivalent-Circuit Model with Universal Differential Equations for Robust Battery Voltage Prediction under Operating-Condition Shift

    eess.SY 2026-05 unverdicted novelty 7.0

    A residual-corrected ECM-UDE hybrid model outperforms standalone ECM and LSTM baselines in battery terminal voltage prediction, with the largest gains under temperature and drive-cycle distribution shifts.

  3. Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglement

    cs.CV 2026-05 unverdicted novelty 7.0

    NOVA represents world states as INR weights for decoder-free rendering, compactness, and unsupervised disentanglement of background, foreground, and motion in video world models.

  4. A solver-in-the-loop framework for end-to-end differentiable coastal hydrodynamics

    physics.flu-dyn 2026-04 unverdicted novelty 7.0

    AegirJAX embeds a coastal hydrodynamics solver as a continuous computational graph in JAX to enable end-to-end optimization and inversion for bathymetry recovery, breakwater design, wave cancellation, and model correction.

  5. MPINeuralODE: Multiple-Initial-Condition Physics-Informed Neural ODEs for Globally Consistent Dynamical System Learning

    cs.LG 2026-05 unverdicted novelty 6.0

    MPINeuralODE combines soft physics residuals with multiple-initial-condition training to reduce out-of-sample and long-horizon errors in dynamical system learning.

  6. Frequency Bias and OOD Generalization in Neural Operators under a Variable-Coefficient Wave Equation

    cs.LG 2026-05 unverdicted novelty 6.0

    FNO exhibits strong frequency bias with sharp OOD error growth on high-frequency inputs in wave equations, while DeepONet shows milder degradation despite higher baseline error.

  7. Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglement

    cs.CV 2026-05 unverdicted novelty 6.0

    NOVA represents scene states as INR weights for analytical rendering without decoders and achieves structural disentanglement of content and dynamics in video world models.

  8. Physics-Informed Neural Networks for Biological $2\mathrm{D}{+}t$ Reaction-Diffusion Systems

    cs.LG 2026-04 unverdicted novelty 6.0

    BINNs are extended to 2D+t systems and combined with symbolic regression to recover reaction-diffusion models of lung cancer cell dynamics from time-lapse microscopy data.

  9. Dissipative Latent Residual Physics-Informed Neural Networks for Modeling and Identification of Electromechanical Systems

    cs.LG 2026-04 unverdicted novelty 6.0

    DiLaR-PINN learns dissipative effects in electromechanical systems via a skew-dissipative latent residual PINN that guarantees non-increasing energy and uses recurrent curriculum training for partial observations.

  10. Predicting Power-System Dynamic Trajectories with Foundation Models

    cs.AI 2026-04 unverdicted novelty 6.0

    LASS-ODE-Power is a pretrained model that predicts power-system dynamic trajectories across regimes in a zero-shot manner after large-scale ODE pretraining and targeted fine-tuning.

  11. Learning to Test: Physics-Informed Representation for Dynamical Instability Detection

    cs.LG 2026-04 unverdicted novelty 6.0

    A physics-informed neural representation is learned from safe data to support distributional hypothesis testing for dynamical instability in stochastic DAE systems without repeated simulations.

  12. Learning Post-Newtonian Corrections from Numerical Relativity

    gr-qc 2025-11 conditional novelty 6.0

    A PINN learns higher-order corrections to the TaylorT4 PN model from eight NR surrogate waveforms, reducing phase and amplitude errors in the inspiral while enforcing physical symmetries.

  13. A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series

    cs.LG 2025-11 unverdicted novelty 6.0

    The Weak Penalty Neural ODE uses a weak form loss to filter noise and learn stable chaotic dynamics from noisy observations.

  14. Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurements

    cs.LG 2025-08 unverdicted novelty 6.0

    Neptune infers spatiotemporal parameter fields in PDEs from as few as 45 sparse measurements using independent coordinate neural networks, outperforming PINNs and neural operators with lower errors and better extrapolation.

  15. PG-LRF: Physiology-Guided Latent Rectified Flow for Electro-Hemodynamic PPG-to-ECG Generation

    eess.SP 2026-05 unverdicted novelty 5.0

    PG-LRF generates signal-faithful and physiologically plausible ECGs from PPG inputs by structuring a latent space with an electro-hemodynamic simulator and enforcing consistency in a rectified flow model.

  16. Knowledge Integration in Differentiable Models: A Comparative Study of Data-Driven, Soft-Constrained, and Hard-Constrained Paradigms for Identification and Control of the Single Machine Infinite Bus System

    cs.LG 2026-02 unverdicted novelty 5.0

    Hard-constrained differentiable programming achieves faster convergence, better generalization, and more accurate LQR controllers than soft-constrained PINNs or data-driven NODEs on the SMIB benchmark.

  17. Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl

    astro-ph.IM 2023-05 accept novelty 5.0

    PySR delivers a distributed evolutionary symbolic regression tool with a new EmpiricalBench for recovering historical scientific equations from data.

  18. Digital twin-based hybrid framework for steam generator clogging prognostics

    stat.CO 2026-04 unverdicted novelty 4.0

    Hybrid framework integrates physics simulation, heterogeneous sparse data, and uncertainty quantification to estimate remaining useful life for steam generator clogging in nuclear reactors.

  19. Experimental Design for Missing Physics

    stat.ML 2026-03 unverdicted novelty 4.0

    A sequential experimental design technique discriminates between model structures from symbolic regression to discover missing physics in process systems such as bioreactors.

Reference graph

Works this paper leans on

140 extracted references · 140 canonical work pages · cited by 18 Pith papers · 13 internal anchors

  1. [1]

    Deep convolution neural network for image recognition

    Boukaye Boubacar Traore, Bernard Kamsu-Foguem, and Fana Tangara. Deep convolution neural network for image recognition. Ecological infor- matics, 48:257–268, 2018

    work page 2018

  2. [2]

    M. T. Islam, B. M. N. Karim Siddique, S. Rahman, and T. Jabid. Image recognition with deep learning. In 2018 International Conference on Intel- ligent Informatics and Biomedical Sciences (ICIIBMS) , volume 3, pages 106–110, Oct 2018. 18

    work page 2018

  3. [3]

    This looks like that: deep learning for interpretable image recognition

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. In Advances in Neural Information Processing Systems, pages 8928–8939, 2019

    work page 2019

  4. [4]

    Recent trends in deep learning based natural language processing

    Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria. Recent trends in deep learning based natural language processing. ieee Computational intelligenCe magazine , 13(3):55–75, 2018

    work page 2018

  5. [5]

    A survey of the usages of deep learning in natural language processing

    Daniel W Otter, Julian R Medina, and Jugal K Kalita. A survey of the usages of deep learning in natural language processing. arXiv preprint arXiv:1807.10854, 2018

    work page arXiv 2018

  6. [6]

    Deep learning and natural language processing

    Y Tsuruoka. Deep learning and natural language processing. Brain and nerve= Shinkei kenkyu no shinpo , 71(1):45, 2019

    work page 2019

  7. [7]

    Deep learning in bioinformatics: Introduction, application, and perspec- tive in the big data era

    Yu Li, Chao Huang, Lizhong Ding, Zhongxiao Li, Yijie Pan, and Xin Gao. Deep learning in bioinformatics: Introduction, application, and perspec- tive in the big data era. Methods, 2019

    work page 2019

  8. [8]

    Recent advances of deep learning in bioinformatics and computational biology

    Binhua Tang, Zixiang Pan, Kang Yin, and Asif Khateeb. Recent advances of deep learning in bioinformatics and computational biology. Frontiers in Genetics, 10, 2019

    work page 2019

  9. [9]

    A primer on deep learning in genomics

    James Zou, Mikael Huss, Abubakar Abid, Pejman Mohammadi, Ali Torka- mani, and Amalio Telenti. A primer on deep learning in genomics. Nature genetics, 51(1):12–18, 2019

    work page 2019

  10. [10]

    Deep learning for computational biology.Molecular systems biology, 12(7), 2016

    Christof Angermueller, Tanel P¨ arnamaa, Leopold Parts, and Oliver Stegle. Deep learning for computational biology.Molecular systems biology, 12(7), 2016

    work page 2016

  11. [11]

    Bioinformatics and Medicine in the Era of Deep Learning

    Davide Bacciu, Paulo JG Lisboa, Jos´ e D Mart´ ın, Ruxandra Stoean, and Alfredo Vellido. Bioinformatics and medicine in the era of deep learning. arXiv preprint arXiv:1802.09791 , 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  12. [12]

    Workshop report on basic research needs for scientific machine learning: Core technologies for artificial intelligence

    Nathan Baker, Frank Alexander, Timo Bremer, Aric Hagberg, Yannis Kevrekidis, Habib Najm, Manish Parashar, Abani Patra, James Sethian, Stefan Wild, et al. Workshop report on basic research needs for scientific machine learning: Core technologies for artificial intelligence. Technical report, USDOE Office of Science (SC), Washington, DC (United States), 2019

    work page 2019

  13. [13]

    Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 378:686–707, 2019

    work page 2019

  14. [14]

    Hamiltonian neural networks

    Sam Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. arXiv preprint arXiv:1906.01563 , 2019. 19

    work page arXiv 1906

  15. [15]

    Symplec- tic ode-net: Learning hamiltonian dynamics with control

    Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplec- tic ode-net: Learning hamiltonian dynamics with control. arXiv preprint arXiv:1909.12077, 2019

    work page arXiv 1909

  16. [16]

    Understanding and mit- igating gradient pathologies in physics-informed neural networks

    Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mit- igating gradient pathologies in physics-informed neural networks. arXiv preprint arXiv:2001.04536, 2020

    work page arXiv 2001

  17. [17]

    Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems

    Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv preprint arXiv:1801.01236 , 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  18. [18]

    Deepxde: A deep learning library for solving differential equations

    Lu Lu, Xuhui Meng, Zhiping Mao, and George E Karniadakis. Deepxde: A deep learning library for solving differential equations. arXiv preprint arXiv:1907.04502, 2019

    work page arXiv 1907

  19. [19]

    Nvidia simnetˆ{TM}: an ai-accelerated multi-physics simulation framework

    Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Ak- shay Subramaniam, Kaustubh Tangsali, Max Rietmann, Jose del Aguila Ferrandis, Wonmin Byeon, Zhiwei Fang, and Sanjay Choudhry. Nvidia simnetˆ{TM}: an ai-accelerated multi-physics simulation framework. arXiv preprint arXiv:2012.07938 , 2020

    work page arXiv 2012

  20. [20]

    Resnet with one-neuron hidden layers is a universal approximator

    Hongzhou Lin and Stefanie Jegelka. Resnet with one-neuron hidden layers is a universal approximator. InAdvances in Neural Information Processing Systems, pages 6169–6178, 2018

    work page 2018

  21. [21]

    Performance of deep and shallow neural networks, the universal approximation theorem, activity cliffs, and QSAR

    David A Winkler and Tu C Le. Performance of deep and shallow neural networks, the universal approximation theorem, activity cliffs, and QSAR. Molecular informatics, 36(1-2):1600118, 2017

    work page 2017

  22. [22]

    The general approximation theorem

    Alexander N Gorban and Donald C Wunsch. The general approximation theorem. In 1998 IEEE International Joint Conference on Neural Net- works Proceedings. IEEE World Congress on Computational Intelligence (Cat. No. 98CH36227) , volume 2, pages 1271–1274. IEEE, 1998

    work page 1998

  23. [23]

    Approximation theory of the mlp model in neural networks [j]

    Pinkus Allan. Approximation theory of the mlp model in neural networks [j]. Acta Numerica, 8:143–195, 1999

    work page 1999

  24. [24]

    Minimum width for universal approximation

    Sejun Park, Chulhee Yun, Jaeho Lee, and Jinwoo Shin. Minimum width for universal approximation. arXiv preprint arXiv:2006.08859 , 2020

    work page arXiv 2006

  25. [25]

    Latent force models

    Mauricio Alvarez, David Luengo, and Neil D Lawrence. Latent force models. In Artificial Intelligence and Statistics , pages 9–16, 2009

    work page 2009

  26. [26]

    Coupled latent differential equation with moderators: Simulation and application

    Yueqin Hu, Steve Boker, Michael Neale, and Kelly L Klump. Coupled latent differential equation with moderators: Simulation and application. Psychological Methods, 19(1):56, 2014

    work page 2014

  27. [27]

    Switched latent force models for movement segmentation

    Mauricio Alvarez, Jan R Peters, Neil D Lawrence, and Bernhard Sch¨ olkopf. Switched latent force models for movement segmentation. In Advances in neural information processing systems , pages 55–63, 2010. 20

    work page 2010

  28. [28]

    Neural ordinary differential equations

    Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Du- venaud. Neural ordinary differential equations. In Advances in neural information processing systems, pages 6571–6583, 2018

    work page 2018

  29. [29]

    Latent ODEs for Irregularly-Sampled Time Series

    Yulia Rubanova, Ricky TQ Chen, and David Duvenaud. Latent odes for irregularly-sampled time series. arXiv preprint arXiv:1907.03907 , 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  30. [30]

    Neural controlled differential equations for irregular time series

    Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. arXiv preprint arXiv:2005.08926, 2020

    work page arXiv 2005

  31. [31]

    Arahal and E.F

    M.R. Arahal and E.F. Camacho. Neural network adaptive control of non- linear plants. IFAC Proceedings Volumes, 28(13):239 – 244, 1995. 5th IFAC Symposium on Adaptive Systems in Control and Signal Processing 1995, Budapest, Hungary, 14-16 June, 1995

    work page 1995

  32. [32]

    Neural network augmented physics models for systems with partially unknown dynamics: Application to slider-crank mechanism

    Wannes De Groote, Edward Kikken, Erik Hostens, Sofie Van Hoecke, and Guillaume Crevecoeur. Neural network augmented physics models for systems with partially unknown dynamics: Application to slider-crank mechanism. arXiv preprint arXiv:1910.12212 , 2019

    work page arXiv 1910

  33. [33]

    Modelingtoolkit: A composable graph transformation system for equation-based modeling

    Yingbo Ma, Shashi Gowda, Ranjan Anantharaman, Chris Laughman, Viral Shah, and Chris Rackauckas. Modelingtoolkit: A composable graph transformation system for equation-based modeling. arXiv preprint arXiv:2103.05244, 2021

    work page arXiv 2021

  34. [34]

    High- performance symbolic-numerics via multiple dispatch

    Shashi Gowda, Yingbo Ma, Alessandro Cheli, Maja Gwozdz, Vi- ral B Shah, Alan Edelman, and Christopher Rackauckas. High- performance symbolic-numerics via multiple dispatch. arXiv preprint arXiv:2105.03949, 2021

    work page arXiv 2021

  35. [35]

    Differentialequations.jl – a per- formant and feature-rich ecosystem for solving differential equations in julia

    Christopher Rackauckas and Qing Nie. Differentialequations.jl – a per- formant and feature-rich ecosystem for solving differential equations in julia. The Journal of Open Research Software , 5(1), 2017. Exported from https://app.dimensions.ai on 2019/05/05

    work page 2017

  36. [36]

    Adam: A Method for Stochastic Optimization

    Diederik P Kingma and Jimmy Ba. Adam A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980 , 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  37. [37]

    On the limited memory BFGS method for large scale optimization

    Dong C Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1-3):503–528, 1989

    work page 1989

  38. [38]

    What is an adjoint model? Bulletin of the American Meteorological Society, 78(11):2577–2592, 1997

    Ronald M Errico. What is an adjoint model? Bulletin of the American Meteorological Society, 78(11):2577–2592, 1997

    work page 1997

  39. [39]

    A review of adjoint methods for sensitivity analysis, un- certainty quantification and optimization in numerical codes

    Gr´ egoire Allaire. A review of adjoint methods for sensitivity analysis, un- certainty quantification and optimization in numerical codes. Ingenieurs de l’Automobile, 836:33–36, July 2015. 21

    work page 2015

  40. [40]

    Computational science and engineering , volume 791

    Gilbert Strang. Computational science and engineering , volume 791. Wellesley-Cambridge Press Wellesley, 2007

    work page 2007

  41. [41]

    SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers

    Alan C Hindmarsh, Peter N Brown, Keith E Grant, Steven L Lee, Radu Serban, Dan E Shumaker, and Carol S Woodward. SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers. ACM Transactions on Mathematical Software (TOMS) , 31(3):363–396, 2005

    work page 2005

  42. [42]

    Notes on adjoint methods for 18.335

    Steven G Johnson. Notes on adjoint methods for 18.335

    work page

  43. [43]

    Efficient gradient computation for dynamical models

    Biswa Sengupta, Karl J Friston, and William D Penny. Efficient gradient computation for dynamical models. NeuroImage, 98:521–527, 2014

    work page 2014

  44. [44]

    A comparison of automatic differ- entiation and continuous sensitivity analysis for derivatives of differential equation solutions

    Christopher Rackauckas, Yingbo Ma, Vaibhav Dixit, Xingjian Guo, Mike Innes, Jarrett Revels, and Vijay Ivaturi. A comparison of automatic differ- entiation and continuous sensitivity analysis for derivatives of differential equation solutions. arXiv preprint arXiv:1812.01892 , 2018

    work page arXiv 2018

  45. [45]

    Confederated modular differential equation apis for accelerated algorithm development and benchmarking

    Christopher Rackauckas and Qing Nie. Confederated modular differential equation apis for accelerated algorithm development and benchmarking. Advances in Engineering Software, 132:1–6, 2019

    work page 2019

  46. [46]

    Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. Inter- national Conference on Artificial Intelligence and Statistics , 2020

    work page 2020

  47. [47]

    ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs

    Amir Gholami, Kurt Keutzer, and George Biros. Anode: Uncondition- ally accurate memory-efficient gradients for neural odes. arXiv preprint arXiv:1902.10298, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1902

  48. [48]

    Stiff neural ordinary differential equations

    Suyong Kim, Weiqi Ji, Sili Deng, Yingbo Ma, and Christopher Rack- auckas. Stiff neural ordinary differential equations. Chaos: An Interdisci- plinary Journal of Nonlinear Science , 31(9):093122, 2021

    work page 2021

  49. [49]

    Discrete adjoint sensitivity analysis of hybrid dynamical systems with switching

    Hong Zhang, Shrirang Abhyankar, Emil Constantinescu, and Mihai An- itescu. Discrete adjoint sensitivity analysis of hybrid dynamical systems with switching. IEEE Transactions on Circuits and Systems I: Regular Papers, 64(5):1247–1259, 2017

    work page 2017

  50. [50]

    The discrete adjoint method for parameter identification in multibody system dynamics

    Thomas Lauß, Stefan Oberpeilsteiner, Wolfgang Steiner, and Karin Nach- bagauer. The discrete adjoint method for parameter identification in multibody system dynamics. Multibody system dynamics , 42(4):397–410, 2018

    work page 2018

  51. [51]

    Forward-Mode Automatic Differentiation in Julia

    J. Revels, M. Lubin, and T. Papamarkou. Forward-mode automatic dif- ferentiation in Julia. arXiv:1607.07892 [cs.MS], 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  52. [52]

    A Differentiable Programming System to Bridge Machine Learning and Scientific Computing

    Mike Innes, Alan Edelman, Keno Fischer, Chris Rackauckus, Elliot Saba, Viral B Shah, and Will Tebbutt. Zygote: A differentiable programming system to bridge machine learning and scientific computing.arXiv preprint arXiv:1907.07587, 2019. 22

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  53. [53]

    Discretize-Optimize vs

    Derek Onken and Lars Ruthotto. Discretize-Optimize vs. Optimize- Discretize for time-series regression and continuous normalizing flows. arXiv preprint arXiv:2005.13420 , 2020

    work page arXiv 2005

  54. [54]

    Hairer, S

    E. Hairer, S. P. Nørsett, and G. Wanner. Solving Ordinary Differen- tial Equations I (2nd Revised. Ed.): Nonstiff Problems . Springer-Verlag, Berlin, Heidelberg, 1993

    work page 1993

  55. [55]

    PDE-Net: Learning PDEs from Data

    Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. Pde-net: Learning pdes from data. arXiv preprint arXiv:1710.09668 , 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  56. [56]

    Dynamic mode decomposition of numerical and experi- mental data

    Peter J Schmid. Dynamic mode decomposition of numerical and experi- mental data. Journal of fluid mechanics , 656:5–28, 2010

    work page 2010

  57. [57]

    A data–driven approximation of the koopman operator: Extending dy- namic mode decomposition

    Matthew O Williams, Ioannis G Kevrekidis, and Clarence W Rowley. A data–driven approximation of the koopman operator: Extending dy- namic mode decomposition. Journal of Nonlinear Science , 25(6):1307– 1346, 2015

    work page 2015

  58. [58]

    Ex- tended dynamic mode decomposition with dictionary learning: A data- driven adaptive spectral decomposition of the Koopman operator

    Qianxiao Li, Felix Dietrich, Erik M Bollt, and Ioannis G Kevrekidis. Ex- tended dynamic mode decomposition with dictionary learning: A data- driven adaptive spectral decomposition of the Koopman operator. Chaos: An Interdisciplinary Journal of Nonlinear Science , 27(10):103111, 2017

    work page 2017

  59. [59]

    Learning Koopman invariant subspaces for dynamic mode decomposition

    Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learning Koopman invariant subspaces for dynamic mode decomposition. In Ad- vances in Neural Information Processing Systems, pages 1130–1140, 2017

    work page 2017

  60. [60]

    Learning partial differential equations via data discov- ery and sparse optimization

    Hayden Schaeffer. Learning partial differential equations via data discov- ery and sparse optimization. Proceedings of the Royal Society A: Mathe- matical, Physical and Engineering Sciences , 473(2197):20160446, 2017

    work page 2017

  61. [61]

    Prediction of dynamical systems by symbolic regression

    Markus Quade, Markus Abel, Kamran Shafi, Robert K Niven, and Bernd R Noack. Prediction of dynamical systems by symbolic regression. Physical Review E, 94(1):012214, 2016

    work page 2016

  62. [62]

    Evolution- ary modeling of systems of ordinary differential equations with genetic programming

    Hongqing Cao, Lishan Kang, Yuping Chen, and Jingxian Yu. Evolution- ary modeling of systems of ordinary differential equations with genetic programming. Genetic Programming and Evolvable Machines , 1(4):309– 337, 2000

    work page 2000

  63. [63]

    Evolutionary algorithms in genetic regulatory networks model

    Khalid Raza and Rafat Parveen. Evolutionary algorithms in genetic reg- ulatory networks model. CoRR, abs/1205.1986, 2012

    work page internal anchor Pith review Pith/arXiv arXiv 1986

  64. [64]

    Extracting sparse high- dimensional dynamics from limited data

    Hayden Schaeffer, Giang Tran, and Rachel Ward. Extracting sparse high- dimensional dynamics from limited data. SIAM Journal on Applied Math- ematics, 78(6):3279–3295, 2018. 23

    work page 2018

  65. [65]

    A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations

    Ramakrishna Tipireddy, Paris Perdikaris, Panos Stinis, and Alexandre M. Tartakovsky. A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations. CoRR, abs/1904.04058, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  66. [66]

    Discovering gov- erning equations from data by sparse identification of nonlinear dynamical systems

    Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering gov- erning equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15):3932– 3937, 2016

    work page 2016

  67. [67]

    Inferring biological networks by sparse identification of nonlinear dynamics

    Niall M Mangan, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Transactions on Molecular, Biological and Multi-Scale Communications, 2(1):52–63, 2016

    work page 2016

  68. [68]

    Model selection for dynamical systems via sparse regression and in- formation criteria

    Niall M Mangan, J Nathan Kutz, Steven L Brunton, and Joshua L Proc- tor. Model selection for dynamical systems via sparse regression and in- formation criteria. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 473(2204):20170009, 2017

    work page 2017

  69. [69]

    R. A. Fisher. The wave of advance of advantageous genes. Annals of Eugenics, 7(4):355–369, 1937

    work page 1937

  70. [70]

    Grindrod

    P. Grindrod. The Theory and Applications of Reaction-Diffusion Equa- tions: Patterns and Waves . Oxford applied mathematics and computing science series. Clarendon Press, 1996

    work page 1996

  71. [71]

    Rudy, Steven L

    Samuel H. Rudy, Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Data-driven discovery of partial differential equations. Science Advances, 3(4), 2017

    work page 2017

  72. [72]

    Dgm: A deep learning algorithm for solving partial differential equations

    Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of Computa- tional Physics, 375:1339–1364, Dec 2018

    work page 2018

  73. [73]

    Artificial neural networks for solving ordinary and partial differential equations

    Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks , 9(5):987–1000, 1998

    work page 1998

  74. [74]

    Deep learning-based numer- ical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations

    E Weinan, Jiequn Han, and Arnulf Jentzen. Deep learning-based numer- ical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Communications in Math- ematics and Statistics , 5(4):349–380, 2017

    work page 2017

  75. [75]

    Solving high-dimensional par- tial differential equations using deep learning

    Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensional par- tial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018. 24

    work page 2018

  76. [76]

    Weak adver- sarial networks for high-dimensional partial differential equations

    Yaohua Zang, Gang Bao, Xiaojing Ye, and Haomin Zhou. Weak adver- sarial networks for high-dimensional partial differential equations. arXiv preprint arXiv:1907.08272, 2019

    work page arXiv 1907

  77. [77]

    Some machine learn- ing schemes for high-dimensional nonlinear pdes

    Cˆ ome Hur´ e, Huyˆ en Pham, and Xavier Warin. Some machine learn- ing schemes for high-dimensional nonlinear pdes. arXiv preprint arXiv:1902.01599, 2019

    work page arXiv 1902

  78. [78]

    Stochastic optimization in continuous time

    Fwu-Ranq Chang. Stochastic optimization in continuous time. Cambridge University Press, 2004

    work page 2004

  79. [79]

    Tyrone E. Duncan. Linear-exponential-quadratic gaussian control. IEEE Transactions on Automatic Control, 58(11):2910–2911, 2013

    work page 2013

  80. [80]

    An adaptive timestepping algorithm for stochastic differential equations

    H Lamba. An adaptive timestepping algorithm for stochastic differential equations. Journal of computational and applied mathematics, 161(2):417– 430, 2003

    work page 2003

Showing first 80 references.