Pith. sign in

REVIEW 4 minor 1 cited by

Space-time error estimates for deep neural network approximations for differential equations

T0 review · 0 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read ReLU networks approximate Euler trajectories in time and space.

desk verdict Solid, fully written-up first space-time error estimates for ReLU-network Euler schemes; the central proof holds up, with the usual caveat that it is about Eulerization, not PDEs. read the letter →

arxiv 1908.03833 v1 pith:VFEXWTVF submitted 2019-08-11 math.NA cs.LGcs.NA

classification math.NAcs.LGcs.NA MSC 65D1568T07
keywords deepneuralnetworksReLUEulerapproximationspace-timeerrorestimatesANNcalculusdifferentialequationsrectifiedlinearunitstheory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proves that rectified deep neural networks can approximate the space-time evolution of a large class of differential equations: for any prescribed tolerance, dimension, and number of Euler steps, there is an explicit ReLU network whose output is uniformly close to the Euler approximation of the solution across the whole time interval and all of R^d, not just at a fixed time. The error is controlled simultaneously in space and time by a single parameter epsilon, with the network parameter count bounded by an explicit polynomial in the number of Euler steps and a logarithmic factor in 1/epsilon. This closes a gap left by earlier rigorous results, which gave spatial error estimates for neural-network PDE approximations but no temporal component. If the estimates hold, they supply a first rigorous consistency check for using deep networks as space-time solvers rather than as spatial function approximators.

What carries the argument

The argument is carried by an ANN calculus, a compositional algebra for building networks realizing sums, parallelizations, compositions, and Euler updates, together with explicit ReLU approximations of the square function x -> $x^{2}$ and of the product (t,x) -> tx. The product network is the space-time bridge: it converts each Euler step into a network operation, hat functions interpolate between grid times, and a discrete Gronwall estimate keeps the trajectory growth polynomial in ||x||. The rectifier activation a(x)=max{x,0} is used throughout.

What would settle it

Check the claim for d=1, N=1, T=1, with drift R(Phi)(x)=max{x,0}. The Euler map is Y_{t,x}=x+t*max{x,0}, so the network must realize the piecewise-quadratic map (t,x) -> x+t*max{x,0} on [0,1] x R. Compute the minimal ReLU network parameters needed to approximate this map to error epsilon and compare with the asserted bound P(Psi) <= C(1+|ln(epsilon)|^2); if the required parameter count grows faster than log(1/epsilon) as epsilon tends to 0, the epsilon-dependence in Theorem 1.1 fails.

Watch

Extended reading notes

Core claim

The main result, Theorem 3.12 with the special case Theorem 1.1, asserts that for every epsilon in (0,1], every dimension d, and every number N of Euler steps, there exist ReLU networks $Psi^{{epsilon,d,N}}$ such that for all t in [0,T] and x in R^d, ||$Y^{{d,N}}$_{t,x} - R($Psi^{{epsilon,d,N}}$)(t,x)|| <= C $d^{{1/2}}$ $N^{{3/2}}$ epsilon (1+||x||^3), while the network size satisfies P($Psi^{{epsilon,d,N}}$) <= C $d^{{16+8d}}$ $N^{6}$ (1+|ln(epsilon)|^2). Here $Y^{{d,N}}$ is the Euler approximation of an ordinary differential equation whose drift is itself given by a network with linear growth. The approximation is space-time in the strong sense: the same network realizes the solution map (t,x) -> Y_{t,x} continuously on [0,T] x R^d, and the error is uniform in t and x.

Load-bearing premise

The drift network must grow at most linearly, with ||R(Phi)(x)|| <= C(1+||x||), because that bound is the input to the Gronwall step that keeps the Euler trajectories finite and makes the a priori bounds enter the error estimate; if it is violated, the claimed uniform bounds can blow up.

Editorial extensions

If this is right

  • For every tolerance epsilon, dimension d, and number of Euler steps N, a ReLU network of explicitly bounded size realizes the Euler map uniformly on [0,T] x R^d.
  • The time direction is controlled by the same accuracy parameter epsilon as the spatial direction, closing the gap left by purely spatial estimates.
  • The construction is explicit enough to be read as an algorithm: compose networks for the square, the product, hat functions, and the Euler update.
  • The network size grows polynomially in N and logarithmically in 1/epsilon, but exponentially in the dimension d.
  • The a priori bounds used in the proof are uniform in the perturbation sequence y, so the same network family covers perturbed Euler schemes in one stroke.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper stops at Euler approximations of ODEs; the announced next step is to feed these space-time networks into PDE approximation schemes. A testable extension is to insert the constructed network into a splitting or backward-solver pipeline and check whether the observed error obeys the same epsilon-dependence.
  • The size bound d^{16+8d} N^6 (1+|ln(epsilon)|^2) grows exponentially in d, so the result is a consistency and expressivity statement for fixed dimension, not a curse-of-dimensionality breakthrough; a follow-up would need to reduce this to a polynomial-in-d bound.
  • Because the construction is explicit and deterministic, it provides a training-free baseline for neural emulation of differential-equation solvers: the weights can be hard-coded and evaluation cost compared against classical Euler integration.
  • The requirement that the drift network grow at most linearly is inherited from the Gronwall step; weakening it would likely need a different a priori control and would change the polynomial-in-N factors in the error bound.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper claims to provide the first space-time error estimates for deep ReLU network approximations of Euler approximations of perturbed differential equations. The main result, Theorem 3.12, asserts that for an Euler scheme driven by a ReLU drift network with linear growth, there exist ReLU networks whose space-time map approximates the Euler flow with explicit error bounds of the form eps times a polynomial in d, N, and the a priori growth bound, together with explicit parameter-count bounds polynomial in N and d and quadratic in |log eps|. The proof develops an ANN calculus for compositions, parallelizations, and sums of ReLU networks; constructs product-approximation networks from a square-function approximant; and combines a posteriori error estimates via time hat-function interpolation with a discrete Gronwall a priori bound. Corollary 3.13 and Theorem 1.1 give simplified closed-form bounds, including the y=0 case stated in the introduction. The paper explicitly states that the estimates concern Euler approximations rather than the differential equation or PDE solution itself.

Significance. If correct, the result is a useful foundational contribution: it appears to be the first space-time error estimate, rather than a purely spatial one, for DNN approximation of an Euler scheme, and it supplies reusable ingredients with fully written proofs and explicitly tracked constants. The central statement is a conditional existence result with no free parameters: given any ReLU drift network, the constructed network and the error and size bounds are explicit. The weakest-assumption concern raised in review, namely the linear growth bound on the drift network, is not fragile: every fixed-depth ReLU network is globally Lipschitz, so the bound holds automatically with C equal to the network's Lipschitz constant, and the constant only enters through exp(CT). The main limitation, that the result concerns Euler approximations and not the exact ODE/PDE solution, is stated honestly in the abstract and introduction and does not undermine the central claim.

minor comments (4)
  1. [Proposition 3.5(v), Section 3.2.1] The displayed intermediate bound in Proposition 3.5(v) appears as "+ 1/(q-2) - 252", while the proof around Eq. (306) establishes the same expression with "+ 720/(q-2) - 252". The final bound is unaffected, but the statement should be corrected for consistency.
  2. [Section 2 and Proposition 3.10] The paper says that every result in Section 2 is proved in detail, yet Proposition 3.10 invokes the existence of identity networks I_d with D(I_d)=(d,2d,d) and R(I_d)=id by citing [25, Lemma 5.4], and Lemma 2.14 treats such networks as an assumption. Since the construction is elementary, either include it or explicitly acknowledge this external result in the self-containedness statement.
  3. [Theorem 1.1 and Corollary 3.13] The final constant is stated as "C in R", but the displayed inequalities are only meaningful when the constant is nonnegative; the statements should read "C in [0, infinity)".
  4. [Title and abstract] The title refers to "differential equations", but the theorem concerns Euler approximations rather than solutions of the differential equation; although the abstract and introduction make this clear, the title could be more precise, e.g., by mentioning Euler schemes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main theorem is a fully self-contained conditional existence result.

full rationale

I walked the derivation chain from the ANN calculus in Section 2 through the square-function and product approximations in Section 3.1–3.2 to the a posteriori estimate (Proposition 3.10), the Gronwall-based a priori bound (Lemma 3.11), and the main Theorem 3.12. Each step is proved in the paper with explicit constants, and the final estimate is a genuine conditional existence statement: given any ReLU drift network Φ with linear growth, the paper constructs a ReLU network Ψ whose realization approximates the Euler scheme of Φ in space-time with explicit error and parameter bounds. No parameter is fitted to the target quantity Y, and no equation is used as both input and output. The square-function approximant is built from an explicit hat-function interpolation (Lemmas 3.1–3.2), the product approximant from the polarization identity and the square approximant, and the Euler representation from a hat-function interpolation of the trajectory; all of these are independent constructions rather than restatements of the desired estimate. The paper does cite prior work from the same research group, notably [25] for an elementary identity-network lemma and for inspiration in the Euler representation, but the relevant identity network is a trivial ReLU construction and the Euler-representation argument is re-proved in the paper (Proposition 2.32 and Corollary 2.31). Such citations are not load-bearing in the sense of importing the target theorem. The acknowledged limitation—that the estimates concern Euler approximations rather than the differential-equation solution itself—is stated in the abstract and introduction and does not create circularity. No self-definitional step, fitted-input-called-prediction step, or imported-uniqueness argument was found.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper proves its results from standard mathematical background and a few explicit hypotheses. It relies on the discrete Gronwall inequality and on prior constructions of ReLU identity networks and square function approximations, all cited with proofs in the literature. The drift growth bound and the size bound on the drift network are stated hypotheses, not derived results. No free parameters are fitted to data, and no new entities are postulated.

assumptions (5)
  • standard math Discrete Gronwall inequality (Hutzenthaler et al. [23, Lemma 2.1])
    Used in Lemma 3.11 to obtain the a priori bound on the Euler trajectory; standard result cited from prior literature.
  • standard math Existence of a ReLU identity network I_d with D(I_d)=(d,2d,d) and R(I_d)(x)=x (Jentzen-Salimova-Welti [25, Lemma 5.4])
    Used in Proposition 3.10 to extend networks to a common depth; proved in cited prior work, not in this paper.
  • standard math ReLU approximation of the square function on [0,1] with error ε and O(log ε^{-1}) parameters (Yarotsky [37], Elbrächter et al. [11])
    Basis for the product approximation results in Propositions 3.3-3.7; the paper reformulates and proves the needed versions, but the original technique is due to prior work.
  • domain assumption Growth bound on the drift network: ||R(Φ)(x)|| ≤ C(1+||x||)
    Stated as a hypothesis in Theorem 3.12 and used in the Gronwall estimate in Lemma 3.11; without it the trajectory bound g_n fails.
  • domain assumption Size bound on the drift network: P(Φ_d) ≤ C d^d in Corollary 3.13 / Theorem 1.1
    Stated as a hypothesis; it enters the final parameter bound P(Ψ) ≤ C d^{16+8d} N^6(1+|ln ε|^2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Space-time error estimates for deep neural network approximations for differential equations." pith.science (2026). https://pith.science/paper/VFEXWTVF

@misc{pith2026190803833,
  author       = {Pith},
  title        = {Pith review of: Space-time error estimates for deep neural network approximations for differential equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFEXWTVF}},
  note         = {Machine review of arXiv:1908.03833}
}
abstract

Over the last few years deep artificial neural networks (DNNs) have very successfully been used in numerical simulations for a wide variety of computational problems including computer vision, image classification, speech recognition, natural language processing, as well as computational advertisement. In addition, it has recently been proposed to approximate solutions of partial differential equations (PDEs) by means of stochastic learning problems involving DNNs. There are now also a few rigorous mathematical results in the scientific literature which provide error estimates for such deep learning based approximation methods for PDEs. All of these articles provide spatial error estimates for neural network approximations for PDEs but do not provide error estimates for the entire space-time error for the considered neural network approximations. It is the subject of the main result of this article to provide space-time error estimates for DNN approximations of Euler approximations of certain perturbed differential equations. Our proof of this result is based (i) on a certain artificial neural network (ANN) calculus and (ii) on ANN approximation results for products of the form $[0,T]\times \mathbb{R}^d\ni (t,x)\mapsto tx\in \mathbb{R}^d$ where $T\in (0,\infty)$, $d\in \mathbb{N}$, which we both develop within this article.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Deep neural network approximations for Monte Carlo algorithms

    math.NA 2019-08 conditional novelty 6.0 of 10

    A general theorem shows that neural networks inherit the absence of the curse of dimensionality from any discrete Monte Carlo scheme they can emulate, with applications to Kolmogorov PDEs.

Reference graph

Works this paper leans on

37 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [25]

    Jentzen, A., Salimova, D., and Welti, T. A proof that deep artificial neural networks overcome the curse of dimensionality in the numer ical approx- imation of Kolmogorov partial differential equations with constant d iffusion and nonlinear drift coefficients. arXiv:1809.07321 (2018), 48 pages

  2. [1]

    Deep splitting method for parabolic PDEs

    Beck, C., Becker, S., Cheridito, P., Jentzen, A., and Neufeld, A. Deep splitting method for parabolic PDEs. arXiv:1907.03452 (2019), 40 pages

  3. [2]

    Solving stochastic differential equations and Kolmogorov equations by means of deep learning

    Beck, C., Becker, S., Grohs, P., Jaafari, N., and Jentzen, A. Solving stochastic differential equations and Kolmogorov equations by means of deep learning. arXiv:1806.00421 (2018), 56 pages

  4. [3]

    Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Eq ua- tions and Second-order Backward Stochastic Differential Equatio ns

    Beck, C., E, W., and Jentzen, A. Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Eq ua- tions and Second-order Backward Stochastic Differential Equatio ns. Journal of Nonlinear Science (Jan 2019). doi: 10.1007/s00332-018-9525-3

  5. [4]

    Deep Optimal Stopping

    Becker, S., Cheridito, P., and Jentzen, A. Deep Optimal Stopping. Journal of Machine Learning Research 20 , 74 (2019), 1–25. doi: 10.3929/ethz- b-000344707. 83

  6. [5]

    Solving high- dimensional optimal stopping problems using deep learning

    Becker, S., Cheridito, P., Jentzen, A., and Welti, T. Solving high- dimensional optimal stopping problems using deep learning. arXiv:1908.01602 (2019), 42 pages

  7. [6]

    A unified deep artificial neural network ap- proach to partial differential equations in complex geometries

    Berg, J., and Nystr ¨om, K. A unified deep artificial neural network ap- proach to partial differential equations in complex geometries. Neurocomput- ing 317 (2018), 28–41. doi:10.1016/j.neucom.2018.06.056

  8. [7]

    Berner, J., Grohs, P., and Jentzen, A. Analysis of the generalization error: Empirical risk minimization over deep artificial neural networ ks over- comes the curse of dimensionality in the numerical approximation of B lack- Scholes partial differential equations. arXiv:1809.03062 (2018), 35 pages

Show all 37 references
  1. [8]

    Machine Learning for Semi Linear PDEs

    Chan-W ai-Nam, Q., Mikael, J., and W arin, X. Machine Learning for Semi Linear PDEs. Journal of Scientific Computing 79 , 3 (Jun 2019), 1667–1712. doi: 10.1007/s10915-019-00908-3

  2. [9]

    Deep Learning-Based Numerical Meth- ods for High-Dimensional Parabolic Partial Differential Equations an d Back- ward Stochastic Differential Equations

    E, W., Han, J., and Jentzen, A. Deep Learning-Based Numerical Meth- ods for High-Dimensional Parabolic Partial Differential Equations an d Back- ward Stochastic Differential Equations. Commun. Math. Stat. 5 , 4 (2017), 349–380. doi: 10.1007/s40304-017-0117-6

  3. [10]

    The deep Ritz method: A deep learning-based numer- ical algorithm for solving variational problems

    E, W., and Yu, B. The deep Ritz method: A deep learning-based numer- ical algorithm for solving variational problems. arXiv:1710.00211 (2017), 14 pages

  4. [11]

    DNN Expression Rate Analysis of High-dimensional PDEs: Application to Op tion Pricing

    Elbr¨achter, D., Grohs, P., Jentzen, A., and Schwab, C. DNN Expression Rate Analysis of High-dimensional PDEs: Application to Op tion Pricing. arXiv:1809.07669v2 (2018), 50 pages

  5. [12]

    F arahmand, A.-m., Nabi, S., and Nikovski, D. N. Deep reinforcement learning for partial differential equation control. American Control Conference (ACC) (2017), 3120–3127. doi: 10.23919/ACC.2017.7963427

  6. [13]

    Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDE s

    Fujii, M., Takahashi, A., and Takahashi, M. Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDE s. Asia-Pacific Financial Markets (Mar 2019). doi: 10.1007/s10690-019-09271-7

  7. [14]

    Deep Learning

    Goodfellow, I., Bengio, Y., and Courville, A. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org

  8. [15]

    Machine Learning for Pricing American Options in High Dimension

    Goudenege, L., Molent, A., and Zanette, A. Machine Learning for Pricing American Options in High Dimension. arXiv:1903.11275 (2019), 11 pages. 84

  9. [16]

    A proof that artificial neural networks overcome the curse of dim ensionality in the numerical approximation of Black-Scholes partial differential e quations

    Grohs, P., Hornung, F., Jentzen, A., and von Wurstemberger, P. A proof that artificial neural networks overcome the curse of dim ensionality in the numerical approximation of Black-Scholes partial differential e quations. arXiv:1809.02362 (2018), 124 pages

  10. [17]

    Deep Neural Network Approximation Theory

    Grohs, P., Perekrestenko, D., Elbr ¨achter, D., and B ¨olcskei, H. Deep Neural Network Approximation Theory. arXiv:1901.02220 (2019), 60 pages

  11. [18]

    Solving high-dimensional partial differ- ential equations using deep learning

    Han, J., Jentzen, A., and E, W. Solving high-dimensional partial differ- ential equations using deep learning. Proceedings of the National Academy of Sciences 115 , 34 (2018), 8505–8510. doi: 10.1073/pnas.1718942115

  12. [19]

    Convergence of the deep BSDE method for coupled FBSDEs

    Han, J., and Long, J. Convergence of the deep BSDE method for coupled FBSDEs. arXiv:1811.01165 (2018), 26 pages

  13. [20]

    Deep Primal-Dual Algorithm for BSDEs: Appli- cations of Machine Learning to CV A and IM

    Henry-Labordere, P. Deep Primal-Dual Algorithm for BSDEs: Appli- cations of Machine Learning to CV A and IM. Preprint, SSRN–id3071506 (2017), 16 pages

  14. [21]

    Some machine learning schemes for high-dimensional nonlinear PDEs

    Hur´ e, C., Pham, H., and W arin, X. Some machine learning schemes for high-dimensional nonlinear PDEs. arXiv:1902.01599 (2019), 33 pages

  15. [22]

    Hutzenthaler, M., Jentzen, A., Kruse, T., and Nguyen, T. A. A proof that rectified deep neural networks overcome the curse of di- mensionality in the numerical approximation of semilinear heat equatio ns. arXiv:1901.10854 (2019), 24 pages

  16. [23]

    Over- coming the curse of dimensionality in the approximative pricing of finan cial derivatives with default risks

    Hutzenthaler, M., Jentzen, A., and von Wurstemberger, P. Over- coming the curse of dimensionality in the approximative pricing of finan cial derivatives with default risks. arXiv:1903.05985 (2019), 71 pages

  17. [24]

    Deep PPDEs for rough local stochastic volatility

    Jacquier, A., and Oumgari, M. Deep PPDEs for rough local stochastic volatility. arXiv:1906.02551 (2019), 21 pages

  18. [26]

    A theoretical analysis of deep neural networks and parametric PD Es

    Kutyniok, G., Petersen, P., Raslan, M., and Schneider, R. A theoretical analysis of deep neural networks and parametric PD Es. arXiv:1904.00377 (2019), 42 pages. 85

  19. [27]

    Better Approximations of High Di- mensional Smooth Functions by Deep Neural Networks with Rectifie d Power Units

    Li, B., Shanshan, T., and Yu, H. Better Approximations of High Di- mensional Smooth Functions by Deep Neural Networks with Rectifie d Power Units. arXiv:1903.05858v3 (2019), 21 pages

  20. [28]

    PDE-Net: Learning PDEs from Data

    Long, Z., Lu, Y., Ma, X., and Dong, B. PDE-Net: Learning PDEs from Data. arXiv:1710.09668 (2017), 15 pages

  21. [29]

    O., Mishra, S., and Ray, D

    Lye, K. O., Mishra, S., and Ray, D. Deep learning observables in computational fluid dynamics. arXiv:1903.03040 (2019), 57 pages

  22. [30]

    Neural Networks Trained to Solve Differential Equations Learn General Representations

    Magill, M., Qureshi, F., and de Haan, H. Neural Networks Trained to Solve Differential Equations Learn General Representations. Advances in Neural Information Processing Systems (2018), 4075–4085

  23. [31]

    Topological prop- erties of the set of functions generated by neural networks of fi xed size

    Petersen, P., Raslan, M., and Voigtlaender, F. Topological prop- erties of the set of functions generated by neural networks of fi xed size. arXiv:1806.08459v2 (2018), 56 pages

  24. [32]

    Optimal approximation of piece- wise smooth functions using deep ReLU neural networks

    Petersen, P., and Voigtlaender, F. Optimal approximation of piece- wise smooth functions using deep ReLU neural networks. arXiv:1709.05289 (2017), 54 pages

  25. [33]

    Neural networks-based backward scheme for fully nonlinear PDEs

    Pham, H., and W arin, X. Neural networks-based backward scheme for fully nonlinear PDEs. arXiv:1908.00412v1 (2019), 15 pages

  26. [34]

    Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

    Raissi, M. Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations. arXiv:1801.06637 (2018), 26 pages

  27. [35]

    Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-s um games of nonlinear stiff systems

    Reisinger, C., and Zhang, Y. Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-s um games of nonlinear stiff systems. arXiv:1903.06652 (2019), 34 pages

  28. [36]

    DGM: A deep learning algorithm for solving partial differential equations

    Sirignano, J., and Spiliopoulos, K. DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics 375 (2018), 1339–1364. doi: 10.1016/j.jcp.2018.08.029

  29. [37]

    Error bounds for approximations with deep ReLU networks

    Yarotsky, D. Error bounds for approximations with deep ReLU networks. Neural Networks 94 (2017), 103–114. doi: 10.1016/j.neunet.2017.07.002. 86

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.