pith. sign in

arxiv: 2309.16131 · v1 · submitted 2023-09-28 · 💻 cs.LG · cs.NE· math.SP

A Spectral Approach for Learning Spatiotemporal Neural Differential Equations

Mingtao Xia , Xiangting Li , Qijing Shen , Tom Chou This is my paper

Pith reviewed 2026-05-24 06:50 UTC · model grok-4.3

classification 💻 cs.LG cs.NEmath.SP
keywords spectral methodneural ODEspatiotemporal differential equationsPDE learningintegro-differential equationsunbounded domainsnonlocal interactionsmachine learning for DEs
0
0 comments X

The pith

Spectral expansions let neural networks learn spatiotemporal differential equations on unbounded domains without any spatial grid.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a neural-ODE method that represents the spatial dependence of a solution through spectral expansions rather than a discrete mesh. The neural network then learns only the time evolution of the spectral coefficients. Because no grid is required, the same framework can reconstruct both PDEs and integro-differential equations whose interactions extend over infinite domains. On standard test problems restricted to bounded domains the spectral approach reaches accuracy levels comparable to recent grid-based machine-learning methods.

Core claim

By expanding the spatial part of the solution in a spectral basis and training a neural network to advance the resulting coefficient vector, the method reconstructs the governing operator of a spatiotemporal differential equation directly from data, without ever discretizing space.

What carries the argument

Spectral neural DE learning, in which spatial dependence is encoded by spectral expansions and the neural ODE models the dynamics of the expansion coefficients.

If this is right

  • The learned model can include long-range nonlocal spatial interactions.
  • The same procedure applies to equations posed on unbounded spatial domains.
  • The framework extends to integro-differential equations in addition to PDEs.
  • Accuracy on bounded-domain PDE benchmarks matches that of current grid-based neural methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could be applied to data from open physical systems such as waves in infinite media or population dynamics across large territories.
  • Different choices of spectral basis could be substituted to match the symmetry or decay properties of a particular equation family.
  • Hybrid models that switch between spectral and local representations might handle mixed bounded-unbounded geometries.

Load-bearing premise

The solutions of the target equations admit accurate representations by spectral expansions in space.

What would settle it

Training the model on data generated by a differential equation whose solution cannot be approximated well by any modest number of spectral modes and observing that prediction error remains large even after extensive training.

Figures

Figures reproduced from arXiv: 2309.16131 by Mingtao Xia, Qijing Shen, Tom Chou, Xiangting Li.

Figure 1
Figure 1. Figure 1: (a) A 1D example of the spectral expansion in an unbounded domain with scaling factor β and displacement x0 (Eq. (7)). (b) The evolution of the coefficient c0(t) and the two tuning parameters β(t), x0(t). (c) A schematic of how to reconstruct Eq. (1) satisfied by the spectral expansion approximation u β N,x0 . The time t, expansion coefficients ci , and tuning variables β(t), and x0 are inputs of the neura… view at source ↗
Figure 2
Figure 2. Figure 2: (a) Errors using different loss functions Eqs. (9) and (19). (b) Average dynamics found from using Eqs. (9) and (19). (c) Errors with λ and σ. (d) Errors on the testing set with random times ti ∼ U(0, 1.5). [25, 26] to regularize the neural network structure. Dropout regularization does not reduce either the training error or the testing error probably because even with a feedforward neural network, the er… view at source ↗
Figure 3
Figure 3. Figure 3: (a,b) Mean relative L 2 errors for N = 14 and γ = −∞, −1, 0, 1/2. (c-f) Saliency maps showing the mean absolute values of the partial derivative of the loss function w.r.t. to {ci,ℓ(0)} for γ = −∞, −1, 0, 1/2. 1We take derivatives w.r.t. to only the coefficients {ci,ℓ(0)} of u β˜m(0) N,x˜0(0),m(x, 0; Θ) in Eq. (19) and not with w.r.t. the expansion coefficients of the observational data u(x, 0). 11 [PITH_… view at source ↗
read the original abstract

Rapidly developing machine learning methods has stimulated research interest in computationally reconstructing differential equations (DEs) from observational data which may provide additional insight into underlying causative mechanisms. In this paper, we propose a novel neural-ODE based method that uses spectral expansions in space to learn spatiotemporal DEs. The major advantage of our spectral neural DE learning approach is that it does not rely on spatial discretization, thus allowing the target spatiotemporal equations to contain long range, nonlocal spatial interactions that act on unbounded spatial domains. Our spectral approach is shown to be as accurate as some of the latest machine learning approaches for learning PDEs operating on bounded domains. By developing a spectral framework for learning both PDEs and integro-differential equations, we extend machine learning methods to apply to unbounded DEs and a larger class of problems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a spectral neural-ODE framework that represents the spatial component of spatiotemporal DEs via spectral expansions, allowing the dynamics to be learned entirely in coefficient space without any spatial grid or discretization. This is claimed to enable learning of both standard PDEs and integro-differential equations on unbounded domains with nonlocal interactions. The abstract asserts that the resulting accuracy is comparable to recent machine-learning methods restricted to bounded domains and that the framework extends the scope of data-driven DE discovery.

Significance. If the central spectral-representation assumption holds with controlled truncation error, the work would provide a principled route to data-driven modeling of nonlocal operators on unbounded domains, a setting where grid-based neural PDE methods are inapplicable. The explicit separation of spatial spectral projection from temporal neural-ODE evolution is a clean architectural choice that could be reused in other spectral settings.

major comments (2)
  1. [Abstract / method description] Abstract and method description: the central claim that the approach extends machine-learning methods to unbounded DEs and integro-differential equations rests on the unverified premise that target solutions admit accurate finite spectral expansions whose truncation error remains controlled under the learned dynamics. No explicit projection-error bound, choice of basis (Hermite/Laguerre/Fourier, etc.), or numerical demonstration that the learned coefficient ODE remains faithful when the true solution has energy outside the retained modes is supplied; this assumption is load-bearing for the unbounded-domain extension.
  2. [Abstract] Abstract: the statement that the spectral approach 'is shown to be as accurate as some of the latest machine learning approaches for learning PDEs operating on bounded domains' is presented without any quantitative comparison, error tables, datasets, or baseline references. Because this accuracy claim is used to position the contribution, its evidentiary support must be supplied in the main text.
minor comments (2)
  1. Notation for the spectral coefficients and the precise form of the neural ODE on those coefficients should be introduced with an equation number in the methods section.
  2. The abstract would be clearer if it named the concrete spectral bases employed and the neural-network architecture used to evolve the coefficients.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments. We address each major point below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Abstract / method description] Abstract and method description: the central claim that the approach extends machine-learning methods to unbounded DEs and integro-differential equations rests on the unverified premise that target solutions admit accurate finite spectral expansions whose truncation error remains controlled under the learned dynamics. No explicit projection-error bound, choice of basis (Hermite/Laguerre/Fourier, etc.), or numerical demonstration that the learned coefficient ODE remains faithful when the true solution has energy outside the retained modes is supplied; this assumption is load-bearing for the unbounded-domain extension.

    Authors: We agree the truncation control assumption is central. Section 3.1 specifies Hermite functions as the basis for unbounded domains due to their orthogonality and decay properties. Section 4 provides numerical evidence on nonlocal problems where the learned dynamics remain accurate. We will add an explicit subsection on basis choice and new ablation experiments varying retained modes to demonstrate fidelity when higher-mode energy is small. revision: yes

  2. Referee: [Abstract] Abstract: the statement that the spectral approach 'is shown to be as accurate as some of the latest machine learning approaches for learning PDEs operating on bounded domains' is presented without any quantitative comparison, error tables, datasets, or baseline references. Because this accuracy claim is used to position the contribution, its evidentiary support must be supplied in the main text.

    Authors: Quantitative comparisons, including error tables and baseline references on bounded-domain PDE benchmarks, appear in Section 4. We will revise the abstract to reference these results explicitly (e.g., 'as demonstrated in Section 4...') so the claim is directly supported by the main text. revision: yes

Circularity Check

0 steps flagged

No circularity: method rests on explicit spectral assumption and external comparisons

full rationale

The paper's derivation chain begins from the standard premise that solutions admit accurate spectral expansions (explicitly stated as the enabling assumption for evolving coefficients without discretization) and combines it with neural ODEs; this premise is not derived from the learned dynamics or fitted outputs. No equations reduce a prediction to a fitted parameter by construction, no load-bearing uniqueness theorems or ansatzes are imported via self-citation, and accuracy is asserted via comparison to other ML methods rather than definitional equivalence. The approach is therefore self-contained against external benchmarks and does not exhibit any of the enumerated circular patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Review is based on abstract only; no specific free parameters, axioms, or invented entities are described in the provided text.

pith-pipeline@v0.9.0 · 5667 in / 1130 out tokens · 24440 ms · 2026-05-24T06:50:57.299584+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 2 internal anchors

  1. [1]

    L. Bar, N. Sochen, Unsupervised deep learning algorithm for PDE-based forward and inverse problems, arXiv preprint arXiv:1904.05417 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  2. [2]

    Hsieh, S

    J.-T. Hsieh, S. Zhao, S. Eismann, L. Mirabella, S. Ermon, Learning neural PDE solvers with convergence guarantees, in: International Conference on L earning Representations, 2018

    work page 2018

  3. [3]

    Ruthotto, E

    L. Ruthotto, E. Haber, Deep neural networks motivated by partial differential equations, Journal of Mathematical Imaging and Vision 62 (3) (2020) 352–364. 12

    work page 2020

  4. [4]

    Sirignano, J

    J. Sirignano, J. F. MacArt, J. B. Freund, DPM: A deep learn ing PDE augmentation method with application to large-eddy simulation, Journal of Computat ional Physics 423 (2020) 109811

    work page 2020

  5. [5]

    Brandstetter, D

    J. Brandstetter, D. E. Worrall, M. Welling, Message pass ing neural PDE solvers, in: International Conference on Learning Representations, 2021

    work page 2021

  6. [6]

    M. Xia, S. Shao, T. Chou, Efficient scaling and moving techn iques for spectral methods in un- bounded domains, SIAM Journal on Scientific Computing 43 (5) (2021) A3244–A3268

    work page 2021

  7. [7]

    K. J. Burns, G. M. Vasil, J. S. Oishi, D. Lecoanet, B. P. Bro wn, Dedalus: A flexible framework for numerical simulations with spectral methods, Physical Review Research 2 (2) (2020) 023068

    work page 2020

  8. [8]

    J. Shen, T. Tang, Tao, L.-L. Wang, Spectral Methods: Algo rithms, Analysis and Applications, Springer Science & Business Media, New York, 2011

    work page 2011

  9. [9]

    Z. Long, Y. Lu, X. Ma, B. Dong, PDE-net: Learning PDEs from data, in: International Confer- ence on Machine Learning, PMLR, 2018, pp. 3208–3216

    work page 2018

  10. [10]

    Z. Long, Y. Lu, B. Dong, PDE-net 2.0: Learning PDEs from d ata with a numeric-symbolic hybrid deep network, Journal of Computational Physics 399 (2019) 1 08925

    work page 2019

  11. [11]

    H. Xu, H. Chang, D. Zhang, DL-PDE: Deep-learning based d ata-driven discovery of partial differential equations from discrete and noisy data, Communi cations in Computational Physics 29 (3) (2021) 698–728

    work page 2021

  12. [12]

    Anandkumar, K

    A. Anandkumar, K. Azizzadenesheli, K. Bhattacharya, N . Kovachki, Z. Li, B. Liu, A. Stuart, Neural operator: Graph kernel network for partial differenti al equations, in: ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equatio ns, 2020

    work page 2020

  13. [13]

    Z. Li, N. B. Kovachki, K. Azizzadenesheli, K. Bhattacha rya, A. Stuart, A. Anandkumar, et al., Fourier neural operator for parametric partial differential equations, in: International Conference on Learning Representations, 2020

    work page 2020

  14. [14]

    Fanaskov, I

    V. Fanaskov, I. Oseledets, Spectral neural operators, arXiv preprint arXiv:2205.10573 (2022)

    work page arXiv 2022

  15. [15]

    M. Xia, L. B¨ ottcher, T. Chou, Spectrally adapted physi cs-informed neural networks for solving unbounded domain problems, Machine Learning: Science and T echnology 4 (2) (2023) 025024

    work page 2023

  16. [16]

    R. T. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, Ne ural ordinary differential equations, in: Proceedings of the 32nd International Conference on Neu ral Information Processing Systems, 2018, pp. 6572–6583

    work page 2018

  17. [17]

    Tang, The Hermite spectral method for Gaussian-type functions, SIAM Journal on Scientific Computing 14 (3) (1993) 594–606

    T. Tang, The Hermite spectral method for Gaussian-type functions, SIAM Journal on Scientific Computing 14 (3) (1993) 594–606

    work page 1993

  18. [18]

    M. Xia, S. Shao, T. Chou, A frequency-dependent p-adapt ive technique for spectral methods, Journal of Computational Physics 446 (2021) 110627

    work page 2021

  19. [19]

    T. Chou, S. Shao, M. Xia, Adaptive Hermite spectral meth ods in unbounded domains, Applied Numerical Mathematics 183 (2023) 201–220

    work page 2023

  20. [20]

    Shen, L.-L

    J. Shen, L.-L. Wang, Sparse spectral approximations of high-dimensional problems based on hyperbolic cross, SIAM Journal on Numerical Analysis 48 (3) (2010) 1087–1109. 13

    work page 2010

  21. [21]

    J. Shen, H. Yu, Efficient spectral sparse grid methods and applications to high-dimensional elliptic problems, SIAM Journal on Scientific Computing 32 (6) (2010) 3228–3250

    work page 2010

  22. [22]

    Bateman, Some recent researches on the motion of fluid s, Monthly Weather Review 43 (4) (1915) 163–170

    H. Bateman, Some recent researches on the motion of fluid s, Monthly Weather Review 43 (4) (1915) 163–170

    work page 1915

  23. [23]

    Antoine, A

    X. Antoine, A. Arnold, C. Besse, M. Ehrhardt, A. Sch¨ adl e, A review of transparent and artificial boundary conditions techniques for linear and nonlinear Sc hr¨ odinger equations, Communications in Computational Physics 4 (4) (2008) 729–796

    work page 2008

  24. [24]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning fo r image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recogn ition, 2016, pp. 770–778

    work page 2016

  25. [25]

    G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskeve r, R. R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors , arXiv preprint arXiv:1207.0580 (2012)

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  26. [26]

    Srivastava, G

    N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskeve r, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting, Journal of M achine Learning Research 15 (2014) 1929–1958

    work page 2014

  27. [27]

    Z. Chen, Y. Xiong, S. Shao, Numerical methods for the Wig ner equation with unbounded poten- tial, Journal of Scientific Computing 79 (1) (2019) 345–368

    work page 2019

  28. [28]

    S. Shao, T. Lu, W. Cai, Adaptive conservative cell avera ge spectral element methods for transient Wigner equation in quantum transport, Communications in Co mputational Physics 9 (3) (2011) 711–739

    work page 2011

  29. [29]

    Bungartz, M

    H.-J. Bungartz, M. Griebel, Sparse grids, Acta Numeric a 13 (2004) 147–269

    work page 2004

  30. [30]

    Zenger, W

    C. Zenger, W. Hackbusch, Sparse grids, in: Proceedings of the Research Workshop of the Israel Science Foundation on Multiscale Phenomenon, Modelling an d Computation, 1991, p. 86

    work page 1991

  31. [31]

    Simonyan, A

    K. Simonyan, A. Vedaldi, A. Zisserman, Deep Inside Conv olutional Networks: Visualising Image Classification Models and Saliency Maps, in: Proceedings of the International Conference on Learning Representation (ICLR), 2014, pp. 1–8

    work page 2014

  32. [32]

    B. Jin, W. Rundell, A tutorial on inverse problems for an omalous diffusion processes, Inverse problems 31 (3) (2015) 035003. 14 Appendix A. Using F ourier neural operator to reconstruct unbounded domain DEs We shall show through a simple example that it is usually diffic ult to apply bounded domain DE reconstructing methods to reconstruct unbounded domain D...

    work page 2015

  33. [33]

    01+t ) (A.2) is the analytic solution to Eq. (A.1). For this problem, we wi ll assume ξ ∼ U (1, 3 2 ). Since the Fourier neural operator (FNO) method relies on spa tial discretization and grids, and cannot be directly applied to unbounded domain problems, we truncate the unbounded domain. Suppose one is interested in the solution’s behavior for x ∈ [− 1, ...

    work page

  34. [34]

    resolution-invariance

    0101 ± 0. 0006 (0 . 0112 ± 0. 0005) 0 . 0083 ± 0. 0005 (0 . 0099 ± 0. 0004) p Method Dropout Dropout & ResNet 0.1 0. 0146 ± 0. 0006 (0 . 0153 ± 0. 0007) 0 . 0125 ± 0. 0004 (0 . 0139 ± 0. 0004) 0.5 0. 0314 ± 0. 0021 (0 . 0327 ± 0. 0023) 0 . 0275 ± 0. 0018 (0 . 0286 ± 0. 0021) tested are λ = 10 − 2, 10− 1. 5, 10− 1, 10− 0. 5. The mean relative errors on the...

    work page 1977