REVIEW 5 major objections 6 minor 1 cited by
A Neural Operator-Based Approach to Symbolic Discovery of PDEs
T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read NOMTO substitutes pretrained neural operators for exact library operations inside a sparse symbolic graph, letting symbolic regression discover special functions, derivatives, and second-order nonlinear PDEs from data.
desk verdict NOMTO is a genuinely novel architecture, but the evidence is all in-sample, the recovered equations are often structurally wrong, and the abstract overclaims; it deserves serious peer review but needs major revisions before the claims hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the neural operator block: a pretrained Fourier Neural Operator or Convolutional Neural Operator that maps a discretized input function to the result of one library operation, wrapped with a cropping layer and an inverse tanh projection so values stay in a bounded, differentiable range. These blocks are wired into a fully connected layered graph; every edge carries a scalar weight, and each node applies its library operation to a weighted sum of the previous layer's outputs, so the graph represents all nonlinear combinations of the library up to a chosen depth. The search itself is sparse weight optimization with RMSProp under a modified $\ell^{1/2}$ penalty, sometimes with periodic pruning of low-energy weights. The surrogates are frozen throughout, which is what supplies gradients for operations such as division, $\Gamma$, and $\mathrm{Ai}$ that are singular, undefined on parts of the domain, or otherwise hard to differentiate exactly.
What would settle it
A direct test is to compute the surrogate approximation error of each frozen neural operator on the actual target data rather than on the Gaussian-mixture training functions and compare it with the target expression's typical magnitude; if the error ratio tracks the near-zero and negative $R^2$ values in Table 3, the surrogates are the bottleneck. Re-running the Burgers rediscovery with exact numerical derivatives replacing the surrogate derivative nodes would settle whether the frozen surrogates, rather than the optimization, prevent NOMTO-CNO from recovering the equation.
Extended reading notes
Core claim
The central discovery is that a neural operator trained once on random Gaussian-mixture input functions can serve as a fixed, differentiable node inside a symbolic expression graph, standing in for a library operation. Because the surrogate acts on discretized functions rather than on point values, the same block tolerates operations with undefined regions (square root, logarithm), singularities (division, Gamma), special functions (Airy), and derivatives. On the paper's evidence, this design rediscovers the heat equation as $\partial u/\partial t = 1.0\,\partial^2u/\partial x^2 + 1.0\,\partial^2u/\partial y^2$ with NOMTO-CNO and identifies the Burgers equations with FNO surrogates, e.g. $\partial u/\partial t = -0.9u\,\partial u/\partial x - 0.9v\,\partial u/\partial y + 2.0\,\partial^2u/\partial x^2 + 2.0\,\partial^2u/\partial y^2$; the CNO variant fails on Burgers. The paper presents this as evidence that symbolic discovery of second-order nonlinear PDEs is achievable without manual construction of nonlinear combinations.
Load-bearing premise
The load-bearing premise is that the frozen neural operator surrogates, trained only on random Gaussian-mixture functions, approximate every library operation accurately enough on the target data for the sparse graph optimization to recover the correct operator structure and coefficients; the paper's own f2 and f4 results and the NOMTO-CNO Burgers failure show this premise can fail.
Editorial extensions
If this is right
- NOMTO can discover compact equations containing nonlocal, singular, or special operator terms directly from discretized field data, without a manually constructed library of candidate nonlinear terms.
- Once a library operation is learned on random Gaussian-mixture functions, the same pretrained surrogate block can be frozen into any new NOMTO graph, so discovery tasks share the same operator library.
- The redefined versions of functions such as $\sqrt{x}$ and $\ln x$ make the search robust to undefined regions, which the paper argues is acceptable because experimental measurements are well defined on the domain of interest.
- With FNO surrogates the method recovers the structure and coefficients of the two-dimensional heat and Burgers equations; with CNO surrogates it recovers the heat equation but not Burgers, indicating that surrogate choice is decisive.
- A two-layer computational graph, together with the $\ell^{1/2}$ sparsity penalty, is sufficient to rediscover second-order nonlinear PDEs on a $50 \times 50 \times 50$ spatio-temporal grid from 1,000 simulations.
Reading between the lines
- Beyond the paper: because neural operators can approximate operators rather than only pointwise functions, the same graph construction should extend to integro-differential or delayed equations, a regime the paper does not test.
- Beyond the paper: the contrast between FNO and CNO on Burgers suggests surrogate fidelity is the dominant bottleneck; an immediate testable extension is to retrain the surrogates on the target data distribution before running the sparse optimization.
- Beyond the paper: the recovered coefficients can be no more accurate than the surrogate's approximation error at the target input magnitudes, which the clipping and tanh projection in the paper partially mitigate but do not remove.
- Beyond the paper: adding temporal integral nodes to the library would let the framework attempt memory-effect equations of the sort the abstract motivates, since the derivative nodes already demonstrate that differential operators are discoverable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces NOMTO, a symbolic-regression architecture in which pretrained neural operators (FNO or CNO) replace the exact operations of an Equation Learner-type graph. Each library operation (identity, multiplication, division, derivatives, sin, cos, sqrt, ln, Airy, Gamma) is approximated by a frozen neural operator trained on random Gaussian-mixture input functions, and the graph weights are optimized under an l1/2 sparsity penalty. The authors evaluate NOMTO on selected Nguyen/Keijzer benchmarks, on expressions containing derivatives and special functions, and on rediscovery of the 2-D heat and Burgers equations from simulations. The central claim is that NOMTO extends symbolic model discovery to nonlocal operators, singularities, special functions, and second-order nonlinear PDEs.
Significance. The idea of using neural operators as differentiable surrogate nodes for symbolic operations is novel and potentially useful, and it is clearly specified in the methodology. The paper is generally well written, the architecture is described in detail, and the PDE simulation protocols are documented in the appendices. If validated out-of-sample, NOMTO could extend symbolic regression to derivatives, special functions, and operator-level terms that existing EQL/KAN approaches cannot easily represent. However, the current evaluation is entirely in-sample, the CNO variant fails on Burgers rediscovery, and the reported failures for f2/f4 are incompatible with the abstract's unconditional claims. The nonlocal-operator and memory-effect promises in the abstract are not tested in any experiment.
major comments (5)
- [§2.1, Tables 1–2; §2.3, Fig. 2 and Appendix D] All reported MSE and R2 values are computed on the same data used to optimize the computational graph; no held-out split, random seed, or repeated-run statistics are provided. In Table 2, NOMTO-FNO 'recovers' Nguyen-7 as a mixture of sqrt, x, and cosine terms with R2=0.9604, and Keijzer-14 as a structurally different expression with R2=0.8151, yet both are described as successful. In Table 3, f2 and f4 have R2=-0.05 and 0.038, respectively, while the text states that the structure was captured. Since structurally wrong equations can achieve high in-sample R2, the reported metrics do not establish symbolic rediscovery; structural-equivalence rates and out-of-sample residual evaluation are needed.
- [§2.3.2, Eqs. (4)–(7)] For the Burgers rediscovery experiment, NOMTO-CNO produces Eq. (6), which includes spurious terms such as 0.5u^2 du/dx, -6.0(du/dx)^2, and 20 du/dx, and omits the u du/dx term with coefficient close to -1 as well as the du/dy diffusion term. This is not a rediscovery of Eq. (2). The abstract's unconditional claim that NOMTO 'accurately rediscover[s] second-order non-linear partial differential equations' is therefore supported only by the FNO variant, and even that variant's convective coefficients are 0.84–0.93 rather than 1. The claims should either be restricted to the successful conditions or the CNO failure should be analyzed and addressed.
- [Abstract and §1; §2.3] The abstract and introduction motivate NOMTO with 'nonlocal differential operators' and 'temporal memory effects', but none of the experiments involve a nonlocal operator or a memory/integral term. The heat and Burgers rediscovery tasks in Section 2.3 are local PDEs, and the symbolic-expression benchmarks in Sections 2.1 and 2.2 use only pointwise algebraic operations, derivatives, and special functions. The nonlocal and memory-effect claims should be either removed or supported by dedicated experiments with operators such as fractional derivatives, integral terms, or auxiliary-field couplings.
- [§4.2, Eq. (8)] The surrogate library uses redefined versions of sqrt and ln: sqrt(x)=0 for x<=0 and ln(x)=ln|x| for x!=0. The extracted expressions in Tables 2 and 3, however, are written with the standard mathematical symbols sqrt and ln. For negative arguments, the standard and redefined functions differ, so it is not justified to claim that NOMTO 'successfully identify[ies] symbolic expressions containing elementary functions with singularities'. At best, the method identifies the redefined surrogate operations, which is a different target. This issue also affects the singularity-handling claims in Sections 2.1 and 3.
- [§4.2 vs. §2.1] The pretraining distribution for the neural operator surrogates coincides with the input distribution used for the benchmarks: both use random mixtures of Gaussians on [-10,10] with 100 discretization points. Consequently, the reported successes may reflect interpolation within the surrogate's training distribution rather than general operator approximation. At least one evaluation with input functions drawn from a different family, or with analytic functions outside the Gaussian-mixture class, is needed to establish that the surrogate nodes are not simply memorizing the training distribution.
minor comments (6)
- [§4.4, Eq. (9)] The two cases in Eq. (9) both read 'for |w| >= a'; the second branch should be 'for |w| < a'.
- [§3] The Discussion section refers to 'NOMAD' instead of 'NOMTO'.
- [§4.2] There are typos: 'tahn' should be 'tanh', and the dataset size '10 5 samples' should be written as 10^5 samples.
- [§2.3.1] The phrase 'rediscover the patial differential equation' contains a typo: 'partial'.
- [§2.1] The text refers to 'test expressions', but no held-out test set is described; 'benchmark expressions' would be more accurate.
- [§5] The data-availability statement says code and data will be released upon acceptance; providing a public artifact or an anonymous repository with the submission would improve reproducibility.
Circularity Check
No circular reduction found: NOMTO's symbolic search is an empirical fitting procedure validated against external benchmark expressions and independently simulated PDE targets.
full rationale
I walked the claimed derivation chain—Sections 2.1–2.3 as empirical demonstrations, Section 4.2 as the surrogate-training step, and Section 4.3–4.4 as the graph optimization and expression extraction—and found no step that reduces to its own inputs by construction. The neural-operator surrogates are pre-trained on random Gaussian-mixture inputs with outputs computed from exact library operations (Section 4.2); this is supervised pretraining of subroutines, not a derivation of the target expressions, and the target expressions are not used to train the surrogates. The benchmark experiments compare against external Nguyen and Keijzer ground truths, and the PDE experiments use independently simulated heat and Burgers solutions with random initial conditions; the target PDEs are not inserted into the library or used to set the graph coefficients. The only author-overlapping citation is [23] for convolutional neural operators, but it is used as a standard architecture component and not as a load-bearing uniqueness theorem or as the justification that the discovered equation must hold. The main weaknesses are validity issues rather than circularity: all reported metrics are in-sample (e.g., Figure 2 and Appendix D report MSE on the training simulations), some recovered expressions have high R2 while being structurally wrong (Table 2: Nguyen-7, Nguyen-2), and NOMTO-CNO fails on Burgers (Eqs. 6–7). These concerns bear on generalization and reproducibility, but they do not make any derivation equivalent to its inputs. Therefore the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (5)
- tanh projection clipping bound =
[-tanh(1000), tanh(1000)]
- l1/2 penalty smoothing constant a =
0.01
- pruning schedule and fraction =
prune 10% lowest-energy weights every 50 epochs after epoch 100
- sparsity regularization coefficient lambda =
not reported
- output clipping range =
[-100, 100] for labels
assumptions (5)
- domain assumption Neural operator surrogates trained on random Gaussian mixtures provide sufficiently accurate approximations of library operations on target data.
- domain assumption The library of operations and graph depth are chosen to contain the operations needed to express the target equation.
- domain assumption Second-order central finite differences provide accurate reference derivatives.
- ad hoc to paper Redefined versions of sqrt and ln in Eq. (8) preserve symbolic discovery capability for functions with undefined regions.
- ad hoc to paper For Burgers, replacing surrogate multiplication with exact PyTorch multiplication does not change the discovered structure.
Cite this review
Pith. "Pith review of A Neural Operator-Based Approach to Symbolic Discovery of PDEs." pith.science (2026). https://pith.science/paper/GNFI6XZ4
@misc{pith2026250108086,
author = {Pith},
title = {Pith review of: A Neural Operator-Based Approach to Symbolic Discovery of PDEs},
year = {2026},
howpublished = {\url{https://pith.science/paper/GNFI6XZ4}},
note = {Machine review of arXiv:2501.08086}
}
read the original abstract
Discovering governing equations from data remains challenging when the underlying dynamics involve nonlocal differential operators, field interactions governed by auxiliary equations, or temporal memory effects. We propose Neural Operator-based symbolic Model approximaTion and discOvery (NOMTO), a framework that extends Equation Learner-type symbolic architectures by incorporating pretrained neural operators as nodes in the symbolic network. NOMTO represents candidate equations as sparse differentiable computational graphs that combine algebraic operations with fixed neural operator surrogates pretrained to approximate nonlinear operators. We evaluate the method on model-discovery problems involving nonlocal spatial operators, couplings mediated by auxiliary field equations, and temporal integral terms representing memory effects. The results show that NOMTO can recover compact governing equations containing nonlocal operator terms, thereby extending symbolic model discovery beyond libraries restricted to local derivatives and point-wise algebraic combinations.
Figures
Forward citations
Cited by 1 Pith paper
-
Neuro-Symbolic ODE Discovery with Latent Grammar Flow
Latent Grammar Flow embeds grammar-based ODE representations into a discrete latent space with a behavioural loss and samples candidate equations via discrete flow to fit observed data.
Reference graph
Works this paper leans on
-
[1]
Brenden K Petersen, Mikel Landajuela, T Nathan Mundhenk, Claudio P Santiago, Soo K Kim, and Joanne T Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019
arXiv 1912
-
[2]
End-to-end symbolic regression with transformers
Pierre-Alexandre Kamienny, St´ ephane d’Ascoli, Guillaume Lample, and Fran¸ cois Charton. End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems, 35:10269– 10281, 2022
work page 2022
-
[3]
Neural symbolic regression that scales
Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascandolo. Neural symbolic regression that scales. In International Conference on Machine Learning, pages 936–945. Pmlr, 2021
2021
-
[4]
Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework
Yuan Tian, Wenqi Zhou, Hao Dong, David S Kammer, and Olga Fink. Sym-q: Adaptive symbolic regression via sequential decision-making. arXiv preprint arXiv:2402.05306, 2024
work page Pith review arXiv 2024
-
[5]
Symformer: End-to-end symbolic regression using transformer-based architecture
Martin Vastl, Jon´ aˇ s Kulh´ anek, Jiˇ r ´ ı Kubal ´ ık, Erik Derner, and Robert Babuˇ ska. Symformer: End-to-end symbolic regression using transformer-based architecture. IEEE Access, 2024
work page 2024
-
[6]
Contemporary symbolic regression methods and their relative performance
William La Cava, Bogdan Burlacu, Marco Virgolin, Michael Kommenda, Patryk Orzechowski, Fabr ´ ıcio Olivetti de Fran¸ ca, Ying Jin, and Jason H Moore. Contemporary symbolic regression methods and their relative performance. Advances in neural information processing systems, 2021(DB1):1, 2021
work page 2021
-
[7]
Zihan Yu, Jingtao Ding, and Yong Li. Symbolic regression via mdlformer-guided search: from minimizing prediction error to minimizing description length. arXiv preprint arXiv:2411.03753, 2024
arXiv 2024
-
[8]
Genetic programming as a means for programming computers by natural selection
John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87–112, 1994
1994
Show all 30 references
-
[9]
Analytical modeling of exoplanet transit spectroscopy with dimensional analysis and symbolic regression
Konstantin T Matchev, Katia Matcheva, and Alexander Roman. Analytical modeling of exoplanet transit spectroscopy with dimensional analysis and symbolic regression. The Astrophysical Journal, 930(1):33, 2022
2022
-
[10]
Orbital anomaly reconstruction using deep symbolic regression
Matteo Manzi and Massimiliano Vasile. Orbital anomaly reconstruction using deep symbolic regression. In 71st International Astronautical Congress, 2020. 11
2020
-
[11]
Simple descriptor derived from symbolic regression accelerating the discovery of new perovskite catalysts
Baicheng Weng, Zhilong Song, Rilong Zhu, Qingyu Yan, Qingde Sun, Corey G Grice, Yanfa Yan, and Wan- Jian Yin. Simple descriptor derived from symbolic regression accelerating the discovery of new perovskite catalysts. Nature communications, 11(1):3513, 2020
2020
-
[12]
Machine learning and symbolic regression investigation on stability of mxene materials
Mu He and Lei Zhang. Machine learning and symbolic regression investigation on stability of mxene materials. Computational Materials Science, 196:110578, 2021
2021
-
[13]
Machine learning to guide the use of adjuvant therapies for breast cancer.Nature Machine Intelligence, 3(8):716–726, 2021
Ahmed M Alaa, Deepti Gurdasani, Adrian L Harris, Jem Rashbass, and Mihaela van der Schaar. Machine learning to guide the use of adjuvant therapies for breast cancer.Nature Machine Intelligence, 3(8):716–726, 2021
2021
-
[14]
Hemoglobin and glucose level estimation from ppg characteristics features of fingertip video using mggp-based model
Md Asaf-uddowla Golap, SM Taslim Uddin Raju, Md Rezwanul Haque, and MMA Hashem. Hemoglobin and glucose level estimation from ppg characteristics features of fingertip video using mggp-based model. Biomedical Signal Processing and Control, 67:102478, 2021
2021
-
[15]
Artificial intelligence in physical sci- ences: Symbolic regression trends and perspectives
Dimitrios Angelis, Filippos Sofos, and Theodoros E Karakasidis. Artificial intelligence in physical sci- ences: Symbolic regression trends and perspectives. Archives of Computational Methods in Engineering, 30(6):3845–3865, 2023
2023
-
[16]
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016
2016
-
[17]
A review of data-driven discovery for dynamic systems
Joshua S North, Christopher K Wikle, and Erin M Schliep. A review of data-driven discovery for dynamic systems. International Statistical Review, 91(3):464–492, 2023
2023
-
[18]
Learning equations for extrapolation and control
Subham Sahoo, Christoph Lampert, and Georg Martius. Learning equations for extrapolation and control. In International Conference on Machine Learning, pages 4442–4450. Pmlr, 2018
2018
-
[19]
Kan: Kolmogorov-arnold networks
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljaˇ ci´ c, Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756, 2024
2024 arXiv
-
[20]
Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability
Kunpeng Xu, Lifei Chen, and Shengrui Wang. Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability. arXiv preprint arXiv:2406.02496, 2024
2024 arXiv
-
[21]
Kan-odes: Kolmogorov–arnold network ordinary differen- tial equations for learning dynamical systems and hidden physics
Benjamin C Koenig, Suyong Kim, and Sili Deng. Kan-odes: Kolmogorov–arnold network ordinary differen- tial equations for learning dynamical systems and hidden physics. Computer Methods in Applied Mechanics and Engineering, 432:117397, 2024
2024
-
[22]
Learning nonlin- ear operators via deeponet based on the universal approximation theorem of operators
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlin- ear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218–229, 2021
2021
-
[23]
Convolutional neural operators for robust and accurate learning of pdes
Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de B´ ezenac. Convolutional neural operators for robust and accurate learning of pdes. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[24]
Laplace neural operator for solving differential equations
Qianying Cao, Somdatta Goswami, and George Em Karniadakis. Laplace neural operator for solving differential equations. Nature Machine Intelligence, 6(6):631–640, 2024
2024
-
[25]
Fourier neural operator for parametric partial differential equations
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020
2010 arXiv
-
[26]
Semantically-based crossover in genetic programming: application to real-valued symbolic regression
Nguyen Quang Uy, Nguyen Xuan Hoai, Michael O’Neill, Robert I McKay, and Edgar Galv´ an-L´ opez. Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Ge- netic Programming and Evolvable Machines, 12:91–119, 2011
2011
-
[27]
Improving symbolic regression with interval arithmetic and linear scaling
Maarten Keijzer. Improving symbolic regression with interval arithmetic and linear scaling. In European Conference on Genetic Programming, pages 70–82. Springer, 2003
2003
-
[28]
Neural operator: Learning maps between function spaces with applications to pdes
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023
2023
-
[29]
Learning integral operators via neural integral equations
Emanuele Zappala, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Andrew Henry Moberly, Michael James Higley, Jessica Cardin, and David van Dijk. Learning integral operators via neural integral equations. Nature Machine Intelligence, 6(9):1046–1062, 2024. 12
2024
-
[30]
Neural networks for machine learning
Geoffrey Hinton, Nitsh Srivastava, and Kevin Swersky. Neural networks for machine learning. Coursera, video lectures, 264(1):2146–2153, 2012. 13 A Predicted symbolic expressions Table 4: Symbolic expressions recovered by Kolmogorov-Arnold Network. Benchmark Predicted expressio...
2012
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.