{"id":"844a995a-df72-4248-bd99-2999d5295e52","arxiv_id":"2501.08086","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"NOMTO combines pretrained neural operators with a sparse symbolic graph to rediscover PDEs like heat and Burgers equations from simulated data.","lead":"This paper introduces NOMTO, a symbolic regression method that uses pretrained neural operators as building blocks to discover equations from data. It can recover partial differential equations such as the heat and Burgers equations, extending equation discovery beyond simple algebraic formulas.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NOMTO's rediscovery claim is unvalidated out-of-sample: all reported metrics are in-sample, structurally wrong equations score well, and no seeds or held-out data are provided.","rationale":"The reader's weakest assumption focuses on surrogate approximation error. That is a real and visible problem, but it is not the most load-bearing issue: even with perfect surrogates, the paper's evaluation protocol cannot distinguish true symbolic rediscovery from flexible in-sample fitting. Every quantitative result in Sections 2.1-2.3 is reported on the training distribution, with no test split, no repeated runs, and no structural fidelity metric. Table 3 already shows two failures (f2 and f4) that are glossed over, and the NOMTO-CNO Burgers equations (Eqs. 6-7) would be accepted by the reported protocol if their training MSE happened to be low. The paper's own Section 5 states that code and data are withheld until acceptance, which prevents independent verification of even the successful cases. My concern is thus about the validity of the inference from the reported numbers to the central claim: the evidence as presented does not establish that NOMTO reliably recovers compact governing equations rather than fitting the training data with alternative expressions. This reinforces the reader's conditional verdict rather than overturning it; the requested revisions should include out-of-sample evaluation and seed-robust structural recovery rates.","tokens_in":15530,"tokens_out":7539,"duration_ms":80812,"concrete_test":"Split the 1,000 PDE simulation samples (heat and Burgers) into 800 training and 200 holdout sets. After standard NOMTO optimization on training data, evaluate each extracted expression exactly (not through the neural operator surrogates) on the holdout set, computing R2 and the maximum normalized residual of the PDE residual. Also report structural recovery rates (support and coefficients matching Eqs. 2-3 within 10%) across 20 random seeds per variant. If structurally wrong expressions such as Eqs. 6-7 achieve near-training accuracy on holdout, or if structural recovery occurs in fewer than half of seeds, the 'rediscovery' claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is not merely surrogate accuracy; it is that the reported MSE/R2 demonstrate true rediscovery rather than in-sample fitting. The paper never evaluates the extracted expressions on held-out data. The PDE rediscovery (Section 2.3) optimizes on the same 1,000 simulations used to report residual MSE (Fig. 2; Appendix D), and the benchmarks use Gaussian-mixture inputs from the same distribution on which the surrogate operators were trained. Under this protocol, structurally wrong equations can score well: Table 2 shows NOMTO-FNO 'recovering' Nguyen-7 as a mixture of sqrt/sin/cos and Nguyen-2 with an x^2.5 term, both with R2 > 0.96; Table 3 f2/f4 have R2 near zero yet are discussed as capturing structure; and the NOMTO-CNO Burgers result (Eqs. 6-7) is a qualitatively different PDE than the target Eqs. 2-3. No random seeds, error bars, or structural-equivalence rates are reported. The central claim—that NOMTO 'can accurately rediscover' PDEs and 'successfully identify' symbolic expressions—therefore is not established; the observed successes (heat, FNO-Burgers) could be selected favorable runs or in-sample fits.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces NOMTO, a symbolic-regression architecture in which pretrained neural operators (FNO or CNO) replace the exact operations of an Equation Learner-type graph. Each library operation (identity, multiplication, division, derivatives, sin, cos, sqrt, ln, Airy, Gamma) is approximated by a frozen neural operator trained on random Gaussian-mixture input functions, and the graph weights are optimized under an l1/2 sparsity penalty. The authors evaluate NOMTO on selected Nguyen/Keijzer benchmarks, on expressions containing derivatives and special functions, and on rediscovery of the 2-D heat and Burgers equations from simulations. The central claim is that NOMTO extends symbolic model discovery to nonlocal operators, singularities, special functions, and second-order nonlinear PDEs.","tokens_in":15785,"tokens_out":5709,"duration_ms":57184,"significance":"The idea of using neural operators as differentiable surrogate nodes for symbolic operations is novel and potentially useful, and it is clearly specified in the methodology. The paper is generally well written, the architecture is described in detail, and the PDE simulation protocols are documented in the appendices. If validated out-of-sample, NOMTO could extend symbolic regression to derivatives, special functions, and operator-level terms that existing EQL/KAN approaches cannot easily represent. However, the current evaluation is entirely in-sample, the CNO variant fails on Burgers rediscovery, and the reported failures for f2/f4 are incompatible with the abstract's unconditional claims. The nonlocal-operator and memory-effect promises in the abstract are not tested in any experiment.","major_comments":[{"comment":"All reported MSE and R2 values are computed on the same data used to optimize the computational graph; no held-out split, random seed, or repeated-run statistics are provided. In Table 2, NOMTO-FNO 'recovers' Nguyen-7 as a mixture of sqrt, x, and cosine terms with R2=0.9604, and Keijzer-14 as a structurally different expression with R2=0.8151, yet both are described as successful. In Table 3, f2 and f4 have R2=-0.05 and 0.038, respectively, while the text states that the structure was captured. Since structurally wrong equations can achieve high in-sample R2, the reported metrics do not establish symbolic rediscovery; structural-equivalence rates and out-of-sample residual evaluation are needed.","section":"§2.1, Tables 1–2; §2.3, Fig. 2 and Appendix D"},{"comment":"For the Burgers rediscovery experiment, NOMTO-CNO produces Eq. (6), which includes spurious terms such as 0.5u^2 du/dx, -6.0(du/dx)^2, and 20 du/dx, and omits the u du/dx term with coefficient close to -1 as well as the du/dy diffusion term. This is not a rediscovery of Eq. (2). The abstract's unconditional claim that NOMTO 'accurately rediscover[s] second-order non-linear partial differential equations' is therefore supported only by the FNO variant, and even that variant's convective coefficients are 0.84–0.93 rather than 1. The claims should either be restricted to the successful conditions or the CNO failure should be analyzed and addressed.","section":"§2.3.2, Eqs. (4)–(7)"},{"comment":"The abstract and introduction motivate NOMTO with 'nonlocal differential operators' and 'temporal memory effects', but none of the experiments involve a nonlocal operator or a memory/integral term. The heat and Burgers rediscovery tasks in Section 2.3 are local PDEs, and the symbolic-expression benchmarks in Sections 2.1 and 2.2 use only pointwise algebraic operations, derivatives, and special functions. The nonlocal and memory-effect claims should be either removed or supported by dedicated experiments with operators such as fractional derivatives, integral terms, or auxiliary-field couplings.","section":"Abstract and §1; §2.3"},{"comment":"The surrogate library uses redefined versions of sqrt and ln: sqrt(x)=0 for x<=0 and ln(x)=ln|x| for x!=0. The extracted expressions in Tables 2 and 3, however, are written with the standard mathematical symbols sqrt and ln. For negative arguments, the standard and redefined functions differ, so it is not justified to claim that NOMTO 'successfully identify[ies] symbolic expressions containing elementary functions with singularities'. At best, the method identifies the redefined surrogate operations, which is a different target. This issue also affects the singularity-handling claims in Sections 2.1 and 3.","section":"§4.2, Eq. (8)"},{"comment":"The pretraining distribution for the neural operator surrogates coincides with the input distribution used for the benchmarks: both use random mixtures of Gaussians on [-10,10] with 100 discretization points. Consequently, the reported successes may reflect interpolation within the surrogate's training distribution rather than general operator approximation. At least one evaluation with input functions drawn from a different family, or with analytic functions outside the Gaussian-mixture class, is needed to establish that the surrogate nodes are not simply memorizing the training distribution.","section":"§4.2 vs. §2.1"}],"minor_comments":[{"comment":"The two cases in Eq. (9) both read 'for |w| >= a'; the second branch should be 'for |w| < a'.","section":"§4.4, Eq. (9)"},{"comment":"The Discussion section refers to 'NOMAD' instead of 'NOMTO'.","section":"§3"},{"comment":"There are typos: 'tahn' should be 'tanh', and the dataset size '10 5 samples' should be written as 10^5 samples.","section":"§4.2"},{"comment":"The phrase 'rediscover the patial differential equation' contains a typo: 'partial'.","section":"§2.3.1"},{"comment":"The text refers to 'test expressions', but no held-out test set is described; 'benchmark expressions' would be more accurate.","section":"§2.1"},{"comment":"The data-availability statement says code and data will be released upon acceptance; providing a public artifact or an anonymous repository with the submission would improve reproducibility.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a promising architecture, but the current evidence does not support the abstract's broad claims. The most urgent fixes are an out-of-sample validation protocol with structural-equivalence metrics, a restriction or correction of the nonlocal/memory claims, and an honest treatment of the CNO Burgers failure. I do not see grounds for rejection, because the core idea is sound and the gaps appear addressable within the manuscript's scope. I would also encourage the authors to check the internal consistency of the redefined library operations and the extracted symbolic expressions, since that issue directly affects the validity of the singularity-handling claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the core idea is solid and worth talking about—pretrained FNO/CNO surrogates as frozen nodes inside an EQL-style sparse graph for symbolic regression and PDE discovery. That combination is new, as far as I know from the cited literature. The heat equation rediscovery, especially with CNO, produces coefficients accurate to one decimal place, and the derivative/special-function cases f1, f3, and f5 come out clean. The authors also deserve credit for being transparent about some failures; they explicitly note that predicted expressions often do not replicate ground-truth structure.\n\nThe problem is that the evidence does not back the abstract's strongest claims. Every reported MSE/R2 is in-sample. There is no held-out evaluation anywhere in the paper. R2 is a poor proxy for structural correctness here: Nguyen-7 gets R2 0.96 with a recovered expression that is a mix of sqrt/sin/cos rather than logs, and f2/f4 in Table 3 have near-zero or negative R2 but are still discussed as capturing structure. The NOMTO-CNO Burgers result is not a rediscovery—it is a qualitatively different PDE. No seeds, no error bars, no multiple runs, so we cannot tell whether the successes are signal or selected favorable runs. The surrogates are trained on the same Gaussian-mixture function family used to generate the benchmark inputs, which makes the generalization claims circular. And the metadata abstract mentions nonlocal operators and memory effects that simply do not appear in the full text; that overclaim has to go.\n\nThere are also a couple of concrete technical blemishes. Equation 9 defines the modified l1/2 norm with the same inequality, |w| >= a, on both branches; presumably the second branch should be |w| < a. Section 3 also calls the method \"NOMAD\" once, a minor typo.\n\nIf I were the editor, I would send this to referees—the architecture deserves scrutiny and the failure analysis is useful—but I would expect referees to require held-out evaluation, structural-equivalence metrics, seeds and error bars, and code/data release. Without those, this is a promising prototype, not a validated method.","headline":"NOMTO is a genuinely novel architecture, but the evidence is all in-sample, the recovered equations are often structurally wrong, and the abstract overclaims; it deserves serious peer review but needs major revisions before the claims hold.","tokens_in":16293,"tokens_out":4505,"would_cite":false,"duration_ms":43792,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NOMTO substitutes pretrained neural operators for exact library operations inside a sparse symbolic graph, letting symbolic regression discover special functions, derivatives, and second-order nonlinear PDEs from data.","keywords":["symbolic regression","neural operator","partial differential equation discovery","Fourier neural operator","convolutional neural operator","sparse computational graph","special functions","Burgers equation"],"falsifier":"A direct test is to compute the surrogate approximation error of each frozen neural operator on the actual target data rather than on the Gaussian-mixture training functions and compare it with the target expression's typical magnitude; if the error ratio tracks the near-zero and negative $R^2$ values in Table 3, the surrogates are the bottleneck. Re-running the Burgers rediscovery with exact numerical derivatives replacing the surrogate derivative nodes would settle whether the frozen surrogates, rather than the optimization, prevent NOMTO-CNO from recovering the equation.","tokens_in":15266,"feed_emoji":"🧮","tokens_out":9386,"duration_ms":85232,"temperature":0.7,"pith_summary":"NOMTO is a symbolic regression framework that turns candidate equations into sparse, differentiable computational graphs whose nodes are neural operators pretrained to approximate library operations such as division, the Gamma function, the Airy function, and derivatives. The paper's central claim is that replacing exact algebraic nodes with these frozen surrogates lets gradient-based search discover symbolic models that Equation Learner (EQL) and Kolmogorov-Arnold Network (KAN) architectures miss, especially expressions with singularities, special functions, and differential operators. The authors support the claim on benchmark expressions and on simulation data, reporting recovery of the two-dimensional heat equation and the coupled two-dimensional Burgers equations with coefficients close to the true values. If correct, NOMTO would extend data-driven governing-equation discovery from local, pointwise operations to nonlocal and special operators without a hand-built library of nonlinear candidate terms.","feed_headline":"Recover heat and Burgers equations from simulation data","feed_subtitle":"NOMTO treats pretrained neural operators as symbolic nodes, extending discovery to special functions and derivatives.","key_machinery":"The load-bearing object is the neural operator block: a pretrained Fourier Neural Operator or Convolutional Neural Operator that maps a discretized input function to the result of one library operation, wrapped with a cropping layer and an inverse tanh projection so values stay in a bounded, differentiable range. These blocks are wired into a fully connected layered graph; every edge carries a scalar weight, and each node applies its library operation to a weighted sum of the previous layer's outputs, so the graph represents all nonlinear combinations of the library up to a chosen depth. The search itself is sparse weight optimization with RMSProp under a modified $\\ell^{1/2}$ penalty, sometimes with periodic pruning of low-energy weights. The surrogates are frozen throughout, which is what supplies gradients for operations such as division, $\\Gamma$, and $\\mathrm{Ai}$ that are singular, undefined on parts of the domain, or otherwise hard to differentiate exactly.","core_discovery":"The central discovery is that a neural operator trained once on random Gaussian-mixture input functions can serve as a fixed, differentiable node inside a symbolic expression graph, standing in for a library operation. Because the surrogate acts on discretized functions rather than on point values, the same block tolerates operations with undefined regions (square root, logarithm), singularities (division, Gamma), special functions (Airy), and derivatives. On the paper's evidence, this design rediscovers the heat equation as $\\partial u/\\partial t = 1.0\\,\\partial^2u/\\partial x^2 + 1.0\\,\\partial^2u/\\partial y^2$ with NOMTO-CNO and identifies the Burgers equations with FNO surrogates, e.g. $\\partial u/\\partial t = -0.9u\\,\\partial u/\\partial x - 0.9v\\,\\partial u/\\partial y + 2.0\\,\\partial^2u/\\partial x^2 + 2.0\\,\\partial^2u/\\partial y^2$; the CNO variant fails on Burgers. The paper presents this as evidence that symbolic discovery of second-order nonlinear PDEs is achievable without manual construction of nonlinear combinations.","pith_inferences":["Beyond the paper: because neural operators can approximate operators rather than only pointwise functions, the same graph construction should extend to integro-differential or delayed equations, a regime the paper does not test.","Beyond the paper: the contrast between FNO and CNO on Burgers suggests surrogate fidelity is the dominant bottleneck; an immediate testable extension is to retrain the surrogates on the target data distribution before running the sparse optimization.","Beyond the paper: the recovered coefficients can be no more accurate than the surrogate's approximation error at the target input magnitudes, which the clipping and tanh projection in the paper partially mitigate but do not remove.","Beyond the paper: adding temporal integral nodes to the library would let the framework attempt memory-effect equations of the sort the abstract motivates, since the derivative nodes already demonstrate that differential operators are discoverable."],"forward_implications":["NOMTO can discover compact equations containing nonlocal, singular, or special operator terms directly from discretized field data, without a manually constructed library of candidate nonlinear terms.","Once a library operation is learned on random Gaussian-mixture functions, the same pretrained surrogate block can be frozen into any new NOMTO graph, so discovery tasks share the same operator library.","The redefined versions of functions such as $\\sqrt{x}$ and $\\ln x$ make the search robust to undefined regions, which the paper argues is acceptable because experimental measurements are well defined on the domain of interest.","With FNO surrogates the method recovers the structure and coefficients of the two-dimensional heat and Burgers equations; with CNO surrogates it recovers the heat equation but not Burgers, indicating that surrogate choice is decisive.","A two-layer computational graph, together with the $\\ell^{1/2}$ sparsity penalty, is sufficient to rediscover second-order nonlinear PDEs on a $50 \\times 50 \\times 50$ spatio-temporal grid from 1,000 simulations."],"supporting_citations":[{"why":"supplies the Fourier Neural Operator used as the surrogate in NOMTO-FNO.","marker":"[25]"},{"why":"supplies the Convolutional Neural Operator used as the surrogate in NOMTO-CNO and as the accuracy benchmark for surrogate selection.","marker":"[23]"},{"why":"provides the Equation Learner architecture that NOMTO generalizes by replacing exact library operations with neural operator surrogates.","marker":"[18]"},{"why":"provides the Kolmogorov-Arnold Network baseline that NOMTO is compared against on benchmark expressions.","marker":"[19]"},{"why":"frames sparse identification of governing equations, the alternative approach NOMTO improves on by avoiding hand-built nonlinear libraries.","marker":"[16]"},{"why":"supplies the Nguyen benchmark expressions used to evaluate NOMTO on elementary symbolic regression.","marker":"[26]"},{"why":"supplies the Keijzer benchmark expressions, including the rational functions with singularities used in the experiments.","marker":"[27]"}],"fun_headline_variants":["Neural operator nodes extend symbolic PDE discovery to nonlocal terms","Rediscover heat and Burgers equations with neural operator surrogates","NOMTO: neural operator surrogates as symbolic building blocks","Symbolic modeling with neural operators finds nonlocal PDEs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the frozen neural operator surrogates, trained only on random Gaussian-mixture functions, approximate every library operation accurately enough on the target data for the sparse graph optimization to recover the correct operator structure and coefficients; the paper's own f2 and f4 results and the NOMTO-CNO Burgers failure show this premise can fail.","fun_headline_variants_meta":{"raw":{"variants":["Neural operator nodes extend symbolic PDE discovery to nonlocal terms","Rediscover heat and Burgers equations with neural operator surrogates","NOMTO: neural operator surrogates as symbolic building blocks","Symbolic modeling with neural operators finds nonlocal PDEs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1387,"prompt_tokens":923,"completion_tokens":464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":392}},"tokens_in":539,"tokens_out":464,"duration_ms":4769,"temperature":1.0,"reasoning_tokens":392,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:29:11.660122+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to compute the surrogate approximation error of each frozen neural operator on the actual target data rather than on the Gaussian-mixture training functions and compare it with the target expression's typical magnitude; if the error ratio tracks the near-zero and negative $R^2$ values in Table 3, the surrogates are the bottleneck. Re-running the Burgers rediscovery with exact numerical derivatives replacing the surrogate derivative nodes would settle whether the frozen surrogates, rather than the optimization, prevent NOMTO-CNO from recovering the equation.","supporting_citations":[{"cited_title":"Convolutional neural operators for robust and accurate learning of pdes","cited_arxiv_id":null,"evidence_quote":"supplies the Convolutional Neural Operator used as the surrogate in NOMTO-CNO and as the accuracy benchmark for surrogate selection."},{"cited_title":"Semantically-based crossover in genetic programming: application to real-valued symbolic regression","cited_arxiv_id":null,"evidence_quote":"supplies the Nguyen benchmark expressions used to evaluate NOMTO on elementary symbolic regression."},{"cited_title":"Improving symbolic regression with interval arithmetic and linear scaling","cited_arxiv_id":null,"evidence_quote":"supplies the Keijzer benchmark expressions, including the rational functions with singularities used in the experiments."}],"review_version":1}