Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A neural operator that couples linear attention with Fourier-domain convolution can simultaneously predict PDE solutions and recover hidden parameter fields, and it does so with far lower cost than its quadratic-attention predecessor.

desk verdict A solid incremental architecture with clean complexity analysis, but the zero-shot generalization claim is undermined by a train/test split that leaks tasks. read the letter →

arxiv 2505.23106 v1 pith:AWGUA5HU submitted 2025-05-29 cs.LG cs.NAmath.NA

Ning Liu , Yue Yu This is my paper
classification cs.LGcs.NAmath.NA MSC 68T0735R3065N21
keywords neuraloperatorlinearattentionFourierinversePDEzero-shotgeneralizationinterpretablekernelDarcyflowMechanicalMNIST
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a neural operator can simultaneously solve forward PDE problems (predict the solution field from the loading) and inverse PDE problems (recover the hidden parameter field or kernel) across many physical systems at once, with no retraining for a new system. The proposed architecture, NIPS, replaces the quadratic-complexity attention and explicit pairwise interaction matrix of Nonlocal Attention Operators with linear attention and a learnable Fourier-domain kernel, turning spatial interactions into FFT operations. On Darcy flow, Mechanical MNIST, and synthetic tissue benchmarks, NIPS reports lower test error than NAO and AFNO while using comparable or fewer parameters, and it scales to 14,641 tokens on one GPU where the quadratic NAO runs out of memory. If correct, this is a concrete step toward interpretable, resolution-invariant models that infer material properties from deformation or flow data.

What carries the argument

The load-bearing object is the attention-based kernel map $K[u_{1:d}, f_{1:d}](x,y)$ built from $L$ iterative blocks that alternately apply channel-independent convolution in Fourier space, via a learnable spectral kernel $R_l$, and linear attention over the $d$ context pairs. The key algebraic step rewrites the nonlocal kernel integral as the product of a Fourier-filtered query term and a key-value inner product, which removes the explicit $N \times N$ pairwise matrix; normalization and random permutation of the $d$ pairs enforce the permutation symmetry $K[u_{1:d}, f_{1:d}] = K[u_{\sigma(1:d)}, f_{\sigma(1:d)}]$. This kernel map carries both tasks: applied to a loading field it solves the forward problem, and read off as a matrix it is interpreted as an inverse stiffness operator whose row sums reveal the hidden permeability or fiber microstructure.

What would settle it

Train NIPS only on diffusion problems with symmetric kernels and then evaluate it zero-shot on an advection-dominated system whose stiffness matrix is nonsymmetric; the paper's own limitation predicts a large accuracy drop and a wrongly symmetric recovered kernel, which would directly bound the zero-shot claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that a data-dependent kernel operator built from iterative Fourier-convolution and linear-attention blocks can learn the mapping from context input-output pairs $(u,f)$ to the PDE kernel $K_b$, so one trained model serves as both a forward solver and an inverse solver. By writing the attention integral as $\mathcal{F}^{-1}(R \cdot \mathcal{F}(pW^Q))(x)$ times an inner product over the key-value pairs, NIPS avoids forming the $N \times N$ attention and projection matrix: the cost per block drops from $O(N^2 d)$ to $O(N d^2 + N d \log N)$, and memory scales with the number of retained Fourier modes $m \ll N$. Experiments show consistent accuracy gains over NAO variants (for example, 1.03% versus 1.45% test error at four layers on Darcy flow, and 1.11% versus 5.27% at four layers on Mechanical MNIST), generalization to out-of-distribution loadings and microstructures and to noisy data, zero-shot transfer to unseen systems, and recovery of two-phase permeability microstructures from the learned kernel, with interpretability assessed against the inverse stiffness matrix $K^{-1}$.

Load-bearing premise

The zero-shot claim stands or falls on whether a new system's governing law resembles the family of equations used in training; the authors state that a diffusion-trained model that expects symmetric interactions will likely fail on advection-dominated systems with non-symmetric interactions.

Editorial extensions

If this is right

  • Zero-shot transfer: a model trained on a family of systems can predict solutions and recover kernels for a new unseen parameter field without retraining, as long as the new kernel is structurally similar to the training family.
  • Scalability on a single GPU: NIPS runs at 14,641 tokens where the quadratic NAO exceeds 40 GB of memory, and at 1,681 tokens it reduces per-epoch time from 58.8 seconds to 8.4 seconds.
  • Interpretable recovery: the learned kernel approximates $K^{-1}$, and row-summing followed by thresholding reconstructs two-phase microstructures with roughly 8 to 16 percent error, degrading only gradually when observational noise is added.
  • Random permutation of context pairs is a necessary ingredient: on Darcy flow the test error falls from 42.04% with one permutation to 2.28% with 100 permutations, confirming that the kernel must be invariant to the ordering of the input-output pairs.
  • The comparison protocol used in the paper holds parameter counts roughly equal across models, so the reported accuracy gains are not attributable simply to larger models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the kernel is learned as a continuous function of $x$ and $y$, the same architecture could be probed for closed-form or symbolic PDE discovery, such as reading off Green's functions or constitutive tensors; the paper only demonstrates this numerically for two-phase fields.
  • A testable consequence the paper does not run: train only on symmetric diffusion problems and test on an advection-dominated problem with a nonsymmetric stiffness matrix; the stated limitation predicts failure, and the error gap would quantify how much of the zero-shot claim rests on kernel-structure priors.
  • The Fourier-mode parameterization suggests a natural resolution-invariance check at much finer grids: if the number of retained modes $m$ stays fixed while $N$ grows, the test error should stay roughly flat if the learned kernel is truly continuous.
  • Connecting to neighboring problems, the same mechanism could serve as a differentiable surrogate for per-specimen material identification in digital twins, where an inverse solve currently requires re-optimizing the model for each new specimen.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper introduces Neural Interpretable PDEs (NIPS), an attention-based neural operator that extends the Nonlocal Attention Operator (NAO) by combining a linear-attention mechanism with learnable Fourier-domain kernels. The central architectural claims are that factoring the NAO kernel-weighted integral reduces the computational cost from quadratic-in-tokens to O(N d^2 + N d log N), and that learning the projection kernel directly in Fourier space reduces the parameter footprint. The paper presents the derivation in Section 3.3, Algorithm 1, and experiments on three benchmarks (Darcy flow, Mechanical MNIST, synthetic tissue learning), reporting that NIPS consistently outperforms NAO and its variants in accuracy and runtime. Additional results cover scalability to 14,641 tokens on a single A100, robustness to noise, an ablation of the permutation augmentation parameter nrand and projection dimension dk, and interpretability experiments that recover a hidden permeability field from the learned kernel. The paper further claims zero-shot generalization to unseen physical systems and the amortization of pairwise interaction costs via Fourier convolution.

Significance. If the zero-shot and accuracy claims are substantiated, NIPS is a useful architectural step for multi-system operator learning: the linear-attention reformulation of NAO is conceptually clean, the O(N d^2 + N d log N) complexity derivation in Section 3.3 is transparent and plausible, and Table 6 gives concrete evidence that the method operates well beyond the memory limits of quadratic NAO (NAO exceeds memory at 3,721 tokens, while NIPS reaches 14,641 tokens on one A100 with 14.4 GB peak memory). The interpretability validation against an externally computed stiffness matrix is a good-faith check of the kernel interpretation, and Table 2's parametric study of nrand and dk is a useful ablation. The explicit Limitations statement about the structural-similarity assumption is a strength. However, the headline generalization claim currently rests on an experimental design that, for the flagship Darcy experiment, does not hold out tasks; the reported numbers therefore support in-context kernel retrieval for seen tasks more strongly than zero-shot transfer to unseen systems.

major comments (4)
  1. [Appendix A.1, Tables 1 and 3] The Darcy-flow evaluation, which is the paper's primary evidence for generalization, does not hold out physical systems. Appendix A.1 describes generating 100 microstructures (tasks), each with 100 loading/solution pairs, then taking 100 random permutations of the pair indices per task, yielding 10,000 samples of which 9,000 are used for training and 1,000 for testing. Because this is a random sample split over the 10,000 permuted samples, not a split over the 100 tasks, every microstructure appears in both training and test (approximately 90 training and 10 test permutations per task). The test tasks in Tables 1 and 3 are therefore not unseen physical systems as claimed in the abstract and in Section 4.1: the model has been trained on other permutations of the same loading/solution pairs for the same permeability field, so the reported ID errors measure in-context retrieval and the permutation-invariance property in Eq. (12), not zero-shot generalization. I ask the authors to re-run the Darcy experiments with a task-level holdout (e.g., train on 80 or 90 microstructures and test on the remaining 20 or 10, reporting the number of held-out tasks), and to state explicitly whether the OOD rows of Table 3 use a task-level split or the same random sample split. This is not a request for a new algorithm; it is a correction of the experimental protocol that supports the paper's central generalization claim.
  2. [Abstract, Sections 4 and 5] The claim that the experiments demonstrate generalization to unseen governing equations is not supported by the reported benchmarks. Each experiment stays within a single PDE family (elliptic Darcy flow; Neo-Hookean hyperelasticity in MMNIST; the synthetic tissue constitutive model), and the test-time variation is in parameter fields or loadings, not in the governing law. The evidence that is consistent with a zero-shot claim is the synthetic tissue experiment (Appendix A.3 uses a 450/50 split over material sets) and possibly the MMNIST experiment, whose split is not described; the OOD scenarios in Table 3 change the covariance of the loading and the permeability field but do not cross PDE families. Given that the paper's own Limitations section concedes that the approach assumes structural similarity between the target and training kernels, the abstract and Section 4 phrasing should be revised to describe what is actually demonstrated (transfer across parameter-field distributions within a family), or supplemented with a cross-family experiment.
  3. [Section 4.1, Tables 1 and 7, Appendix B] Errors of 52.09%, 52.84%, and 98.70% for the AFNO baseline are far outside the range typical of Fourier token-mixing architectures on Darcy-type operator problems, and this casts doubt on whether the comparison is a fair one. Since the paper explicitly states that it follows the fairness protocol of McGreivy and Hakim (2024), the authors should describe how the d context function pairs are encoded for AFNO, whether AFNO is being asked to solve the same transductive in-context task as NIPS (construct a kernel from the context and predict the remaining pairs), and what hyperparameter search was performed. If AFNO is evaluated under a different task formulation (e.g., no context or per-sample regression), the comparison should be reframed or replaced with a correctly configured baseline; as presented, the claim that NIPS consistently surpasses this baseline rests on a configuration that may be mis-specified.
  4. [Tables 1, 3, 4, 5 and Appendix Table 7] Most accuracy results are reported for a single run: only Appendix Table 7 provides mean and standard deviation, and only for the first Darcy configuration. The claim that NIPS consistently surpasses NAO would be stronger with repeated-seed statistics for all tables, particularly for the Table 3 ID configuration where the NIPS/NAO test-error difference is small (4.09% versus 4.15%) and NAO actually reports a slightly better microstructure error (7.09% versus 7.92%). In addition, the configuration behind Table 3 is not reconciled with Table 1: Table 3 lists 327,744 parameters for NIPS, while Table 1 lists 108,692 parameters for the 4-layer, 441-token case and 366,932 for the 1,681-token case; the grid resolution, context size d, and layer count used for Table 3 should be stated so the discrepancy can be understood.
minor comments (7)
  1. [Abstract] The abstract's code URL (https://github.com/fishmoon1234/Nonlocal-Attention-Operator) points to a repository named after the NAO baseline rather than NIPS; please link the actual NIPS implementation and indicate which files correspond to NIPS versus the baselines.
  2. [Appendix A.1] Appendix A.1 states that p(x) is the source term and g(x) is the hydraulic height (the solution), which reverses the roles in Eq. (11), where g is the source and p is the solution; please correct.
  3. [Table 3, Section 4.1] In the ID no-noise row of Table 3, NIPS's microstructure error (7.92%) is larger than NAO's (7.09%), so the sentence that NIPS achieves the lowest errors across nearly all scenarios should name this exception explicitly.
  4. [Abstract and Section 3.3] The phrase channel-independent convolution in Fourier space is not evidently consistent with Eq. (10), in which the learnable R multiplies F(g W^Q) after a channel-mixing linear map; please clarify what is channel-independent in the implementation.
  5. [Table 3] In Table 3, the OOD Scenario 1 with sigma = 0.1 reports a lower test error (3.84%) than the ID case with sigma = 0.01 (4.40%) and the ID case with sigma = 0.1 (9.98%); a brief explanation of this nonmonotonicity would prevent the reader from doubting the noise-robustness claim.
  6. [Eq. (12)] Equation (12) uses sigma(1:d) without definition; please define sigma as a permutation of the index set {1, ..., d}.
  7. [Algorithm 1, Section 4] Algorithm 1 and the text should state explicitly that the test-time kernel is constructed from the same d pairs whose prediction error is being computed (a transductive, in-context protocol); this affects the interpretation of all reported errors, and it is relevant to the task-split issue raised in the major comments.

Circularity Check

1 steps flagged · score 6.0 of 10

Forward zero-shot metric is self-referential: the kernel is built from the same test solutions it is asked to predict, so the reported forward errors measure in-context reconstruction rather than generalization.

  1. self definitional [Section 3.2, Eq. (6); Algorithm 1 Test Phase, Section 4]
    "we propose to replace Kb with a kernel map K[u1:d, f1:d] in the form: G_Kb[f](x) = ∫_Ω K[u1:d, f1:d](x,y) f(y) dy (6) ... Compute prediction error on the test dataset ... E_test_forward := (1/(T_test d)) Σ_{η,i} ||∫_Ω K[˜f^η_{1:d}, ˜u^η_{1:d}; θ*](x,y) ˜f^η_i(y)dy − ˜u^η_i(x)|| / ||˜u^η_i(x)||."

    The forward 'prediction' of u_i is produced by a kernel K whose construction takes u_{1:d} as input, including the very u_i being scored. The predicted target is therefore part of the predictor's definition; the test error evaluates how well the kernel reconstructs its own context, not how well G maps a held-out f to an unseen u. The training loss (Eq. 9) and the test metric are the same functional, so the reported zero-shot forward errors are self-consistency scores. In addition, A.1 builds 10,000 samples by permuting pairs within the same 100 microstructures and then splits 9,000/1,000, so the same microstructure appears in both train and test, further undermining the 'unseen physical system' reading.

full rationale

The derivation of NIPS from the integral-kernel formulation is algebraically self-contained: Eq. (5) is the standard Green's-function ansatz; Eqs. (6)-(8) replace the static kernel with an attention-based kernel map; the linear-attention factorization and the substitution of F(W^P) by a learnable Fourier kernel R are explicit identities that do not assume the target result. The efficiency gains (O(N d^2 + N d log N)) follow from these identities and are corroborated by runtime/memory tables independent of accuracy claims. The interpretability experiment is validated by an external finite-difference solve for B from the learned kernel, not by the training loss. No load-bearing result depends on a self-citation: citations to NAO and Lu & Yu (2025) motivate the design but the architecture and complexity analysis stand alone. The main circularity is in the forward 'zero-shot' evaluation: the kernel is built from the full context including the test pair, so the prediction of u_i is defined in terms of u_i itself; the test metric is the training objective. The additional ID split issue (same microstructures in train/test) is a generalization-evidence gap rather than circularity. Because one central claim—zero-shot forward prediction on unseen systems—reduces by construction, the score is 6.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the expressivity of the chosen kernel parameterization and on the similarity between training and test PDE systems. No new physical entities are introduced. Hyperparameters such as nrand, dk, and the number of Fourier modes are design choices that affect performance but are not evidence of circular reasoning.

free parameters (4)
  • nrand (number of random permutations) = 100 in main experiments
    Hyperparameter for data augmentation; ablations show performance improves with larger nrand, so it is tuned to affect generalization.
  • dk (query-key projection dimension) = 40
    Hyperparameter controlling information compression; ablation shows performance improves with larger dk until saturation.
  • number of retained Fourier modes m = Not specified explicitly; stated m << N
    The learnable Fourier kernel R retains m modes, affecting memory and approximation quality. The value is chosen by the implementation.
  • Layer depth L = 2 or 4 in experiments
    Architecture depth is a design choice; results are reported for different depths.
assumptions (3)
  • domain assumption The data-dependent kernel map K[u_{1:d}, f_{1:d}] is sufficiently expressive to approximate the true Green's function or inverse operator of the PDEs.
    Section 3.2 assumes this parameterization can capture the mapping; no universality or identifiability proof is provided.
  • domain assumption Target kernels in zero-shot tests share structural similarities with training kernels.
    Explicitly stated in the Limitations paragraph: performance degrades when the target kernel significantly deviates from the training distribution.
  • standard math The Fourier convolution of W^P is a valid approximation for the integral over the domain.
    Uses the Fast Fourier Transform to compute convolution, standard for periodic or appropriately padded domains; assumes suitable boundary handling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery." pith.science (2026). https://pith.science/paper/AWGUA5HU

@misc{pith2026250523106,
  author       = {Pith},
  title        = {Pith review of: Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AWGUA5HU}},
  note         = {Machine review of arXiv:2505.23106}
}
read the original abstract

Attention mechanisms have emerged as transformative tools in core AI domains such as natural language processing and computer vision. Yet, their largely untapped potential for modeling intricate physical systems presents a compelling frontier. Learning such systems often entails discovering operators that map between functional spaces using limited instances of function pairs -- a task commonly framed as a severely ill-posed inverse PDE problem. In this work, we introduce Neural Interpretable PDEs (NIPS), a novel neural operator architecture that builds upon and enhances Nonlocal Attention Operators (NAO) in both predictive accuracy and computational efficiency. NIPS employs a linear attention mechanism to enable scalable learning and integrates a learnable kernel network that acts as a channel-independent convolution in Fourier space. As a consequence, NIPS eliminates the need to explicitly compute and store large pairwise interactions, effectively amortizing the cost of handling spatial interactions into the Fourier transform. Empirical evaluations demonstrate that NIPS consistently surpasses NAO and other baselines across diverse benchmarks, heralding a substantial leap in scalable, interpretable, and efficient physics learning. Our code and data accompanying this paper are available at https://github.com/fishmoon1234/Nonlocal-Attention-Operator.

Figures

Figures reproduced from arXiv: 2505.23106 by the authors.

Figure 1
Figure 1. Illustration of the NIPS architecture. NIPS applies physics-informed data augmentation by randomly permuting embedding dimensions to prevent features from being tied to a specific sequence. The data is then tokenized and normalized, followed by several Fourier and linear attention layers that implicitly extract hidden prior knowledge from multiple physical systems. For each system, the final layer maps the last iter… view at source ↗
Figure 2
Figure 2. Interpretable microstructure discovery in experiment 1. ∂Ω) in all samples, which prevents the measurement pairs (p(x), g(x)) from containing information near ∂Ω, making it impossible to identify the kernel at the boundaries. With additional physics knowledge, one can quantitatively assess the interpretability provided by the recovered ker￾nel. Specifically, the learned kernel should correspond to K−1 , where K is t… view at source ↗
Figure 3
Figure 3. Illustration of exemplar MMNIST samples in experiment 2. (a): material parameter field corresponding to different b. (b): displacement fields (second row) ux corresponding to the same loading field (first row) fx. (c): displacement fields (second row) uy corresponding to the same loading field (first row) fy. A.3. Experiment 3 - Synthetic Tissue Learning We generate synthetic tissue data by sampling the fiber orient… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Learning-based Domain Decomposition Method

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A single pretrained neural operator can act as a reusable local solver inside additive Schwarz iterations, solving elliptic PDEs with random microstructures on large, non-convex domains.

Reference graph

Works this paper leans on

29 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [2]

    (b): displacement fields (second row) ux corresponding to the same loading field (first row) fx

    (a): material parameter field corresponding to different b. (b): displacement fields (second row) ux corresponding to the same loading field (first row) fx. (c): displacement fields (second row) uy corresponding to the same loading field (first row) fy. A.3. Experiment 3 - Synthetic Tissue Learning We generate synthetic tissue data by sampling the fiber o...

    work page 2011

  2. [3]

    Note that NIPS is conceptually related to the Performer (Choromanski et al., 2020), which introduces kernel-based approx- imations for efficient self-attention

    All other parameters such as the sparsity threshold and the hard thresholding fraction are set to the default values of 0.01 and 1, respectively. Note that NIPS is conceptually related to the Performer (Choromanski et al., 2020), which introduces kernel-based approx- imations for efficient self-attention. Performer replaces the standard softmax attention ...

    work page 2020

  3. [5]

    Coupling deep learning with full waveform inversion

    Ding, W., Ren, K., and Zhang, L. Coupling deep learning with full waveform inversion. arXiv preprint arXiv:2203.01799,

    work page arXiv

  4. [7]

    Adaptive fourier neural operators: Efficient token mixers for transformers

    Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A., and Catanzaro, B. Adaptive fourier neural operators: Efficient token mixers for transformers. arXiv preprint arXiv:2111.13587,

    arXiv

  5. [8]

    Reinforced inverse scat- tering

    Jiang, H., Khoo, Y ., and Yang, H. Reinforced inverse scat- tering. arXiv preprint arXiv:2206.04186,

    work page arXiv

  6. [9]

    Polysketchformer: Fast transformers via sketches for polynomial kernels

    10 Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery Kacham, P., Mirrokni, V ., and Zhong, P. Polysketchformer: Fast transformers via sketches for polynomial kernels. arXiv preprint arXiv:2310.01655,

    arXiv

  7. [12]

    Deep Neural Operator Enabled Digital Twin Modeling for Additive Manufacturing

    Liu, N., Jafarzadeh, S., and Yu, Y . Domain agnostic fourier neural operators. Advances in Neural Information Pro- cessing Systems, 36, 2023a. Liu, N., Yu, Y ., You, H., and Tatikola, N. Ino: Invariant neu- ral operators for learning complex physical systems with momentum conservation. In International Conference on Artificial Intelligence and Statistics,...

    work page Pith review arXiv

  8. [13]

    and Yu, Y

    Lu, F. and Yu, Y . Transformer learns the cross-task prior and regularization for in-context learning. arXiv preprint arXiv:2505.12138,

    arXiv

Show all 29 references
  1. [15]

    M., Letey, M

    Lu, Y . M., Letey, M. I., Zavatone-Veth, J. A., Maiti, A., and Pehlevan, C. Asymptotic theory of in-context learning by linear attention. arXiv preprint arXiv:2405.11751,

  2. [16]

    Neural inverse operators for solving pde inverse problems

    Molinaro, R., Yang, Y ., Engquist, B., and Mishra, S. Neural inverse operators for solving pde inverse problems. arXiv preprint arXiv:2301.11167,

    arXiv

  3. [17]

    Deep synthe- sis regularization of inverse problems

    Obmann, D., Schwab, J., and Haltmeier, M. Deep synthe- sis regularization of inverse problems. arXiv preprint arXiv:2002.00155,

    2002 arXiv

  4. [18]

    Z., Shen, Z., and Yang, H

    11 Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery Ong, Y . Z., Shen, Z., and Yang, H. IAE-Net: integral autoencoders for discretization-invariant learning. arXiv preprint arXiv:2203.05142,

    arXiv

  5. [19]

    Towards a foundation model for partial differential equation: Multi- operator learning and extrapolation

    Sun, J., Liu, Y ., Zhang, Z., and Schaeffer, H. Towards a foundation model for partial differential equation: Multi- operator learning and extrapolation. arXiv preprint arXiv:2404.12355,

    arXiv

  6. [20]

    Monotone peridynamic neural operator for nonlinear material modeling with conditionally unique solutions

    Wang, J., Tian, X., Zhang, Z., Silling, S., Jafarzadeh, S., and Yu, Y . Monotone peridynamic neural operator for nonlinear material modeling with conditionally unique solutions. arXiv preprint arXiv:2505.01060,

    arXiv

  7. [21]

    J., Zhang, Y ., Luo, T., Xiao, Y ., and Ma, Z

    Xu, Z.-Q. J., Zhang, Y ., Luo, T., Xiao, Y ., and Ma, Z. Fre- quency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06523,

    1901 arXiv

  8. [22]

    and Osher, S

    Yang, L. and Osher, S. J. Pde generalization of in-context operator networks: A study on 1d scalar nonlinear con- servation laws. arXiv preprint arXiv:2401.07364,

    arXiv

  9. [23]

    Pdeformer: Towards a foundation model for one- dimensional partial differential equations

    Ye, Z., Huang, X., Chen, L., Liu, H., Wang, Z., and Dong, B. Pdeformer: Towards a foundation model for one- dimensional partial differential equations. arXiv preprint arXiv:2402.12652,

    arXiv

  10. [26]

    Bold numbers highlight the best method

    Test errors and number of trainable parameters for the Darcy flow problem. Bold numbers highlight the best method. Model NIPS NAO AFNO Test error 2.31%±0.03% 3.25%±0.14% 52.92% ±0.72% 13 Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and In...

    2020

  11. [64]

    • NAO-f: The NAO-f model follows the same configuration as NAO, except that LayerNorm is applied across both the token and projection dimensions in all layers

    and LeakyReLU activation functions. • NAO-f: The NAO-f model follows the same configuration as NAO, except that LayerNorm is applied across both the token and projection dimensions in all layers. • NAO-W p: The NAO-W p model follows the same configuration as NAO, except that a...

    2021

  12. [1991]

    Goswami, S., Bora, A., Yu, Y ., and Karniadakis, G. E. Physics-informed neural operators. 2022 arXiv preprint arXiv:2207.05748,

    2022 arXiv

  13. [2001]

    Non- local kernel network (NKN): A stable and resolution- independent deep neural network

    You, H., Yu, Y ., D’Elia, M., Gao, T., and Silling, S. Non- local kernel network (NKN): A stable and resolution- independent deep neural network. Journal of Compu- tational Physics, pp. arXiv preprint arXiv:2201.02217, 2022a. You, H., Zhang, Q., Ross, C. J., Lee, C.-H., and Yu...

    arXiv

  14. [2018]

    Modno: Multi operator learning with distributed neural operators

    Zhang, Z. Modno: Multi operator learning with distributed neural operators. arXiv preprint arXiv:2404.02892,

    arXiv

  15. [2019]

    let data talk: data- regularized operator learning theory for inverse problems

    Chen, K., Wang, C., and Yang, H. let data talk: data- regularized operator learning theory for inverse problems. arXiv preprint arXiv:2310.09854,

    arXiv

  16. [2020]

    Neural operator: Graph kernel network for partial differential equations

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020a. Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Stuart, ...

    2003 arXiv

  17. [2021]

    Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations’ operator learning. arXiv preprint arXiv:2205.13671,

    arXiv

  18. [2022]

    Polynormer: Polynomial- expressive graph transformer in linear time

    Deng, C., Yue, Z., and Zhang, Z. Polynormer: Polynomial- expressive graph transformer in linear time. arXiv preprint arXiv:2403.01232,

    arXiv

  19. [2023]

    Cho, S. W. and Son, H. Physics-informed deep inverse op- erator networks for solving pde inverse problems. arXiv preprint arXiv:2412.03161,

    arXiv

  20. [2024]

    Rethinking attention with performers

    Choromanski, K., Likhosherstov, V ., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794,

    2009 arXiv

  21. [2025]

    Lu, L., Jin, P., and Karniadakis, G. E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of opera- tors. arXiv preprint arXiv:1910.03193,

    1910 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.