Pith. sign in

REVIEW 4 major objections 7 minor 98 references

A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read This paper claims a transformer neural operator trained only on PDE residuals generalizes to unseen initial and boundary conditions, with relative errors one-fifth to one-third of the leading physics-informed baseline in five flow problems.

desk verdict A real architectural contribution (cross-attention boundary conditioning) whose headline generalization numbers are inflated by tuning on the unseen set and a favorable error metric; worth reviewing with claims corrected. read the letter →

arxiv 2412.09009 v4 pith:23NBUFEL submitted 2024-12-12 cs.LG physics.comp-ph

classification cs.LGphysics.comp-ph
keywords physics-informedneuraloperatortransformercross-attentionlearningPDEgeneralizationinitialandboundaryconditionsNavier-Stokesequationsphysicsloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PINTO is a neural operator designed to learn the solution map of an initial boundary value problem for a whole family of initial and boundary conditions, rather than for one instance. The central claim is that this map can be learned using only the PDE residual as the training signal, with no simulation or observation data, because a cross-attention mechanism makes each predicted point's internal representation depend on the full boundary function. On advection, Burgers, Kovasznay, Beltrami, and lid-driven cavity flows, the paper reports relative errors on unseen conditions roughly one-fifth to one-third those of the leading physics-informed operator baseline. It also reports accurate solutions for advection and Burgers at times beyond the training horizon. If the claim holds, a single trained model could replace repeated numerical solves for new conditions, which is the practical payoff of operator learning.

What carries the argument

The load-bearing component is the iterative kernel integral operator unit, a multihead cross-attention mechanism implementing the operator-theoretic layer $\mu_{j+1}(X) = \sigma(W_j\mu_j(X) + K_j\mu_j(X) + \gamma_j(X))$. In each unit, the encoded interior query point is scored against encoded boundary coordinates by a scaled dot product; softmax over all boundary tokens yields attention scores, and the output is the attention-weighted sum of encoded boundary values, added through a residual connection and Swish activation. Because the boundary function enters as a variable-length sequence of keys and values, repeated units produce a boundary-aware hidden representation, and a projection MLP maps that representation to the solution field. Lifting operators for the query point, boundary coordinates, and boundary values are all MLPs, and with a single query point the attention cost is $O(N m^2)$ rather than $O(MN m^2)$.

What would settle it

Train PINTO on advection or Burgers with the paper's sinusoidal or random-field initial conditions, then evaluate on an initial condition from a different family, such as a narrow Gaussian pulse or a discontinuous square wave, comparing against a high-resolution numerical solver; if the relative error jumps well above the reported 2–5% range, the generalization is limited to the training distribution. A second, cleaner check is to redo the hyperparameter selection without using the unseen-condition errors (Appendix B) and see whether the reported error gap over the baseline persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that the map $\mathcal{G}: \mathcal{A}\to\mathcal{H}$ from initial/boundary conditions to PDE solutions can be learned by a transformer-based operator, PINTO, trained exclusively with the physics loss (the PDE residual and boundary residual in Eq. 3), with no simulation or observation data. The reported result is that this works: for the advection equation, Burgers equation, Kovasznay flow, Beltrami flow, and lid-driven cavity flow, the trained model achieves mean relative errors on unseen conditions of roughly 0.41% to 5.24%, against 2.26% to 11.85% for the physics-informed operator baseline, about one-fifth to one-third of the baseline's error. For advection and Burgers, the same model also produces accurate solutions at times $t>1$, beyond the temporal collocation points used in training, which the baseline does not. The paper attributes this behavior to its cross-attention kernel integral operator units, which make the representation of every interior query point depend on the whole initial/boundary function rather than on a fixed-size encoding.

Load-bearing premise

The argument rests on assuming that minimizing the PDE residual at finitely many collocation points and finitely many sampled initial/boundary conditions produces an operator that stays accurate for all other conditions in the same family, and the paper provides no error bound for that step.

Editorial extensions

If this is right

  • A single PINTO model trained with physics loss can be reused for new initial and boundary conditions without retraining and without simulation data, turning PDE solving into fast inference.
  • For advection and Burgers, the model predicts correct solutions at $t>1$, beyond the training collocation domain, indicating that the learned operator carries temporal dynamics rather than memorizing the training window.
  • Because boundary conditions are encoded as variable-length sequences, the same trained architecture accepts different numbers of boundary points at test time without architectural changes.
  • The cross-attention unit is a general conditioning mechanism that can be inserted into other neural operators, such as Fourier or time-marching operators, to make them aware of input functions.
  • With reported inference times of tens to hundreds of milliseconds, the approach is a candidate replacement for repeated numerical solves in engineering settings such as wind-farm layout and digital twins.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The reported generalization is demonstrated only within the same parametric family used for training (sinusoidal initial conditions, Reynolds numbers in a bounded range, lid velocities near the training values), so the title-level claim of 'any' initial and boundary condition is stronger than the evidence shown.
  • Inference: Because Appendix B reports selecting hyperparameters by validation on unseen-condition errors, a conservative re-evaluation with hyperparameters fixed before seeing the test set would clarify how much of the reported error gap is architectural rather than tuning advantage.
  • Inference: The time-extrapolation results suggest a testable extension: check whether error growth beyond $t>6$ stays bounded and whether the model can resolve shock formation in Burgers without additional training.
  • Inference: The same key-value conditioning idea could be applied to geometry, PDE coefficients, or forcing terms; a concrete next experiment would be training PINTO on obstacle positions encoded as boundary sequences, which the paper notes but does not demonstrate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PINTO, a transformer-based neural operator whose cross-attention 'iterative kernel integral' units condition the solution-domain query points on sequences of initial/boundary coordinates and values. The model is trained purely on strong-form PDE residuals at collocation points, without simulation data, and is evaluated on advection, Burgers, Kovasznay, Beltrami, and lid-driven cavity problems, with PI-DeepONet as the baseline. The authors claim relative errors of one-fifth to one-third of PI-DeepONet's on unseen initial/boundary conditions and an ability to extrapolate the advection and Burgers solutions to times beyond the training domain. The architecture is plausible and the experiments are extensive, but two load-bearing problems undermine the headline claims: the ratios in the paper's own Table 2 contradict the stated 20-33% range for Burgers (44%) and the lid-driven cavity (46%), and the hyperparameters were selected using the same unseen-condition set that later serves as the reported test set (Appendix B), so the reported generalization is partly selected-for.

Significance. The goal addressed by PINTO is genuinely useful: a single model trained without simulation data that solves a PDE family across initial/boundary conditions. The paper's strengths include the public code and data release (GitHub and Zenodo), the documented hyperparameter studies, the matched parameter budgets between PINTO and PI-DeepONet, and the honest disclosure of the modified error metric in Appendix A. The cross-attention conditioning of query points on boundary tokens is a sensible and transferable architectural idea, and the temporal-extrapolation experiments are interesting. If, after a clean re-evaluation, the generalization advantage over physics-informed DeepONet holds at the stated margins, this would be a useful empirical contribution to simulation-free operator learning. At present, however, the abstract and conclusion overstate the evidence: only one of the five test cases falls in the claimed one-fifth-to-one-third ratio range, and the evaluation protocol makes the 'unseen' numbers partially selected, so the contribution cannot be assessed until the quantitative claims are redone.

major comments (4)
  1. [Abstract, §3.2, §5, Table 2] The paper's central quantitative claim — 'relative errors during testing ... only one-fifth to one-third of other leading physics informed operator learning methods' (Abstract) and 'only 20% to 33% of the relative error of PI-DeepONet's solutions' (Section 5) — is contradicted by the paper's own Table 2. Computing the unseen-condition ratios from that table gives: advection 2.85/11.26 = 25%, Burgers 5.24/11.85 = 44%, Kovasznay 0.41/2.26 = 18%, Beltrami 0.60/4.89 = 12%, and lid-driven cavity 2.78/6.08 = 46%. Only the advection case lies in the claimed 20-33% range; Burgers and the lid-driven cavity are close to half the baseline error, and Kovasznay and Beltrami fall below one-fifth. Additionally, Section 3.2 states that the PI-DeepONet error is 'three times' that of PINTO for Burgers, whereas the actual ratios from Table 2 are 2.26x (unseen) and 2.66x (seen), and for seen advection conditions PINTO (2.11%) is actually worse than PI-DeepONet (1.35%). The quantitative claims in the abstract, Section 3, and Section 5 must be corrected case-by-case to match the data.
  2. [§3.1, Appendix B, Table B.5] The generalization evidence is compromised by the hyperparameter-selection protocol. Section 3.1 states that the number of CAUs, sequence length, learning rate, and activation function were chosen 'using validation on unseen initial conditions', and Appendix B confirms that the authors 'chosen the hyperparameters that gave good predictions with low relative errors on the unseen conditions', with Table B.5 reporting the mean relative error on exactly those unseen conditions as the selection criterion. The 20 held-out conditions used to compute the 'unseen' errors in Table 2 are therefore the same set used for model selection; consistently, the validation errors in Table B.5 for advection (2.47-2.61%) nearly coincide with the reported unseen error in Table 2 (2.85%). The paper should either use a train/validation/test three-way split with the test set untouched during selection, or explicitly label the Table 2 'unseen' column as validation performance and temper the generalization claim. It should also document whether PI-DeepONet received equivalent validation-based tuning; as reported, the comparison is asymmetric because only PINTO's hyperparameters were tuned on the evaluation set.
  3. [Appendix A, Eq. A.5, Tables 2, A.3-A.4] The reported 'relative error' is the adjusted metric |h - h_hat|/(1 + |h|), which Appendix A itself describes as a modification to handle near-zero true values; it is not a standard relative L2 error, and the denominator compresses errors at solution values of order one or larger. The abstract and conclusion state the 'one-fifth to one-third' claim without this qualification, and even with this favorable metric the claim fails for two test cases (see major comment above). The comparison should be recomputed under a standard relative L2 error; Tables A.3 and A.4 show that the ratios are also not consistently in the claimed range under RMSE, nRMSE, or MAPE (e.g., the Kovasznay unseen nRMSE ratio is 0.000715/0.001758 = 0.41). In addition, the standard deviations in Table 2 frequently exceed the means (e.g., advection unseen 2.85% +/- 4.73%, Burgers seen 4.81% +/- 4.43%), indicating large variability across the 20 test conditions, and no significance test is reported for the PINTO-versus-PI-DeepONet differences. Per-condition error distributions and confidence intervals should be reported.
  4. [§2.1, Eq. 3; §1.2; §5] The paper claims an operator that generalizes 'for any initial and boundary condition', but no consistency or error-bound argument links the training objective in Eq. 3 (strong-form residual over a fixed set of collocation points and K sampled conditions) to the operator condition in Eq. 2 for all b in A. As an empirical paper this gap is acceptable only if the claims are scoped: each test family is narrow (superpositions of sinusoids on the advection equation, Gaussian random fields for Burgers, Reynolds-number intervals, and lid speeds), only 20 unseen conditions are tested per case, and the unseen conditions are interpolating within those families. I recommend explicitly stating that generalization is demonstrated for the sampled parametric families, and either deferring 'any condition' claims or adding a brief discussion of relevant operator-learning generalization theory. Relatedly, the claim in Section 5 that temporal extrapolation is 'the first result of this nature in the neural operator literature' is not substantiated by a literature search and should be softened to 'to our knowledge'.
minor comments (7)
  1. [Abstract, §1 and §3] The Abstract and Section 1 refer to 'other leading physics informed operator learning methods' in the plural, but the only baseline evaluated is PI-DeepONet; please reword to refer to the single baseline or add additional physics-informed operator baselines.
  2. [Table 2, §2.1, §4] There are several typos that should be corrected: 'Condtions' in the Table 2 header, 'emperical' in Section 2.1, and 'diffrence' in Section 4.
  3. [§2.1, Eq. 3] The notation in Eq. 3 uses G(Theta; X) while Section 2.1 defines G_theta(X, b; Theta); please unify the notation.
  4. [§3.3.1 and §3.3.3 vs Table C.8] Sections 3.3.1 and 3.3.3 state that errors are evaluated on a 64x64 grid, but Table C.8 lists 256x256 grids for the Kovasznay and lid-driven cavity flows; please clarify which grids the reported metrics in Table 2 are computed on.
  5. [Table C.8] Table C.8 is titled 'Training and inference time' but contains only inference time; please report training times as well, since Section 4's complexity discussion compares training behavior.
  6. [§3.1, Eq. 9b] Equation (9b) and the surrounding text should specify the value of n_max used to generate the training and testing initial conditions and confirm that the unseen conditions are drawn from the same wavenumber and amplitude range, so the interpolation-versus-extrapolation status of the test set is explicit.
  7. [Appendix D] The finite-difference interpretation in Appendix D omits the softmax normalization in Eq. 7; since the attention weights are normalized unlike the row-stochastic matrix shown, the derivation should be labeled as an intuition rather than an equivalence.

Circularity Check

1 steps flagged · score 3.0 of 10

Generalization claim is partly selected on the same unseen-condition set used for evaluation, though the core physics-loss training is independent.

  1. fitted input called prediction [Section 3.1 (Advection) and Appendix B, Table B.5; results reported in Table 2]
    "Hyperparameter tuning was performed to select the number of CAUs, sequence length, learning rate, and activation functions using validation on unseen initial conditions, as described in Table B.5. ... Since here we are emphasizing the applicability of our proposed architecture to generalize for unseen initial and boundary conditions, we have chosen the hyperparameters that gave good predictions with low relative errors on the unseen conditions. ... Here, the mean of relative error across all the test cases is indicated as the validation metrics."

    The 20 initial conditions that Section 3.1 calls 'unseen situations' are the same conditions on which Appendix B selects hyperparameters (CAUs, sequence length, learning rate, activation) by minimizing the mean relative error reported as 'validation metrics' in Table B.5. Table 2 then reports the relative error on 'Unseen Conditions' as if it were an untouched holdout. The headline generalization claim is therefore partly a selected-for value: the configuration was chosen to minimize exactly the error later exhibited as a prediction. This is fitted-input-called-prediction at the evaluation level, although the PDE-residual training objective itself is not circular.

full rationale

The core derivation is not circular: PINTO is trained by minimizing physics and boundary residuals (Eq. 3) without supervision from target solutions, and the validation uses external PDEBENCH, analytical, and finite-volume references; no load-bearing self-citation chain is present. One evaluation-level circularity exists: hyperparameters were tuned on the very unseen-condition set later reported as the test set, so the reported 'unseen' errors are not a fresh holdout. This is a partial leakage rather than a reduction of the method to its inputs; the architecture could still fail on truly unseen conditions. Separately, the abstract's 'one-fifth to one-third' claim is not supported by Table 2 for Burgers (5.24/11.85 = 0.44) and lid-driven cavity (2.78/6.08 = 0.46), and the reported metric is the modified Eq. A.5 rather than a standard relative L2 error; those are correctness concerns, not additional circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the usual neural operator approximation assumptions, on an unproved generalization assumption for physics-loss training, and on a set of hand-selected hyperparameters. No new physical entities are introduced.

free parameters (5)
  • Physics loss weights lambda_1 and lambda_2 = Not reported for most cases; boundary loss weight 100 for lid-driven cavity zero boundary
    Eq. 3 balances the PDE residual and the initial/boundary residual; this balance affects which solution the network converges to, and the values are chosen by hand.
  • Number of cross-attention units (CAUs) = 1 for Navier-Stokes cases, 2 to 3 for Burgers, 1 to 2 for advection
    Selected by validation on unseen conditions in Table B.5, not derived from theory.
  • Sequence length L of BPE and BVE tokens = 40 to 100 depending on the test case
    Longer sequences reduce validation error (Fig. 4) and the length is chosen per case in Table B.5.
  • Embedding dimension, number of attention heads, and key dimension = 64, 2 heads, key dimension 64 for all cases
    Architecture hyperparameters chosen by hand with no sensitivity analysis reported.
  • Learning rate, epochs, optimizer and scheduler = 1e-5 to 1e-3, 20,000 to 50,000 epochs, Adam or AdamW with per-case schedules
    Selected per test case in Table B.7 using validation on unseen conditions.
assumptions (4)
  • standard math Neural operators of the form in Eq. 4, with lifting, kernel integration, and projection layers, can approximate the solution map G.
    Treated as established from prior neural operator theory (Li et al. [25]); not proved in this paper.
  • domain assumption The PDEs considered are well-posed and have unique solutions for every sampled initial or boundary condition b in A.
    Needed for Eqs. 1 through 3 to define a single-valued map G; this is standard for Navier-Stokes and scalar conservation laws in the considered regimes.
  • domain assumption A finite discretization of the boundary at L points is sufficient to represent the initial or boundary condition for the cross-attention operator.
    Attention keys and values use L samples, and the paper provides no error analysis linking L to approximation accuracy.
  • ad hoc to paper Minimizing the strong-form PDE residual at random collocation points is sufficient to enforce the PDE for all points in the domain and for all b in A.
    This is the central training assumption for physics-informed operators; the paper gives empirical evidence but no theoretical guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems." pith.science (2026). https://pith.science/paper/23NBUFEL

@misc{pith2026241209009,
  author       = {Pith},
  title        = {Pith review of: A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23NBUFEL}},
  note         = {Machine review of arXiv:2412.09009}
}
read the original abstract

Initial boundary value problems arise commonly in applications with engineering and natural systems governed by nonlinear partial differential equations (PDEs). Operator learning is an emerging field for solving these equations by using a neural network to learn a map between infinite dimensional input and output function spaces. These neural operators are trained using a combination of data (observations or simulations) and PDE-residuals (physics-loss). A major drawback of existing neural approaches is the requirement to retrain with new initial/boundary conditions, and the necessity for a large amount of simulation data for training. We develop a physics-informed transformer neural operator (named PINTO) that efficiently generalizes to unseen initial and boundary conditions, trained in a simulation-free setting using only physics loss. The main innovation lies in our new iterative kernel integral operator units, implemented using cross-attention, to transform the PDE solution's domain points into an initial/boundary condition-aware representation vector, enabling efficient learning of the solution function for new scenarios. The PINTO architecture is applied to simulate the solutions of important equations used in engineering applications: advection, Burgers, and steady and unsteady Navier-Stokes equations (three flow scenarios). For these five test cases, we show that the relative errors during testing under challenging conditions of unseen initial/boundary conditions are only one-fifth to one-third of other leading physics informed operator learning methods. Moreover, our PINTO model is able to accurately solve the advection and Burgers equations at time steps that are not included in the training collocation points. The code is available at https://github.com/quest-lab-iisc/PINTO

Figures

Figures reproduced from arXiv: 2412.09009 by the authors.

Figure 1
Figure 1. Schematic of PINTO: Our neural operator has three stages (i) Query Point, Initial/Boundary Point and Initial/Boundary Value (lifting operators) (ii) Cross Attention Units (iterative kernel intergral operators) and (iii) output projection dense layers. 2.3. Practical Implementation [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Advection Equation: Initial conditions (first row), PINTO solutions (second row), corresponding PDEBENCH data (third row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions. For training PINTO, 2000 collocation, and 250 initial and boundary points are considered in the domain [0, 1] × [0, 1]. For comparison, the PI-DeepONet model is also tr… view at source ↗
Figure 3
Figure 3. Advection Equation: PINTO, PI-DeepONet and numerical solutions (PDEBENCH) for seen and unseen initial conditions at t=0.01, 0.5, 2. The first two columns are results for seen and the last columns are for the unseen initial conditions. A landmark A is shown in the solution for unseen initial conditions (ICs) to visualize how the wave is propagating in time [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Learning curves during PINTO training for advection equation: a) Training (solid line) and validation (dashed line) loss curves for different learning rates. b) Training (solid line) and validation (dashed line) loss curves for different sequence lengths of initial con…
Figure 5
Figure 5. Figure 5: Burgers Equation: Initial conditions (first row), PINTO solutions (second row), corresponding numerical solution (third row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions. 3.3.1. Kovaszna…
Figure 6
Figure 6. Figure 6: Burgers Equation: PINTO, PI-DeepONet and numerical solutions for seen and unseen initial conditions at t=0.01, 0.5, 2. The first two columns are results for seen and the last columns are for the unseen initial conditions. A landmark A is shown in the solution for unsee…
Figure 7
Figure 7. Figure 7: Burgers Equation: Evolution of mean relative error in time for 20 different unseen initial conditions. 40, 000 epochs using the Adam optimizer with a learning rate of 5e − 4. The PI-DeepONet is also trained in the same setting for comparison. We used the relative error…
Figure 8
Figure 8. Figure 8: Kovasznay Flow: PINTO solutions (first row), corresponding analytical solution (second row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions (Re). Flow streamlines are overlaid on a backgrou…
Figure 9
Figure 9. Figure 9: Beltrami Flow: PINTO solutions (first row), corresponding analytical solution (second row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions (Re) at solution time step t = 0.5. Flow streamlin…
Figure 10
Figure 10. Figure 10: Mean relative error of PINTO predictions for the Kovasznay and Beltrami flows at di [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Lid Driven Cavity Flow: PINTO solutions (first row), corresponding analytical solution (second row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) boundary conditions (Lid Velocity). Flow streamlines are ov…
Figure 12
Figure 12. Figure 12: Lid Driven Cavity Flow: PINTO and PI-DeepONet solutions (first row), corresponding numerical solution (second row) and relative error (third row) for unseen lid velocity interpolation (1.2) and extrapolation (3.5). Flow streamlines are overlaid on a background of the …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

98 extracted references · 54 canonical work pages

  1. [1]

    Q. Wang, M. Ihme, Y .-F. Chen, J. Anderson, A tensorflow simulation framework for scientific computing of fluid flows on tensor processing units, Computer Physics Communications 274 (2022) 108292

  2. [2]

    I. G. Ion, D. Loukrezis, H. De Gersem, Tensor train based isogeometric analysis for pde approximation on parameter dependent geometries, Computer Methods in Applied Mechanics and Engineering 401 (2022) 115593

  3. [3]

    Richter, L

    L. Richter, L. Sallandt, N. Nüsken, Solving high-dimensional parabolic pdes using the tensor train format, in: International Conference on Machine Learning, PMLR, 2021, pp. 8998–9009

  4. [4]

    D. W. Heermann, A. N. Burkitt, Parallel algorithms in computational science, V ol. 24, Springer Science & Business Media, 2012

  5. [5]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Neural operator: Graph kernel network for partial differential equations, arXiv preprint arXiv:2003.03485 (2020)

  6. [6]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv preprint arXiv:2010.08895 (2020)

  7. [7]

    L. Lu, P. Jin, G. Pang, Z. Zhang, G. E. Karniadakis, Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nature machine intelligence 3 (3) (2021) 218–229

  8. [8]

    H. You, Y . Yu, M. D’Elia, T. Gao, S. Silling, Nonlocal kernel network (nkn): A stable and resolution-independent deep neural network, Journal of Computational Physics 469 (2022) 111536

Show all 98 references
  1. [9]

    Kovachki, Z

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, A. Anandkumar, Neural operator: Learning maps between function spaces with applications to pdes, Journal of Machine Learning Research 24 (89) (2023) 1–97

  2. [10]

    Raissi, P

    M. Raissi, P. Perdikaris, G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial di fferential equations, Journal of Computa- tional Physics 378 (2019) 686–707. doi:https://doi.org/10.101...

  3. [11]

    L. Lu, X. Meng, Z. Mao, G. E. Karniadakis, Deepxde: A deep learning library for solving differential equations, SIAM review 63 (1) (2021) 208–228

  4. [12]

    G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nature Reviews Physics 3 (6) (2021) 422–440

  5. [13]

    L. Yang, X. Meng, G. E. Karniadakis, B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data, Journal of Computational Physics 425 (2021) 109913

  6. [14]

    S. Cai, Z. Mao, Z. Wang, M. Yin, G. E. Karniadakis, Physics-informed neural networks (pinns) for fluid me- chanics: A review, Acta Mechanica Sinica 37 (12) (2021) 1727–1738

  7. [15]

    S. A. Faroughi, N. M. Pawar, C. Fernandes, M. Raissi, S. Das, N. K. Kalantari, S. Kourosh Mahjour, Physics- guided, physics-informed, and physics-encoded neural networks and operators in scientific computing: Fluid and solid mechanics, Journal of Computing and Information Scie...

  8. [16]

    Yamazaki, A

    Y . Yamazaki, A. Harandi, M. Muramatsu, A. Viardin, M. Apel, T. Brepols, S. Reese, S. Rezaei, A finite element- based physics-informed operator learning framework for spatiotemporal partial di fferential equations on arbi- trary domains, Engineering with Computers 41 (1) (2025) 1–29

  9. [17]

    Kaewnuratchadasorn, J

    C. Kaewnuratchadasorn, J. Wang, C.-W. Kim, Physics-informed neural operator solver and super-resolution for solid mechanics, Computer-Aided Civil and Infrastructure Engineering 39 (22) (2024) 3435–3451. 24

  10. [18]

    V . M. Nguyen-Thanh, X. Zhuang, T. Rabczuk, A deep energy method for finite deformation hyperelasticity, European Journal of Mechanics-A/Solids 80 (2020) 103874

  11. [19]

    D. W. Abueidda, S. Koric, R. A. Al-Rub, C. M. Parrott, K. A. James, N. A. Sobh, A deep learning energy method for hyperelasticity and viscoelasticity, European Journal of Mechanics-A/Solids 95 (2022) 104639

  12. [20]

    J. He, D. Abueidda, R. A. Al-Rub, S. Koric, I. Jasiuk, A deep learning energy-based method for classical elastoplasticity, International Journal of Plasticity 162 (2023) 103531

  13. [21]

    Pathak, S

    J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, et al., Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators, arXiv preprint arXiv:2202.11214 (2022)

  14. [22]

    G. Wen, Z. Li, K. Azizzadenesheli, A. Anandkumar, S. M. Benson, U-fno—an enhanced fourier neural operator- based deep-learning model for multiphase flow, Advances in Water Resources 163 (2022) 104180

  15. [23]

    Bonev, T

    B. Bonev, T. Kurth, C. Hundt, J. Pathak, M. Baust, K. Kashinath, A. Anandkumar, Spherical fourier neural operators: Learning stable dynamics on the sphere, in: International conference on machine learning, PMLR, 2023, pp. 2806–2823

  16. [24]

    H. You, Q. Zhang, C. J. Ross, C.-H. Lee, Y . Yu, Learning deep implicit fourier neural operators (ifnos) with applications to heterogeneous material modeling, Computer Methods in Applied Mechanics and Engineering 398 (2022) 115296

  17. [25]

    Z. Li, D. Z. Huang, B. Liu, A. Anandkumar, Fourier neural operator with learned deformations for pdes on general geometries, Journal of Machine Learning Research 24 (388) (2023) 1–26

  18. [26]

    R. J. George, J. Zhao, J. Kossaifi, Z. Li, A. Anandkumar, Incremental spatial and spectral learning of neural operators for solving large-scale pdes (2024). arXiv:2211.15188

  19. [27]

    Kovachki, S

    N. Kovachki, S. Lanthaler, S. Mishra, On universal approximation and error bounds for fourier neural operators, Journal of Machine Learning Research 22 (290) (2021) 1–76

  20. [28]

    Kovachki, Z

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, A. Anandkumar, Neural operator: Learning maps between function spaces with applications to pdes, Journal of Machine Learning Research 24 (89) (2023) 1–97. URL http://jmlr.org/papers/v24/21-1524.html

  21. [29]

    Lehmann, F

    F. Lehmann, F. Gatti, M. Bertin, D. Clouteau, 3d elastic wave propagation with a factorized fourier neural operator (f-fno), Computer Methods in Applied Mechanics and Engineering 420 (2024) 116718

  22. [30]

    S. Wang, H. Wang, P. Perdikaris, Learning the solution operator of parametric partial differential equations with physics-informed deeponets, Science advances 7 (40) (2021) eabi8605

  23. [31]

    Lanthaler, S

    S. Lanthaler, S. Mishra, G. E. Karniadakis, Error estimates for deeponets: A deep learning framework in infinite dimensions, Transactions of Mathematics and Its Applications 6 (1) (2022) tnac001

  24. [32]

    J. He, S. Kushwaha, J. Park, S. Koric, D. Abueidda, I. Jasiuk, Sequential deep operator networks (s-deeponet) for predicting full-field solutions under time-dependent loads, Engineering Applications of Artificial Intelligence 127 (2024) 107258

  25. [33]

    W. Xu, Y . Lu, L. Wang, Transfer learning enhanced deeponet for long-time prediction of evolution equations, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 37, 2023, pp. 10629–10636

  26. [34]

    W. Li, M. Z. Bazant, J. Zhu, Phase-field deeponet: Physics-informed deep operator neural network for fast simulations of pattern formation governed by gradient flows of free-energy functionals, Computer Methods in Applied Mechanics and Engineering 416 (2023) 116299. 25

  27. [35]

    Goswami, M

    S. Goswami, M. Yin, Y . Yu, G. E. Karniadakis, A physics-informed variational deeponet for predicting crack path in quasi-brittle materials, Computer Methods in Applied Mechanics and Engineering 391 (2022) 114587

  28. [36]

    Goswami, A

    S. Goswami, A. Bora, Y . Yu, G. E. Karniadakis, Physics-informed deep neural operator networks, in: Machine Learning in Modeling and Simulation: Methods and Applications, Springer, 2023, pp. 219–254

  29. [37]

    Goswami, A

    S. Goswami, A. D. Jagtap, H. Babaee, B. T. Susi, G. E. Karniadakis, Learning sti ff chemical kinetics using extended deep neural operators, Computer Methods in Applied Mechanics and Engineering 419 (2024) 116674

  30. [38]

    Haghighat, U

    E. Haghighat, U. bin Waheed, G. Karniadakis, En-deeponet: An enrichment approach for enhancing the ex- pressivity of neural operators with applications to seismology, Computer Methods in Applied Mechanics and Engineering 420 (2024) 116681

  31. [39]

    Kobayashi, J

    K. Kobayashi, J. Daniell, S. B. Alam, Improved generalization with deep neural operators for engineering sys- tems: Path towards digital twin, Engineering Applications of Artificial Intelligence 131 (2024) 107844

  32. [40]

    Zhang, S

    J. Zhang, S. Zhang, J. Shen, G. Lin, Energy-dissipative evolutionary deep operator neural networks, Journal of Computational Physics 498 (2024) 112638

  33. [41]

    J. He, S. Koric, D. Abueidda, A. Najafi, I. Jasiuk, Geom-deeponet: A point-cloud-based deep operator network for field predictions on 3d parameterized geometries, Computer Methods in Applied Mechanics and Engineering 429 (2024) 117130

  34. [42]

    Koric, D

    S. Koric, D. W. Abueidda, Data-driven and physics-informed deep learning operators for solution of heat conduc- tion equation with parametric heat source, International Journal of Heat and Mass Transfer 203 (2023) 123809

  35. [43]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, A. Stuart, K. Bhattacharya, A. Anandkumar, Multipole graph neural operator for parametric partial differential equations, Advances in Neural Information Processing Systems 33 (2020) 6755–6766

  36. [44]

    Raonic, R

    B. Raonic, R. Molinaro, T. De Ryck, T. Rohner, F. Bartolucci, R. Alaifari, S. Mishra, E. de Bézenac, Convo- lutional neural operators for robust and accurate learning of pdes, Advances in Neural Information Processing Systems 36 (2024)

  37. [45]

    Tiwari, N

    K. Tiwari, N. Krishnan, et al., Cono: Complex neural operator for continuous dynamical systems, arXiv preprint arXiv:2310.02094 (2023)

  38. [46]

    Gupta, X

    G. Gupta, X. Xiao, P. Bogdan, Multiwavelet-based operator learning for di fferential equations, Advances in neural information processing systems 34 (2021) 24048–24062

  39. [47]

    Tripura, S

    T. Tripura, S. Chakraborty, Wavelet neural operator for solving parametric partial differential equations in com- putational mechanics problems, Computer Methods in Applied Mechanics and Engineering 404 (2023) 115783

  40. [48]

    J. Rani, T. Tripura, H. Kodamana, S. Chakraborty, P. K. Tamboli, Fault detection and isolation using probabilistic wavelet neural operator auto-encoder with application to dynamic processes, Process Safety and Environmental Protection 173 (2023) 215–228

  41. [49]

    G. Pang, M. D’Elia, M. Parks, G. E. Karniadakis, npinns: nonlocal physics-informed neural networks for a parametrized nonlocal universal laplacian operator. algorithms and applications, Journal of Computational Physics 422 (2020) 109760

  42. [50]

    Q. Cao, S. Goswami, G. E. Karniadakis, Laplace neural operator for solving di fferential equations, Nature Machine Intelligence 6 (6) (2024) 631–640

  43. [51]

    Peyvan, V

    A. Peyvan, V . Oommen, A. D. Jagtap, G. E. Karniadakis, Riemannonets: Interpretable neural operators for riemann problems, Computer Methods in Applied Mechanics and Engineering 426 (2024) 116996. 26

  44. [52]

    Z. Li, N. Kovachki, C. Choy, B. Li, J. Kossaifi, S. Otta, M. A. Nabian, M. Stadler, C. Hundt, K. Azizzadenesheli, et al., Geometry-informed neural operator for large-scale 3d pdes, Advances in Neural Information Processing Systems 36 (2024)

  45. [53]

    Z. Zhao, C. Liu, Y . Li, Z. Chen, X. Liu, Diffeomorphism neural operator for various domains and parameters of partial differential equations, arXiv preprint arXiv:2402.12475 (2024)

  46. [54]

    V . S. Fanaskov, I. V . Oseledets, Spectral neural operators, in: Doklady Mathematics, V ol. 108, Springer, 2023, pp. S226–S232

  47. [55]

    Rafiq, G

    M. Rafiq, G. Rafiq, H.-Y . Jung, G. S. Choi, Ssno: Spatio-spectral neural operator for functional space learning of partial differential equations, IEEE Access 10 (2022) 15084–15095

  48. [56]

    Z. Hao, Z. Wang, H. Su, C. Ying, Y . Dong, S. Liu, Z. Cheng, J. Song, J. Zhu, Gnot: A general neural operator transformer for operator learning, in: International Conference on Machine Learning, PMLR, 2023, pp. 12556– 12569

  49. [57]

    Z. Li, K. Meidani, A. B. Farimani, Transformer for partial di fferential equations’ operator learning, arXiv preprint arXiv:2205.13671 (2022)

  50. [58]

    B. Shih, A. Peyvan, Z. Zhang, G. E. Karniadakis, Transformers as neural operators for solutions of di fferential equations with finite regularity, Computer Methods in Applied Mechanics and Engineering 434 (2025) 117560

  51. [59]

    H. Li, X. Ye, P. Jiang, G. Qin, T. Wang, Local neural operator for solving transient partial differential equations on varied domains, Computer Methods in Applied Mechanics and Engineering 427 (2024) 117062

  52. [60]

    Jafarzadeh, S

    S. Jafarzadeh, S. Silling, N. Liu, Z. Zhang, Y . Yu, Peridynamic neural operators: A data-driven nonlocal con- stitutive model for complex material responses, Computer Methods in Applied Mechanics and Engineering 425 (2024) 116914

  53. [61]

    Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizzadenesheli, A. Anandkumar, Physics-informed neural operator for learning partial differential equations, ACM/JMS Journal of Data Science 1 (3) (2024) 1–27

  54. [62]

    Chennault, A

    A. Chennault, A. A. Popov, A. N. Subrahmanya, R. Cooper, A. H. M. Rafid, A. Karpatne, A. Sandu, Adjoint- matching neural network surrogates for fast 4d-var data assimilation, arXiv preprint arXiv:2111.08626 (2021)

  55. [63]

    Arcucci, L

    R. Arcucci, L. Moutiq, Y .-K. Guo, Neural assimilation, in: Computational Science–ICCS 2020: 20th Interna- tional Conference, Amsterdam, The Netherlands, June 3–5, 2020, Proceedings, Part VI 20, Springer, 2020, pp. 155–168

  56. [64]

    P. Wu, X. Chang, W. Yuan, J. Sun, W. Zhang, R. Arcucci, Y . Guo, Fast data assimilation (fda): Data assimilation by machine learning for faster optimize model state, Journal of Computational Science 51 (2021) 101323

  57. [65]

    Fablet, B

    R. Fablet, B. Chapron, L. Drumetz, E. Mémin, O. Pannekoucke, F. Rousseau, Learning variational data assimi- lation models and solvers, Journal of Advances in Modeling Earth Systems 13 (10) (2021) e2021MS002572

  58. [66]

    Chowdhury, R

    R. Chowdhury, R. Murugan, D. Subramani, Intelligent onboard routing in stochastic dynamic environments using transformers, in: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, 2023, pp. 1688–1696

  59. [67]

    D. H. Kelley, J. H. Thomas, Cerebrospinal fluid flow, Annual Review of Fluid Mechanics 55 (1) (2023) 237–264. arXiv:https://doi.org/10.1146/annurev-fluid-120720-011638 , doi:10.1146/ annurev-fluid-120720-011638 . URL https://doi.org/10.1146/annurev-fluid-120720-011638 27

  60. [68]

    Mittal, K

    R. Mittal, K. Breuer, J. H. Seo, The flow physics of face masks, Annual Review of Fluid Mechanics 55 (1) (2023) 193–211. arXiv:https://doi.org/10.1146/annurev-fluid-120720-035029 , doi:10.1146/ annurev-fluid-120720-035029 . URL https://doi.org/10.1146/annurev-fluid-120720-035029

  61. [69]

    Breton, J

    S.-P. Breton, J. Sumner, J. N. Sørensen, K. S. Hansen, S. Sarmast, S. Ivanell, A survey of modelling methods for high-fidelity wind farm simulations using large eddy simulation, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences ...

  62. [70]

    Gafoor CTP, S

    A. Gafoor CTP, S. Kumar Boya, R. Jinka, A. Gupta, A. Tyagi, S. Sarkar, D. N. Subramani, A physics-informed neural network for turbulent wake simulations behind wind turbines, Physics of Fluids 37 (1) (2025)

  63. [71]

    P. Jin, S. Meng, L. Lu, Mionet: Learning multiple-input operators via tensor product, SIAM Journal on Scientific Computing 44 (6) (2022) A3490–A3514

  64. [72]

    Mandl, S

    L. Mandl, S. Goswami, L. Lambers, T. Ricken, Separable deeponet: Breaking the curse of dimensionality in physics-informed machine learning, arXiv preprint arXiv:2407.15887 (2024)

  65. [73]

    M. Zhu, S. Feng, Y . Lin, L. Lu, Fourier-deeponet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness, Computer Methods in Applied Mechanics and Engineering 416 (2023) 116300

  66. [74]

    Cao, Choose a transformer: Fourier or galerkin, in: M

    S. Cao, Choose a transformer: Fourier or galerkin, in: M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, J. W. Vaughan (Eds.), Advances in Neural Information Processing Systems, V ol. 34, Curran Associates, Inc., 2021, pp. 24924–24940

  67. [75]

    Harandi, H

    A. Harandi, H. Danesh, K. Linka, S. Reese, S. Rezaei, A spectral-based physics-informed finite operator learning for prediction of mechanical behavior of microstructures, arXiv preprint arXiv:2410.19027 (2024)

  68. [76]

    Navaneeth, S

    N. Navaneeth, S. Chakraborty, Stochastic projection based approach for gradient free physics informed learning, Computer Methods in Applied Mechanics and Engineering 406 (2023) 115842

  69. [77]

    B. S. Kumar, quest-lab-iisc /pinto: Pinto (Dec. 2024). doi:10.5281/zenodo.14330860. URL https://doi.org/10.5281/zenodo.14330860

  70. [78]

    Takamoto, T

    M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger, M. Niepert, Pdebench: An exten- sive benchmark for scientific machine learning, Advances in Neural Information Processing Systems 35 (2022) 1596–1611

  71. [79]

    Kovachki, Z

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, A. Anandkumar, Neural operator: Learning maps between function spaces, arXiv preprint arXiv:2108.08481 (2021)

  72. [80]

    L. I. G. Kovasznay, Laminar flow behind a two-dimensional grid, Mathematical Proceedings of the Cambridge Philosophical Society 44 (1) (1948) 58–62. doi:10.1017/S0305004100023999

  73. [81]

    Taylor, Lxxv

    G. Taylor, Lxxv. on the decay of vortices in a viscous fluid, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 46 (274) (1923) 671–674

  74. [82]

    M. P. Ueckermann, P. F. J. Lermusiaux, 2.29 Finite V olume MATLAB Framework Documentation, MSEAS Report 14, Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA (2012). URL http://mseas.mit.edu/?p=2567

  75. [83]

    K. Song, Y . Jung, D. Kim, I.-C. Moon, Implicit kernel attention, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 35, 2021, pp. 9713–9721. 28

  76. [84]

    Rishi, A

    J. Rishi, A. Gafoor, S. Kumar, D. Subramani, Discrete residual loss functions for training physics-informed neural networks, in: International Conference on Computational Science, Springer, 2024, pp. 363–377

  77. [85]

    T. Li, Y . Zou, S. Zou, X. Chang, L. Zhang, X. Deng, Learning to solve pdes with finite volume-informed neural networks in a data-free approach, Journal of Computational Physics 530 (2025) 113919

  78. [86]

    Anandh, D

    T. Anandh, D. Ghose, A. Tyagi, A. Gupta, S. Sarkar, S. Ganesan, An e fficient hp-variational pinns framework for incompressible navier-stokes equations, arXiv preprint arXiv:2409.04143 (2024)

  79. [87]

    X. Jin, S. Cai, H. Li, G. E. Karniadakis, Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations, Journal of Computational Physics 426 (2021) 109951

  80. [88]

    A. Daw, J. Bu, S. Wang, P. Perdikaris, A. Karpatne, Mitigating propagation failures in physics-informed neural networks using retain-resample-release (r3) sampling, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023

  81. [89]

    J. Hou, Y . Li, S. Ying, Enhancing pinns for solving pdes via adaptive collocation point movement and adaptive loss weighting, Nonlinear Dynamics 111 (16) (2023) 15233–15261

  82. [90]

    Maddu, D

    S. Maddu, D. Sturm, C. L. Müller, I. F. Sbalzarini, Inverse dirichlet weighting enables reliable training of physics informed neural networks, Machine Learning: Science and Technology 3 (1) (2022) 015026

  83. [91]

    Deguchi, M

    S. Deguchi, M. Asai, Dynamic & norm-based weights to normalize imbalance in back-propagated gradients of physics-informed neural networks, Journal of Physics Communications 7 (7) (2023) 075005

  84. [92]

    K. Tang, X. Wan, C. Yang, Das-pinns: A deep adaptive sampling method for solving high-dimensional partial differential equations, Journal of Computational Physics 476 (2023) 111868

  85. [93]

    Moseley, A

    B. Moseley, A. Markham, T. Nissen-Meyer, Finite basis physics-informed neural networks (fbpinns): a scalable domain decomposition approach for solving di fferential equations, Advances in Computational Mathematics 49 (4) (2023) 62

  86. [94]

    S. Wang, Y . Teng, P. Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM Journal on Scientific Computing 43 (5) (2021) A3055–A3081

  87. [95]

    Bergstra, R

    J. Bergstra, R. Bardenet, Y . Bengio, B. Kégl, Algorithms for hyper-parameter optimization, Advances in neural information processing systems 24 (2011)

  88. [96]

    Escapil-Inchauspé, G

    P. Escapil-Inchauspé, G. A. Ruz, Hyper-parameter tuning of physics-informed neural networks: Application to helmholtz problems, Neurocomputing 561 (2023) 126826

  89. [97]

    Kaplarevi ´c-Mališi´c, B

    A. Kaplarevi ´c-Mališi´c, B. Andrijevi´c, F. Bojovi´c, S. Nikoli´c, L. Krsti´c, B. Stojanovi´c, M. Ivanovi´c, Identifying optimal architectures of physics-informed neural networks by evolutionary strategy, Applied Soft Computing 146 (2023) 110646

  90. [98]

    Y . Wang, L. Zhong, Nas-pinn: neural architecture search-guided physics-informed neural network for solving pdes, Journal of Computational Physics 496 (2024) 112603. 29

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.