Pith. sign in

REVIEW 4 major objections 6 minor 47 references

SUPRA: Subspace Parameterized Attention for Neural Operator on General Domains

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Attention between functions cuts PDE error by up to 33 percent.

desk verdict Clean function-space attention framing with a correct approximation argument, but the Airfoil basis-reuse gap and a table inconsistency need fixing before the 'general domains' claim is credible. read the letter →

arxiv 2504.15897 v1 pith:CXF45V25 submitted 2025-04-22 cs.LG

Zherui Yang , Zhengyang Xue , Ligang Liu This is my paper · ORCID
classification cs.LG
keywords neuraloperatorattentionmechanismfunctionspaceLaplacianeigenfunctionsmodelreductionirregulardomainsPDEsurrogate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes treating the functions that neural operators learn to map as the tokens of an attention mechanism, rather than treating mesh points as tokens. It observes that standard self-attention on vectors is built from a bilinear form and a linear operator, and it redefines both on $L^2(\Omega)$ so attention acts directly between functions. Because that infinite-dimensional attention cannot be computed exactly, the paper approximates it inside a finite-dimensional subspace spanned by $N$ basis functions, chosen as Laplacian eigenfunctions on irregular domains so the basis respects the geometry. The resulting SUPRA operator is claimed to be as expressive as full attention while costing $O(C^2N + CM)$ rather than $O(CM^2)$, and experiments report relative $L^2$ errors up to 33 percent lower than four recent baselines on five PDE benchmarks.

What carries the argument

The load-bearing object is the subspace-parameterized attention (SUPRA) block: a projection of each input function onto $N$ basis coefficients $\hat{u}_i$, standard attention applied to those coordinates using matrices $A$ and $B$ (decomposed into query/key/value matrices for multi-head use), and a reconstruction back to function values. The basis on general domains is the Laplacian eigensubspace, the span of the smallest eigenfunctions of $-\Delta$ on $\Omega$, which generalizes the Fourier basis and guarantees continuity and quasi-optimal approximation for smooth functions. The convergence argument rests on the boundedness (continuity) of the bilinear form and linear operator, so that coefficient truncation commutes with the attention computation in the limit.

What would settle it

Project the ground-truth output fields of an irregular-domain benchmark onto the best $N$-term Laplacian-eigenfunction subspace and measure the relative $L^2$ residual; if at $N=256$ the residual exceeds the reported test error, the claimed accuracy cannot be achieved by any attention mechanism confined to this subspace, and the paper's Airfoil result would then rest on cross-sample transfer of the eigensubspace rather than on the subspace approximation argument.

Watch

Extended reading notes

Core claim

The central claim is that attention between functions, defined exactly as attention between vectors via a learned bilinear form $a(\cdot,\cdot)$ and a learned linear operator $b(\cdot)$, can be carried out faithfully in a low-dimensional subspace: truncating the basis at $N$ terms turns the bilinear form into a matrix $A$ and the operator into a matrix $B$ acting on the coordinate vectors, so attention reduces to standard multi-head attention on those coordinates. Appendix A proves that as $N \to \infty$ the output converges to the infinite-dimensional attention. On irregular domains the paper uses the low-frequency eigenfunctions of the Laplace operator as the basis, which are continuous, orthonormal, and optimal approximants for smooth functions; this is what lets SUPRA avoid the discontinuity artifacts that FFT-based spectral convolution suffers when irregular domains are cut and flattened. On five standard benchmarks the method reports the lowest relative $L^2$ error on four of them and near-linear cost, including sharp-feature cases like transonic shock waves.

Load-bearing premise

The load-bearing premise is that the solution fields, and the learned relations between them, are well approximated by just 64 to 256 smooth basis functions (Laplacian, Fourier, or Chebyshev), a quantitative sufficiency asserted from ablations rather than derived for each problem.

Editorial extensions

If this is right

  • SUPRA removes the need to cut irregular domains for spectral convolution, since the Laplacian eigenfunctions are defined directly on the physical domain.
  • The complexity $O(C^2N + CM)$ makes attention practical at mesh sizes where point-token attention at $O(CM^2)$ is prohibitive.
  • Because the same code path as standard multi-head attention can be reused, the method inherits optimized GPU attention implementations.
  • On the reported benchmarks, the method attains lower relative $L^2$ error than four recent baselines on four of five datasets, and the ablations show instance normalization and basis choice substantially affect accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper does not develop: the same subspace parameterization could be plugged into other operator architectures beyond the FNO-style backbone, e.g. as a drop-in replacement for the spectral convolution layer, provided the basis projection cost stays manageable.
  • The Laplacian-eigenfunction choice suggests a natural error monitor: track the projection residual of the input and output functions; when it is large, SUPRA's accuracy ceiling is set by the subspace, not by attention.
  • The Airfoil practice of computing the eigensubspace once on the mean mesh and reusing it across shapes is a strong assumption; if it transfers, it implies geometries in the dataset are near-isospectral perturbations of the mean shape, which could be checked by measuring basis-consistency across samples.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SUPRA, a neural operator that reformulates standard attention as a bilinear form plus a linear operator on L2(Ω), truncates the resulting infinite-dimensional attention to an N-dimensional subspace, and uses Laplacian eigenfunctions as the subspace on irregular domains. The authors claim that this yields a function-space attention mechanism with state-of-the-art accuracy (up to 33% lower relative L2 error) and near-linear computational cost on both regular and irregular PDE benchmarks, and Appendix A gives a convergence argument as N increases. Experiments cover Darcy, Navier-Stokes, Plasticity, Airfoil, and Pipe benchmarks, with ablations on basis count, basis type, normalization, model size, and cost.

Significance. If the claims hold, the paper offers a clean conceptual bridge between attention mechanisms and function-space operator learning: the bilinear-form/linear-operator formulation is principled, no fitted constants enter the convergence argument, and the Laplacian eigensubspace is a natural geometry-aware analogue of Fourier bases. The theoretical derivation in Section 4 and Appendix A is largely sound under stated boundedness assumptions, and the idea of parameterizing attention on subspace coordinates is potentially useful. However, the irregular-domain support is weakened by the mean-mesh Airfoil basis transfer, the complexity analysis omits an N factor in projection/reconstruction, and the experimental evidence is single-run without error bars. These issues are local enough to be fixable, but they are load-bearing for the paper's two headline claims: general-domain accuracy and near-linear cost.

major comments (4)
  1. [Section 4.3 and Appendix C.1, Eq. (12)] The construction of the Laplacian eigensubspace for Airfoil is not geometry-adaptive as claimed. Appendix C.1 states that the eigensubspace is computed once on the mean mesh and reused for all airfoil shapes, but the Laplace–Beltrami operator is defined on each sample domain Ω_s; a basis that is orthonormal in L²(Ω_mean) is not orthonormal in L²(Ω_s) and, absent an explicit mapping from each Ω_s to Ω_mean, is not even a basis there. Consequently Eq. (12) is not an L² projection on the sample domain, and the convergence argument in Appendix A, which requires the basis to be complete in the space containing the target functions, does not apply to the Airfoil experiments. The paper needs either per-sample basis computation, an explicit transfer-error analysis, or a restriction of the irregular-domain claim.
  2. [Table 10 vs Tables 3 and 5] The reported best Airfoil configuration is internally inconsistent. Table 10 lists #Basis = 64, but Table 3 reports Airfoil error 0.446 at 64 basis versus 0.340 at 128/192, and Table 5 uses 128 Laplacian eigenfunctions; the Table 1 value of 0.34 therefore cannot be reproduced from the stated optimal configuration. Please correct the configuration table or the ablation numbers.
  3. [Section 4.4, Complexity Analysis] The complexity accounting omits the factor N in projection and reconstruction. In Eq. (12), computing N coefficients for one function requires sums over M sample points, i.e., O(NM) per function, and reconstruction from N coefficients to M points is also O(NM) per function. For C functions the total is O(C²N + CNM), not O(C²N + CM), and for the Laplacian basis on irregular meshes there is no fast transform to remove the N factor. This undermines the 'near-linear' efficiency claim and should be restated.
  4. [Section 5.1, Table 1] The experimental support for the headline accuracy improvement is thin. The baseline numbers are taken from prior papers rather than rerun under matched settings, and no error bars, standard deviations, or number of seeds are reported; for example, the Navier-Stokes gap between Transolver (7.83) and SUPRA (6.25) could be within run-to-run variation. Please provide repeated-run statistics or clearly label the comparison as previously reported values.
minor comments (6)
  1. [Section 4.3] The sentence 'we provide a Laplacian-based method to construct proper basis functions (?).' contains a literal '(?)' placeholder for a citation; please fill in the reference.
  2. [Definition 3.2] The definition uses uppercase A(·,·) in the axioms but the rest of the paper uses lowercase a(·,·); align the notation.
  3. [Appendix A, Eq. (17)] Boundedness of a bilinear form should be stated as |a(u,v)| ≤ C||u|| ||v||, not a(u,v) ≤ C||u|| ||v||, since bilinear forms may be negative.
  4. [Theorem B.1] The statement that the infimum is achieved 'if and only if φ is an eigenfunction' is imprecise; scalar multiples of the eigenfunction also achieve it, and the naming of φ_k is ambiguous.
  5. [Section 5.2, Table 3] The claim that SUPRA 'will not degenerate when the number of basis is small' is not supported by the Airfoil row, which degrades from 0.340 to 0.446 when reducing from 128 to 64 basis functions; please qualify the claim.
  6. [Table 5] The header of Table 5 reports units of ×10⁻³, while Tables 1 and 3 use ×10⁻²; this makes cross-comparison needlessly confusing. Unify the units or state the conversion explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SUPRA's subspace truncation is a standard approximation argument, and its convergence proof does not depend on fitted parameter values.

full rationale

The paper's central derivation is self-contained. Equations (9)-(11) define attention on function spaces by substituting a basis expansion into a bounded bilinear form and linear operator, and the finite-N parameterization is the same mathematical object truncated. Appendix A's convergence claim follows from basis completeness plus boundedness of the operator and form, with no fitted coefficient entering the proof; the learnable matrices A and B are trained from data, but the approximation statement in Equation (20) concerns the underlying operator, not the fitted values. The Laplacian eigensubspace claim is supported by classical external results (Gilbarg & Trudinger 2013; Evans 2010) and by standard spectral approximation theory, not by a self-citation chain. The mean-mesh reuse on Airfoil (Appendix C.1) and the apparent configuration discrepancy between Table 10 (64 basis) and Table 3 (best at 128) are correctness and consistency concerns, not circularity: they do not make any predicted quantity equivalent to a fitted input by construction. No load-bearing step reduces the paper's claims to its own definitions or to unverified self-citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method itself is a construction; its convergence rests on standard spectral theory and on the sufficiency of a 64 to 256 dimensional subspace for the target PDE fields. The main domain-specific assumptions are the optimal approximation claim and the mean-mesh transfer for Airfoil.

free parameters (4)
  • Number of basis functions N = Darcy: 100; Navier-Stokes: 144; Plasticity: 144; Airfoil: 64; Pipe: 144 (optimal configs, App. C.2)
    Chosen per dataset by validation; controls the trade-off between approximation quality and computational cost. The paper states N=64 to 256 is typically sufficient.
  • Basis choice = Chebyshev for Darcy; Fourier for Navier-Stokes, Plasticity, Pipe; Laplacian eigensubspace for Airfoil
    Selected per dataset by validation; Tables 4 and 5 show the choice affects accuracy, e.g., Laplacian beats Fourier on Airfoil.
  • Loss weighting for Darcy = L2 + 0.1 H1
    The H1 coefficient 0.1 is hand-tuned for Darcy and is not used for other cases.
  • Normalization scheme = LayerNorm for Darcy and Airfoil; InstanceNorm for Navier-Stokes, Plasticity, Pipe
    Chosen per dataset; Table 6 shows results vary with normalization and training can blow up without it.
assumptions (4)
  • standard math Laplacian eigenfunctions of a bounded domain form an orthonormal basis of L^2(Ω).
    Invoked in Section 4.3 and Appendix B to justify using eigenfunctions as the subspace basis; standard result from Gilbarg & Trudinger.
  • domain assumption The Laplacian eigensubspace spanned by the smallest eigenfunctions has optimal approximation properties for smooth functions.
    Stated in Section 4.3 and used to argue the subspace is suitable; the paper relies on the min-max theorem (Evans) but does not quantify the error bound for the specific PDE datasets.
  • domain assumption The learned bilinear form and linear operator are bounded (continuous) on L^2(Ω), ensuring convergence of the truncated attention.
    Assumed in Appendix A to prove convergence of SUPRA to the function-space attention as N grows; not verified for trained matrices.
  • ad hoc to paper For Airfoil, the Laplacian eigensubspace computed on the mean mesh is a valid basis for every sample geometry.
    Appendix C.1 states the subspace is precomputed only once on the mean mesh, not per sample; this transfer assumption is untested and load-bearing for the Airfoil results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SUPRA: Subspace Parameterized Attention for Neural Operator on General Domains." pith.science (2026). https://pith.science/paper/CXF45V25

@misc{pith2026250415897,
  author       = {Pith},
  title        = {Pith review of: SUPRA: Subspace Parameterized Attention for Neural Operator on General Domains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXF45V25}},
  note         = {Machine review of arXiv:2504.15897}
}
read the original abstract

Neural operators are efficient surrogate models for solving partial differential equations (PDEs), but their key components face challenges: (1) in order to improve accuracy, attention mechanisms suffer from computational inefficiency on large-scale meshes, and (2) spectral convolutions rely on the Fast Fourier Transform (FFT) on regular grids and assume a flat geometry, which causes accuracy degradation on irregular domains. To tackle these problems, we regard the matrix-vector operations in the standard attention mechanism on vectors in Euclidean space as bilinear forms and linear operators in vector spaces and generalize the attention mechanism to function spaces. This new attention mechanism is fully equivalent to the standard attention but impossible to compute due to the infinite dimensionality of function spaces. To address this, inspired by model reduction techniques, we propose a Subspace Parameterized Attention (SUPRA) neural operator, which approximates the attention mechanism within a finite-dimensional subspace. To construct a subspace on irregular domains for SUPRA, we propose using the Laplacian eigenfunctions, which naturally adapt to domains' geometry and guarantee the optimal approximation for smooth functions. Experiments show that the SUPRA neural operator reduces error rates by up to 33% on various PDE datasets while maintaining state-of-the-art computational efficiency.

Figures

Figures reproduced from arXiv: 2504.15897 by the authors.

Figure 1
Figure 1. Discontinuities induced by domain cuts. The top row shows the mapping between a physical domain and a computation domain in the NACA Airfoil problem. To define the continuous map, the physical domain must be cut along the homology loop (red dashed line). Figures in the bottom row visualize a continu￾ous function f(x, y) = sin(4πx) cos(4πy) in different domains, where x, y are coordinates in the computational domains… view at source ↗
Figure 2
Figure 2. Overall design of SUPRA neural operator. We adopt the architecture proposed in (Kossaifi et al., 2023) while replacing spectral convolutions with SUPRA blocks. All trainable modules are colored green. LN stands for LayerNorm, WV corresponds to the matrix B, and W⊤ Q WK corresponds to the matrix A defined in Equation (10). Although LayerNorm (Ba et al., 2016) is a common choice, InstanceNorm (Ulyanov et al., 2016) ca… view at source ↗
Figure 3
Figure 3. Comparison between Laplacian eigenfunctions and Fourier basis. The eigenfunctions guarantee continuity across the physical domain, while the Fourier basis defined on the com￾putational mesh does not. Basis Construction For irregular domains, the eigenvectors of the Laplace matrix resulting from FEM discretization are precomputed once by using classical methods. For regular grids, the basis is computed from a tensor … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparisons between our prediction and ground truth at the first and last step in the Navier Stokes problem. Although the previous input is smooth, the output can be very sharp. each method on different test cases is shown in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: SUPRA neural operator accurately captures the vortex and shock wave, demonstrating its ability to capture complex dynamics even for sharp solutions [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison between ground truth in Plasticity and Pipe test set. Airfoil Euler equation models the transonic flow over an airfoil. The input is mesh point locations, and the output is the Mach number on a structured mesh. We extract the average shape as the mean mesh t…
Figure 7
Figure 7. Figure 7: Laplacian eigensubspace construction in Airfoil. We use input/output normalizers in each case, which is a common strategy to guarantee the stability of training progress. We use AdamW optimizer(Loshchilov & Hutter, 2017) and a OneCycle scheduler(Smith & Topin, 2018) fo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 26 canonical work pages

  1. [1]

    R., and Hinton, G

    Ba, J., Kiros, J. R., and Hinton, G. E. Layer normalization. ArXiv, abs/1607.06450, 2016

    arXiv 2016

  2. [2]

    A survey of projection-based model reduction methods for parametric dynamical systems

    Benner, P., Gugercin, S., and Willcox, K. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review, 57 0 (4): 0 483--531, 2015. doi:10.1137/130932715

    doi:10.1137/130932715 2015

  3. [3]

    Model Reduction and Approximation

    Benner, P., Ohlberger, M., Cohen, A., and Willcox, K. Model Reduction and Approximation. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2017. doi:10.1137/1.9781611974829

    doi:10.1137/1.9781611974829 2017

  4. [4]

    and Maday, Y

    Bernardi, C. and Maday, Y. Spectral methods. In Techniques of Scientific Computing (Part 2), volume 5 of Handbook of Numerical Analysis, pp.\ 209--485. Elsevier, 1997

    work page 1997

  5. [5]

    Spherical fourier neural operators: Learning stable dynamics on the sphere

    Bonev, B., Kurth, T., Hundt, C., Pathak, J., Baust, M., Kashinath, K., and Anandkumar, A. Spherical fourier neural operators: Learning stable dynamics on the sphere. In International Conference on Machine Learning, 2023

    work page 2023

  6. [6]

    and Scott, R

    Brenner, S. and Scott, R. The Mathematical Theory of Finite Element Method, volume 15. 01 2008. ISBN 978-1-4757-4340-1. doi:10.1007/978-1-4757-4338-8

    doi:10.1007/978-1-4757-4338-8 2008

  7. [7]

    L., Noack, B

    Brunton, S. L., Noack, B. R., and Koumoutsakos, P. Machine learning for fluid mechanics. ArXiv, abs/1905.11075, 2019

    arXiv 1905

  8. [8]

    Choose a transformer: Fourier or galerkin

    Cao, S. Choose a transformer: Fourier or galerkin. In Neural Information Processing Systems, 2021

    work page 2021

Show all 47 references
  1. [9]

    Gradient-based constrained optimization using a database of linear reduced-order models

    Choi, Y., Boncoraglio, G., Anderson, S., Amsallem, D., and Farhat, C. Gradient-based constrained optimization using a database of linear reduced-order models. Journal of Computational Physics, 423: 0 109787, 2020. ISSN 0021-9991. doi:https://doi.org/10.1016/j.jcp.2020.109787

    2020

  2. [10]

    Fluid simulation using laplacian eigenfunctions

    De Witt, T., Lessig, C., and Fiume, E. Fluid simulation using laplacian eigenfunctions. ACM Trans. Graph., 31 0 (1), February 2012. ISSN 0730-0301. doi:10.1145/2077341.2077351

    2012

  3. [11]

    Partial Differential Equations

    Evans, L. Partial Differential Equations. Graduate studies in mathematics. American Mathematical Society, 2010. ISBN 9780821849743

    2010

  4. [12]

    and Trudinger, N

    Gilbarg, D. and Trudinger, N. Elliptic Partial Differential Equations of Second Order. Grundlehren der mathematischen Wissenschaften. Springer Berlin Heidelberg, 2013. ISBN 9783642963797

    2013

  5. [13]

    Gnot: A general neural operator transformer for operator learning

    Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceed...

    2023

  6. [14]

    Hesthaven, J. S. Numerical Methods for Conservation Laws. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2018. doi:10.1137/1.9781611975109

    2018 doi

  7. [15]

    and Mitsume, N

    Horie, M. and Mitsume, N. Graph neural PDE solvers with conservation and similarity-equivariance. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning...

    2024

  8. [16]

    E., Kevrekidis, I

    Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., and Yang, L. Physics-informed machine learning. Nature Reviews Physics, 3: 0 422 -- 440, 2021

    2021

  9. [17]

    B., Azizzadenesheli, K., and Anandkumar, A

    Kossaifi, J., Kovachki, N. B., Azizzadenesheli, K., and Anandkumar, A. Multi-grid tensorized fourier neural operator for high-resolution pdes. ArXiv, abs/2310.00120, 2023

    2023 arXiv

  10. [18]

    Neural operator: Learning maps between function spaces with applications to pdes

    Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24 0 (89): 0 1--97, 2023

    2023

  11. [19]

    and Zhang, H

    L\' e vy, B. and Zhang, H. R. Spectral mesh processing. In ACM SIGGRAPH 2010 Courses, SIGGRAPH '10, New York, NY, USA, 2010. Association for Computing Machinery. ISBN 9781450303958. doi:10.1145/1837101.1837109

    2010

  12. [20]

    Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations' operator learning. Trans. Mach. Learn. Res., 2023, 2022 a

    2023

  13. [21]

    Scalable transformer for pde surrogate modeling

    Li, Z., Shu, D., and Barati Farimani, A. Scalable transformer for pde surrogate modeling. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 28010--28039. Curran Associates, In...

    2023

  14. [22]

    B., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A

    Li, Z.-Y., Kovachki, N. B., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. M., and Anandkumar, A. Fourier neural operator for parametric partial differential equations. ArXiv, abs/2010.08895, 2020

    2010 arXiv

  15. [23]

    Z., Liu, B., and Anandkumar, A

    Li, Z.-Y., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for pdes on general geometries. J. Mach. Learn. Res., 24: 0 388:1--388:26, 2022 b

    2022

  16. [24]

    B., Choy, C., Li, B., Kossaifi, J., Otta, S

    Li, Z.-Y., Kovachki, N. B., Choy, C., Li, B., Kossaifi, J., Otta, S. P., Nabian, M. A., Stadler, M., Hundt, C., Azizzadenesheli, K., and Anandkumar, A. Geometry-informed neural operator for large-scale 3d pdes. ArXiv, abs/2309.00583, 2023 b

    2023 arXiv

  17. [25]

    E., Michelis, M

    Lingsch, L. E., Michelis, M. Y., De Bezenac, E., M. Perera, S., Katzschmann, R. K., and Mishra, S. Beyond regular grids: F ourier-based neural operators on arbitrary domains. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F....

    2024

  18. [26]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations, 2017

    2017

  19. [27]

    Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3: 0 218 -- 229, 2019

    2019

  20. [28]

    A., George, R

    Rahman, M. A., George, R. J., Elleithy, M., Leibovici, D., Li, Z., Bonev, B., White, C., Berner, J., Yeh, R. A., Kossaifi, J., Azizzadenesheli, K., and Anandkumar, A. Pretraining codomain attention neural operators for solving multiphysics pdes. Advances in Neural Information ...

    2024

  21. [29]

    Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017

    2017 arXiv

  22. [30]

    Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378: 0 686--707, 2019

    2019

  23. [31]

    Challenges in training PINN s: A loss landscape perspective

    Rathore, P., Lei, W., Frangella, Z., Lu, L., and Udell, M. Challenges in training PINN s: A loss landscape perspective. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conferen...

    2024

  24. [32]

    Efficient spectral methods for pdes with spectral fractional laplacian

    Sheng, C., Cao, D., and Shen, J. Efficient spectral methods for pdes with spectral fractional laplacian. Journal of Scientific Computing, 88, 07 2021. doi:10.1007/s10915-021-01491-2

    2021 doi

  25. [33]

    and Barbic, J

    Sifakis, E. and Barbic, J. Fem simulation of 3d deformable solids: a practitioner's guide to theory, discretization and model reduction. In ACM SIGGRAPH 2012 Courses, SIGGRAPH '12, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450316781. doi:10.1145/23...

    2012

  26. [34]

    Turbulence and the dynamics of coherent structures

    Sirovich, L. Turbulence and the dynamics of coherent structures. i. coherent structures. Quarterly of Applied Mathematics, 45: 0 561--571, 1987

    1987

  27. [35]

    F., Bj rstad, P

    Smith, B. F., Bj rstad, P. E., and Gropp, W. D. Domain decomposition: parallel multilevel methods for elliptic partial differential equations. Cambridge University Press, USA, 1996. ISBN 052149589X

    1996

  28. [36]

    Smith, L. N. and Topin, N. Super-convergence: very fast training of neural networks using large learning rates. In Defense + Commercial Sensing, 2018

    2018

  29. [37]

    and Shakarchi, R

    Stein, E. and Shakarchi, R. Real Analysis: Measure Theory, Integration, and Hilbert Spaces. Princeton University Press, 2009. ISBN 9781400835560

    2009

  30. [38]

    Trefethen, L. N. Approximation theory and approximation practice. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2013. ISBN 978-1-611972-39-9

    2013

  31. [39]

    Ulyanov, D., Vedaldi, A., and Lempitsky, V. S. Instance normalization: The missing ingredient for fast stylization. CoRR, abs/1607.08022, 2016

    2016 arXiv

  32. [40]

    M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A

    Vaswani, A., Shazeer, N. M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Neural Information Processing Systems, 2017

    2017

  33. [41]

    BENO : Boundary-embedded neural operators for elliptic PDE s

    Wang, H., Jiaxin, L., Dwivedi, A., Hara, K., and Wu, T. BENO : Boundary-embedded neural operators for elliptic PDE s. In The Twelfth International Conference on Learning Representations, 2024

    2024

  34. [42]

    and Wang, C

    Wang, T. and Wang, C. Latent neural operator for solving forward and inverse pde problems. In Advances in Neural Information Processing Systems, 2024

    2024

  35. [43]

    Wen, G., Li, Z., Azizzadenesheli, K., Anandkumar, A., and Benson, S. M. U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163: 0 104180, 2022. ISSN 0309-1708. doi:https://doi.org/10.1016/j.advwatres.2022.104180

    2022

  36. [44]

    B., Javed, M., Montanelli, H., and Trefethen, L

    Wright, G. B., Javed, M., Montanelli, H., and Trefethen, L. N. Extension of C hebfun to periodic functions. SIAM J. Sci. Comput., 37 0 (5): 0 C554--C573, 2015. ISSN 1064-8275,1095-7197. doi:10.1137/141001007. URL https://doi.org/10.1137/141001007

    2015 doi

  37. [45]

    Transolver: A fast transformer solver for PDE s on general geometries

    Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. Transolver: A fast transformer solver for PDE s on general geometries. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Confere...

    2024

  38. [46]

    Improved operator learning by orthogonal attention

    Xiao, Z., Hao, Z., Lin, B., Deng, Z., and Su, H. Improved operator learning by orthogonal attention. ArXiv, abs/2310.12487, 2023

    2023 arXiv

  39. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.