Pith. sign in

REVIEW 3 major objections 6 minor 45 references

Sequential Neural Operator Transformer for High-Fidelity Surrogates of Time-Dependent Non-linear Partial Differential Equations

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that replacing the dot-product fusion in Sequential DeepONet with cross-attention between the encoded loading sequence and spatial query points consistently improves prediction accuracy on three challenging time-dependent…

desk verdict S-NOT is a plausible incremental architecture but the empirical claim of consistent superiority over S-DON needs a fairer, multi-seed comparison before it is established. read the letter →

arxiv 2507.03272 v1 pith:PEJIKV5H submitted 2025-07-04 physics.comp-ph

Qibang Liu , Seid Koric This is my paper
classification physics.comp-ph MSC 68T0765M99
keywords TransformerSequentialNeuralOperatorDeepLearningAttentionMechanismSurrogateModelingTime-dependentPDEsGRU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Sequential Neural Operator Transformer (S-NOT), a surrogate architecture for predicting full solution fields of time-dependent nonlinear PDEs. The central claim is that replacing the dot-product fusion used by Sequential DeepONet (S-DON) with cross-attention between the encoded loading history and spatial query points yields consistently lower prediction errors on three challenging problems: multiphysics steel solidification, a 3D lug under time-varying pressure, and a dog-bone specimen under sequential displacement. The reported improvements are substantial: mean stress relative $L^2$ error drops from $18.1\%$ to $4.3\%$ on steel solidification, from $11.6\%$ to $5.31\%$ on the lug, and from $2.01\%$ to $1.13\%$ on the dog-bone, while parameter count and GPU inference time stay comparable and remain orders of magnitude faster than finite element simulation. The paper argues this makes S-NOT a practical surrogate for workflows that require many forward evaluations, such as uncertainty quantification and optimization.

What carries the argument

The central mechanism is cross-attention between the encoded loading history and the spatial query points, defined as $\mathrm{Attention}(Q,K,V) = \mathrm{softmax}(QK^T/\sqrt{d_e})V$. The GRU stack preserves temporal causality in the loading sequence, the sinusoidal positional encoding injects time information explicitly, self-attention lets the encoder re-weight the sequence, and cross-attention lets each spatial point aggregate the most relevant parts of the history instead of relying on a single inner product. This replacement of dot-product fusion by attention is the component the paper credits for the accuracy gain.

What would settle it

Retrain both models on the same three datasets with each model's hyperparameters tuned separately using equal budgets and several random seeds, then compute mean and 90th-percentile errors with confidence intervals. If S-DON matches or beats S-NOT in mean or tail errors under matched tuning, the claim that attention-based fusion is more accurate would be refuted.

Watch

Extended reading notes

Core claim

S-NOT uses a GRU encoder to process a sequence of loading values in time, adds sinusoidal positional encodings, and refines the representation with self-attention blocks; the resulting sequence serves as keys and values for the decoder. In the decoder, each spatial query point is embedded with a NeRF-style positional encoding followed by an MLP to form a query, and cross-attention selects the relevant portions of the load history for that point. This replaces the fixed dot product of branch and trunk outputs in S-DON. In the paper's tests, the change consistently improves accuracy, and the largest gains appear on high-error outlier samples, so the error tails become shorter.

Load-bearing premise

The comparison assumes that S-DON was trained with the same hyperparameters, training budget, and data splits as S-NOT; the paper does not report those settings, so if S-DON was undertuned the reported accuracy gap could shrink or reverse.

Editorial extensions

If this is right

  • On all three reported benchmarks, S-NOT attains a lower mean error than S-DON for stress, temperature, and plastic strain fields.
  • S-NOT's test error distributions have shorter tails, so the model is more reliable on difficult or outlier samples where S-DON's accuracy degrades.
  • S-NOT keeps roughly the same number of parameters and inference speed as S-DON, so the accuracy gain does not add much deployment cost.
  • Surrogate inference (about $10^{-3}$ seconds per sample on GPU) is orders of magnitude faster than the reference finite element runs (20 to 333 seconds per sample on CPU), enabling repeated forward evaluations.
  • The design is supported by the theoretical result that cross-attention transformers are universal operator approximators, giving the attention-based fusion a principled basis beyond the empirical gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the accuracy gain concentrates in the error tails, S-NOT should be most valuable in safety-critical applications such as predicting hot tearing in continuous casting, where rare large errors dominate risk.
  • The cross-attention decoder is agnostic to the choice of sequence encoder, so S-NOT could be combined with other temporal encoders or with physics-informed loss terms without changing the core architecture.
  • A natural next test is whether S-NOT retains its advantage on loading histories sampled from distributions held out during training, a stronger out-of-distribution check than the outlier analysis reported here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes the Sequential Neural Operator Transformer (S-NOT), a surrogate model that combines GRU-based sequential encoding with transformer-style self-attention and cross-attention to map time-dependent loading histories to full-field PDE solutions. The model is benchmarked on three FEM-generated datasets: multiphysics steel solidification, a 3D lug specimen, and a dog-bone specimen under path-dependent loading. The central claim, stated in the abstract and Section 1, is that S-NOT consistently achieves higher prediction accuracy than S-DON across all test cases, including outlier samples. Table 1 reports lower mean errors for S-NOT on all datasets, and Table 2 indicates comparable inference time and parameter counts. However, the evaluation is based on a single run per model, reports no training hyperparameters, uses a nonstandard split for the dog-bone dataset, and provides only qualitative evidence for the outlier claim, leaving the comparative claim insufficiently supported.

Significance. The attention-based cross-attention decoder is a natural and potentially impactful modification of S-DON for sequential operator learning, and the three realistic nonlinear PDE benchmarks are well chosen. If the claimed accuracy gains are confirmed under controlled comparison, S-NOT could provide a useful surrogate for path-dependent material simulations. The paper also makes its code and trained models available on GitHub and the datasets on Zenodo, which is a strength for reproducibility. The main weakness is that the empirical comparison is not yet rigorous enough to establish the central claim: missing training configurations, single-run results with large per-sample standard deviations, and an unmatched data split for the dog-bone dataset leave room for alternative explanations of the reported accuracy gap.

major comments (3)
  1. [Section 4, Table 1] The central claim that S-NOT 'consistently achieves a higher prediction accuracy than S-DON' is not supported by the present evaluation because no training hyperparameters (optimizer, learning rate, batch size, epochs, early stopping, random seeds) are reported for either model, and only a single run per model is presented. The per-sample standard deviations are large (e.g., steel stress: 4.3 ± 15.3 for S-NOT vs. 18.1 ± 23.2 for S-DON), so the observed differences may be within run-to-run stochasticity. Please provide full training configurations for both models, repeat each model over multiple seeds, and report paired statistics (e.g., per-sample error-difference distributions or a signed-rank test) to substantiate the consistency claim.
  2. [Section 3.3, Dog-bone split] The dog-bone dataset uses a 3,200/800 train/test split, which is neither the 80/20 split used for the other two datasets nor justified relative to the original 15,000 samples. The manuscript does not state whether the reported S-DON dog-bone results were reproduced on this exact split or quoted from He et al. [27] under a different split. This makes the S-DON comparison potentially unequal and could bias the reported accuracy gap. Please justify the split and ensure both models are trained and evaluated on identical data partitions.
  3. [Abstract, Section 4.3, Conclusion] The claim of superior accuracy 'even for data outliers' is supported only by qualitative percentile plots (Figs. 6, 9, 10, 12, 13) and error histograms, with no quantitative tail metrics (e.g., 90th/95th percentile errors, tail-mean, or the fraction of test samples where S-NOT outperforms S-DON). Given the heavy-tailed error distributions in Table 1, the robustness claim requires such statistics to be substantiated.
minor comments (6)
  1. [Abstract] The abstract contains the phrase 'a architecture' and should read 'an architecture'; it also uses 'S-DeepONet' while the main text predominantly uses 'S-DON', which should be made consistent.
  2. [Section 3.3] The text says 'the finnal state of von Mises stress' and should be corrected to 'the final state'.
  3. [Table 2] The dog-bone row for S-NOT parameter count reads '7,90952', which appears to be a typo for '790,952'.
  4. [Section 3.1, Eq. (7)] Equation (7) is introduced with 'given in Eq. (5)', which is incorrect; it should refer to its own equation number.
  5. [Section 4] The sentence 'S-NOT offers similar inference speed and parameter count to S-DON' is imprecise because Table 2 shows S-NOT has fewer parameters in the 3D Lug case but more in the other two cases; the statement should be qualified.
  6. [Section 2.2] The attention formula in Eq. (2) is labeled generically; the text should explicitly clarify that cross-attention in the decoder uses the same formula with Q derived from query points and K, V from the encoder, as described in the prose.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the S-NOT advantage over S-DON rests on held-out FEM benchmarks, not on a self-referential construction.

full rationale

The paper is an empirical architecture comparison rather than a derivation: S-NOT's output is defined by the attention operation in Eq. (2) combined with a GRU sequential encoder, and the reported accuracies are computed on held-out FEM samples using the error metrics in Eqs. (10) and (11). No fitted parameter is later renamed as a prediction, and no equation defines the claimed advantage in terms of the S-DON output or the evaluation metric. The theoretical support for cross-attention operator models is attributed to the external work of Shih et al. [34], not to the authors' own prior results. The self-citations [16], [27], [33], and [35] supply prior baselines, datasets, and related neural-operator transformer variants, but the central S-NOT-versus-S-DON comparison is presented as this paper's own benchmark rather than imported as a cited conclusion. Concerns about the dog-bone split (3,200/800) and the absence of reported hyperparameter settings are legitimate threats to the fairness and statistical strength of the comparison, but they do not make the prediction equivalent to its input by construction. Accordingly, no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

S-NOT introduces no new physical entities. Its central claim rests on the supervised training setup: the fidelity of the FEM-generated labels, the coverage of the loading distribution, and the fairness of the S-DON baseline. These are domain assumptions, not derived results. The only listed free parameter group is the set of architecture and training hyperparameters that the paper leaves unreported.

free parameters (1)
  • Model hyperparameters (embedding dimension, number of attention heads, number of GRU layers, learning rate, batch size) = not reported in text (available in code repository)
    These choices directly affect the accuracy comparison between S-NOT and S-DON. The paper reports neither the values nor the tuning procedure, which is necessary to assess fairness of the comparison.
assumptions (4)
  • standard math Transformers with cross-attention have the universal approximation property for operator learning (Shih et al. [34]).
    Invoked in Section 2.2 to justify that the cross-attention decoder has sufficient capacity to represent the operator. This is a cited theorem from the literature, not proved in this paper.
  • domain assumption The FEM-generated datasets are accurate ground-truth solutions of the target PDE problems.
    Section 3 describes data generation with Abaqus and a user-defined UMAT. No experimental validation of the FEM models is provided, so the surrogate can only be as good as the simulation data it learns from.
  • domain assumption The random loading profiles used to generate training data are sampled from the same distribution as the test profiles.
    Sections 3.1 and 3.2 sample loading histories from Gaussian random fields and split data randomly into 80% training and 20% test. The generalization claim depends on this distributional match.
  • domain assumption The S-DON baseline is implemented and tuned with comparable effort to S-NOT.
    Section 4 reports only final errors and gives no training budgets, hyperparameters, or number of trials for either model. The comparative conclusion holds only if this assumption is true.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sequential Neural Operator Transformer for High-Fidelity Surrogates of Time-Dependent Non-linear Partial Differential Equations." pith.science (2026). https://pith.science/paper/PEJIKV5H

@misc{pith2026250703272,
  author       = {Pith},
  title        = {Pith review of: Sequential Neural Operator Transformer for High-Fidelity Surrogates of Time-Dependent Non-linear Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PEJIKV5H}},
  note         = {Machine review of arXiv:2507.03272}
}
read the original abstract

Partial differential equations (PDEs) are fundamental to modeling complex and nonlinear physical phenomena, but their numerical solution often requires significant computational resources, particularly when a large number of forward full solution evaluations are necessary, such as in design, optimization, sensitivity analysis, and uncertainty quantification. Recent progress in operator learning has enabled surrogate models that efficiently predict full PDE solution fields; however, these models often struggle with accuracy and robustness when faced with highly nonlinear responses driven by sequential input functions. To address these challenges, we propose the Sequential Neural Operator Transformer (S-NOT), a architecture that combines gated recurrent units (GRUs) with the self-attention mechanism of transformers to address time-dependent,nonlinear PDEs. Unlike S-DeepONet (S-DON), which uses a dot product to merge encoded outputs from the branch and trunk sub-networks, S-NOT leverages attention to better capture intricate dependencies between sequential inputs and spatial query points. We benchmark S-NOT on three challenging datasets from real-world applications with plastic and thermo-viscoplastic highly nonlinear material responses: multiphysics steel solidification, a 3D lug specimen, and a dogbone specimen under temporal and path-dependent loadings. The results show that S-NOT consistently achieves a higher prediction accuracy than S-DON even for data outliers, demonstrating its accuracy and robustness for drastically accelerating computational frameworks in scientific and engineering applications.

Figures

Figures reproduced from arXiv: 2507.03272 by the authors.

Figure 1
Figure 1. S-DON architecture. capturing the sequence of the input data. Gating methods in GRUs address vanishing gradient issues and facilitate the learning of long-term dependencies, rendering them more effective for sequential learning than conventional recurrent neural networks. Although the training duration was extended compared to the original DeepONet, the S￾DON model substantially enhanced predictive accuracy [27] and… view at source ↗
Figure 2
Figure 2. Overview of the S-NOT architecture. The solution decoder generates predictions at the query points by leveraging the sequential representations from the encoder. Each query point is first embedded using a NeRF-style positional encoding [36], followed by a multilayer perceptron (MLP), to produce the QUERY matrix of size 𝑁𝑞 × 𝑑𝑒 , where 𝑁𝑞 is the number of query points and 𝑑𝑒 is the embedding dimension. This QUERY is … view at source ↗
Figure 3
Figure 3. ‘Slice modeling domain in caster with boundary value conditions. ’ which accounts for latent heat due to solid-state transformations such as 𝛿-ferrite to austenite transitions. 𝜌 𝜕𝐻 𝜕𝑡 = ∇ ⋅ (𝑘∇𝑇 ) (3) Given the negligible influence of inertia during solidification, mechanical equilibrium is treated as quasi-static in Eq. (4): ∇ ⋅ 𝝈(𝒙) + b = 0 (4) Here, 𝜎 is the Cauchy stress tensor, and b is the body force density.… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: The 3D LUG specimen for FEM simulation. (a) The lug geometry, FE mesh, and time-dependent loading profile. (b) the material properties used in the simulation. In the second example, we consider a 3D lug specimen subjected to time-dependent pressure loading. The geometr…
Figure 5
Figure 5. Figure 5: The dog bone specimen. 4. Results In this section, we present the prediction results for the three benchmark datasets introduced in Section 3. To evaluate model performance, we use the relative 𝐿2 error for stress predictions, defined as 𝐿 2 = 1 𝑁𝑃 ∑ 𝑁𝑃 𝑖=1 ‖ ‖ ‖ 𝑺 tru…
Figure 6
Figure 6. Figure 6: Stress Comparison, Multiphysics Solidification. 4.1. Multiphysics Steel Solidification Both S-DON and S-NOT accurately predict the temperature distributions, which are relatively smooth, as indicated by the low mean relative 𝐿2 norm error across all test samples (Eq. (…
Figure 7
Figure 7. Figure 7: Test L2 errors Distribution (histograms) of the Multiphysics Slice. 4.2. 3D-LUG For the 3D-LUG example, model accuracy is evaluated using the relative 𝐿2 error for stress and the mean absolute error (MAE) for PEEQ. S-NOT achieves a relative 𝐿2 error of 5.31% for stress…
Figure 8
Figure 8. Figure 8: test L2 errors Distribution (histograms) of the 3D LUG. The histograms of test error distributions for both PEEQ and stress in [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Stress comparison for the 3D LUG example: True vs S-NOT vs S-DON. Each row corresponds to a test sample at the 50th, 75th, and 99th percentiles (by relative 𝐿2 error). Columns show: (1) true stress, (2) S-NOT prediction, (3) S-DON prediction, (4) error between S-NOT an…
Figure 10
Figure 10. Figure 10: PEEQ comparison for the 3D LUG example: True vs S-NOT vs S-DON. Rows correspond to the 50th, 75th, and 99th percentile test samples (by relative 𝐿2 error). Columns show: (1) true PEEQ, (2) S-NOT prediction, (3) S-DON prediction, (4) error between S-NOT and true, and (…
Figure 11
Figure 11. Figure 11: test L2 errors Distribution (histograms) of the Dog-bone. 50th percentile 75th percentile 99th percentile [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Stress comparison for the dog-bone example: True vs S-NOT vs S-DON. Rows correspond to the 50th, 75th, and 99th percentile test samples (by relative 𝐿2 error). Columns show: (1) true stress, (2) S-NOT prediction, (3) S-DON prediction, (4) error between S-NOT and true,…
Figure 13
Figure 13. Figure 13: PEEQ comparison for the dog-bone example: True vs S-NOT vs S-DON. Rows correspond to the 50th, 75th, and 99th percentile test samples (by relative 𝐿2 error). Columns show: (1) true PEEQ, (2) S-NOT prediction, (3) S-DON prediction, (4) error between S-NOT and true, and…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 38 canonical work pages

  1. [27]

    J.He,S.Kushwaha,J.Park,S.Koric,D.Abueidda,I.Jasiuk, Sequentialdeepoperatornetworks(s-deeponet)forpredictingfull-fieldsolutions under time-dependent loads, Engineering Applications of Artificial Intelligence 127 (2024) 107258

    work page 2024

  2. [1]

    Kiranyaz, J

    S. Kiranyaz, J. Malik, H. B. Abdallah, T. Ince, A. Iosifidis, M. Gabbouj, Exploiting heterogeneity in operational neural networks by synaptic plasticity, Neural Computing and Applications 33 (2021) 7997–8015. doi:10.1007/s00521-020-05543-w. Q. Liu et al.:Preprint submitted to Elsevier Page 12 of 14 S-NOT

    work page doi:10.1007/s00521-020-05543-w 2021

  3. [2]

    doi:10.1126/sciadv.abd7416

    Z.Yang,C.-H.Yu,M.J.Buehler, Deeplearningmodeltopredictcomplexstressandstrainfieldsinhierarchicalcomposites, ScienceAdvances 7 (2021) eabd7416. doi:10.1126/sciadv.abd7416

    doi:10.1126/sciadv.abd7416 2021

  4. [3]

    H. Sun, L. Peng, S. Huang, S. Li, Y. Long, S. Wang, W. Zhao, Development of a physics-informed doubly fed cross-residual deep neural network for high-precision magnetic flux leakage defect size estimation, IEEE Transactions on Industrial Informatics 18 (2021) 1629–1640. doi:10.1109/TII.2021.3089333

    work page arXiv 2021

  5. [4]

    Z. Peng, B. Yang, Y. Xu, F. Wang, L. Liu, Y. Zhang, Rapid surrogate modeling of electromagnetic data in frequency domain using neural operator, IEEE Transactions on Geoscience and Remote Sensing 60 (2022) 1–12. doi:10.1109/TGRS.2022.3222507

    work page arXiv 2022

  6. [5]

    Konuk, J

    T. Konuk, J. Shragge, Physics-guided deep learning using fourier neural operators for solving the acoustic vti wave equation, in: 82nd EAGE annualconference&exhibition,volume2021,EuropeanAssociationofGeoscientists&Engineers,2021,pp.1–5.doi: 10.3997/2214-4609. 202113304

    work page doi:10.3997/2214-4609 2021

  7. [6]

    Borrel-Jensen, S

    N. Borrel-Jensen, S. Goswami, A. P. Engsig-Karup, G. E. Karniadakis, C.-H. Jeong, Sound propagation in realistic interactive 3d scenes with parameterized sources using deep neural operators, Proceedings of the National Academy of Sciences 121 (2024) e2312159120. doi:doi.org/10.1073/pnas.2312159120

    doi:10.1073/pnas.2312159120 2024

  8. [7]

    doi:10.1021/acs.jpcb.3c07714

    Q.Liu,D.Abueidda,S.Vyas,Y.Gao,S.Koric,P.H.Geubelle, Adaptivedata-drivendeep-learningsurrogatemodelforfrontalpolymerization in dicyclopentadiene, The Journal of Physical Chemistry B 128 (2024) 1220–1230. doi:10.1021/acs.jpcb.3c07714

    work page doi:10.1021/acs.jpcb.3c07714 2024

Show all 45 references
  1. [8]

    P.Cai,S.Liu,Q.Liu,P.H.Geubelle,R.Gomez-Bombarelli, Towardslongrolloutofneuraloperatorswithlocalattentionandflowmatching- inspiredcorrection:AnexampleinfrontalpolymerizationPDEs, in:MachineLearningandthePhysicalSciencesWorkshop@NeurIPS2024,

  2. [9]

    Abdar, F

    M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, et al., A review of uncertainty quantification in deep learning: Techniques, applications and challenges, Information fusion 76 (2021) 243–297. doi:10.10...

    2021 doi

  3. [10]

    Cheng, C

    S. Cheng, C. Quilodrán-Casas, S. Ouala, A. Farchi, C. Liu, P. Tandeo, R. Fablet, D. Lucor, B. Iooss, J. Brajard, et al., Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review, IEEE/CAA Journal of Automatica Sinica 10 (2023) 1361–1387

    2023

  4. [11]

    doi:/10.1061/(ASCE)MT.1943-5533.0003843

    K.Li,Y.Long,H.Wang,Y.-F.Wang, Modelingandsensitivityanalysisofconcretecreepwithmachinelearningmethods, JournalofMaterials in Civil Engineering 33 (2021) 04021206. doi:/10.1061/(ASCE)MT.1943-5533.0003843

    2021

  5. [12]

    doi:10.1016/j.chemosphere.2023.138830

    S.Wang,Y.Ren,B.Xia,K.Liu,H.Li, Predictionofatmosphericpollutantsinurbanenvironmentbasedoncoupleddeeplearningmodeland sensitivity analysis, Chemosphere 331 (2023) 138830. doi:10.1016/j.chemosphere.2023.138830

    2023

  6. [13]

    E. M. Dogo, O. Afolabi, N. Nwulu, B. Twala, C. Aigbavboa, A comparative analysis of gradient descent-based optimization algorithms on convolutionalneuralnetworks, in:2018internationalconferenceoncomputationaltechniques,electronicsandmechanicalsystems(CTEMS), IEEE, 2018, pp. 92...

    2018

  7. [14]

    M. S. Daoud, M. Shehab, H. M. Al-Mimi, L. Abualigah, R. A. Zitar, M. K. Y. Shambour, Gradient-based optimizer (gbo): a review, theory, variants,andapplications, ArchivesofComputationalMethodsinEngineering30(2023)2431–2449.doi: 10.1007/s11831-022-09872-y

    2023 doi

  8. [15]

    Q. Liu, P. Cai, D. Abueidda, S. Vyas, S. Koric, R. Gomez-Bombarelli, P. Geubelle, Univariate conditional variational autoencoder for morphogenic pattern design in frontal polymerization-based manufacturing, Computer Methods in Applied Mechanics and Engineering 438 (2025) 11784...

    2025

  9. [16]

    Q. Liu, S. Koric, D. Abueidda, H. Meidani, P. Geubelle, Towards signed distance function based metamaterial design: Neural operator transformer for forward prediction and diffusion model for inverse design, arXiv preprint arXiv:2504.01195 (2025). doi:10.48550/arXiv. 2504.01195

    2025 doi arXiv

  10. [17]

    L. Lu, P. Jin, G. Pang, et al., Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nature Machine Intelligence 3 (2021) 218–229

    2021

  11. [18]

    Koric, A

    S. Koric, A. Viswanath, D. W. Abueidda, N. A. Sobh, K. Khan, Deep learning operator network for plastic deformation with variable loads and material properties, Engineering with Computers 40 (2024) 917–929

    2024

  12. [19]

    Goswami, M

    S. Goswami, M. Yin, Y. Yu, G. E. Karniadakis, A physics-informed variational deeponet for predicting crack path in quasi-brittle materials, Computer Methods in Applied Mechanics and Engineering 391 (2022) 114587

    2022

  13. [20]

    T. Zhao, W. Qian, J. Lin, H. Chen, H. Ao, G. Chen, L. He, Learning mappings from iced airfoils to aerodynamic coefficients using a deep operator network, Journal of Aerospace Engineering 36 (2023) 04023035

    2023

  14. [21]

    L. Xu, H. Zhang, M. Zhang, Training a deep operator network as a surrogate solver for two-dimensional parabolic-equation models, The Journal of the Acoustical Society of America 154 (2023) 3276–3284

    2023

  15. [22]

    Haghighat, U

    E. Haghighat, U. b. Waheed, G. Karniadakis, En-deeponet: An enrichment approach for enhancing the expressivity of neural operators with applications to seismology, Computer Methods in Applied Mechanics and Engineering 420 (2024) 116681

    2024

  16. [23]

    Kobayashi, J

    K. Kobayashi, J. Daniell, S. B. Alam, Improved generalization with deep neural operators for engineering systems: Path towards digital twin, Engineering Applications of Artificial Intelligence 131 (2024) 107844

    2024

  17. [24]

    Sahin, C

    I. Sahin, C. Moya, A. Mollaali, G. Lin, G. Paniagua, Deep operator learning-based surrogate models with uncertainty quantification for optimizing internal cooling channel rib profiles, International Journal of Heat and Mass Transfer 219 (2024) 124813

    2024

  18. [25]

    S.Koric,D.W.Abueidda, Data-drivenandphysics-informeddeeplearningoperatorsforsolutionofheatconductionequationwithparametric heat source, International Journal of Heat and Mass Transfer 203 (2023) 123809

    2023

  19. [26]

    R. B. Hossain, F. Ahmed, K. Kobayashi, S. Koric, D. Abueidda, S. B. Alam, Virtual sensing-enabled digital twin framework for real-time monitoring of nuclear systems leveraging deep neural operators, npj Materials Degradation 9 (2025) 21

    2025

  20. [28]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems,...

    2017

  21. [29]

    Cao, Choose a transformer: Fourier or galerkin, Advances in neural information processing systems 34 (2021) 24924–24940

    S. Cao, Choose a transformer: Fourier or galerkin, Advances in neural information processing systems 34 (2021) 24924–24940

    2021

  22. [30]

    X. Liu, B. Xu, L. Zhang, Ht-net: Hierarchical transformer based operator learning model for multiscale pdes (2022)

    2022

  23. [31]

    Farimani, Transformerforpartial differentialequations’ operatorlearning, arXiv preprintarXiv:2205.13671(2022)

    Z.Li, K.Meidani, A.B. Farimani, Transformerforpartial differentialequations’ operatorlearning, arXiv preprintarXiv:2205.13671(2022). doi:10.48550/arXiv.2205.13671

    2022 doi arXiv

  24. [32]

    12556–12569

    Z.Hao,Z.Wang,H.Su,C.Ying,Y.Dong,S.Liu,Z.Cheng,J.Song,J.Zhu, Gnot:Ageneralneuraloperatortransformerforoperatorlearning, in: International Conference on Machine Learning, PMLR, 2023, pp. 12556–12569

    2023

  25. [33]

    Q. Liu, V. Zhong, H. Meidani, D. Abueidda, S. Koric, P. Geubelle, Geometry-informed neural operator transformer, arXiv preprint arXiv:2504.19452 (2025). doi:doi.org/10.48550/arXiv.2504.19452

    2025 doi

  26. [34]

    doi:10.1016/j.cma.2024.117560

    B.Shih,A.Peyvan,Z.Zhang,G.E.Karniadakis, Transformersasneuraloperatorsforsolutionsofdifferentialequationswithfiniteregularity, Computer Methods in Applied Mechanics and Engineering 434 (2025) 117560. doi:10.1016/j.cma.2024.117560

    2025

  27. [35]

    Kushwaha, J

    S. Kushwaha, J. Park, S. Koric, J. He, I. Jasiuk, D. Abueidda, Advanced deep operator networks to predict multiphysics solution fields in materials processing and additive manufacturing, Additive Manufacturing (2024) 104266

    2024

  28. [36]

    doi:10.48550/arXiv.2003.08934

    B.Mildenhall,P.P.Srinivasan,M.Tancik,J.T.Barron,R.Ramamoorthi,R.Ng, Nerf:Representingscenesasneuralradiancefieldsforview synthesis, Communications of the ACM 65 (2021) 99–106. doi:10.48550/arXiv.2003.08934

    2021 doi arXiv

  29. [37]

    Kozlowski, B

    P. Kozlowski, B. Thomas, J. Azzi, H. Wang, Simple constitutive equations for steel at high temperature, Metallurgical and Materials Transactions A 23 (1992) 903–918

    1992

  30. [38]

    Zhu, Coupled Thermo-Mechanical Finite-Element Model with Application to Initial Solidification, Ph.D

    H. Zhu, Coupled Thermo-Mechanical Finite-Element Model with Application to Initial Solidification, Ph.D. thesis, University of Illinois at Urbana-Champaign, Urbana, IL, USA, 1996

    1996

  31. [39]

    Dassault Systèmes Simulia Corp., Abaqus/Standard User’s Manual, Version 2022, Johnston, RI, USA, 2022

    2022

  32. [40]

    Koric, B

    S. Koric, B. G. Thomas, Efficient thermo-mechanical model for solidification processes, International Journal for Numerical Methods in Engineering 66 (2006) 1955–1989

    2006

  33. [41]

    M.Zappulla,S.Cho,S.Koric,H.Lee,S.Kim,B.Thomas, Multiphysicsmodelingofcontinuouscastingofstainlesssteel, JournalofMaterials Processing Technology 278 (2020) 116469

    2020

  34. [42]

    D.W.Abueidda,S.Koric,N.A.Sobh,H.Sehitoglu, Deeplearningforplasticityandthermo-viscoplasticity, InternationalJournalofPlasticity 136 (2021) 102852

    2021

  35. [43]

    NCSA, Delta user documentation, 2025.https://docs.ncsa.illinois.edu/systems/delta/en/latest/index.html

    2025

  36. [44]

    NCSA, Delta ai user documentation, 2025.https://docs.ncsa.illinois.edu/systems/deltaai/en/latest/index.html. Q. Liu et al.:Preprint submitted to Elsevier Page 14 of 14

    2025

  37. [2024]

    URL:https://openreview.net/forum?id=CFNgeuXzPx

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.