Pith. sign in

REVIEW 5 major objections 5 minor 45 references

An Attention-based Spatio-Temporal Neural Operator for Evolving Physics

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Splitting temporal extrapolation from spatial correction yields a neural surrogate that generalizes zero-shot to unseen physics.

desk verdict A promising architectural idea undermined by an unfair comparison: ASNO sees future forcing while baselines do not. read the letter →

arxiv 2506.11328 v1 pith:Y5V4I5QA submitted 2025-06-12 cs.LG cs.CE

classification cs.LGcs.CE MSC 68T0765L06
keywords spatio-temporalneuraloperatorbackwarddifferentiationformulaimplicit-explicitschemenonlocalattentionzero-shotgeneralizationscientificmachinelearningDarcyflow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a surrogate model for evolving physical systems predicts better, generalizes to unseen conditions, and reveals its own mechanics when its architecture mirrors a classical numerical scheme instead of treating prediction as an undifferentiated black box. The scheme in question is the implicit-explicit backward differentiation formula: an explicit step extrapolates the next state from past states, and an implicit step corrects that extrapolation using external forcing and spatial coupling. ASNO instantiates this split with a Transformer encoder for the explicit extrapolation and a nonlocal attention operator that learns a spatial kernel for the correction, yielding the composition $X_{m+1} = \text{NAO}(\text{TE}(X_{m-4},\dots,X_m), F_{m+1})$. On Darcy flow, the Lorenz system, Navier–Stokes, and a directed-energy-deposition manufacturing dataset, the paper reports the lowest test losses, the lowest out-of-distribution losses, and slower error growth in long autoregressive rollouts than the compared baselines. A sympathetic reader would care because zero-shot adaptation to new physical parameters, and components that can be checked against known physics, are exactly what surrogate models for digital twins and engineering design currently lack.

What carries the argument

The load-bearing object is the ASNO composition $X_{m+1} = \text{NAO}(\text{TE}(X_{m-4},\dots,X_m), F_{m+1})$, which encodes the implicit-explicit BDF split. In the explicit step the Transformer encoder compresses the past five states into a latent $H_{m+1}$ that plays the role of the homogeneous extrapolation $\tilde{X}_{m+1}$ in BDF. In the implicit step the Nonlocal Attention Operator (NAO), an attention mechanism that estimates a data-dependent nonlocal kernel $K[H_{1:d}, F_{1:d}](y,z)$ and outputs $X_{m+1}(y) = \int K(y,z) F_{m+1}(z)\,dz$, plays the role of the static PDE solver that corrects the extrapolation under forcing. The kernel map is built from stacked attention layers with residual connections (Eqs. 12–14), and the whole pipeline is trained end-to-end on the $L^2$ prediction loss over all systems (Eq. 16). The separation is what does the work: each module has a reduced job, one learning a multistep linear recurrence and the other a compact spatial correction, and the paper credits this split for both the generalization and the interpretability results.

What would settle it

On the Darcy benchmark, test ASNO with the true next-step forcing $F_{m+1}$ withheld — set it to zero or to the previous step's forcing $F_m$ while keeping the same past five states — and compare the loss jump to the OOD-f shift; if the error rises to or beyond the baseline models' levels, the reported zero-shot performance depends on the model being handed the future forcing and the claim collapses to interpolation of a known input. Separately, recompute the learned NAO kernel on permeability microstructures drawn from a different GRF family and check it against $K_{\text{true}}$ of Eq. (22): the interpretability claim fails if the kernel agrees only on training-distribution data.

Watch

Extended reading notes

Core claim

The paper's central claim is that spatio-temporal physics admits a separable decomposition that can be learned directly from data: the next state is a linear multistep extrapolation of recent history, corrected by a nonlocal operation that depends on that extrapolation and on the external loading at the next step. ASNO learns both pieces — a Transformer encoder produces the latent extrapolation $H_{m+1}$, and the Nonlocal Attention Operator (NAO) learns a kernel $K[H_{1:d}, F_{1:d}](y,z)$ and applies it to the forcing field to produce the output — and the paper reports that the jointly trained components rediscover known structure. On dynamic Darcy flow, the learned latent closely matches the BDF5 extrapolation (Eq. 19) and the learned kernel agrees with the closed-form kernel $K_{\text{true}} = -(60/137)\Delta t (I + (60/137)\Delta t A)^{-1}$ from the Darcy discretization (Eq. 22). Empirically the paper finds that ASNO achieves the lowest best, OOD-forcing, and OOD-permeability test losses among FNO, U-Net, Transolver, GNOT, DeepONet, and Transformer variants, and that this advantage persists on the Lorenz system, the Navier–Stokes benchmark, and a DED melt-pool temperature task. The intended consequence is that a single trained model can predict unseen initial conditions, loadings, and environments zero-shot while exposing temporal and spatial contributions separately for physics discovery.

Load-bearing premise

The load-bearing assumption is that the next state truly decomposes as a linear multistep extrapolation of past states corrected by a function that depends only on that extrapolation and on the next-step forcing (Eq. 17), an inductive bias imposed by the architecture rather than derived from the dynamics, and one that requires the future forcing $F_{m+1}$ to be known at prediction time.

Editorial extensions

If this is right

  • On dynamic Darcy flow the model posts the lowest best, OOD-forcing, and OOD-permeability losses (0.0368, 0.0673, 0.0982), beating the strongest baseline Transolver (0.0428, 0.0721, 0.1535) with fewer trainable parameters.
  • The learned components are inspectable: the Transformer latent tracks the BDF5 extrapolation and the NAO kernel matches the closed-form Darcy kernel, so the model can be read as having rediscovered the time integrator and the kernel structure of the PDE.
  • Long-term autoregressive rollouts stay stable, with cumulative error growing more slowly than for the baselines on Darcy and Navier–Stokes, which matters when predictions are fed back as inputs.
  • The same architecture transfers to an engineering use: DED melt pool temperature fields are predicted with 2.50% MAPE from laser power, scan rate, and laser location inputs.
  • On the Lorenz system the separation of a linear multistep forecast from a small nonlinear correction keeps chaotic trajectories near the true attractor longer than monolithic models, with test loss 0.000794 versus 0.000835 for the next best model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the temporal component would extract the learned attention weights from the Transformer encoder and compare them coefficient-by-coefficient against the BDF5 coefficients of Eq. (19); the paper shows visual agreement on Darcy only, and a coefficient-level match would confirm genuine integration-scheme discovery.
  • The requirement that $F_{m+1}$ be known at inference marks a real boundary of the method: in forecasting settings where future forcing is itself unknown, ASNO would need a forcing forecaster, and the paper does not test that regime; a stress test with lagged or noised forcing would map that boundary.
  • The OOD-f experiment shifts forcing amplitude by a factor of 200 in one jump; sweeping the amplitude continuously between the training value and the OOD value would reveal whether generalization degrades gradually or snaps at a threshold, which is the practically useful curve for engineering deployment.
  • The kernel-versus-theory comparison protocol could be repeated on systems with analytically known kernels, such as the heat or wave equation, to sharpen the physics-discovery claim beyond the single Darcy example.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ASNO, a spatio-temporal neural operator that combines a Transformer encoder for temporal extrapolation with an attention-based nonlocal operator for spatial correction, inspired by the implicit-explicit BDF scheme. The authors claim that ASNO generalizes zero-shot to unseen physical parameters, outperforms existing neural operators and transformers on Darcy flow, Lorenz, Navier–Stokes, and additive manufacturing benchmarks, and yields interpretable components that match a BDF5 extrapolation and a Darcy-derived kernel. The central architectural idea is interesting, but the empirical evaluation has a serious input-information asymmetry: ASNO receives the future forcing F_{m+1}, while all baselines receive only the current forcing F_m. Additional issues include overstated OOD claims, inconsistent dataset arithmetic in the Lorenz experiment, and the absence of error bars. The current evidence does not support the paper's main claims.

Significance. If the claims were supported, ASNO would be a meaningful contribution: a separable, BDF-inspired architecture that isolates temporal and spatial effects, with potential for interpretability and zero-shot generalization in scientific ML. The paper also addresses a practically relevant problem (forecasting with known future loadings, e.g., in additive manufacturing). The architecture itself is plausible and the experimental breadth is commendable. However, the significance is currently undercut by the comparison protocol: because ASNO is given F_{m+1} while baselines are not, the reported accuracy and OOD gains could stem entirely from this information advantage. The interpretability results are also post-hoc and not quantitatively validated. The paper would be significant only after a fair comparison and a more careful treatment of OOD claims.

major comments (5)
  1. [§4.1, Eq. (17); §4.2] The central comparison is unfair. Eq. (17) defines ASNO(X_m,...,X_{m-n+1}, F_{m+1}), and Figure 1 shows F_{m+1} as an input, but Section 4.1 states that all baselines receive the past five states and the current forcing field F_m; Section 4.2 repeats that each model receives the five previous states plus F_m. Thus ASNO has direct access to the forcing at the time step it must predict, whereas the baselines do not. In the Darcy OOD-f experiment the forcing is time-varying (g(t,x)=g~(x) sin(t)), so F_{m+1} carries information that F_m does not. The reported margins over Transolver, GNOT, and other baselines in Tables 1–3 could therefore be explained by input asymmetry rather than architectural merit. The paper provides no ablation with ASNO restricted to F_m and no experiment giving F_{m+1} to baselines. This must be fixed before any performance claim can be evaluated.
  2. [§1 and §4.1–4.3] The paper repeatedly claims zero-shot generalization to unseen initial conditions (e.g., the abstract and contribution list), but no experiment with unseen initial conditions is reported. The Darcy OOD tests vary the forcing distribution and the permeability field, the Lorenz experiment uses the same initial condition (0,1,0) throughout, and the Navier–Stokes section reports only test loss without any OOD shift. Either an explicit out-of-distribution initial-condition experiment must be added, or the claim must be revised to refer only to the tested generalization modes.
  3. [§4.2, Lorenz dataset] The Lorenz dataset arithmetic is inconsistent. The text says there are 2000 profiles from 100 coefficient combinations times 20 loading functions, with 80% for training and 20% for testing. That implies 1600 training profiles, yet the calculation given is '80×995×20 = 1,592,000 training samples', which uses 80 profiles, not 1600. If the intended split is 1600 profiles, the number of training samples should be 1600×995×20 = 31,840,000 before any subsampling; if the intended split is indeed 80/20 profiles, the statement that the dataset has 2000 profiles is wrong. This makes the reported sample counts and the resulting test-loss numbers unreliable.
  4. [Tables 1–3] All performance tables report only the 'Best Test Loss' over what appears to be a single run or a selected checkpoint. No standard deviations, no seed counts, and no training/validation protocol are given. Without repeated seeds or error bars, the claim that ASNO 'achieves the lowest' loss and 'outperforms' baselines is not statistically supported, especially where margins are small (e.g., Lorenz: 0.000794 vs 0.000835 for Transolver). The paper should report mean and standard deviation over multiple random seeds and a defined model-selection rule.
  5. [§4.1, Eqs. (19)–(22), Figures 2–3] The interpretability results are post-hoc and not quantitatively validated. The claim that the Transformer latent 'closely matches' BDF5 and that the NAO kernel 'agrees with' the Darcy kernel is based on visual comparison only; no error metric, no confidence interval, and no test on out-of-distribution systems is provided. Moreover, the NAO kernel is compared with a kernel derived from a specific Darcy discretization using the same equation family on which the model was trained, so the alignment may reflect memorization of the training distribution rather than discovery of a general law. A quantitative comparison and a held-out test are needed before the physics-discovery claim can be supported.
minor comments (5)
  1. [§2.2 and Table 2] The Lorenz section says 'We did not include architectures such as U-Net, FNO, and GNOT in our Lorenz comparison', but Table 2 includes GNOT. This contradiction should be resolved.
  2. [§4.1, Table 1] The list of baselines in the text mentions 'Transformer Encoder + NAO', but the table lists 'Linear Enc. + NAO' instead. Clarify which baseline was used and whether the naming is consistent.
  3. [Eq. (19) and Figure 2] Equation (19) writes the BDF approximation as \tilde X_m, while the text and Figure 2 compare the latent H_{m+1} with \tilde X_{m+1}; the index shift should be made explicit, and the formula should be written for the same time index as the comparison.
  4. [Abstract and §5] There are several grammatical and typographical issues, e.g., 'ASNO outperforms over existing models' in the abstract and 'outpertforms' in the conclusion. The paper should be proofread.
  5. [§4.4] The additive manufacturing experiment reports a single MAPE value of 2.50% with no comparison to baselines, no error bars, and no description of how the MAPE is computed across the 290,400 test samples. Include a table or comparison to make the result interpretable.

Circularity Check

1 steps flagged · score 5.0 of 10

ASNO is fed the future forcing F_{m+1} while baselines receive only F_m; the reported OOD/accuracy gains reduce by construction to this information asymmetry, with the BDF/kernel 'discovery' results being post-hoc self-consistency checks rather than independent predictions.

  1. other [Section 3.4, Eq. (17); Section 4.1, baseline input description; Section 4.2 Lorenz setup]
    "X out m+1 = NAO Hm+1, Fm+1 = NAO TE(Xm, Xm−1, . . . , Xm−n+1), Fm+1 = ASNO(Xm, Xm−1, . . . , Xm−n+1, Fm+1) (17) ... All baseline models (FNO, U-Net, Transolver, GNOT, DeepONet, Transformer Encoder, Transformer Encoder + NAO) use the same inputs: the past five states Xm−4, . . . , Xm and the current forcing field Fm (Sec. 4.1)."

    By construction, the ASNO prediction X_out_{m+1} is a function of F_{m+1}, the forcing at the very time step being predicted, whereas every baseline is restricted to F_m. Since the source is time-varying in the OOD-f experiment (g(t,x)=g~(x) sin(t)) and in the Lorenz experiments (g_i(t) time-dependent), F_{m+1} carries information about the target state X_{m+1} that F_m does not. Thus the reported lower test/OOD losses in Tables 1 and 2, and the conclusion of strongest OOD generalization, are confounded by an information asymmetry: the comparison is between a model given the future loading and models denied it. No ablation restricts ASNO to F_m or grants baselines F_{m+1}, so the margin is explained by the input protocol rather than by the architecture.

full rationale

The paper's model derivation is an ansatz, not a circular derivation: BDF motivates the TE+NAO split (Eqs. 1-17), the model is trained end-to-end, and its prediction operator is a learned map rather than a restatement of its inputs. The interpretability results (Figures 2-3, Eqs. 19-22) are post-hoc consistency checks on the trained model, not parameter-free law extraction; they do not by themselves make the derivation circular, but they also do not independently confirm physics discovery. The NAO component is cited from Yu et al. (2024), a prior paper by a co-author, yet the current paper evaluates ASNO empirically, so this self-citation is not the sole load-bearing evidence. The one serious circularity-adjacent defect is the input asymmetry in the benchmark protocol: Eq. (17) feeds F_{m+1} to ASNO, while Section 4.1 states that all baselines receive only F_m. In time-varying forcing experiments, the future forcing is not inferable from F_m, so the reported accuracy and OOD advantages are explained by construction by the extra input rather than by the separable architecture. This makes the central empirical claim partially circular in its validation logic, even though the model's internal derivation is not formally circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central design rests on the assumed separability of temporal and spatial physics, the availability of future forcing, and the inferability of hidden states from observations. These are domain assumptions introduced by the paper, not derived facts. The hand-chosen hyperparameters (window length, permutation count) and the latent hidden state are the main additional commitments.

free parameters (3)
  • history window length n = 5
    Chosen by hand for all experiments; the model and baselines all use five past states, and performance depends on this choice.
  • random permutation count per trajectory = 20
    Data augmentation multiplier used for all benchmarks; chosen without sensitivity analysis; inflates sample counts by 20x.
  • NAO attention steps T = not specified
    Number of iterative attention steps in the NAO (Eq. 12) is a hyperparameter; the paper does not report its value or sensitivity.
assumptions (4)
  • ad hoc to paper The next state can be factored as a linear multistep extrapolation of past states followed by a nonlinear spatial correction depending only on the extrapolated latent and the future forcing: X_{m+1}=NAO(TE(X_{m-4},...,X_m), F_{m+1}).
    This is the central inductive bias of the architecture, introduced in Section 3.4 Eq. (17). It is not derived from the PDE structure.
  • domain assumption The external forcing F_{m+1} at the time step to be predicted is available as input at inference.
    Used in Eq. (17) and Eq. (30); in real forecasting F_{m+1} is usually unknown, and baselines are only given F_m.
  • domain assumption The hidden system state S^{(eta)} is inferable from the observation sequence without labels.
    Section 3.1 goal (2); the paper assumes the model can discover S directly from (X,F) observations, but no inverse-problem evaluation is provided.
  • standard math Standard BDF/IMEX stability and consistency properties transfer to the learned analog.
    The motivation in Section 3.1 relies on BDF theory, but no theorem guarantees the learned components inherit these properties.
invented entities (1)
  • Hidden system state S^{(eta)}
    purpose: Accounts for unseen environment and PDE coefficients in the ODE system of Eq. (4); inferred from data without supervision.
    It is a latent variable introduced to justify generalization; no falsifiable prediction or measurement is tied to it, so it can absorb any unexplained variation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Attention-based Spatio-Temporal Neural Operator for Evolving Physics." pith.science (2026). https://pith.science/paper/Y5V4I5QA

@misc{pith2026250611328,
  author       = {Pith},
  title        = {Pith review of: An Attention-based Spatio-Temporal Neural Operator for Evolving Physics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y5V4I5QA}},
  note         = {Machine review of arXiv:2506.11328}
}
read the original abstract

In scientific machine learning (SciML), a key challenge is learning unknown, evolving physical processes and making predictions across spatio-temporal scales. For example, in real-world manufacturing problems like additive manufacturing, users adjust known machine settings while unknown environmental parameters simultaneously fluctuate. To make reliable predictions, it is desired for a model to not only capture long-range spatio-temporal interactions from data but also adapt to new and unknown environments; traditional machine learning models excel at the first task but often lack physical interpretability and struggle to generalize under varying environmental conditions. To tackle these challenges, we propose the Attention-based Spatio-Temporal Neural Operator (ASNO), a novel architecture that combines separable attention mechanisms for spatial and temporal interactions and adapts to unseen physical parameters. Inspired by the backward differentiation formula (BDF), ASNO learns a transformer for temporal prediction and extrapolation and an attention-based neural operator for handling varying external loads, enhancing interpretability by isolating historical state contributions and external forces, enabling the discovery of underlying physical laws and generalizability to unseen physical environments. Empirical results on SciML benchmarks demonstrate that ASNO outperforms over existing models, establishing its potential for engineering applications, physics discovery, and interpretable machine learning.

Figures

Figures reproduced from arXiv: 2506.11328 by the authors.

Figure 1
Figure 1. The ASNO framework with the Transformer Encoder for capturing temporal dependencies and the Non-Local Attention Operator (NAO) for modeling spatial interactions, following a Backward Differentiation Formula (BDF) approach. 3.2. Explicit Step: Temporal Extrapolation As the first component of ASNO, we employ a Transformer Encoder to resemble the explicit step (2). In particular, it processes time series data by captur… view at source ↗
Figure 2
Figure 2. Comparison of the latent prediction from the Transformer Encoder (Hm+1), the BDF approximation (X˜m+1), and the ASNO￾predicted output at Timestep 10. The latent prediction and BDF approximation closely align, and the final ASNO output differs as the NAO adjusts the prediction using field-based corrections. As the Transformer Encoder processes the past observations and outputs the latent variable Hm+1 using Equation … view at source ↗
Figure 3
Figure 3. Left: Cumulative error over time for various models on the Darcy flow benchmark, illustrating ASNO’s long-term stability. Right: Comparison between the ground-truth Darcy kernel and the spatial kernel learned by the Nonlocal Attention Operator (NAO), demonstrating alignment with the underlying physics. 4.2. Chaotic ODE: Lorenz System In this section, we evaluate the performance of ASNO on the Lorenz system, a well-k… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Long Time prediction comparison of Predicted vs. true time-series trajectories (x, y, z) for models listed in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Left: Cumulative error comparison across models listed in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Prediction results on the DED AM case study with the ASNO prediction on the left, and target results on the right. external load impacts. To ensure robust generalization across diverse physical systems, ASNO integrates the NAO, which facilitates adaptation to varying P…
Figure 7
Figure 7. Figure 7: Comparison of ASNO model output and target temperature distributions for the melt pool in Directed Energy Deposition at Timesteps 10, 25, and 36. Each row represents a different timestep, with the model output shown on the left and the target on the right. The color sc…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 33 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in ":" * " " * FUNCTION f...

  2. [2]

    M., Ruuth, S

    Ascher, U. M., Ruuth, S. J., and Wetton, B. T. (1995). Implicit-explicit methods for time-dependent partial differential equations. SIAM Journal on Numerical Analysis , 32(3):797--823

  3. [3]

    Azizzadenesheli, K., Kovachki, N., Li, Z., Liu-Schiaffini, M., Kossaifi, J., and Anandkumar, A. (2024). Neural operators for accelerating scientific simulations and design. Nature Reviews Physics , pages 1--9

  4. [4]

    Cao, Q., Goswami, S., and Karniadakis, G. E. (2024). Laplace neural operator for solving differential equations. Nature Machine Intelligence , 6(6):631--640

  5. [5]

    Cao, S. (2021). Choose a transformer: Fourier or galerkin. Advances in neural information processing systems , 34:24924--24940

  6. [6]

    Real-Time Decision-Making for Digital Twin in Additive Manufacturing with Model Predictive Control using Time-Series Deep Neural Networks

    Chen, Y.-P., Karkaria, V., Tsai, Y.-K., Rolark, F., Quispe, D., Gao, R. X., Cao, J., and Chen, W. (2025). Real-time decision-making for digital twin in additive manufacturing with model predictive control using time-series deep neural networks. arXiv preprint arXiv:2501.07601

  7. [7]

    Cheng, C.-W., Huang, J., Zhang, Y., Yang, G., Sch \"o nlieb, C.-B., and Aviles-Rivero, A. I. (2024). Mamba neural operator: Who wins? transformers vs. state-space models for pdes. arXiv preprint arXiv:2410.02113

  8. [8]

    M., and Aloise, D

    Fournier, Q., Caron, G. M., and Aloise, D. (2023). A practical survey on faster and lighter transformers. ACM Computing Surveys , 55(14s):1--40

Show all 45 references
  1. [9]

    Fredebeul, C. (1998). A-bdf: a generalization of the backward differentiation formulae. SIAM journal on numerical analysis , 35(5):1917--1938

  2. [10]

    and Graves, A

    Graves, A. and Graves, A. (2012). Long short-term memory. Supervised sequence labelling with recurrent neural networks , pages 37--45

  3. [11]

    Gunasegaram, D., Barnard, A., Matthews, M., Jared, B., Andreaco, A., Bartsch, K., and Murphy, A. (2024). Machine learning-assisted in-situ adaptive strategies for the control of defects and anomalies in metal additive manufacturing. Additive Manufacturing , page 104013

  4. [12]

    Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. (2023). Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning , pages 12556--12569. PMLR

  5. [13]

    G., Pretorius, J

    Kapteyn, M. G., Pretorius, J. V., and Willcox, K. E. (2021). A probabilistic graphical model foundation for enabling predictive digital twins at scale. Nature Computational Science , 1(5):337--347

  6. [14]

    X., and Chen, W

    Karkaria, V., Goeckner, A., Zha, R., Chen, J., Zhang, J., Zhu, Q., Cao, J., Gao, R. X., and Chen, W. (2024a). Towards a digital twin framework in additive manufacturing: Machine learning and bayesian optimization for time series process optimization. Journal of Manufacturing Systems

  7. [15]

    Karkaria, V., Tsai, Y.-K., Chen, Y.-P., and Chen, W. (2024b). An optimization-centric review on integrating artificial intelligence and digital twin technologies in manufacturing. Engineering Optimization , pages 1--47

  8. [16]

    and Lee, J

    Kim, D. and Lee, J. (2024). A review of physics informed neural networks for multiscale analysis and inverse problems. Multiscale Science and Engineering , pages 1--11

  9. [17]

    Kitaev, N., Kaiser, ., and Levskaya, A. (2020). Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451

  10. [18]

    Ko, H., Kim, J., Lu, Y., Shin, D., Yang, Z., and Oh, Y. (2022). Spatial-temporal modeling using deep learning for real-time monitoring of additive manufacturing. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference ,...

  11. [19]

    Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2023). Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research , 24(89):1--97

  12. [20]

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2020). Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895

  13. [21]

    Liao, S., Golgoon, A., Mozaffar, M., and Cao, J. (2023). Efficient gpu-accelerated thermomechanical solver for residual stress prediction in additive manufacturing. Computational Mechanics , 71(5):879--893

  14. [22]

    \"O ., Loeff, N., and Pfister, T

    Lim, B., Ar k, S. \"O ., Loeff, N., and Pfister, T. (2021). Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting , 37(4):1748--1764

  15. [23]

    and Yu, Y

    Lu, F. and Yu, Y. (2025). Transformer learns the cross-task prior and regularization for in-context learning. arXiv preprint arXiv:2505.12138

  16. [24]

    Lu, L., Jin, P., and Karniadakis, G. E. (2019). Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193

  17. [25]

    Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. (2021). Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence , 3(3):218--229

  18. [26]

    Mandl, L., Goswami, S., Lambers, L., and Ricken, T. (2024). Separable deeponet: Breaking the curse of dimensionality in physics-informed machine learning. arXiv preprint arXiv:2407.15887

  19. [27]

    Molinaro, R., Yang, Y., Engquist, B., and Mishra, S. (2023). Neural inverse operators for solving pde inverse problems. arXiv preprint arXiv:2301.11167

  20. [28]

    Molnar, C. (2020). Interpretable machine learning . Lulu. com

  21. [29]

    Nelson, B. K. (1998). Time series analysis using autoregressive integrated moving average (arima) models. Academic emergency medicine , 5(7):739--744

  22. [30]

    Niu, Z., Zhong, G., and Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing , 452:48--62

  23. [31]

    Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., and Zhong, C. (2022). Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys , 16:1--85

  24. [32]

    and Matteson, D

    Tang, B. and Matteson, D. S. (2021). Probabilistic transformer for time series analysis. Advances in Neural Information Processing Systems , 34:23592--23608

  25. [33]

    D., Todd, M

    Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C., and Hu, Z. (2022). A comprehensive review of digital twin—part 1: modeling and twinning enabling technologies. Structural and Multidisciplinary Optimization , 65(12):354

  26. [34]

    van Beek, A., Nevile Karkaria, V., and Chen, W. (2023). Digital twins for the designs of systems: a perspective. Structural and Multidisciplinary Optimization , 66(3):49

  27. [35]

    N., Kaiser, L., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems , 30

  28. [36]

    Wang, Q., Ren, P., Zhou, H., Liu, X.-Y., Deng, Z., Zhang, Y., Chengze, R., Liu, H., Wang, Z., Wang, J.-X., et al. (2024). P ^2 c ^2 net: Pde-preserved coarse correction network for efficient prediction of spatiotemporal dynamics. arXiv preprint arXiv:2411.00040

  29. [37]

    Wen, G., Li, Z., Azizzadenesheli, K., Anandkumar, A., and Benson, S. M. (2022). U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources , 163:104180

  30. [38]

    Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. (2024). Transolver: A fast transformer solver for pdes on general geometries. arXiv preprint arXiv:2402.02366

  31. [39]

    J., Lee, C.-H., and Yu, Y

    You, H., Zhang, Q., Ross, C. J., Lee, C.-H., and Yu, Y. (2022). Learning deep implicit fourier neural operators ( IFNOs ) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering , 398:115296

  32. [40]

    Yu, Y., Liu, N., Lu, F., Gao, T., Jafarzadeh, S., and Silling, S. (2024). Nonlocal attention operator: Materializing hidden knowledge towards interpretable physics discovery. arXiv preprint arXiv:2408.07307

  33. [41]

    Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., and Eickhoff, C. (2021). A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages 2114--2124

  34. [42]

    Zhang, Y., Zhang, Y., Zhang, Z., Bao, J., and Song, Y. (2018). Human activity recognition based on time series analysis using u-net. arXiv preprint arXiv:1809.08113

  35. [43]

    Zhao, T., Fang, L., Ma, X., Li, X., and Zhang, C. (2024). Tfformer: A time-frequency domain bidirectional sequence-level attention based transformer for interpretable long-term sequence forecasting. Pattern Recognition , page 110994

  36. [44]

    Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11106--11115

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.