Pith. sign in

REVIEW 3 major objections 5 minor 41 references

Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By aligning the governing equations discovered from ground-truth and predicted windows, PITA reduces long-trajectory prediction error of auto-regressive PDE foundation models by about 30%.

desk verdict The paper's central consistency loss has zero gradient under the stated alternating optimization, which undermines the reported ablation unless the code uses an undocumented surrogate; still worth a peer review round to pin down the mechanism. read the letter →

arxiv 2505.10930 v2 pith:554S4HIC submitted 2025-05-16 cs.LG

Congcong Zhu , Xiaoyan Xu , Jiayue Han , Jingrun Chen This is my paper · ORCID
classification cs.LG
keywords physics-informedtemporalalignmentauto-regressivePDEfoundationmodelsshortcutlearningerroraccumulationdiscoverysparseregressionself-supervisedout-of-distributiongeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Auto-regressive PDE foundation models tend to learn a shortcut: instead of evolving the field, they copy the previous frame, and errors pile up over long rollouts, especially on out-of-distribution data. This paper proposes a self-supervised training signal, PITA, that makes the model check its own output against the governing equation it can infer from the data. After each rollout, sparse regression on a library of candidate derivative and polynomial terms recovers a coefficient vector from the ground-truth window and from each predicted window; the model is penalized when these vectors disagree, and also when the predicted field fails to satisfy its own recovered equation. On twelve datasets and models up to 500 million parameters, the method reduces long-trajectory error by about 30% on average and improves out-of-distribution accuracy on a previously unseen Burgers dataset by 37.8%.

What carries the argument

The load-bearing object is the discovered coefficient vector $\Lambda$ from sparse regression on a candidate library. Given compressed data $U(\theta)$, the method builds a matrix $\Phi(\theta)$ whose columns are candidate terms—state values, spatial derivatives, low-degree polynomials, and a constant—and solves $\partial_t U(\theta) = \Phi(\theta) \Lambda$ under $\ell^0$ sparsity via thresholded ridge regression. PITA computes $\Lambda^*$ from a full-resolution ground-truth window and $\Lambda_i$ from each downsampled predicted window, then penalizes their difference; this temporal alignment is the new supervision mechanism. An alternating-direction optimizer alternates sparse-regression updates of $\Lambda$ with gradient updates of the neural operator, and uncertainty-based weights balance data, physics, and consistency losses.

What would settle it

Finetune a PDE foundation model with PITA on a trajectory whose governing equation includes a term deliberately left out of the candidate library, for example a cubic damping term when the library only contains quadratic terms. If the consistency loss then forces the recovered coefficients toward a $\Lambda^*$ that cannot represent the true law, and prediction error stays at or above the plain auto-regressive baseline, the reported gains rely on assuming the library contains the relevant terms rather than on the alignment mechanism itself.

Watch

Extended reading notes

Core claim

The central claim is that the shortcut problem in auto-regressive PDE prediction can be suppressed without any known-physics prior by aligning the physics discovered at different times on each trajectory. Because any segment of a trajectory is governed by the same law, the coefficient vector $\Lambda^*$ recovered by sparse regression from the first ten full-resolution ground-truth frames should equal the coefficient vector $\Lambda_i$ recovered from each downsampled predicted window. PITA enforces this equality through a consistency loss $\|\Lambda^* - \Lambda_i\|^2$ and, in parallel, a physics residual loss that requires the predicted field to satisfy the recovered equation. The authors report that this alignment reduces the average nRMSE of long-trajectory predictions by about 30.22%, and on the out-of-distribution viscous Burgers task it beats plain auto-regressive finetuning by 37.8% across model sizes.

Load-bearing premise

The whole physics signal rests on the assumption that sparse regression over a short, downsampled window—three temporal frames and a quarter of the spatial points—recovers a coefficient vector that is comparable to the vector from ten full-resolution ground-truth frames, and that the candidate library is normalized identically across windows.

Editorial extensions

If this is right

  • On long trajectories PITA reduces total nRMSE by about 30.22% on average across models and datasets, and the per-step error accumulation is visibly slowed in rolling-MSE plots.
  • The gains hold across model scales from 7M to 500M parameters and across diverse benchmarks, and the extra training cost does not grow with model size.
  • On out-of-distribution tasks, PITA improves accuracy even when the pretraining physics differs: about 5.93% average nRMSE reduction on compressible Navier-Stokes with different viscosities and about 37.8% on the unseen viscous Burgers equation.
  • PITA is data-efficient in downstream finetuning: from a pretrained checkpoint it reaches the same accuracy with roughly 500 samples that plain auto-regressive finetuning needs 1000 samples to reach.
  • The method remains competitive when the discovered physics is partly corrupted or the candidate library is incomplete; in ablations it degrades gracefully to around or above the auto-regressive baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to apply the same coefficient-vector alignment to non-PDE auto-regressive models, such as any dynamical system with a parametric generative law; if the law is identifiable from a short window, consistency between windows should reduce compounding errors there as well.
  • The reported gains all involve finetuning pretrained checkpoints, so an untested but direct corollary of the paper's design is using PITA during pretraining itself, where the same alignment signal might shape the foundation model before downstream adaptation.
  • The method's reliance on a hand-built candidate library could be relaxed by generative or genetic library search; the paper's own ablation results suggest performance degrades gracefully when the library is incomplete, so an adaptive library might preserve gains without specialist tuning.
  • Because the overhead of PDE discovery is constant and independent of model size, the relative cost of PITA shrinks as models grow, making it a plausible plug-in for much larger PDE foundation models than the 500M-parameter case tested.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PITA, a self-supervised framework that adds physics-informed temporal alignment to auto-regressive PDE foundation models. The method first discovers a sparse coefficient vector Λ* from the initial ground-truth frames of a trajectory via sparse regression, then, during training, discovers coefficient vectors Λ_i(θ) from predicted rollout windows and penalizes their deviation from Λ* with a consistency loss; a physics residual loss and uncertainty-based weighting are added, and an alternating direction scheme is used. Experiments on FNO, PDEBench, PDEArena, and CFDBench across DPOT, MPP, and FNO backbones report consistent nRMSE reductions, including roughly 30% average improvement on long trajectories and 37.8% on an out-of-distribution Burgers task.

Significance. If the mechanism works as claimed, PITA is a practically valuable plug-in for improving long-horizon accuracy and OOD robustness of autoregressive PDE foundation models, requiring no known PDE form. The paper's strengths are its broad empirical coverage (multiple foundation models, model sizes, and datasets), component-wise ablations, robustness and noise experiments, and a computational-cost analysis. The central claim, however, rests on the consistency loss being able to influence the network parameters, and the manuscript does not specify how gradients flow through the sparse-regression step; it also does not justify the comparability of coefficient vectors obtained at different resolutions and sampling densities. These are load-bearing gaps rather than presentation issues.

major comments (3)
  1. [Sec. 3.5, Eqs. (10)-(11)] As written, the alternating-direction procedure makes LCon unable to influence θ. In Eq. (10), Λ^{k+1} is computed from the frozen θ^k, and in Eq. (11) θ is updated with that Λ^{k+1} held fixed. Since the only θ-dependence of LCon in Eq. (8) enters through Λ_i(θ), the consistency term is constant during the θ update unless Λ_i(θ) is differentiated through Algorithm 1. Algorithm 1 uses hard thresholding and a pseudoinverse, with no stated differentiation rule, unrolling, or relaxation. Table 3, Task 1, reports that LData+LCon reduces nRMSE from 0.00301/0.00437 to 0.00165/0.00249, which is impossible if the term has zero gradient. Please specify the actual gradient path (e.g., straight-through estimator, soft-thresholded surrogate, implicit differentiation, or unrolled sparse regression) and update the algorithm description accordingly.
  2. [Sec. 3.3 and Sec. 3.4, Eq. (8)] The consistency loss compares Λ* obtained from the first ten full-resolution ground-truth frames with Λ_i(θ) obtained from predicted windows that are downsampled to TC=3 temporal frames and one-quarter of the spatial points. Sparse-regression coefficients are not invariant to the finite-difference stencil spacing used for derivative approximation, nor to the column scaling of the library Φ(θ). The manuscript does not state any column normalization or resampling correction. If the columns of Φ have different norms in the ground-truth and predicted-window constructions, ∥Λ*−Λ_i(θ)∥² is a scale-sensitive quantity and may not measure physical disagreement. Please specify the normalization and justify comparability, or the consistency target is not well-defined.
  3. [Sec. 3.4, Eq. (7)] The physics loss LPhy is written with Λ_i, which appears to be the sparse-regression solution for the same predicted window from Eq. (4). In that case the residual ∥∂tU_i−Φ_iΛ_i∥² is already minimized by Algorithm 1 and is approximately zero, so its role in Eq. (11) is unclear. If Λ_i is instead a free variable in Eq. (7), the alternating procedure in Eqs. (10)-(11) does not state how it is updated in the θ-step. Please clarify the status of Λ_i in LPhy and how gradients flow through it; this is needed to interpret the Task 1 ablation attribution between LPhy and LCon.
minor comments (5)
  1. [Table 1 and Appendix E.12] The main results table reports no error bars or standard deviations, whereas Appendix E.12 provides them only for long-trajectory datasets; please state in the main text or table caption that variance information is deferred to the appendix, or add representative error bars.
  2. [Sec. 4.4, Task 5] The text says "One-Order" removes only second-order derivatives, but the table and surrounding discussion suggest that first-order derivative terms are present in that configuration; please clarify the exact library composition for each row of Task 5.
  3. [Appendix F.3 and F.4] There are typos in the appendix captions: "visulize" (F.3), "naive-stokes" (F.4), and "PDE-Bench CNS-(M,η)" is inconsistent with the "PDEBench-CNS" naming used elsewhere; these should be corrected.
  4. [Eqs. (4) and (7)] The scalar α appears both in the sparse-regression objective Eq. (4) and in LPhy Eq. (7); please state explicitly whether these are the same hyperparameter and how α is set in the experiments.
  5. [Fig. 2 and Sec. 3.3] The diagonal downsampling matrix D is mentioned in the figure and in Sec. 3.3 but is not used in any subsequent equation; either remove it or define its role formally in the compressed-data notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PITA's physics targets are self-supervised discoveries from the same trajectory, not fitted values renamed as predictions.

full rationale

I examined the claimed derivation chain for circular reduction. The central mechanism is that sparse regression discovers a coefficient vector Lambda* from the ground-truth initial frames of a trajectory (Sec. 3.3, Eq. (4), Algorithm 1), and the same sparse-regression procedure produces Lambda_i(theta) from each predicted rollout window. The consistency loss in Eq. (8), ||Lambda* - Lambda_i(theta)||^2, then supervises the predictor to produce rollouts whose discovered dynamics match the dynamics discovered from the observed prefix. This is a self-supervised alignment objective: Lambda* is an auxiliary target obtained from observation data, not a quantity that the model is asked to predict from its own outputs, and the final evaluation is against held-out ground-truth frames (Eq. (12)). The physics loss in Eq. (7) is a standard residual-style regularizer, and the data loss in Eq. (6) is an independent supervised term. None of the losses reduces to the reported nRMSE numbers by construction; the reported improvements are empirical outcomes of training. The manuscript does not rely on a load-bearing self-citation chain, and no uniqueness theorem or ansatz is imported from the authors' prior work. The only notable issue in the paper is a potential implementation inconsistency in the alternating-direction optimization (Eqs. (10)-(11)) regarding whether gradient flows through the sparse-regression mapping Lambda_i(theta); that is a correctness or reproducibility concern, not a circularity concern, because it does not make the prediction equivalent to its input by definition. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two kinds of input: hyperparameters chosen by hand (downsampling l=4, temporal window TC=3, library content, sparsity thresholds) and domain assumptions about the validity of sparse PDE discovery from short, downsampled windows. The method introduces no new physical entities.

free parameters (4)
  • Spatial downsampling factor l = 4
    Chosen in ablation Task 2 on FNO-NS-1e-3 (l=4 gave nRMSE 0.00157/0.00223 vs l=1 0.00169/0.00238), then applied to all experiments.
  • Temporal retention window TC = 3
    Task 3 ablation found TC=7 best (0.00155/0.00221) but TC=3 (0.00157/0.00223) chosen to reduce cost; this window determines the data used for every PDE discovery step.
  • Sparsity threshold beta and L0 penalty alpha = not stated
    Algorithm 1 depends on beta and alpha but no numeric values or selection procedure are reported.
  • Candidate library composition = polynomials, first/second derivatives, products
    The set of candidate terms is a modeling choice that determines which PDEs can be discovered; the paper tests only removal of derivative groups, not alternative libraries.
assumptions (4)
  • domain assumption The governing PDE is exactly a sparse linear combination of the candidate library terms (Eq 3).
    Underlies all discovery steps; if the true dynamics include a term missing from the library, Lambda* and Lambda_i are biased in the same direction and the alignment target is wrong.
  • domain assumption The same functional form of the PDE governs the initial observation window and the later predicted windows within each trajectory.
    This is the premise of temporal alignment, stated in Sec 3.1 and used by the consistency loss; it holds for the benchmark PDEs but is an assumption for arbitrary data.
  • domain assumption Finite-difference derivatives computed from one quarter of the spatial points and three temporal frames are accurate enough for sparse regression to identify stable coefficients.
    The paper provides no formal error analysis; noise sensitivity of finite differences is acknowledged in Sec 4.5 as a limitation.
  • domain assumption The alternating-direction update in Eq (11) gives a working gradient signal even though Lambda is the solution of a non-differentiable sparse regression.
    The paper does not explain how, or whether, gradients flow through Eq (10); without this, the consistency loss would not influence theta.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models." pith.science (2026). https://pith.science/paper/554S4HIC

@misc{pith2026250510930,
  author       = {Pith},
  title        = {Pith review of: Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/554S4HIC}},
  note         = {Machine review of arXiv:2505.10930}
}
read the original abstract

Auto-regressive partial differential equation (PDE) foundation models have shown great potential in handling time-dependent data. However, these models suffer from the shortcut problem deeply rooted in auto-regressive prediction, causing error accumulation. The challenge becomes particularly evident for out-of-distribution data, as the pretraining performance may approach random model initialization for downstream tasks with long-term dynamics. To deal with this problem, we propose physics-informed temporal alignment (PITA), a self-supervised learning framework inspired by inverse problem solving. Specifically, PITA aligns the physical dynamics discovered at different time steps on each given PDE trajectory by integrating physics-informed constraints into the self-supervision signal. The alignment is derived from observation data without relying on known physics priors, indicating strong generalization ability to the out-of-distribution data. Extensive experiments show that PITA significantly enhances the accuracy and robustness of existing foundation models on diverse time-dependent PDE data. The code is available at https://github.com/SCAILab-USTC/PITA.

Figures

Figures reproduced from arXiv: 2505.10930 by the authors.

Figure 1
Figure 1. Insight of the proposed framework. Existing (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Work flow of PITA. The proposed framework integrates auto-regressive prediction and PDE discovery with [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of scalability for different model sizes. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Rolling step MSE on long trajectory datasets. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Illustration of Error Propagation in Auto-regressive Prediction. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Intrinsic connection between error accumulation and shortcut learning. On the left, we present the shortcut issue on [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Results of PITA and Auto-Regressive Methods on Long Trajectory Datasets. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Results of scaling experiments for different dataset sizes. In the first row, we present results for PDEBench-SWE [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Results of Computational Costs Between PITA and Auto-regressive Method. In the left, we present the CPU [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Mean Value and Standard Deviation on Datasets with Long Trajectory. [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 13 canonical work pages

  1. [1]

    This decline in performance can be attributed to the shortcut problem that occurs during training. 22 Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models Table 11: Comparisons of PITA and Auto-regressive Strategies Across Various Models and Datasets (Tar = 10). FNO-NS-ν PDEBench PDEBench CNS-(η,ζ) PDEArena CFDBench 1e-5 1e-4 1e-3...

    work page arXiv 1997

  2. [2]

    The total length of the testing dataset consists of 20 steps

    =−∇p+∇·µ [ ∇u+ (∇u)⊤] , ∇·(ρu) = 0, (18) whereρ is the density andµ is the dynamic viscosity,u is the velocity field, andp is the pressure. The total length of the testing dataset consists of 20 steps. The task is to predict the velocity fieldu(x,t)given the initial 10 time steps. C.7. Magnet-Viscous Burgers To evaluate whether PITA can effectively handle...

    work page 2024

  3. [8]

    Gnot: A general neu- ral operator transformer for operator learning

    Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y ., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neu- ral operator transformer for operator learning. InInter- national Conference on Machine Learning, pp. 12556– 12569. PMLR, 2023a. Hao, Z., Yao, J., Su, C., Su, H., Wang, Z., Lu, F., Xia, Z., Zhang, Y ., Liu, S., Lu, L., et al. Pinnacle: A comprehe...

    arXiv

  4. [9]

    L., Mobahi, H., Fel, T., and Mozer, M

    Hermann, K. L., Mobahi, H., Fel, T., and Mozer, M. C. On the foundations of shortcut learning.arXiv preprint arXiv:2310.16228,

    arXiv

  5. [11]

    C., Kumar, A., and Pathak, D

    Li, A. C., Kumar, A., and Pathak, D. Generative classifiers avoid shortcut solutions. InICML 2024 Workshop on Structured Probabilistic Inference & Generative Model- ing, 2024a. Li, L., Khodak, M., Balcan, M.-F., and Talwalkar, A. Geometry-aware gradient algorithms for neural architec- ture search.arXiv preprint arXiv:2004.07802, 2020a. Li, X., Li, L., Yue...

    arXiv 2024

  6. [13]

    Scalable trans- former for pde surrogate modeling.Advances in Neural Information Processing Systems, 36, 2024b

    Li, Z., Shu, D., and Barati Farimani, A. Scalable trans- former for pde surrogate modeling.Advances in Neural Information Processing Systems, 36, 2024b. Li, Z., Shu, D., and Barati Farimani, A. Scalable trans- former for pde surrogate modeling.Advances in Neural Information Processing Systems, 36, 2024c. Liebel, L. and K ¨orner, M. Auxiliary tasks in mult...

    arXiv

  7. [14]

    Model sparsity can simplify machine unlearning

    Liu, J., Ram, P., Yao, Y ., Liu, G., Liu, Y ., SHARMA, P., Liu, S., et al. Model sparsity can simplify machine unlearning. Advances in Neural Information Processing Systems, 36, 2024a. Liu, Q. and Ma, J. Foundation models for geophysics: Re- views and perspectives.arXiv preprint arXiv:2406.03163,

    arXiv

  8. [15]

    Prose-fd: A multimodal PDE foundation model for learning multiple operators for forecasting fluid dynamics

    Liu, Y ., Sun, J., He, X., Pinney, G., Zhang, Z., and Schaeffer, H. Prose-fd: A multimodal PDE foundation model for learning multiple operators for forecasting fluid dynamics. arXiv preprint arXiv:2409.09811, 2024b. Long, Z., Lu, Y ., Ma, X., and Dong, B. PDE-net: Learning PDEs from data. InInternational Conference on Machine Learning, pp. 3208–3216. PMLR,

    arXiv

Show all 41 references
  1. [16]

    Lu, L., Jin, P., and Karniadakis, G. E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of opera- tors.arXiv preprint arXiv:1910.03193,

    1910 arXiv

  2. [17]

    Cfdbench: A comprehen- sive benchmark for machine learning methods in fluid dynamics.arXiv preprint arXiv:2310.05963,

    Luo, Y ., Chen, Y ., and Zhang, Z. Cfdbench: A comprehen- sive benchmark for machine learning methods in fluid dynamics.arXiv preprint arXiv:2310.05963,

    arXiv

  3. [18]

    R.-S., Parker, L

    12 Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models McCabe, M., Blancard, B. R.-S., Parker, L. H., Ohana, R., Cranmer, M., Bietti, A., Eickenberg, M., Golkar, S., Krawezik, G., Lanusse, F., et al. Multiple physics pre- training for physical surroga...

    arXiv

  4. [19]

    On causal and anticausal learning.arXiv preprint arXiv:1206.6471,

    Sch¨olkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K., and Mooij, J. On causal and anticausal learning.arXiv preprint arXiv:1206.6471,

    arXiv

  5. [20]

    J., Nicoli- cioiu, A

    Scimeca, L., Rubinstein, A., Teney, D., Oh, S. J., Nicoli- cioiu, A. M., and Bengio, Y . Mitigating shortcut learn- ing with diffusion counterfactuals and diverse ensembles. arXiv preprint arXiv:2311.16176,

    arXiv

  6. [21]

    Ups: Efficiently building foundation models for PDE solving via cross- modal adaptation

    Shen, J., Marwah, T., and Talwalkar, A. Ups: Efficiently building foundation models for PDE solving via cross- modal adaptation. InICML 2024 AI for Science Work- shop,

    2024

  7. [22]

    Shortcut learning in in-context learning: A survey.arXiv preprint arXiv:2411.02018, 2024a

    Song, R., Li, Y ., Giunchiglia, F., and Xu, H. Shortcut learning in in-context learning: A survey.arXiv preprint arXiv:2411.02018, 2024a. Song, Z., Yuan, J., and Yang, H. Fmint: Bridging human designed and data pretrained models for differential equa- tion foundation model.arX...

    arXiv

  8. [23]

    Towards a foundation model for partial differential equation: Multi- operator learning and extrapolation.arXiv preprint arXiv:2404.12355,

    Sun, J., Liu, Y ., Zhang, Z., and Schaeffer, H. Towards a foundation model for partial differential equation: Multi- operator learning and extrapolation.arXiv preprint arXiv:2404.12355,

    arXiv

  9. [24]

    Large language models can be lazy learners: Analyze shortcuts in in- context learning.arXiv preprint arXiv:2305.17256,

    13 Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models Tang, R., Kong, D., Huang, L., and Xue, H. Large language models can be lazy learners: Analyze shortcuts in in- context learning.arXiv preprint arXiv:2305.17256,

    arXiv

  10. [27]

    Dl-PDE: Deep-learning based data-driven discovery of partial differential equa- tions from discrete and noisy data.arXiv preprint arXiv:1908.04463,

    Xu, H., Chang, H., and Zhang, D. Dl-PDE: Deep-learning based data-driven discovery of partial differential equa- tions from discrete and noisy data.arXiv preprint arXiv:1908.04463,

    1908 arXiv

  11. [28]

    PDEformer: Towards a foundation model for one- dimensional partial differential equations.arXiv preprint arXiv:2402.12652, 2024a

    Ye, Z., Huang, X., Chen, L., Liu, H., Wang, Z., and Dong, B. PDEformer: Towards a foundation model for one- dimensional partial differential equations.arXiv preprint arXiv:2402.12652, 2024a. Ye, Z., Huang, X., Chen, L., Liu, Z., Wu, B., Liu, H., Wang, Z., and Dong, B. PDEforme...

    arXiv

  12. [29]

    Do llms overcome shortcut learning? an evaluation of short- cut challenges in large language models.arXiv preprint arXiv:2410.13343,

    Yuan, Y ., Zhao, L., Zhang, K., Zheng, G., and Liu, Q. Do llms overcome shortcut learning? an evaluation of short- cut challenges in large language models.arXiv preprint arXiv:2410.13343,

    arXiv

  13. [30]

    To- wards faithful explanations: Boosting rationalization with shortcuts discovery.arXiv preprint arXiv:2403.07955,

    Yue, L., Liu, Q., Du, Y ., Wang, L., Gao, W., and An, Y . To- wards faithful explanations: Boosting rationalization with shortcuts discovery.arXiv preprint arXiv:2403.07955,

    arXiv

  14. [32]

    Zhao, Z., Ding, X., and Prakash, B. A. Pinnsformer: A transformer-based framework for physics-informed neu- ral networks.arXiv preprint arXiv:2307.11833,

    arXiv

  15. [33]

    and Farimani, A

    Zhou, A. and Farimani, A. B. Masked autoencoders are PDE learners.arXiv preprint arXiv:2403.17728,

    arXiv

  16. [34]

    Zhou, A., Lorsung, C., Hemmasian, A., and Farimani, A. B. Strategies for pretraining neural operators.arXiv preprint arXiv:2406.08473, 2024a. 14 Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models Zhou, H., Ma, Y ., Wu, H., Wang, H., and Long, M. Uni-...

    arXiv

  17. [38]

    The sequentially thresholded least squares (STLS) method (Budi ˇsi´c et al.,

    method achieves sparsity through ℓ1 regularization, its convex relaxation property tends to randomly select collinear features when high correlations exist between columns of the data matrix (as demonstrated in the typical PDE identification scenario in reference (Knowles & Re...

    2014

  18. [39]

    STRidge time

    improves sparse identification through a recursive thresholding mechanism, but its iterative process based on ordinary least squares lacks regularization mechanisms. In the presence of ill-conditioned matrices or highly correlated features (as shown in the experiments of refer...

    2012

  19. [40]

    As shown in Figure 9 (right), the overhead due to PDE discovery (detailed in Section 3.3) is constant and independent of model size. During inference, PITA matches the baseline’s efficiency—because discovery and alignment occur only during training—and consistently outperforms...

    2012

  20. [41]

    When randomly retaining 50% of the library terms, the test loss on PDEBench-SWE increases from a baseline of 0.00137 (achieved with full-term libraries) to 0.00213. Notably, this performance degradation diminishes when preserving 80% of library terms, where the test loss stabi...

  21. [1973]

    Adaptive fourier neural operators: Efficient token mixers for transformers.arXiv preprint arXiv:2111.13587,

    Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A., and Catanzaro, B. Adaptive fourier neural operators: Efficient token mixers for transformers.arXiv preprint arXiv:2111.13587,

    arXiv

  22. [1997]

    and Efros, A

    Torralba, A. and Efros, A. A. Unbiased look at dataset bias. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Wan, H., Wang, Q., Mi, Y ., and Sun, H. Pimrl: Physics- informed multi-scale recurrent learning for spatiotempo- ral prediction.arXiv p...

  23. [2006]

    Towards interpreting and mitigating shortcut learning behavior of nlu models

    Du, M., Manjunatha, V ., Jain, R., Deshpande, R., Dernon- court, F., Gu, J., Sun, T., and Hu, X. Towards interpreting and mitigating shortcut learning behavior of nlu models. arXiv preprint arXiv:2103.06922,

    arXiv

  24. [2008]

    Autoregressive action sequence learning for robotic manipulation.arXiv preprint arXiv:2410.03132, 2024a

    Zhang, X., Liu, Y ., Chang, H., Schramm, L., and Boularias, A. Autoregressive action sequence learning for robotic manipulation.arXiv preprint arXiv:2410.03132, 2024a. Zhang, Z., Moya, C., Lu, L., Lin, G., and Schaeffer, H. Deeponet as a multi-operator extrapolation model: Dis...

    arXiv

  25. [2012]

    Timedit: General- purpose diffusion transformers for time series foundation model.arXiv preprint arXiv:2409.02322,

    Cao, D., Ye, W., Zhang, Y ., and Liu, Y . Timedit: General- purpose diffusion transformers for time series foundation model.arXiv preprint arXiv:2409.02322,

    arXiv

  26. [2017]

    A., and Brendel, W

    Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wich- mann, F. A., and Brendel, W. Imagenet-trained cnns are biased towards texture; increasing shape bias improves ac- curacy and robustness.arXiv preprint arXiv:1811.12231,

    arXiv

  27. [2019]

    Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895, 2020b

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895, 2020b. Li, Z., Meidani, K., and Farimani, A. B. Transformer for parti...

    2010 arXiv

  28. [2020]

    N., and Brunton, S

    Zolman, N., Fasel, U., Kutz, J. N., and Brunton, S. L. Sindy- rl: Interpretable and efficient model-based reinforcement learning.arXiv preprint arXiv:2403.09110,

    arXiv

  29. [2021]

    Gupta, J. K. and Brandstetter, J. Towards multi- spatiotemporal-scale generalized PDE modeling.arXiv preprint arXiv:2209.15616,

    arXiv

  30. [2022]

    Physics-enhanced machine learning: A posi- tion paper for dynamical systems investigations.arXiv preprint arXiv:2405.05987,

    Cicirello, A. Physics-enhanced machine learning: A posi- tion paper for dynamical systems investigations.arXiv preprint arXiv:2405.05987,

    arXiv

  31. [2023]

    and Ruder, S

    Howard, J. and Ruder, S. Universal language model fine-tuning for text classification.arXiv preprint arXiv:1801.06146,

    arXiv

  32. [2024]

    Omniarch: Building the foun- dation model for scientific computing.arXiv preprint arXiv:2402.16014,

    Chen, T., Zhou, H., Li, Y ., Wang, H., Gao, C., Shi, R., Zhang, S., and Li, J. Omniarch: Building the foun- dation model for scientific computing.arXiv preprint arXiv:2402.16014,

    arXiv

  33. [2025]

    and Wu, Z

    Wang, Z. and Wu, Z. Foundation model for chemical process modeling: Meta-learning with physics-informed adaptation.arXiv preprint arXiv:2405.11752,

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.