S$^2$GPT-PINNs: Sparse and Small models for PDEs

Shawn Koohy; Yajie Ji; Yanlai Chen

REVIEW 4 major objections 4 minor 35 references

S$^2$GPT-PINNs: Sparse and Small models for PDEs

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read S2GPT-PINN claims that about $2n-1$ collocation points suffice to match GPT-PINN accuracy, with two to three times less online compute.

desk verdict A solid extension of GPT-PINN with sparse collocation; the accuracy claim is supported by experiments, but the greedy error indicator lacks a proof of fidelity to the full residual. read the letter →

arxiv 2506.15687 v1 pith:UIRY6GVM submitted 2025-05-25 cs.LG stat.ML

Yajie Ji , Yanlai Chen , Shawn Koohy This is my paper · ORCID

classification cs.LGstat.ML MSC 65M7068T07

keywords parametricPDEsphysics-informedneuralnetworksreducedbasismethodempiricalinterpolationsparsecollocationmeta-learningsurrogatemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes S2GPT-PINN, a sparse and small surrogate for parametric partial differential equations, and claims it is as accurate as the dense GPT-PINN while being two to three times faster online. The key move is to evaluate the physics-informed loss on a hyper-reduced collocation set of about $2n-1$ points, where $n$ is the number of hidden neurons in the small network, instead of on the full discretization grid. This set combines interpolation points chosen by a generalized empirical interpolation method with residual points chosen by an empirical interpolation method, and the sparse loss is used both to train the coefficients and to drive the greedy selection of new basis parameters. If the claim holds, parametric surrogate models can be built and queried with orders of magnitude fewer collocation points and much smaller online compute, which matters for real-time and resource-constrained applications such as control, optimization, and uncertainty quantification.

What carries the argument

The load-bearing object is the sparse collocation set $X_m=X_s^n \cup X_r^{n-1}$ of size $2n-1$, together with the reduced loss $L_{\mathrm{S2GPT}} = L_{\mathrm{int}}(X_m, c; \mu) + L_i(c;\mu)$ defined on it. $X_s^n$ is built by the Generalized Empirical Interpolation Method (GEIM), which picks, for each new orthonormalized basis function, the point where its magnitude is largest; $X_r^{n-1}$ is built by the Empirical Interpolation Method (EIM), which picks the point where the PDE residual of the new basis is largest. The same sparse loss serves as the error indicator $\Delta_{\mathrm{S2GPT}}$ that drives the greedy parameter selection in the offline stage, and the online training updates the coefficients $c$ by gradient descent on this sparse loss alone. A second component is the initialization of the network weights using the interpolation residual coefficients $\alpha$ recorded when each new basis function is orthogonalized, which the paper reports gives faster and more stable convergence.

What would settle it

On a parametric problem with a shock or boundary layer whose location depends on the parameter, compute the sparse error indicator $\Delta_{\mathrm{S2GPT}}$ and the full-grid residual for a dense set of unseen parameters; if the parameter that maximizes the sparse indicator is not among the parameters with the largest full-grid residual, or if the coefficients trained on $X_m$ produce full-grid errors clearly larger than GPT-PINN's, the representative-quality assumption fails and the accuracy claim collapses.

Watch

Extended reading notes

Core claim

The central discovery is that GPT-PINN's dense collocation grid is unnecessary: a set $X_m=X_s^n \cup X_r^{n-1}$ of cardinality $2n-1$, formed by the interpolation points of the orthonormalized basis functions and the residual points of the new basis, is enough to reproduce the same accuracy. The paper shows that the worst-case training loss of S2GPT-PINN converges exponentially as neurons are added, and that the loss evaluated on the sparse set tracks the loss on the full grid, indicating the sparsification does not distort the optimization landscape. On four parametric equations (Klein-Gordon, Allen-Cahn, Burgers, and Helmholtz), the pointwise error patterns of S2GPT-PINN and GPT-PINN are very similar, while the online compute time is reduced by a further factor of two to three and the number of collocation points drops from tens of thousands to tens.

Load-bearing premise

The load-bearing premise is that the sparse collocation set $X_m$ yields a loss landscape faithful to the full-grid residual for every parameter in the training domain; the paper itself flags in its conclusion that stability for more complex or chaotic systems and a full theoretical analysis remain open.

Editorial extensions

If this is right

The online cost of training and evaluating the parametric surrogate becomes proportional to the number of basis neurons rather than the size of the full collocation grid, since only $2n-1$ points enter the loss.
The sparse error indicator $\Delta_{\mathrm{S2GPT}}$ is effective enough to drive the greedy basis construction, as evidenced by exponential convergence of the worst-case training loss over the parameter training set.
S2GPT-PINN inherits the reduced-basis interpretability of GPT-PINN: each hidden neuron is a pre-trained PINN for a greedily selected parameter, and the learned representation remains small.
On the four tested equation families, the method achieves the same accuracy as GPT-PINN with two to three times less online compute and with collocation sets orders of magnitude smaller than the full-order PINN discretization.
The interpolation-residual initialization makes online training converge in around 50 epochs with L-BFGS, so the savings include optimization time as well as point evaluations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper does not pursue is a rigorous a posteriori bound relating the sparse loss to the true PDE residual; if such a bound could be derived, the greedy selection would become certified rather than empirically validated.
The same sparsification idea could be applied to other physics-informed loss formulations, such as variational or energy-based losses, wherever a reduced basis and its residual spaces are available.
The factor of two in the sparse-set size ($2n-1$) mirrors the over-collocation ratio in reduced-basis theory, so testing larger constants (e.g., $4n$) on problems with interior layers or shocks would reveal how much safety margin the fixed factor provides.
Because the paper reports that the chosen parameters cluster near the boundary of the parameter domain, an interesting test is whether the sparse error indicator still selects informative parameters in higher-dimensional parameter spaces, where boundary effects dominate.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here is my take on arXiv:2506.15687. The genuinely new piece is the combination of GPT-PINN with R2ROC-style hyper-reduction: S2GPT-PINN builds a sparse collocation set from GEIM interpolation points and EIM residual points, and adds an initialization based on interpolation coefficients. That is a real extension, and the paper supports it with reproducible numerical evidence. Code is public, the four PDE experiments are consistent, and the full-grid loss check in Figure 3 is a good sanity check: on those problems, the sparse loss tracks the full residual. The reported speedups are honest - two to three times over GPT-PINN online, with orders of magnitude fewer collocation sites. The soft spot is exactly what the stress-test note names. The sparse loss is used both as the training objective and as the greedy error indicator in Algorithm 2, and no bound relates the sparse loss to the true residual. If that equivalence fails for a rougher problem, the greedy parameter selection could pick irrelevant parameters. The paper's own conclusion admits that stability for complex or chaotic systems requires further investigation, which is the right caveat. I do not see this as fatal - the empirical corroboration across four equation families is decent evidence for the heuristic - but the abstract's phrase 'mathematically rigorous greedy algorithm' overstates what is actually shown. Also, Section 4.2 contains duplicated text blocks that need cleaning. The citation pattern is fine; the paper builds explicitly on GPT-PINN, TGPT-PINN, and R2ROC, and does not overclaim novelty. The limitation is stated clearly. This paper deserves a serious referee: the method is practical, the experiments are reproducible, and the key question - when does the sparse loss faithfully represent the full residual - is a good topic for a referee to push on. I would accept it for peer review, and ask the authors to either provide a theoretical justification for the error indicator or soften the 'rigorous' claim, and to fix the duplication.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes S2GPT-PINN, a sparse and small surrogate for parametric PDEs. It builds on GPT-PINN's network-of-networks ansatz (Eq. 2.5), replacing the full collocation grid with a set X_m of M = 2n - 1 points constructed from GEIM interpolation points and EIM residual points during an offline greedy loop (Algorithm 2). Online, only the outer coefficients c are trained against the sparse physics-informed loss L_S2GPT (Section 3.2.1), and that same loss is used as the error indicator Delta_n (Eq. 3.1) for greedy parameter selection. In four numerical examples (Klein-Gordon, Allen-Cahn, Burgers, Helmholtz), the method is reported to match GPT-PINN accuracy while using 19-47 collocation points and about two to three times less online compute, and the sparse loss is shown to corroborate the full-grid loss in Figure 3.

Significance. If the reliability of the sparse loss is established, the contribution is significant: it reduces the online training cost of GPT-PINN from scaling with the full collocation set to scaling with the size of the reduced network, and the algorithm is simple to implement. The paper's strengths include a transparent algorithm, a reproducible code repository, and a useful empirical check in Figure 3, where full-grid losses are overlaid on the sparse-grid losses. The four test problems span different regimes (a wave-like equation, a reaction-diffusion equation, viscous Burgers, and a two-dimensional Helmholtz problem), and the reported speedups are consistent across them. However, because the central mechanism -- the equivalence of the sparse loss to the true residual -- is asserted rather than proved or quantitatively validated, the contribution is currently a promising empirical method with a load-bearing open question.

major comments (4)

[§3.2.1, Eq. (3.1); Algorithm 2, steps 5–6] The error indicator Delta_n is defined as the S2GPT-PINN training loss on X_m, and this same quantity drives both greedy parameter selection (Eq. 3.3) and the convergence report (Figure 3). The manuscript gives no bound of the form |R_full(u_n(mu)) - R_sparse(u_n(mu))| <= E(n) and no effectivity estimate relating Delta_n to the true residual. Because the loss being optimized is also the criterion being maximized, the greedy algorithm could select parameters where the sparse loss is hard to minimize rather than where the full-order approximation is poor; a single such parameter would invalidate the reported accuracy transfer. The only evidence offered is the visual agreement in Figure 3. Please add either a proof in the spirit of the R2ROC analysis cited in [6] or, at minimum, a quantitative effectivity study over held-out parameters for each n, reporting the ratio of Delta_n to the full-grid loss and to the actual L2 error on the four test problems. This is load-bearing because the method's central claim is that X_m faithfully represents the full-grid PINN loss.
[§1 (Abstract) and §5 (Conclusion)] The abstract describes a 'mathematically rigorous greedy algorithm', but Section 5 states that 'theoretical analysis' is ongoing and that stability for more complex or chaotic systems 'still requires further investigation'. The rigor of the greedy algorithm rests entirely on the reliability of Delta_n (Eq. 3.1), and no theorem in the paper establishes that reliability. Moreover, the choice M = 2n - 1 is imported from the R2ROC framework [6], whose analysis concerns linear reduced-basis projections and residual-based estimators; the transfer to the nonlinear outer-network training used here is not argued. Either supply the missing analysis, or revise the abstract and Section 3.2.1 so that 'rigorous' refers only to the GEIM/EIM construction steps and explicitly not to the greedy selection criterion.
[§3.2.3] The proposed initialization step is underspecified. The text says that '[alpha, 0]' should be used as the initial values for all S2GPT-PINNs with a (2,n,1) architecture, where alpha is the interpolation residual of the newly added n-th basis. But in Algorithm 2, the parameter sweep at step 5 trains the network at many candidate mu before mu_n is selected, so the target solution at those mu is not available and the corresponding alpha cannot be computed. Please clarify how the initial guess is formed for each candidate mu during the sweep, or state explicitly that this is a heuristic warm-start taken from the most recent basis addition. As written, the procedure is not reproducible.
[§4.2, Table 2 and Figure 6] The runtime claims are presented ambiguously. Table 2 is captioned 'Online compute time, normalized by that of the corresponding PINN solver', while Figure 6's caption refers to 'The testing time ... on a single-core CPU', but the text in §4.2 calls the right panel 'the cumulative runtime'. These are different quantities: the online coefficient-training time for a new parameter and the cumulative offline-plus-online cost. Please define precisely what is measured in Table 2 and in each panel of Figure 6, and report the number of L-BFGS iterations or epochs used for the outer network in both methods. Without this, the speedup factors in Table 2 cannot be independently assessed.

minor comments (4)

[§4.2] Section 4.2 contains a large verbatim duplication: the paragraph beginning 'examples in Table 1...' appears twice, and Figures 2-6 appear with repeated captions. Please remove the duplicated blocks and ensure each figure appears exactly once.
[§3.2.1 and §4] There are typos that should be corrected: 'folows' in Section 3.2.1 and 'Helmhotlz' in the opening paragraph of Section 4.
[§2, around Eq. (2.3)] The text says the loss incorporates 'initial conditions (2.1b), and boundary conditions (2.1c)', but Eq. (2.1b) is the boundary condition and Eq. (2.1c) is the initial condition. Please fix the cross-reference.
[Figure 3] The claim that 'exponential convergence is noticeable' would be more persuasive if the figure included fitted convergence rates or numerical values; as it stands, 'noticeable' is a visual judgment.

Circularity Check

1 steps flagged · score 2.0 of 10

Self-referential error indicator Δ equals the sparse training loss, so the greedy parameter selection is not independently certified; full-grid and pointwise validations keep the central accuracy claim non-circular.

self definitional [Section 3.2.1, Eq. (3.1); Algorithm 2, steps 5–6]
"This loss is also adopted to evaluate the accuracy of the reduced and sparsified network NNs(2,n,1) for approximating the PDE solution at µ, ∆_n^S2GPT(c(µ)) := L_PINN^S2GPT(c(µ)). ... µ_n = argmax_{µ∈Ξ_train\{µ_i,i=1,···,n−1}} ∆_{n−1}^S2GPT(µ)."

The greedy parameter selection that builds the reduced basis is driven by ∆, but ∆ is defined as the same sparse loss L that online training minimizes via Eq. (3.2). Thus the 'error indicator' is the objective being optimized, not an independent estimate of the true residual; no effectivity bound relating the sparse loss to the full-grid residual is stated. This makes the greedy selection self-referential. The paper does separately validate with full-grid losses and pointwise L2 errors, so the main accuracy claim does not reduce to the loss by construction; the circularity is partial and limited to the 'rigorous greedy' justification.

full rationale

The core construction — sparse over-collocation set X_m from GEIM/EIM, coefficient training by minimizing the sparse loss, and greedy parameter selection — applies prior R2ROC [6] and GPT-PINN [8] ideas. The reported speedup and accuracy are checked against GPT-PINN and PINN on full grids and on unseen test parameters (Figures 3–6), so the central claim is not forced by definition. The one self-referential element is Eq. (3.1): the error indicator ∆_n^S2GPT used in Algorithm 2 to choose new parameters is defined as the same sparse loss that training minimizes, with no stated effectivity bound relative to the full residual; this weakens the 'mathematically rigorous' characterization but does not reduce the final empirical accuracy claim to the input. The paper also concedes in Section 5 that stability for more complex or chaotic systems requires further investigation, which is a limitation rather than circularity. Several cited foundations are the authors' own prior work ([6], [7], [8]), but they are used as baselines and building blocks with independent numerical comparison here, so they are not load-bearing in a circular sense.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method borrows its core machinery from reduced basis theory and empirical interpolation without proving that these guarantees transfer to the network training setting; the main uncharged input is the assumption that enforcing the PDE at about 2n-1 carefully chosen points controls the full residual.

free parameters (2)

Terminal number of basis neurons N = KG:12, AC:12, B:10, H:24
Selected by the authors (inherited from GPT-PINN settings) as the stopping point of the greedy algorithm; it controls the size of the small model and the number of sparse collocation points m=2N-1.
Full PINN training configuration = Hidden sizes (1801, 50049, 1341, 1341), L-BFGS learning rate 0.1, 50 epochs
Hand-set hyperparameters from GPT-PINN [8]; the accuracy of the pretrained basis PINNs depends on them, but they are not fitted to the S2GPT loss.

assumptions (4)

domain assumption The parametric PDE (2.1) is well-posed and pre-trained PINNs at sampled parameters are sufficiently accurate to act as high-fidelity basis functions.
Section 2.1.2 and Algorithm 2 treat full PINN solutions as trusted snapshots; no certification of PINN accuracy is provided.
domain assumption Enforcing the residual at the sparse over-collocation set controls the full residual, transferring the R2ROC theory to GPT-PINN.
Section 3.2.1 uses the sparse loss as error indicator without proof; the paper cites R2ROC [6] but does not establish the transfer.
standard math GEIM and EIM produce stable interpolatory approximations for the solution and residual manifolds.
Invoked in Algorithms 3 and 4; relies on stability results of [3,27,28].
domain assumption The initial condition is either mu-independent or smooth enough in mu to admit a finite EIM expansion used in L_i.
Remark 3.1 derives L_i under this assumption; for mu-dependent initial conditions the approximation is argued informally.

how reviews work

0 comments

Cite this review

Pith. "Pith review of S$^2$GPT-PINNs: Sparse and Small models for PDEs." pith.science (2026). https://pith.science/paper/UIRY6GVM

@misc{pith2026250615687,
  author       = {Pith},
  title        = {Pith review of: S$^2$GPT-PINNs: Sparse and Small models for PDEs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UIRY6GVM}},
  note         = {Machine review of arXiv:2506.15687}
}

abstract

We propose S$^2$GPT-PINN, a sparse and small model for solving parametric partial differential equations (PDEs). Similar to Small Language Models (SLMs), S$^2$GPT-PINN is tailored to domain-specific (families of) PDEs and characterized by its compact architecture and minimal computational power. Leveraging a small amount of extremely high quality data via a mathematically rigorous greedy algorithm that is enabled by the large full-order models, S$^2$GPT-PINN relies on orders of magnitude less parameters than PINNs to achieve extremely high efficiency via two levels of customizations. The first is knowledge distillation via task-specific activation functions that are transferred from Pre-Trained PINNs. The second is a judicious down-sampling when calculating the physics-informed loss of the network compressing the number of data sites by orders of magnitude to the size of the small model.

Figures

Figures reproduced from arXiv: 2506.15687 by the authors.

**Figure 1.** Schematics of S2GPT-PINN. Inspired by R2ROC and given the collocation nature of PINN and GPT-PINN, we propose in this work to further enhance GPT-PINN by a sparsification strategy and a novel reduced loss function detailed in Section 3.2.1. This loss function is based on the sparse collocation set and serves as an error indicator in the greedy algorithm to build the meta network. The effectiveness of the reduced lo… view at source ↗

**Figure 2.** The S2GPT-PINN solution for the four examples with (α, β, γ) = (−1.22, 0.22, 0.56), (λ, ϵ) = (0.00091, 2.6), ν = 0.8212, (a1, a2) = (1.95, 3.95) [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** S2GPT-PINN is as accurate as GPT-PINN: Worst case training losses for the S2GPTPINN (evaluated on the sparse and full grid) and GPT-PINN. Shown in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: S2GPT-PINN (left) and GPT-PINN (right) pointwise errors and L2 error for a randomly selected test case. Parameter values for the four examples are (α, β, γ) = (−1.22, 0.22, 0.56), (λ, ϵ) = (0.00091, 2.6), ν = 0.8212, (a1, a2) = (1.95, 3.95). 13 [PITH_FULL_IMAGE:figure…

**Figure 5.** Figure 5: S2GPT-PINN is much sparser than GPT-PINN. Left: Neuron parameters generated by the GPT-PINN (green) and S2GPT-PINN (red). Right: Sparse grid generated by the S2GPT-PINN overlaid with the full grid of GPT-PINN. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: While having comparable accuracy, S Figure 6: While having comparable accuracy, S2 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages

[6]

Y. Chen, S. Gottlieb, L. Ji, and Y. Maday. An EIM-degradation free reduced ba- sis method via over collocation and residual hyper reduction-based error estimation. Journal of Computational Physics, 444:110545, 2021

work page 2021
[1]

Abdin, J

M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv:2404.14219, 2024

arXiv 2024
[2]

Ainsworth and J

M. Ainsworth and J. Dong. Galerkin neural networks: A framework for approximat- ing variational equations with error control.SIAM Journal on Scientific Computing, 43(4):A2474–A2501, 2021

work page 2021
[3]

empirical interpolation

M. Barrault, N. C. Nguyen, Y. Maday, and A. T. Patera. An “empirical interpolation” method: Application to efficient reduced-basis discretization of partial differential equa- tions. C. R. Acad. Sci. Paris, Série I, 339:667–672, 2004

work page 2004
[4]

Binev, A

P. Binev, A. Cohen, W. Dahmen, R. Devore, G. Petrova, and P. Wojtaszczyk. Con- vergence rates for greedy algorithms in reduced basis methods. SIAM Journal on Mathematical Analysis, 43(3):1457–1472, 2011

work page 2011
[5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020. 12 Figure 4: S2GPT-PINN (left) and GPT-PINN (right) pointwise errors and L2 error for a randomly selected test case. Paramete...

work page 1901
[7]

Y. Chen, Y. Ji, A. Narayan, and Z. Xu. TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNs. Computer Methods in Applied Mechanics and Engineering, 430:117198, 2024

work page 2024
[8]

Chen and S

Y. Chen and S. Koohy. GPT-PINN: Generative pre-trained physics-informed neural networks toward non-intrusive meta-learning of parametric PDEs.Finite Elements in Analysis and Design, 228:104047, 2024

work page 2024

Show all 35 references

[9]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

2009
[10]

DeVore, B

R. DeVore, B. Hanin, and G. Petrova. Neural network approximation.Acta Numerica, 30:327–444, 2021

2021
[11]

Dhariwal and A

P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

2021
[12]

Fresca, L

S. Fresca, L. Dede’, and A. Manzoni. A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes.Journal of Scientific Computing, 87:1–36, 2021

2021
[13]

Fresca and A

S. Fresca and A. Manzoni. POD-DL-ROM: Enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition. Computer Methods in Applied Mechanics and Engineering, 388:114181, 2022

2022
[14]

Z. Gao, L. Yan, and T. Zhou. Failure-informed adaptive sampling for PINNs.SIAM Journal on Scientific Computing, 45(4):A1971–A1994, 2023

2023
[15]

J. Han, A. Jentzen, and W. E. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505– 8510, 2018

2018
[16]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

2016
[17]

J. S. Hesthaven, G. Rozza, B. Stamm, et al. Certified reduced basis methods for parametrized partial differential equations, volume 590. Springer, 2016

2016
[18]

J. S. Hesthaven and S. Ubbiali. Non-intrusive reduced order modeling of nonlinear problems using neural networks.Journal of Computational Physics, 363:55–78, 2018

2018
[19]

J. Hou, Y. Li, and S. Ying. Enhancing pinns for solving pdes via adaptive collocation point movement and adaptive loss weighting. Nonlinear Dynamics, 111(16):15233– 15261, 2023

2023
[20]

Huang, Z

X. Huang, Z. Ye, H. Liu, S. Ji, Z. Wang, K. Yang, Y. Li, M. Wang, H. Chu, F. Yu, et al. Meta-auto-decoder for solving parametric partial differential equations.Advances in Neural Information Processing Systems, 35:23426–23438, 2022

2022
[21]

Y. Ji, Y. Chen, and Z. Xu. EGPT-PINN: Entropy-Enhanced Generative Pre-Trained Physics Informed Neural Networks for Parameterized Nonlinear Conservation Laws. Availableat SSRN: https://ssrn.com/abstract=5216939, 2025

2025
[22]

G. K. R. Lau, A. Hemachandra, S.-K. Ng, and B. K. H. Low. PINNACLE: PINN adaptive collocation and experimental points selection. InThe Twelfth International Conference on Learning Representations, 2024. 16

2024
[23]

Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021

2021
[24]

L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. Learning nonlinear opera- tors via deeponet based on the universal approximation theorem of operators.Nature machine intelligence, 3(3):218–229, 2021

2021
[25]

X. Lu, Z. Liu, A. Liusie, V. Raina, V. Mudupalli, Y. Zhang, and W. Beauchamp. Blending is all you need: Cheaper, better alternative to trillion-parameters llm. arXiv:2401.02994, 2024

2024 arXiv
[26]

Z. Lu, X. Li, D. Cai, R. Yi, F. Liu, X. Zhang, N. D. Lane, and M. Xu. Small language models: Survey, measurements, and insights.arXiv:2409.15790, 2024

2024 arXiv
[27]

Maday and O

Y. Maday and O. Mula. A generalized empirical interpolation method: application of reduced basis techniques to data assimilation. InAnalysis and numerics of partial differential equations, volume 4 ofSpringer INdAM Ser., pages 221–235. Springer, Mi- lan, 2013

2013
[28]

Maday, O

Y. Maday, O. Mula, A. T. Patera, and M. Yano. The generalized empirical interpolation method: stability theory on hilbert spaces with an application to the stokes equation. Computer Methods in Applied Mechanics and Engineering, 287:310–334, 2015

2015
[29]

L. D. McClenny and U. M. Braga-Neto. Self-adaptive physics-informed neural networks. Journal of Computational Physics, 474:111722, 2023

2023
[30]

Physics-InformedNeuralNet- works (PINNs) for Parameterized PDEs: A Metalearning Approach.arXiv:2110.13361, 2021

M.Penwardena, S.Zheb, A.Narayanc, andR.M.Kirbya. Physics-InformedNeuralNet- works (PINNs) for Parameterized PDEs: A Metalearning Approach.arXiv:2110.13361, 2021

2021 arXiv
[31]

Quarteroni, A

A. Quarteroni, A. Manzoni, and F. Negri. Reduced Basis Methods for Partial Differential Equations. Springer International Publishing, 2016

2016
[32]

Raissi, P

M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 378:686–707, 2019

2019
[33]

Schick and H

T. Schick and H. Schütze. It’s not just size that matters: Small language models are also few-shot learners.arXiv:2009.07118, 2020

2009 arXiv
[34]

J. W. Siegel, Q. Hong, X. Jin, W. Hao, and J. Xu. Greedy training algorithms for neural networksandapplicationstopdes. JournalofComputationalPhysics, 484:112084, 2023

2023
[35]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need.Advancesin neural information processing systems, 30, 2017. 17

2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [6]

Y. Chen, S. Gottlieb, L. Ji, and Y. Maday. An EIM-degradation free reduced ba- sis method via over collocation and residual hyper reduction-based error estimation. Journal of Computational Physics, 444:110545, 2021

work page 2021

[2] [1]

Abdin, J

M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv:2404.14219, 2024

arXiv 2024

[3] [2]

Ainsworth and J

M. Ainsworth and J. Dong. Galerkin neural networks: A framework for approximat- ing variational equations with error control.SIAM Journal on Scientific Computing, 43(4):A2474–A2501, 2021

work page 2021

[4] [3]

empirical interpolation

M. Barrault, N. C. Nguyen, Y. Maday, and A. T. Patera. An “empirical interpolation” method: Application to efficient reduced-basis discretization of partial differential equa- tions. C. R. Acad. Sci. Paris, Série I, 339:667–672, 2004

work page 2004

[5] [4]

Binev, A

P. Binev, A. Cohen, W. Dahmen, R. Devore, G. Petrova, and P. Wojtaszczyk. Con- vergence rates for greedy algorithms in reduced basis methods. SIAM Journal on Mathematical Analysis, 43(3):1457–1472, 2011

work page 2011

[6] [5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020. 12 Figure 4: S2GPT-PINN (left) and GPT-PINN (right) pointwise errors and L2 error for a randomly selected test case. Paramete...

work page 1901

[7] [7]

Y. Chen, Y. Ji, A. Narayan, and Z. Xu. TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNs. Computer Methods in Applied Mechanics and Engineering, 430:117198, 2024

work page 2024

[8] [8]

Chen and S

Y. Chen and S. Koohy. GPT-PINN: Generative pre-trained physics-informed neural networks toward non-intrusive meta-learning of parametric PDEs.Finite Elements in Analysis and Design, 228:104047, 2024

work page 2024