Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Low spectral bias alone does not guarantee accurate PINN solutions, because broader NTK spectra can bring convergence instability.

desk verdict A useful, honest empirical benchmark of learnable activations and KAN bases for PINNs, with a real cautionary finding about NTK spectra, but the scaling conclusion is over-read from a single fixed training protocol. read the letter →

arxiv 2411.15111 v4 pith:INIQUM34 submitted 2024-11-22 cs.NE cs.LG

Afrah Farea , Mustafa Serdar Celebi This is my paper · ORCID
classification cs.NEcs.LG
keywords physics-informedneuralnetworksKolmogorov-ArnoldlearnableactivationfunctionsspectralbiastangentkernelHessianeigenvaluespartialdifferentialequationstrainingstability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether learnable activation functions—parametric Tanh in MLPs and trainable basis functions in KANs—can overcome the two persistent PINN problems: spectral bias, the tendency to learn low-frequency features slowly, and unstable convergence caused by the multi-term physics loss. Comparing seven model variants on five PDEs with different spectral content, the authors find a trade-off between expressivity and training stability. Fourier and Jacobi bases keep the broadest NTK eigenvalue spectra during training yet show the largest Hessian eigenvalues and unstable losses, while B-splines, which show steep spectral decay, still achieve the best errors on Helmholtz and Cavity problems through localized frequency representation. The upshot is that activation choice is inherently problem-specific and low spectral bias is not a reliable proxy for accuracy.

What carries the argument

The central object is the pair of diagnostics applied to the same trained models: the empirical NTK eigenvalue spectrum, whose decay rate quantifies spectral bias, and the maximum Hessian eigenvalue \(\lambda_{\max}\), estimated by power iteration, which quantifies convergence instability from sharp curvature in the PINN loss. The models under comparison are MLPs with fixed or parametric Tanh and KANs with B-spline, Gaussian RBF, Fourier, Chebyshev, and Jacobi learnable basis functions. All are trained on five PDEs under identical settings, and the two diagnostics are tracked at initialization, mid-training, and the end of training.

What would settle it

Retrain the Fourier and Jacobi KANs on Helmholtz, Cavity, and Wave with per-model tuned learning rates, grid sizes, and iteration budgets chosen so their maximum Hessian eigenvalues stay as low as the B-spline model's; if they then match or beat B-spline test errors, the broad-spectrum instability trade-off is not a general law of these architectures.

Watch

Extended reading notes

Core claim

Using the eigenvalue decay of the empirical NTK to measure spectral bias and the maximum Hessian eigenvalue \(\lambda_{\max}\) to measure convergence stability, the paper shows that these two diagnostics pull in opposite directions. Fourier and Jacobi KAN bases maintain broad NTK spectra throughout training—evidence of high-frequency representational capacity—yet their \(\lambda_{\max}\) values are large and fluctuating, losses oscillate, and test errors are high or even diverge on several equations. B-splines, by contrast, exhibit rapid NTK eigenvalue decay, indicating high spectral bias, but they obtain the lowest errors on Helmholtz and Cavity because their local support confines high-frequency content to small regions, similar to domain decomposition. The paper concludes that low spectral bias alone does not guarantee better accuracy and that activation selection remains inherently problem-specific.

Load-bearing premise

The scaling conclusion rests on the assumption that training every model with the same learning rate, 60,000 iterations, batch size, and no per-KAN tuning is a fair comparison that isolates architecture behavior; if larger KANs simply need lower learning rates or longer schedules, the reported failures could be an artifact of the fixed budget.

Editorial extensions

If this is right

  • Flat NTK spectra should not be used alone as a design target for PINN accuracy; frequency reach must be balanced against training stability.
  • B-spline KANs can outperform Fourier KANs on oscillatory and boundary-layer problems despite higher measured spectral bias, because local support plays the role of domain decomposition.
  • Learnable activations that work well in small networks can destabilize in larger ones; scaling KAN width and depth raises optimization difficulty faster than expressivity helps.
  • Activation selection should follow the PDE's spectral character: Tanh-type smooth activations for wave-like problems, locally supported bases for sharp gradients, and cosines only when global periodicity and stable training align.
  • Hessian diagnostics expose that the physics-loss term stays ill-conditioned across all activations, so activation changes alone will not resolve the optimizer's burden.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Under the paper's fixed training budget, per-architecture tuning of learning rate, grid size, and iterations remains untested; if such tuning rescued the larger Fourier or Jacobi KANs, the 'curse of functional dimensionality' would need to be reinterpreted as an optimization-budget artifact.
  • Because B-spline local support mimics finite-basis domain decomposition, pairing B-spline KAN layers with multigrid-style coarse-to-fine schedules is a natural next experiment for multi-scale PDEs.
  • The diagnostics suggest a cheap selection heuristic: after a few hundred training steps, measure NTK flatness and \(\lambda_{\max}\) on the physics residual and pick the activation with the best stability-adjusted bandwidth; the paper's data imply B-spline or Tanh would win on most of the tested equations.
  • Extending the comparison to inverse problems and operator learning would test whether the expressivity-stability trade-off transfers beyond forward PDE solving, which the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper presents a systematic empirical comparison of fixed and learnable activation functions in physics-informed neural networks (PINNs), covering MLPs with Tanh and parametric-Tanh and KANs with B-spline, GRBF, Fourier, Chebyshev, and Jacobi bases, across five PDEs (Helmholtz, Klein-Gordon, Wave, Convection-diffusion, Cavity). The authors report relative L2 errors, training losses, empirical NTK eigenvalue spectra, and maximum Hessian eigenvalues. They conclude that learnable activations help in smaller architectures but scale poorly in larger ones due to an alleged 'curse of functional dimensionality,' that low spectral bias does not guarantee better accuracy, and that activation function selection remains inherently problem-specific.

Significance. If the empirical findings are robust, the paper provides a valuable counterexample to the common assumption that flatter NTK spectra imply better PINN accuracy, and it offers one of the more extensive comparisons of KAN basis functions on PDEs to date. The public code and Zenodo release support reproducibility. However, the scaling conclusion and the spectral-bias interpretation currently overreach relative to the controlled evidence, so the significance depends on whether the authors can address the training-protocol confounds and the post-hoc selection issues.

major comments (4)
  1. [Section 7 and Section 5.1] The conclusion that learnable activations 'encounter scalability issues in complex networks due to the curse of functional dimensionality' is not supported by the data in Table 4: B-spline(A2) achieves the lowest error on Cavity (4.09%) and 3.03% on Helmholtz, and GRBF(A2) achieves 8.11% on Helmholtz and 14.7% on Wave, so the A2 failures are specific to Fourier, Chebyshev, and Jacobi bases rather than to learnable activations in general. Moreover, all models are trained under a fixed budget (Adam, LR 1e-3, 60,000 iterations, batch 128, no per-basis input normalization; Section 5.1), and the paper itself notes that results could be improved with hyperparameter tuning. The divergences in Table 5 (Lphy ~1e10 to 1e12 for Fourier/Chebyshev/Jacobi A2) are consistent with learning-rate or input-scale mismatch for polynomial/Fourier bases, as the paper acknowledges in Section 3.3. The 'curse of functional dimensionality' is never defined or derived. Please either provide additional experiments (e.g., lower learning rates, input normalization, or grid-size tuning for the failing A2 models) or substantially soften the scalability claim to be about the fixed-protocol behavior of these specific bases.
  2. [Tables 4 and 5] All results are single runs; no variance estimates or multiple seeds are reported. Since the paper ranks models (e.g., 'the lowest velocity error' in Section 5.2.1) and draws qualitative conclusions, the absence of error bars makes the ranking claims statistically unsupported. Please report mean and standard deviation over at least three independent seeds, or explicitly limit claims to the single-run observations.
  3. [Section 6] The NTK and Hessian analysis is performed only on 'best-performing models' selected post-hoc from Table 4, and the experiments use only the basis function without the SiLU term (as stated in Section 6). This selection makes the claimed association between broad NTK spectra and convergence instability (Section 7) vulnerable to selection bias; the paper should either compute spectra for all models or present the claim as a case-specific observation. Additionally, Equations (21)-(26) present infinite-width NTK gradient-flow theory, but the experiments use finite-width empirical NTK with Adam and minibatches; the approximation involved should be stated explicitly.
  4. [Section 5.2.1] The text states that 'KAN models with A2 architecture exhibit instability and failure to solve the Helmholtz equation,' but Table 4 reports relative L2 errors of 3.03% for B-spline(A2) and 8.11% for GRBF(A2) on Helmholtz. This is a direct contradiction and misleads the reader about the scope of the A2 failures; please correct the narrative to align with the data in Table 4.
minor comments (6)
  1. [Equations (8) and (14)] The boundary loss terms are written as norms of sums (e.g., \|u(t,0)+u(t,1)\|), which is not equivalent to enforcing both boundary conditions separately; this appears to be a typo and should be corrected for reproducibility.
  2. [Equation (3)] The Kolmogorov-Arnold representation theorem applies to continuous functions on the unit cube; the domain notation R^n is imprecise and should be refined.
  3. [Figure 13 caption] The caption 'including the zero eigenvalues' is unclear on a log-scale plot; please clarify how zero eigenvalues are handled in the spectral plots.
  4. [Section 5.1] The notation 'weight decay of 10e-6' is unusual; please use '1e-5' for clarity.
  5. [Table 4 and Section 5.2.3] The text in Section 5.2.3 refers to 'GRBF(A1)' while Table 4 and the figure caption refer to 'GRBF(A2)' for the Wave equation; please reconcile this inconsistency.
  6. [Section 6.2] The power iteration method in Equation (28) is standard, but the number of iterations and the convergence criterion used in the experiments are not reported; please provide these details.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical NTK/Hessian findings are direct measurements of trained models, and the only self-citation is a reproducibility code/pretrained-model release carrying no argumentative weight.

full rationale

The paper's central claims are supported by direct measurements rather than by fitted inputs renamed as predictions. Relative L2 errors (Table 4), final training losses (Table 5), empirical NTK eigenvalue spectra (Fig. 13), and Hessian maximum eigenvalues (Fig. 14) are all computed from trained models and reported as observations. No equation reduces one measured quantity to another by construction: the NTK and Hessian diagnostics are descriptive analyses of the same training runs, not parameters fitted to the test errors and then used to predict them. The statement that low spectral bias does not guarantee better accuracy is a negative empirical comparison between measured spectra and measured errors, not a derivation from an assumed relation. The FBPINN analogy for B-spline local support is explicitly post-hoc interpretation ('This can be understood by drawing an analogy'), and the 'curse of functional dimensionality' is an explanatory label, not a quantity fitted to the data. The only self-citation is reference [41], the authors' Zenodo release of source code and pretrained models, which is a reproducibility artifact and does not supply any load-bearing premise. Since the evaluation is anchored to external references (exact PDE solutions and an Ansys Fluent cavity benchmark), the empirical findings are self-contained against external benchmarks, and no circular step is identifiable.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The central claims rest on a series of background theorems (Kolmogorov-Arnold, universal approximation, NTK linearization, power iteration) and on experimental assumptions: exact and CFD reference solutions are correct, minimizing the PINN loss recovers the PDE solution, and the fixed training protocol is a fair comparison across architectures. The hand-chosen loss weights and KAN basis hyperparameters are the main adjustable inputs that shape the results.

free parameters (3)
  • PDE loss weighting coefficients (lambda1, lambda2, lambda3) = Wave (1,100,100); Helmholtz (1,10); Klein-Gordon (1,50,50); Convection-diffusion (1,10,10); Cavity (0.1,2,4)
    Hand-chosen in Section 4 for each problem; they control the balance between physics, boundary, and initial losses and directly affect which models converge.
  • KAN basis hyperparameters = B-spline grid 8 degree 3; GRBF sigma 1 k 8; Fourier k 4; Chebyshev degree 4; Jacobi degree 4 alpha=beta=1
    Fixed for all problems (Table 2); no per-problem tuning, which affects the scalability comparison.
  • Optimizer and training settings = lr 0.001, weight decay 10e-6, batch 128, 60000 iterations
    Identical settings for all models (Section 5.1); the A2 KAN failures are measured under this single budget.
assumptions (7)
  • standard math Kolmogorov-Arnold representation theorem guarantees KAN architectures can represent any continuous function on a bounded domain.
    Invoked in Section 3.3 to motivate KANs; the paper notes it gives no learnability guarantee.
  • standard math Universal approximation theorem for MLPs with a single hidden layer.
    Section 3.2, used as theoretical grounding for MLP baselines.
  • standard math In the infinite-width limit, neural network training under gradient descent follows linear NTK dynamics with exponential mode decay.
    Section 6.1, Eqs. 21-26; used to justify spectral-bias measurement.
  • standard math Power iteration converges to the dominant eigenvalue of the loss Hessian.
    Section 6.2, Eq. 28; used to estimate lambda_max.
  • domain assumption The exact synthetic solutions and the Ansys Fluent solution are accurate references for computing relative L2 errors.
    Section 4 and Section 5; the benchmark's ground truth is assumed correct.
  • domain assumption Minimizing the weighted PINN loss recovers the PDE solution.
    Section 3.4, Eq. 5; standard PINN premise that the optimization problem is well-posed and solvable by Adam.
  • domain assumption Empirical NTK eigenvalues including zero eigenvalues and the maximum Hessian eigenvalue are informative proxies for spectral bias and convergence stability.
    Section 6; the analysis depends on this proxy assumption, which is not itself validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations." pith.science (2026). https://pith.science/paper/INIQUM34

@misc{pith2026241115111,
  author       = {Pith},
  title        = {Pith review of: Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INIQUM34}},
  note         = {Machine review of arXiv:2411.15111}
}
read the original abstract

Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving Partial Differential Equations (PDEs). However, they face challenges related to spectral bias (the tendency to learn low-frequency components while struggling with high-frequency features) and unstable convergence dynamics (mainly stemming from the multi-objective nature of the PINN loss function). These limitations impact their accuracy for problems involving rapid oscillations, sharp gradients, and complex boundary behaviors. We systematically investigate learnable activation functions as a solution to these challenges, comparing Multilayer Perceptrons (MLPs) using fixed and learnable activation functions against Kolmogorov-Arnold Networks (KANs) that employ learnable basis functions. Our evaluation spans diverse PDE types, including linear and non-linear wave problems, mixed-physics systems, and fluid dynamics. Using empirical Neural Tangent Kernel (NTK) analysis and Hessian eigenvalue decomposition, we assess spectral bias and convergence stability of the models. Our results reveal a trade-off between expressivity and training convergence stability. While learnable activation functions work well in simpler architectures, they encounter scalability issues in complex networks due to the higher functional dimensionality. Counterintuitively, we find that low spectral bias alone does not guarantee better accuracy, as functions with broader NTK eigenvalue spectra may exhibit convergence instability. We demonstrate that activation function selection remains inherently problem-specific, with different bases showing distinct advantages for particular PDE characteristics. We believe these insights will help in the design of more robust neural PDE solvers.

Figures

Figures reproduced from arXiv: 2411.15111 by the authors.

Figure 1
Figure 1. This diagram presents a high-level comparison between the MLP and KAN architectures within the PINN [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Training convergence history for the Helmholtz equation. The loss curves for all models are shown, with [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Contour plots for the solution of the Helmholtz equation. The plots depict the reference or actual solution, the [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Training convergence history for the Klein-Gordon equation. The loss curves for the two best converging [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Contour plots for the solution of the Klein-Gordon equation. The plots depict the reference or actual solution, [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Training convergence history for the Wave equation. The loss curves indicate that [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Contour plots for the solution of the Wave equation in the spatio-temporal domain [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Training convergence history for the Convection-diffusion equation. The loss curves for all models are [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Contour plots for the solution of the Convection-diffusion velocity ( [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Training convergence history for the Cavity problem. Loss curves for all models are shown, with [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Contour plots for the solution of the Cavity problem velocity ( [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Contour plots for the solution of the Cavity problem, showing velocity components ( [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Evolution of the NTK eigenvalue spectra during training, including the zero eigenvalues. Each row of a subplot corresponds to a different training iteration: the initial (top), middle, and final (bottom) stages of the training epochs. The horizontal axis of each plot …
Figure 14
Figure 14. Figure 14: Evolution of the maximum eigenvalue of the Hessian of the loss function, [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. KKANs: Kurkova-Kolmogorov-Arnold Networks and Their Learning Dynamics

    cs.LG 2024-12 conditional novelty 6.0 of 10

    KKANs, a two-block KART-based architecture with MLP inner functions and basis-function outer functions, universally approximate continuous functions and empirically outperform MLP and cKAN baselines in regression, PIN...

Reference graph

Works this paper leans on

79 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 378:686–707, 2019

    2019

  2. [2]

    Scientific machine learning through physics–informed neural networks: Where we are and what’s next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022

    2022

  3. [3]

    Physics- informed machine learning.Nature Reviews Physics, 3(6):422–440, 2021

    George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics- informed machine learning.Nature Reviews Physics, 3(6):422–440, 2021

    2021

  4. [4]

    Nsfnets (navier-stokes flow nets): Physics- informed neural networks for the incompressible navier-stokes equations.Journal of Computational Physics, 426:109951, 2021

    Xiaowei Jin, Shengze Cai, Hui Li, and George Em Karniadakis. Nsfnets (navier-stokes flow nets): Physics- informed neural networks for the incompressible navier-stokes equations.Journal of Computational Physics, 426:109951, 2021

    work page 2021

  5. [5]

    Artificial neural networks for solving ordinary and partial differential equations.IEEE transactions on neural networks, 9(5):987–1000, 1998

    Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations.IEEE transactions on neural networks, 9(5):987–1000, 1998

    1998

  6. [6]

    Cambridge University Press, 2023

    Miguel A Mendez, Andrea Ianiro, Bernd R Noack, and Steven L Brunton.Data-driven fluid mechanics: combining first principles and machine learning. Cambridge University Press, 2023

    work page 2023

  7. [7]

    Uncovering near-wall blood flow from sparse data with physics-informed neural networks.Physics of Fluids, 33(7):071905, 2021

    Amirhossein Arzani, Jian-Xun Wang, and Roshan M D’Souza. Uncovering near-wall blood flow from sparse data with physics-informed neural networks.Physics of Fluids, 33(7):071905, 2021

    work page 2021

  8. [8]

    Mostajeran and R

    F. Mostajeran and R. Mokhtari. Deepbhcp: Deep neural network algorithm for solving backward heat conduction problems.Computer Physics Communications, 272:108236, 2022

    work page 2022

Show all 79 references
  1. [9]

    Dinglei Zhang, Ying Li, and Shihui Ying. Trans-net: A transferable pretrained neural networks based on temporal domain decomposition for solving partial differential equations.Computer Physics Communications, 299:109130, 2024

    2024

  2. [10]

    On the spectral bias of neural networks

    Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. InInternational conference on machine learning, pages 5301–5310. PMLR, 2019

  3. [11]

    Frequency principle: Fourier analysis sheds light on deep neural networks.Communications in Computational Physics, 28(5):1746–1767, 2020

    Zhi-Qin John Xu. Frequency principle: Fourier analysis sheds light on deep neural networks.Communications in Computational Physics, 28(5):1746–1767, 2020

    2020

  4. [12]

    Multi-scale deep neural networks for solving high dimensional pdes.Neural Information Processing Systems, 2024

    Wei Cai and Zhi-Qin John Xu. Multi-scale deep neural networks for solving high dimensional pdes.Neural Information Processing Systems, 2024

    2024

  5. [13]

    Fourier neural operator for parametric partial differential equations.International Conference on Learning Representations, 2021

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations.International Conference on Learning Representations, 2021

    2021

  6. [14]

    Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Finite basis physics-informed neural networks (fbpinns): a scalable domain decomposition approach for solving differential equations.Advances in Computational Mathematics, 49(4):62, 2023

    2023

  7. [15]

    Self-scalable tanh (stan): Multi-scale solutions for physics-informed neural networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15588–15603, 2023

    Raghav Gnanasambandam, Bo Shen, Jihoon Chung, Xubo Yue, and Zhenyu Kong. Self-scalable tanh (stan): Multi-scale solutions for physics-informed neural networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15588–15603, 2023. 23

    2023

  8. [16]

    Kan: Kolmogorov-arnold networks.arXiv preprint arXiv:2404.19756, 2024

    Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljaˇci´c, Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov-arnold networks.arXiv preprint arXiv:2404.19756, 2024

    2024 arXiv

  9. [17]

    Challenges in training pinns: A loss landscape perspective.arXiv preprint arXiv:2402.01868, 2024

    Pratik Rathore, Weimu Lei, Zachary Frangella, Lu Lu, and Madeleine Udell. Challenges in training pinns: A loss landscape perspective.arXiv preprint arXiv:2402.01868, 2024

    2024 arXiv

  10. [18]

    Kronecker neural networks overcome spectral bias for pinn-based wavefield computation

    Umair Bin Waheed. Kronecker neural networks overcome spectral bias for pinn-based wavefield computation. IEEE Geoscience and Remote Sensing Letters, 19:1–5, 2022

    2022

  11. [19]

    On the expressiveness and spectral bias of kans.arXiv preprint arXiv:2410.01803, 2024

    Yixuan Wang, Jonathan W Siegel, Ziming Liu, and Thomas Y Hou. On the expressiveness and spectral bias of kans.arXiv preprint arXiv:2410.01803, 2024

    2024 arXiv

  12. [20]

    Benjamin C Koenig, Suyong Kim, and Sili Deng. KAN-ODEs: Kolmogorov–Arnold network ordinary differential equations for learning dynamical systems and hidden physics.Computer Methods in Applied Mechanics and Engineering, 432:117397, 2024

    2024

  13. [21]

    The spectral bias of polynomial neural networks.International Conference on Learning Representations, 2022

    Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos, Julien Mairal, and V olkan Cevher. The spectral bias of polynomial neural networks.International Conference on Learning Representations, 2022

    2022

  14. [22]

    Extrapolation and spectral bias of neural nets with hadamard product: a polynomial net study.Advances in neural information processing systems, 35:26980–26993, 2022

    Yongtao Wu, Zhenyu Zhu, Fanghui Liu, Grigorios Chrysos, and V olkan Cevher. Extrapolation and spectral bias of neural nets with hadamard product: a polynomial net study.Advances in neural information processing systems, 35:26980–26993, 2022

    2022

  15. [23]

    On the activation function dependence of the spectral bias of neural networks.arXiv preprint arXiv:2208.04924, 2022

    Qingguo Hong, Jonathan W Siegel, Qinyang Tan, and Jinchao Xu. On the activation function dependence of the spectral bias of neural networks.arXiv preprint arXiv:2208.04924, 2022

    2022 arXiv

  16. [24]

    Aku Kammonen, Lisi Liang, Anamika Pandey, and Raúl Tempone. Comparing spectral bias and robustness for two-layer neural networks: Sgd vs adaptive random fourier features.International Conference on Scientific Computing and Machine Learning, 2024

    2024

  17. [25]

    SIAM, 2000

    William L Briggs, Van Emden Henson, and Steve F McCormick.A multigrid tutorial. SIAM, 2000

    2000

  18. [26]

    Elsevier, 2000

    Ulrich Trottenberg, Cornelius W Oosterlee, and Anton Schuller.Multigrid. Elsevier, 2000

    2000

  19. [27]

    Self-adaptive physics-informed neural networks.Journal of Computational Physics, 474:111722, 2023

    Levi D McClenny and Ulisses M Braga-Neto. Self-adaptive physics-informed neural networks.Journal of Computational Physics, 474:111722, 2023

    2023

  20. [28]

    Binary structured physics-informed neural networks for solving equations with rapidly changing solutions

    Yanzhi Liu, Ruifan Wu, and Ying Jiang. Binary structured physics-informed neural networks for solving equations with rapidly changing solutions. 518:113341

  21. [29]

    Multilevel domain decomposition- based architectures for physics-informed neural networks.Computer Methods in Applied Mechanics and Engi- neering, 429:117116, 2024

    Victorita Dolean, Alexander Heinlein, Siddhartha Mishra, and Ben Moseley. Multilevel domain decomposition- based architectures for physics-informed neural networks.Computer Methods in Applied Mechanics and Engi- neering, 429:117116, 2024

    2024

  22. [30]

    Adaptive sampling points based multi-scale residual network for solving partial differential equations.Computers & Mathematics with Applications, 169:223–236, 2024

    Jie Wang, Xinlong Feng, and Hui Xu. Adaptive sampling points based multi-scale residual network for solving partial differential equations.Computers & Mathematics with Applications, 169:223–236, 2024

    2024

  23. [31]

    Jinshuai Bai, Gui-Rong Liu, Ashish Gupta, Laith Alzubaidi, Xi-Qiao Feng, and YuanTong Gu. Physics-informed radial basis network (pirbn): A local approximating neural network for solving nonlinear partial differential equations.Computer Methods in Applied Mechanics and Engineer...

    2023

  24. [32]

    Exact enforcement of temporal continuity in sequential physics-informed neural networks.Computer Methods in Applied Mechanics and Engineering, 430:117197, 2024

    Pratanu Roy and Stephen T Castonguay. Exact enforcement of temporal continuity in sequential physics-informed neural networks.Computer Methods in Applied Mechanics and Engineering, 430:117197, 2024

    2024

  25. [33]

    Adaptive activation functions accelerate convergence in deep and physics-informed neural networks.Journal of Computational Physics, 404:109136, 2020

    Ameya D Jagtap, Kenji Kawaguchi, and George Em Karniadakis. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks.Journal of Computational Physics, 404:109136, 2020

    2020

  26. [34]

    A practical pinn framework for multi-scale problems with multi-magnitude loss terms.Journal of Computational Physics, 510:113112, 2024

    Yong Wang, Yanzhong Yao, Jiawei Guo, and Zhiming Gao. A practical pinn framework for multi-scale problems with multi-magnitude loss terms.Journal of Computational Physics, 510:113112, 2024

    2024

  27. [35]

    Shahed Rezaei, Ali Harandi, Ahmad Moeineddin, Bai-Xiang Xu, and Stefanie Reese. A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: Comparison with finite element method.Computer Methods in Applied M...

    2022

  28. [36]

    Finite element interpolated neural networks for solving forward and inverse problems.Computer Methods in Applied Mechanics and Engineering, 418:116505, 2024

    Santiago Badia, Wei Li, and Alberto F Martín. Finite element interpolated neural networks for solving forward and inverse problems.Computer Methods in Applied Mechanics and Engineering, 418:116505, 2024

    2024

  29. [37]

    Loss-attentional physics-informed neural networks.Journal of Computational Physics, 501:112781, 2024

    Yanjie Song, He Wang, He Yang, Maria Luisa Taccari, and Xiaohui Chen. Loss-attentional physics-informed neural networks.Journal of Computational Physics, 501:112781, 2024

    2024

  30. [38]

    Efficient physics-informed neural networks using hash encoding.Journal of Computational Physics, 501:112760, 2024

    Xinquan Huang and Tariq Alkhalifah. Efficient physics-informed neural networks using hash encoding.Journal of Computational Physics, 501:112760, 2024. 24

    2024

  31. [39]

    Pdebench: An extensive benchmark for scientific machine learning.Advances in Neural Information Processing Systems, 35:1596–1611, 2022

    Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning.Advances in Neural Information Processing Systems, 35:1596–1611, 2022

    2022

  32. [40]

    Data-driven physics-informed neural networks: A digital twin perspective.Computer Methods in Applied Mechanics and Engineering, 428:117075, 2024

    Sunwoong Yang, Hojin Kim, Yoonpyo Hong, Kwanjung Yee, Romit Maulik, and Namwoo Kang. Data-driven physics-informed neural networks: A digital twin perspective.Computer Methods in Applied Mechanics and Engineering, 428:117075, 2024

    2024

  33. [41]

    Learnable activation functions in physics-informed neural networks for solving partial differential equations, 2025

    Afrah Farea. Learnable activation functions in physics-informed neural networks for solving partial differential equations, 2025

    2025

  34. [42]

    A survey on modern trainable activation functions.Neural Networks, 138:14–32, 2021

    Andrea Apicella, Francesco Donnarumma, Francesco Isgrò, and Roberto Prevete. A survey on modern trainable activation functions.Neural Networks, 138:14–32, 2021

    2021

  35. [43]

    Searching for activation functions.arXiv preprint arXiv:1710.05941, 2017

    Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions.arXiv preprint arXiv:1710.05941, 2017

    2017 arXiv

  36. [44]

    Adaptive blending units: Trainable activation functions for deep neural networks

    Leon René Sütfeld, Flemming Brieger, Holger Finger, Sonja Füllhase, and Gordon Pipa. Adaptive blending units: Trainable activation functions for deep neural networks. InIntelligent Computing: Proceedings of the 2020 Computing Conference, Volume 3, pages 37–50. Springer, 2020

    2020

  37. [45]

    Learning specialized activation functions for physics-informed neural networks.arXiv preprint arXiv:2308.04073, 2023

    Honghui Wang, Lu Lu, Shiji Song, and Gao Huang. Learning specialized activation functions for physics-informed neural networks.arXiv preprint arXiv:2308.04073, 2023

    2023 arXiv

  38. [46]

    Kan versus mlp on irregular or noisy functions.arXiv preprint arXiv:2408.07906, 2024

    Chen Zeng, Jiahui Wang, Haoran Shen, and Qiao Wang. Kan versus mlp on irregular or noisy functions.arXiv preprint arXiv:2408.07906, 2024

    2024 arXiv

  39. [47]

    A comprehen- sive and fair comparison between mlp and kan representations for differential equations and operator networks

    Khemraj Shukla, Juan Diego Toscano, Zhicheng Wang, Zongren Zou, and George Em Karniadakis. A comprehen- sive and fair comparison between mlp and kan representations for differential equations and operator networks. Journal of Computational Physics, 42(6):789–812, 2024

    2024

  40. [48]

    Adaptive training of grid-dependent physics-informed kolmogorov-arnold networks.arXiv preprint arXiv:2407.17611, 2024

    Spyros Rigas, Michalis Papachristou, Theofilos Papadopoulos, Fotios Anagnostopoulos, and Georgios Alexan- dridis. Adaptive training of grid-dependent physics-informed kolmogorov-arnold networks.arXiv preprint arXiv:2407.17611, 2024

    2024 arXiv

  41. [49]

    Kolmogorov arnold informed neural network: A physics-informed deep learning framework for solving pdes based on kolmogorov arnold networks.arXiv preprint arXiv:2406.11045, 2024

    Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Liu. Kolmogorov arnold informed neural network: A physics-informed deep learning framework for solving pdes based on kolmogorov arnold networks.arXiv prep...

    2024 arXiv

  42. [50]

    Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 2(4):303–314, 1989

    George Cybenko. Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 2(4):303–314, 1989

    1989

  43. [51]

    Andrey N Kolmogorov. On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition.Doklady Akademii Nauk SSSR, 114(5):953–956, 1957

    1957

  44. [52]

    On calculating with b-splines.Journal of Approximation theory, 6(1):50–62, 1972

    Carl De Boor. On calculating with b-splines.Journal of Approximation theory, 6(1):50–62, 1972

    1972

  45. [53]

    Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation.arXiv preprint arXiv:2405.07200, 2024

    Sidharth SS. Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation.arXiv preprint arXiv:2405.07200, 2024

    2024 arXiv

  46. [54]

    Kolmogorov-arnold networks are radial basis function networks.arXiv preprint arXiv:2405.06721, 2024

    Ziyao Li. Kolmogorov-arnold networks are radial basis function networks.arXiv preprint arXiv:2405.06721, 2024

    2024 arXiv

  47. [55]

    Bsrbf-kan: A combination of b-splines and radial basic functions in kolmogorov-arnold networks.arXiv preprint arXiv:2406.11173, 2024

    Hoang-Thang Ta. Bsrbf-kan: A combination of b-splines and radial basic functions in kolmogorov-arnold networks.arXiv preprint arXiv:2406.11173, 2024

    2024 arXiv

  48. [56]

    Fourierkan-gcf: Fourier kolmogorov-arnold network–an effective and efficient feature transformation for graph collaborative filtering.arXiv preprint arXiv:2406.01034, 2024

    Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Wei Wang, Xiping Hu, and Edith C-H Ngai. Fourierkan-gcf: Fourier kolmogorov-arnold network–an effective and efficient feature transformation for graph collaborative filtering.arXiv preprint arXiv:2406.01034, 2024

    2024 arXiv

  49. [57]

    Wav-kan: Wavelet kolmogorov-arnold networks.arXiv preprint arXiv:2405.12832, 2024

    Zavareh Bozorgasl and Hao Chen. Wav-kan: Wavelet kolmogorov-arnold networks.arXiv preprint arXiv:2405.12832, 2024

    2024 arXiv

  50. [58]

    Unveiling the power of wavelets: A wavelet-based kolmogorov-arnold network for hyperspectral image classification.arXiv preprint arXiv:2406.07869, 2024

    Seyd Teymoor Seydi. Unveiling the power of wavelets: A wavelet-based kolmogorov-arnold network for hyperspectral image classification.arXiv preprint arXiv:2406.07869, 2024

    2024 arXiv

  51. [59]

    Softadapt: Techniques for adaptive loss weighting of neural networks with multi-part loss functions.arXiv preprint arXiv:1912.12355, 2019

    A Ali Heydari, Craig A Thompson, and Asif Mehmood. Softadapt: Techniques for adaptive loss weighting of neural networks with multi-part loss functions.arXiv preprint arXiv:1912.12355, 2019

    1912 arXiv

  52. [60]

    Self-adaptive physics-informed neural networks using a soft attention mechanism.arXiv preprint arXiv:2009.04544, 2020

    Levi McClenny and Ulisses Braga-Neto. Self-adaptive physics-informed neural networks using a soft attention mechanism.arXiv preprint arXiv:2009.04544, 2020

    2009 arXiv

  53. [61]

    Understanding and mitigating gradient flow pathologies in physics-informed neural networks.SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021

    Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks.SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021. 25

    2021

  54. [62]

    When and why pinns fail to train: A neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    2022

  55. [63]

    Multi-objective loss balancing for physics-informed deep learning.Computer Methods in Applied Mechanics and Engineering, 439:117914, 2025

    Rafael Bischof and Michael A Kraus. Multi-objective loss balancing for physics-informed deep learning.Computer Methods in Applied Mechanics and Engineering, 439:117914, 2025

    2025

  56. [64]

    Residual- based attention in physics-informed neural networks.Computer Methods in Applied Mechanics and Engineering, 421:116805, 2024

    Sokratis J Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, and George Em Karniadakis. Residual- based attention in physics-informed neural networks.Computer Methods in Applied Mechanics and Engineering, 421:116805, 2024

    2024

  57. [65]

    Simple yet effective adaptive activation functions for physics-informed neural networks.Computer Physics Communications, page 109428, 2024

    Jun Zhang and Chensen Ding. Simple yet effective adaptive activation functions for physics-informed neural networks.Computer Physics Communications, page 109428, 2024

    2024

  58. [66]

    Adaptive-sampling physics-informed neural network for viscoacoustic wavefield simulation.IEEE Geoscience and Remote Sensing Letters, 2024

    Haixia Zhao, Yufan Hao, Wenhao Xu, and Chen Li. Adaptive-sampling physics-informed neural network for viscoacoustic wavefield simulation.IEEE Geoscience and Remote Sensing Letters, 2024

    2024

  59. [67]

    Neural tangent kernel: Convergence and generalization in neural networks.Advances in neural information processing systems, 31, 2018

    Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks.Advances in neural information processing systems, 31, 2018

    2018

  60. [68]

    Wide neural networks of any depth evolve as linear models under gradient descent.Advances in neural information processing systems, 32, 2019

    Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent.Advances in neural information processing systems, 32, 2019

    2019

  61. [69]

    Fourier features let networks learn high frequency functions in low dimensional domains.Advances in neural information processing systems, 33:7537–7547, 2020

    Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains.Advances in neural information proce...

    2020

  62. [70]

    On understanding and overcoming spectral biases of deep neural network learning methods for solving pdes.Journal of Computational Physics, 2025

    Wei Cai Zhi-Qin John Xu, Lulu Zhang. On understanding and overcoming spectral biases of deep neural network learning methods for solving pdes.Journal of Computational Physics, 2025

    2025

  63. [71]

    Reproducing activation function for deep learning

    Senwei Liang, Liyao Lyu, Chunmei Wang, and Haizhao Yang. Reproducing activation function for deep learning. Communications in Mathematical Sciences, 2024

    2024

  64. [72]

    An investigation into neural net optimization via hessian eigenvalue density

    Behrooz Ghorbani, Shankar Krishnan, and Ying Xiao. An investigation into neural net optimization via hessian eigenvalue density. InInternational Conference on Machine Learning, pages 2232–2241. PMLR, 2019

    2019

  65. [73]

    Gradient descent on neural networks typically occurs at the edge of stability.International Conference on Learning Representations, 2021

    Jeremy M Cohen, Simran Kaur, Yuanzhi Li, J Zico Kolter, and Ameet Talwalkar. Gradient descent on neural networks typically occurs at the edge of stability.International Conference on Learning Representations, 2021

    2021

  66. [74]

    Negative eigenvalues of the hessian in deep neural networks.International Conference on Learning Representations, 2019

    Guillaume Alain, Nicolas Le Roux, and Pierre-Antoine Manzagol. Negative eigenvalues of the hessian in deep neural networks.International Conference on Learning Representations, 2019

    2019

  67. [75]

    Hessian eigenspectra of more realistic nonlinear models.Advances in Neural Information Processing Systems, 34:20104–20117, 2021

    Zhenyu Liao and Michael W Mahoney. Hessian eigenspectra of more realistic nonlinear models.Advances in Neural Information Processing Systems, 34:20104–20117, 2021

    2021

  68. [76]

    Sharpness-aware minimization for efficiently improving generalization.International Conference on Learning Representations, 2021

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization.International Conference on Learning Representations, 2021

    2021

  69. [77]

    On the maximum hessian eigenvalue and generalization

    Simran Kaur, Jeremy Cohen, and Zachary Chase Lipton. On the maximum hessian eigenvalue and generalization. InProceedings on, pages 51–65. PMLR, 2023

    2023

  70. [78]

    Neural networks-tricks of the trade second edition

    Grégoire Montavon and Klaus-Robert Müller. Neural networks-tricks of the trade second edition

  71. [79]

    Pyhessian: Neural networks through the lens of the hessian

    Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W Mahoney. Pyhessian: Neural networks through the lens of the hessian. In2020 IEEE international conference on big data (Big data), pages 581–590. IEEE, 2020. 26

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.