Pith. sign in

REVIEW 4 major objections 6 minor 37 references

Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper establishes that a spline-basis KAN layer is exactly a multichannel MLP with biases fixed to the spline knots, and that training the KAN is preconditioned gradient descent in the ReLU basis, which together with geometric…

desk verdict A correct and useful algebraic reading of KANs as preconditioned multichannel MLPs, with a practical multilevel refinement scheme, but the theoretical spectral bound has a repairable scaling error and the deep-net speedup claim outruns the evidence. read the letter →

arxiv 2505.18131 v1 pith:CZ7W2JNW submitted 2025-05-23 cs.LG cs.AI

Jonas A. Actor , Graham Harper , Ben Southworth , Eric C. Cyr This is my paper
classification cs.LGcs.AI MSC 65D0768T0765F35
keywords Kolmogorov-ArnoldnetworksmultichannelMLPsB-splinespreconditionedgradientdescentgeometricrefinementfree-knotsplinesspectralbiasphysics-informedneural
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that the training advantage of spline-based Kolmogorov-Arnold networks over plain multilayer perceptrons is not a matter of expressive power but of optimization geometry. Because the B-spline basis and the truncated-power (ReLU) basis span the same spline space, any KAN layer can be rewritten as a multichannel MLP whose extra bias channels are fixed at the spline knots. Gradient descent on the KAN's spline weights is then exactly gradient descent on the MLP's ReLU weights multiplied by a fixed preconditioner $A A^\top$, where $A$ is the change-of-basis matrix between the two bases. The paper argues that this preconditioning improves the conditioning of the loss landscape, that the local support of splines tames the spectral bias that slows ReLU MLPs, and that nested spline spaces allow cheap geometric refinement during training. On regression, XOR, and physics-informed Burgers benchmarks, the spline-basis and multilevel-trained models reach lower loss for the same training work, and trainable free-knot splines help on nonsmooth problems.

What carries the argument

The load-bearing object is the change-of-basis matrix $A$ between the B-spline basis and the $\mathrm{ReLU}^{\,r-1}$ truncated-power basis of the spline space $S_r(T)$: for uniform knots $A^{[r]}=\frac{h^{1-r}}{(r-1)!}(A^{[1]})^r$ with $A^{[1]}$ bidiagonal, and for nonuniform knots it is built recursively by $A^{[r]}_{ij}=\frac{1}{t_{i+r-1}-t_i}A^{[r-1]}_{ij}-\frac{1}{t_{i+r}-t_{i+1}}A^{[r-1]}_{i+1,j}$. It carries the argument because it converts KAN spline weights into multichannel MLP ReLU weights, relates the Hessians by congruence $H_{\mathrm{spline}} = A H_{\mathrm{ReLU}} A^\top$, and makes gradient descent in the spline basis equal to preconditioned gradient descent in the ReLU basis with preconditioner $A A^\top$. The same matrix, in uniform form, is what the NTK spectral-radius bound $\rho(\mathrm{NTK}_S^{[r]})\le 4\rho(\mathrm{NTK}_R)$ uses, and its banded structure keeps the change of basis cheap ($O(nr)$ per layer).

What would settle it

Run a deep multichannel ReLU MLP and its spline-basis counterpart on the same task and compare per-layer Hessian condition numbers after the change of basis: if any layer's condition number is not improved, or if spline-basis gradient descent and preconditioned ReLU gradient descent do not follow the same trajectory, the layer-wise preconditioning claim fails. Independently, recompute the uniform-knot bound in Appendix B: with $\tilde A^{[r]}=h^{r-1}A^{[r]}$ and generating function $F_r(x)=\frac{r}{(r-1)!}(1-e^{ix})^r$ (Eqs. 49-50), the claimed uniform bound $\|\tilde A^{[r]}\|\le 2^r/(r-1)!\le 4$ appears to drop factors of $h$; explicit spectral norm computation for $r=3,4$ and $n\to\infty$ would settle whether $\rho(\mathrm{NTK}_S^{[r]})\le 4\rho(\mathrm{NTK}_R)$ actually holds for all $r$.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is an exact structural equivalence with a practical payoff. Let $T=\{t_i\}$ be the spline knots and let $S_r(T)$ be the space of order-$r$ splines on an interval; the B-spline basis $B_S=\{b_i^{[r-1]}\}$ and the truncated-power basis $B_R=\{\mathrm{ReLU}(x-t_i)^{r-1}\}$ are both bases of $S_r(T)$, so there is an invertible, banded matrix $A$ with $\tilde W^{(\ell)} \times_3 A = W^{(\ell)}$ mapping each KAN layer's spline weights to ReLU weights. A KAN layer $\sum_{p,i} \tilde W^{(\ell)}_{qpi} b_i(x_p^{(\ell)})$ is therefore a multichannel MLP layer $\sum_{p,i} W^{(\ell)}_{qpi}\mathrm{ReLU}(x_p^{(\ell)}-t_i)^{r-1}$, with one channel per knot and biases fixed to the knots. Minimizing the same loss in the two parameterizations is equivalent, but gradient descent in the spline basis is the preconditioned update $W_{k+1}=W_k-\eta A A^\top \nabla \mathcal{L}(W_k)$, and in the linear least-squares case the spline Hessian condition number is bounded independently of grid size while the ReLU Hessian condition number grows without bound. The paper concludes from this that KANs inherit MLP approximation power while enjoying a better-conditioned optimization problem, that hierarchical multilevel refinement is an exact injection that preserves progress, and that parameterizing knot distances with a softmax makes free-knot splines trainable by backpropagation.

Load-bearing premise

The load-bearing assumption is that the conditioning improvement proven for a one-layer linear least-squares problem still holds in deep nonlinear networks when the spline change of basis is applied independently layer by layer; the paper does not prove this, noting that a rigorous statement is difficult, and instead offers three small experiments.

Editorial extensions

If this is right

  • Any standard multichannel ReLU MLP can be trained with KAN-like speedups by fixing biases at spline knots and preconditioning each layer's gradient by $A A^\top$, without changing the architecture's function class.
  • Geometric refinement via nested spline grids is an exact injection: transferring a trained coarse KAN to a finer grid preserves the learned function, so multilevel training never throws away progress and costs little enough to run during training.
  • Trainable knots parameterized by softmax-spaced distances let a KAN adapt its grid to nonsmooth features, improving accuracy where fixed uniform grids fail.
  • The bounded spline Gram condition number and the NTK spectral-radius comparison imply training updates are spread across localized modes instead of concentrating in the dominant eigen-direction, reversing the spectral-bias failure mode of ReLU MLPs.
  • On equal-FLOP comparisons, spline-basis and multilevel KANs reach lower training loss than comparable MLPs on regression, XOR, and a physics-informed Burger's equation problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the equivalence is exact, the practical implication is broader than KANs: one could wrap any MLP optimizer in the fixed preconditioner $A A^\top$ and reproduce KAN-style training dynamics on standard architectures, making the speedup a drop-in optimization change rather than a new architecture.
  • The paper's layer-wise decoupling of the preconditioning argument suggests a diagnostic: measure per-layer Hessian or NTK conditioning in a deep ReLU MLP before and after the spline change of basis; if some layers do not improve, the benefit may be confined to particular depths or tasks rather than universal.
  • The free-knot softmax parameterization points toward a unified view of adaptive KAN grids and trainable MLP biases: both are the same mechanism, so adaptivity results from one setting should transfer to the other.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper establishes an exact algebraic relationship between KANs built on a B-spline basis and multichannel MLPs with ReLU^{r-1} activations whose biases are fixed to the spline knots. The central identity (Eqs. 12-14) shows that full-batch gradient descent on the spline weights is equivalent to preconditioned gradient descent on the ReLU weights with block-diagonal preconditioner P = AA^T. The paper then argues that this preconditioning improves conditioning and convergence, proposes a geometric refinement scheme for multilevel KAN training using exact injection operators, and introduces a trainable free-knot spline parameterization based on softmax-normalized knot spacings. Experiments on two regression problems and a physics-informed neural network (PINN) report improved accuracy from the spline basis, multilevel refinement, and free knots. The main theoretical support for the deep-network speedup claim is Theorem 3.2, which asserts a spectral-radius bound for the neural tangent kernel under the basis change; the proof of this theorem appears in Appendix B.

Significance. If the central claims are correct, the paper provides a clean, non-circular explanation of KAN training speedups: KANs in a B-spline basis are exactly multichannel MLPs in a better-conditioned parameterization. The equivalence in Section 2 and the preconditioning identity in Section 3 are direct algebraic identities that can be verified independently, and the proposed geometric refinement and free-knot parameterizations are concrete, executable algorithmic contributions. The paper does not adapt existing results by curve-fitting; the derivations are self-contained and cite the relevant spline and Toeplitz literature. However, the advertised theoretical guarantee for deep nonlinear networks rests on a proof that contains a scaling inconsistency, and the experiments use an optimizer different from the one analyzed in the equivalence, so the theoretical portion needs correction before the paper's claims can be fully accepted.

major comments (4)
  1. [Appendix B, Eq. (49)] The generating function for the normalized change-of-basis matrix eA[r] contains a spurious factor n^{r-1}. By Eq. (47), eA[r] = A[r]/h^{1-r}, and with h = 1/n, Corollary B.3 gives eA[r] = (A[1])^r/(r-1)!, which has Fourier symbol (1-e^{ix})^r/(r-1)! and operator norm at most 2^r/(r-1)!. The last equality in Eq. (49), n^{r-1}(1-e^{ix})^r/(r-1)!, is instead the symbol of the unscaled A[r]. As written, this makes Eq. (50) internally inconsistent: the n^{r-1} factor would grow with n, contradicting the claimed uniform bound 2^r/(r-1)! ≤ 4. Consequently, the proof of Theorem 3.2 is invalid as written.
  2. [Theorem 3.2 / Appendix B, after Eq. (50)] Even after removing the spurious n^{r-1} factor, the proof does not establish the stated constant 4. The argument bounds the singular values of eA, but the matrix appearing in NTK_S = JAA^TJ^T is the unscaled change-of-basis matrix A; if one instead works with the scaled basis, the Jacobian J changes by the same scaling, and this change is not tracked. Submultiplicativity gives ρ(NTK_S) = ||JA||^2 ≤ ||J||^2 ||A||^2, so with ||A|| ≤ 4 one obtains ρ(NTK_S) ≤ 16 ρ(NTK_R), not the stated factor 4, unless an additional argument is supplied.
  3. [Section 3, paragraph after Eq. (14)] The step from the linear least-squares problem to deep nonlinear networks is asserted rather than proved. The text states that the preconditioner is 'applied block-wise independently for each layer, thereby improving the conditioning and convergence' and then concedes that rigorously proving the deep-network statement is difficult. Since Theorem 3.2 is the only quantitative support for this extrapolation, and its proof is currently invalid, the theoretical explanation of expedited training in deep networks is incomplete. The authors should either provide a corrected proof of a suitable statement or explicitly reframe the deep-network conditioning claim as an empirical observation supported by the experiments.
  4. [Section 6 and Section 3] The theoretical equivalence in Eqs. (12)-(14) is derived for gradient descent, but the experiments in Section 6 use L-BFGS. It is not shown that the preconditioning result transfers to L-BFGS updates, and the paper does not describe how 'equivalent amounts of work (FLOPs)' are counted. This mismatch makes it difficult to attribute the observed speedups specifically to the preconditioning mechanism, as opposed to other differences between the architectures and optimizers. The authors should address the optimizer mismatch or qualify the claim that the experiments validate the gradient-descent preprocessing analysis.
minor comments (6)
  1. [Section 2, Eq. (9)] The phrase 'low-rank factorization' for the restriction W^{(ℓ)}_{qpi} = δ_{i=p} W^{(ℓ)}_{qp} is inaccurate; this is a sparsity/selection pattern, not a low-rank factorization.
  2. [Tables 1-3] The column header 'FREE# MSE' is garbled and should be split into 'FREE KNOTS?' and '# MSE' or equivalent; several other table headers are crowded and would benefit from clearer formatting.
  3. [Appendix D] There are minor typos: 'we us weight-regularized Adam' should read 'we use weight-regularized Adam', and 'ranging form 0.0001 to 0.001' should read 'ranging from'.
  4. [Appendix C.1, Eq. (51)] The batch-size condition should be stated as a ceiling: the required batch size is the smallest integer greater than or equal to log(1-τ)/log(P(x∉[t_i,t_{i+1}])).
  5. [Theorem 3.2] The sentence 'the dominant eigenvalue is the same order for both bases' is stronger than the inequality that follows; the inequality only provides an upper bound on one spectral radius, not order equivalence in both directions.
  6. [Appendix B, Theorem B.5] The proof applies Tilli's theorem for finite block-Toeplitz matrices, but the finite change-of-basis matrices have boundary rows that are not exact Toeplitz; the authors should justify that the theorem's hypotheses cover their finite matrices or provide a direct norm bound for the banded matrices.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central KAN/MLP equivalence is an algebraic identity grounded in external spline theory, not in the paper's own fitted or cited prior results.

full rationale

The paper's load-bearing claim is the change-of-basis identity in Section 3: with W = A fW, gradient descent in spline weights is exactly preconditioned gradient descent in ReLU weights with preconditioner AA^T (Eqs. 12–14). This is derived from the basis equivalence in Lemma 2.4, which is cited to Chui (1988) and De Boor (1978), i.e., independent external spline theory. Proposition 3.1, on conditioning of Gram matrices, rests on Hong et al. (2022), Unser (1999), De Boor (1978), and Goel et al. (2017), not on the authors' prior work. The hierarchical refinement scheme follows from the exact interpolation identity S_r(T) subset S_r(T') for nested knots (Eqs. 20–21); this is a standard multigrid restriction/prolongation property, and the paper tests it against coarse and fine baselines in Tables 1–3 rather than reporting a fitted quantity as a prediction. Section 5's free-knot parameterization does cite Actor et al. (2024) for the softmax-distance trick, but that is a technical construction device, not a justification of the central equivalence, and the equivalence itself does not assume uniform knots. The paper explicitly concedes that the deep-network conditioning statement is not rigorously proven ("Rigorously proving this statement is difficult..."), which is an acknowledged limitation and a potential correctness concern, but not circularity. The claimed NTK bound in Theorem 3.2 may have a scaling issue in Appendix B, but an invalid or incomplete proof is a correctness deficit, not a reduction of the result to its own inputs. No equation in the paper defines its conclusion in terms of its premise, no fitted parameter is renamed as a prediction, and the self-citations are supportive technical references rather than load-bearing authority for the main contribution.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central equivalence uses standard B-spline and ReLU basis theory, plus the Gram-conditioning results of Hong et al. The main unreserved step is the extension of layer-wise linear preconditioning to deep nonlinear training, which the paper states but does not prove. No new entities are introduced, and no fitted constants are needed for the core derivation.

assumptions (5)
  • standard math The B-spline basis and the truncated power basis ReLU(· - t_i)^{r-1} span the same spline space (Lemma 2.4, citing Chui 1988).
    Invoked in Section 2 to establish the equivalence between KAN layers and multichannel MLP layers.
  • standard math For non-degenerate knot spacing, B-spline Gram matrices have condition number bounded independent of knot count, while ReLU^{r-1} Gram matrices are unbounded (Proposition 3.1, citing Hong et al. 2022, Unser 1999, De Boor 1978).
    Used in Section 3 to argue that the spline basis preconditioning improves conditioning of the least-squares objective.
  • standard math Uniform-knot change-of-basis matrices are Toeplitz, and their spectral norms can be bounded through the generating function maximum (Theorem B.5, citing Tilli 1998).
    Used in Appendix B for the NTK spectral radius bound; this is the proof point where the paper's scaling appears inconsistent.
  • domain assumption Layer-wise preconditioning, established for a linear least-squares problem, transfers to accelerate training of deep nonlinear networks.
    Stated at the end of Section 3; the authors note a rigorous proof is difficult, and only three small benchmarks support it.
  • standard math For nested knots T subset T', S_r(T) subset S_r(T'), so exact refinement via restriction and prolongation operators is possible (Section 4.1).
    Basis of the multilevel KAN refinement scheme; standard spline nesting property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement." pith.science (2026). https://pith.science/paper/CZ7W2JNW

@misc{pith2026250518131,
  author       = {Pith},
  title        = {Pith review of: Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZ7W2JNW}},
  note         = {Machine review of arXiv:2505.18131}
}
abstract

Multilayer perceptrons (MLPs) are a workhorse machine learning architecture, used in a variety of modern deep learning frameworks. However, recently Kolmogorov-Arnold Networks (KANs) have become increasingly popular due to their success on a range of problems, particularly for scientific machine learning tasks. In this paper, we exploit the relationship between KANs and multichannel MLPs to gain structural insight into how to train MLPs faster. We demonstrate the KAN basis (1) provides geometric localized support, and (2) acts as a preconditioned descent in the ReLU basis, overall resulting in expedited training and improved accuracy. Our results show the equivalence between free-knot spline KAN architectures, and a class of MLPs that are refined geometrically along the channel dimension of each weight tensor. We exploit this structural equivalence to define a hierarchical refinement scheme that dramatically accelerates training of the multi-channel MLP architecture. We show further accuracy improvements can be had by allowing the $1$D locations of the spline knots to be trained simultaneously with the weights. These advances are demonstrated on a range of benchmark examples for regression and scientific machine learning.

Figures

Figures reproduced from arXiv: 2505.18131 by the authors.

Figure 1
Figure 1. Example of the learned separating hyperplanes for three different architecture choices: [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of ReLU MLP hyperplanes against the KAN spline grid corresponding to two [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Select convergence history for the nonsmooth and XOR regression examples under [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 7 canonical work pages

  1. [1]

    W., Pantidis, P., and Mobasher, M

    Abueidda, D. W., Pantidis, P., and Mobasher, M. E. Deepokan: Deep operator network based on kolmogorov arnold networks for mechanics problems.arXiv preprint arXiv:2405.19143,

    arXiv

  2. [2]

    (38) We proceed now to the inductive step forr≥2

    satisfies Equation (33), A[2] ij = 1 ti+1 −t i A[1] ij − 1 ti+2 −t i+1 A[1] i+1,j. (38) We proceed now to the inductive step forr≥2 . Suppose the result holds through s= 1, . . . , r−1. By the recurrence relation to define the spline functionsb [r] i (5), we have b[r] i (x) = x−t i ti+r−1 −t i b[r−1] i (x) + ti+r −x ti+r −t i+1 b[r−1] i+1 (x) = 1 ti+r−1 −...

    work page 1998

  3. [6]

    Can kan work? exploring the potential of kolmogorov-arnold networks in computer vision.arXiv preprint arXiv:2411.06727,

    10 Cang, Y ., Shi, L., et al. Can kan work? exploring the potential of kolmogorov-arnold networks in computer vision.arXiv preprint arXiv:2411.06727,

    arXiv

  4. [7]

    Demonstrating the efficacy of kolmogorov-arnold networks in vision tasks.arXiv preprint arXiv:2406.14916,

    Cheon, M. Demonstrating the efficacy of kolmogorov-arnold networks in vision tasks.arXiv preprint arXiv:2406.14916,

    arXiv

  5. [10]

    Detecting Dead Weights and Units in Neural Networks

    Evci, U. Detecting dead weights and units in neural networks.arXiv preprint arXiv:1806.06068,

    work page Pith review arXiv

  6. [11]

    and Tan, V

    Gao, Y . and Tan, V . Y . On the convergence of (stochastic) gradient descent for kolmogorov–arnold networks.arXiv preprint arXiv:2410.08041,

    arXiv

  7. [16]

    A., Jacob, B., Murphy, S

    Howard, A. A., Jacob, B., Murphy, S. H., Heinlein, A., and Stinis, P. Finite basis kolmogorov-arnold networks: domain decomposition for data-driven and physics-informed problems.arXiv preprint arXiv:2406.19662, 2024a. Howard, A. A., Jacob, B., and Stinis, P. Multifidelity kolmogorov-arnold networks.arXiv preprint arXiv:2410.14764, 2024b. Igelnik, B. and P...

    arXiv

  8. [18]

    Gkan: Graph kolmogorov-arnold networks.arXiv preprint arXiv:2406.06470,

    11 Kiamari, M., Kiamari, M., and Krishnamachari, B. Gkan: Graph kolmogorov-arnold networks.arXiv preprint arXiv:2406.06470,

    arXiv

Show all 37 references
  1. [23]

    So, C. C. and Yung, S. P. Higher-order-relu-kans (hrkans) for solving physics-informed neural networks (pinns) more accurately, robustly and faster.arXiv preprint arXiv:2409.14248,

    arXiv

  2. [24]

    A., Islam, M

    Somvanshi, S., Javed, S. A., Islam, M. M., Pandit, D., and Das, S. A survey on kolmogorov-arnold network.arXiv preprint arXiv:2411.06078,

    arXiv

  3. [27]

    J., Blanco, L., Pereira, R., and Caus, M

    Vaca-Rubio, C. J., Blanco, L., Pereira, R., and Caus, M. Kolmogorov-arnold networks (kans) for time series analysis.arXiv preprint arXiv:2405.08790,

  4. [30]

    K., Li, B., and Perdikaris, P

    Wang, S., Bhartari, A. K., Li, B., and Perdikaris, P. Gradient alignment in physics-informed neural networks: A second-order optimization perspective.arXiv preprint arXiv:2502.00604,

  5. [31]

    doi: 10.1016/j.inffus

    ISSN 1566-2535. doi: 10.1016/j.inffus. 2024.102344. URLhttp://dx.doi.org/10.1016/j.inffus.2024.102344. Yu, R., Yu, W., and Wang, X. Kan or mlp: A fairer comparison.arXiv preprint arXiv:2407.16674,

    2024

  6. [32]

    Kan versus mlp on irregular or noisy functions.arXiv preprint arXiv:2408.07906,

    Zeng, C., Wang, J., Shen, H., and Wang, Q. Kan versus mlp on irregular or noisy functions.arXiv preprint arXiv:2408.07906,

    arXiv

  7. [33]

    We define the ReLUr−1 basis functions as ψ[r] i (x) =ReLU(x−t i)r−1.(30) Define A[r] ∈R (n+r−1)×(n+r−1) as the change-of-basis matrix between B[r] = h b[r] 1−r,

    i (x) = 1x≥t i 0else .(28) Forr >1, we define the spline basis function recursively as before: b[r] i (x) = x−t i ti+r−1 −t i b[r−1] i (x) + ti+r −x ti+r −t i+1 b[r−1] i+1 (x).(29) Each functionb [r] i is supported on the interval[t i, ti+r]. We define the ReLUr−1 basis functi...

    2022

  8. [35]

    generating function of eA[r] given by Fr(x) =r rX ℓ=0 (−1)ℓ (ℓ)! (r−ℓ)! e−iℓx =r· (1−e ix)r r! =n r−1 (1−e ix)r (r−1)! .(49) Thus we have forr∈Z +, ∥ eA[r]∥ ≤max x |Fr(x)|= 2r (r−1)! ≤4forr≥1,(50) which completes the proof. 17 C Practical Considerations C.1 Domain Mismatch of ...

    2015

  9. [36]

    and elsewhere. This issue was considered early in KAN literature, and many solutions have arisen, including adaptive spline knot schemes (Liu et al., 2024b,a) and batch / layer normalization (Ioffe & Szegedy, 2015; Ba et al., 2016). Many adaptive spline knot schemes weight gri...

    2015

  10. [37]

    and Chebyshev polynomial bases (SS et al., 2024; Guo et al., 2024; Shukla et al., 2024). In particular, when using Chebyshev polynomials as a basis for KANs, the use of function compositions does not enrich the approximation space: the composition of polynomial functions remai...

    2024

  11. [1958]

    D., Wang, Z., Zou, Z., and Karniadakis, G

    Shukla, K., Toscano, J. D., Wang, Z., Zou, Z., and Karniadakis, G. E. A comprehensive and fair comparison between mlp and kan representations for differential equations and operator networks. arXiv preprint arXiv:2406.02917,

    arXiv

  12. [1959]

    K., Arya, M., Jacobs, K., Bondar, D

    Patra, S., Panda, S., Parida, B. K., Arya, M., Jacobs, K., Bondar, D. I., and Sen, A. Physics informed kolmogorov-arnold neural networks for dynamical analysis via efficent-kan and wav-kan.arXiv preprint arXiv:2407.18373,

    arXiv

  13. [1988]

    Fast and accurate deep network learning by exponential linear units (elus).arXiv preprint arXiv:1511.07289,

    Clevert, D.-A. Fast and accurate deep network learning by exponential linear units (elus).arXiv preprint arXiv:1511.07289,

    arXiv

  14. [1993]

    Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation.arXiv preprint arXiv:2405.07200,

    SS, S., AR, K., KP, A., et al. Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation.arXiv preprint arXiv:2405.07200,

    arXiv

  15. [1998]

    D., Oommen, V ., Varghese, A

    Toscano, J. D., Oommen, V ., Varghese, A. J., Zou, Z., Daryakenari, N. A., Wu, C., and Karniadakis, G. E. From pinns to pikans: Recent advances in physics-informed machine learning.arXiv preprint arXiv:2410.13228,

    arXiv

  16. [1999]

    Relu-kan: New kolmogorov-arnold networks that only need matrix addition, dot multiplication, and relu.arXiv preprint arXiv:2406.02075,

    Qiu, Q., Zhu, T., Gong, H., Chen, L., and Ning, H. Relu-kan: New kolmogorov-arnold networks that only need matrix addition, dot multiplication, and relu.arXiv preprint arXiv:2406.02075,

    arXiv

  17. [2003]

    W., Tan, Q., and Xu, J

    Hong, Q., Siegel, J. W., Tan, Q., and Xu, J. On the activation function dependence of the spectral bias of neural networks.arXiv preprint arXiv:2208.04924,

    arXiv

  18. [2004]

    An expert’s guide to training physics-informed neural networks.arXiv preprint arXiv:2308.08468,

    Wang, S., Sankaran, S., Wang, H., and Perdikaris, P. An expert’s guide to training physics-informed neural networks.arXiv preprint arXiv:2308.08468,

    arXiv

  19. [2009]

    Kagnns: Kolmogorov-arnold networks meet graph learning.arXiv preprint arXiv:2406.18380,

    Bresson, R., Nikolentzos, G., Panagopoulos, G., Chatzianastasis, M., Pang, J., and Vazirgiannis, M. Kagnns: Kolmogorov-arnold networks meet graph learning.arXiv preprint arXiv:2406.18380,

    arXiv

  20. [2013]

    Kan 2.0: Kolmogorov-arnold networks meet science.arXiv preprint arXiv:2408.10205, 2024a

    Liu, Z., Ma, P., Wang, Y ., Matusik, W., and Tegmark, M. Kan 2.0: Kolmogorov-arnold networks meet science.arXiv preprint arXiv:2408.10205, 2024a. Liu, Z., Wang, Y ., Vaidya, S., Ruehle, F., Halverson, J., Soljaˇci´c, M., Hou, T. Y ., and Tegmark, M. Kan: Kolmogorov-arnold netw...

    arXiv

  21. [2015]

    A., and Stinis, P

    Jacob, B., Howard, A. A., and Stinis, P. Spikans: Separable physics-informed kolmogorov-arnold networks.arXiv preprint arXiv:2411.06286,

    arXiv

  22. [2016]

    and Chen, H

    Bozorgasl, Z. and Chen, H. Wav-kan: Wavelet kolmogorov-arnold networks.arXiv preprint arXiv:2405.12832,

    arXiv

  23. [2017]

    Physics-informed kolmogorov-arnold network with chebyshev polynomials for fluid mechanics.arXiv preprint arXiv:2411.04516,

    Guo, C., Sun, L., Li, S., Yuan, Z., and Wang, C. Physics-informed kolmogorov-arnold network with chebyshev polynomials for fluid mechanics.arXiv preprint arXiv:2411.04516,

    arXiv

  24. [2018]

    L., Kiros, J

    Ba, J. L., Kiros, J. R., and Hinton, G. E. Layer normalization.arXiv preprint arXiv:1607.06450,

    arXiv

  25. [2020]

    Mlp-kan: Unifying deep representation and function learning

    He, Y ., Xie, Y ., Yuan, Z., and Sun, L. Mlp-kan: Unifying deep representation and function learning. arXiv preprint arXiv:2410.03027,

    arXiv

  26. [2021]

    Kolmogorov-arnold networks (kan) for time series classification and robust analysis.arXiv preprint arXiv:2408.07314,

    Dong, C., Zheng, L., and Chen, W. Kolmogorov-arnold networks (kan) for time series classification and robust analysis.arXiv preprint arXiv:2408.07314,

    arXiv

  27. [2022]

    and Zhang, D

    Hou, Y . and Zhang, D. A comprehensive survey on kolmogorov arnold networks (kan).arXiv preprint arXiv:2407.11075,

    arXiv

  28. [2023]

    Piratenets: Physics-informed deep learning with residual adaptive networks.arXiv preprint arXiv:2402.00326,

    Wang, S., Li, B., Chen, Y ., and Perdikaris, P. Piratenets: Physics-informed deep learning with residual adaptive networks.arXiv preprint arXiv:2402.00326,

    arXiv

  29. [2024]

    A preliminary study on continual learning in computer vision using kolmogorov-arnold networks.arXiv preprint arXiv:2409.13550,

    Cacciatore, A., Morelli, V ., Paganica, F., Frontoni, E., Migliorelli, L., and Berardini, D. A preliminary study on continual learning in computer vision using kolmogorov-arnold networks.arXiv preprint arXiv:2409.13550,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.