REVIEW 6 cited by
On the expressiveness and spectral bias of KANs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Kolmogorov-Arnold Networks (KAN) \cite{liu2024kan} were very recently proposed as a potential alternative to the prevalent architectural backbone of many deep learning models, the multi-layer perceptron (MLP). KANs have seen success in various tasks of AI for science, with their empirical efficiency and accuracy demostrated in function regression, PDE solving, and many more scientific problems. In this article, we revisit the comparison of KANs and MLPs, with emphasis on a theoretical perspective. On the one hand, we compare the representation and approximation capabilities of KANs and MLPs. We establish that MLPs can be represented using KANs of a comparable size. This shows that the approximation and representation capabilities of KANs are at least as good as MLPs. Conversely, we show that KANs can be represented using MLPs, but that in this representation the number of parameters increases by a factor of the KAN grid size. This suggests that KANs with a large grid size may be more efficient than MLPs at approximating certain functions. On the other hand, from the perspective of learning and optimization, we study the spectral bias of KANs compared with MLPs. We demonstrate that KANs are less biased toward low frequencies than MLPs. We highlight that the multi-level learning feature specific to KANs, i.e. grid extension of splines, improves the learning process for high-frequency components. Detailed comparisons with different choices of depth, width, and grid sizes of KANs are made, shedding some light on how to choose the hyperparameters in practice.
Forward citations
Cited by 6 Pith papers
-
KANEL\'E: Kolmogorov-Arnold Networks for Efficient LUT-based Evaluation
Quantized, pruned Kolmogorov-Arnold Networks can be compiled directly into FPGA lookup tables, achieving extreme latency/resource reductions and matching state-of-the-art LUT-based networks on several benchmarks.
-
Fast, accurate, and differentiable: a neural-network surrogate for NRSur7dq4 precessing binary black hole waveforms
A piecewise MLP surrogate emulates NRSur7dq4 over its full domain at NR-faithful accuracy with ~1 ms GPU latency and a fully differentiable JAX likelihood pipeline.
-
Kolmogorov-Arnold Network for Gene Regulatory Network Inference
scKAN uses Kolmogorov-Arnold networks in a one-vs-rest regression and treats model gradients as signed gene regulation strengths, outperforming baselines on several BEELINE benchmark tasks.
-
Improving Memory Efficiency for Training KANs via Meta Learning
MetaKANs generates each KAN activation function from a shared prompt-conditioned meta-learner, cutting trainable parameters toward MLP level while retaining comparable or better accuracy on tested benchmarks.
-
Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networks
A low tensor-rank adaptation (LoTRA) method and learning-rate guidance enable efficient fine-tuning of Kolmogorov-Arnold networks, validated on PDE solving and representation tasks.
-
PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEs
PG-KINN pairs a KAN trial space with a Petrov–Galerkin test space for forward and inverse PDEs, but the inverse benchmark data is inconsistent with the governing equation and the accuracy claims are not supported by t...
Discussion (0). Continue with ORCID to comment.