Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies
Reviewed by Pithpith:4WKRZJFHopen to challenge →
read the original abstract
The emergence of Kolmogorov-Arnold Networks (KANs) has sparked significant interest and debate within the scientific community. This paper explores the application of KANs in the domain of computer vision (CV). We examine the convolutional version of KANs, considering various nonlinearity options beyond splines, such as Wavelet transforms and a range of polynomials. We propose a parameter-efficient design for Kolmogorov-Arnold convolutional layers and a parameter-efficient finetuning algorithm for pre-trained KAN models, as well as KAN convolutional versions of self-attention and focal modulation layers. We provide empirical evaluations conducted on MNIST, CIFAR10, CIFAR100, Tiny ImageNet, ImageNet1k, and HAM10000 datasets for image classification tasks. Additionally, we explore segmentation tasks, proposing U-Net-like architectures with KAN convolutions, and achieving state-of-the-art results on BUSI, GlaS, and CVC datasets. We summarized all of our findings in a preliminary design guide of KAN convolutional models for computer vision tasks. Furthermore, we investigate regularization techniques for KANs. All experimental code and implementations of convolutional layers and models, pre-trained on ImageNet1k weights are available on GitHub via this https://github.com/IvanDrokin/torch-conv-kan
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition
KAConvNet introduces a Kolmogorov-Arnold Convolutional Layer to build networks competitive with ViTs and CNNs while offering stronger theoretical interpretability.
-
Structural Kolmogorov-Arnold Convolutions: Learnable Function on the Values or the Filter Shape as Parameter-Efficient Alternative to Per-Edge Convolutional KANs
Structural KAN convolutions with shared value functions or wavelet-based adaptive filter shapes match or exceed per-edge KAN accuracy on CIFAR at 0.4M parameters.
-
KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognition
A hybrid KAN-MLP model for IMU-based human activity recognition achieves 5.33% relative macro F1 improvement over pure MLPs on eight datasets by placing KANs at input embedding and classification stages.
-
Light-ResKAN: A Parameter-Sharing Lightweight KAN with Gram Polynomials for Efficient SAR Image Recognition
Light-ResKAN reaches 99.09% accuracy on MSTAR SAR images with 82.9 times fewer FLOPs and 163.78 times fewer parameters than VGG16 by combining KAN convolutions, Gram polynomials, and channel-wise parameter sharing.
-
KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognition
A hybrid KAN-MLP architecture with KAN input embedding and specialized LarctanKAN classification layer yields 5.33% average macro F1 gain over pure-MLP baselines in IMU-based human activity recognition.
-
GroupKAN: Efficient Kolmogorov-Arnold Networks via Grouped Spline Modeling
GroupKAN reduces KAN parameter scaling via intra-group spline mappings, delivering 79.80% average IoU (+1.11% over U-KAN) at 47.6% of the parameters on BUSI, GlaS, and CVC datasets.
-
Interpretable Clinical Classification with Kolmogorov-Arnold Networks
Logistic KAN and KAAM achieve competitive or superior accuracy on clinical datasets compared to linear, tree, and neural baselines while providing built-in interpretability via symbolic forms and feature-wise decompositions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.