Pith. sign in

REVIEW 4 major objections 6 minor 57 references

Hyb-KAN ViT: Hybrid Kolmogorov-Arnold Networks Augmented Vision Transformer

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that replacing a Vision Transformer's MLP layers with hybrid wavelet- and spline-based KAN layers yields state-of-the-art accuracy on ImageNet-1K, COCO, and ADE20K, including 86.3% top-1 for Hyb-KAN ViT-B.

desk verdict Hybrid KAN-ViT idea is reasonable, but unsupported SOTA claims and wrong wavelet gradients make this a desk reject. read the letter →

arxiv 2505.04740 v1 pith:67XZXAD7 submitted 2025-05-07 cs.CV

Sainath Dey , Mitul Goswami , Jashika Sethi , Prasant Kumar Pattnaik This is my paper · ORCID
classification cs.CV
keywords visiontransformerKolmogorov-ArnoldnetworkwavelettransformB-splineimageclassificationobjectdetectionsemanticsegmentationhybridarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that replacing the multi-layer perceptron (MLP) blocks inside a Vision Transformer with Kolmogorov-Arnold Network (KAN) blocks yields higher accuracy and better parameter efficiency. It proposes two modular replacements: Eff-KAN, which uses learnable B-spline activations, and Wav-KAN, which applies orthogonal wavelet transforms for multi-resolution feature extraction. On ImageNet-1K the best hybrid, Hyb-KAN ViT-B, is reported to reach 86.3% top-1 accuracy with 113.6 million parameters, and the paper also reports state-of-the-art or competitive results on COCO object detection and ADE20K semantic segmentation, with standalone Wav-KAN leading segmentation at 52.3 mIoU. If correct, the central insight is that wavelet-driven spectral priors in early encoder layers and spline-based projection in the classification head make KAN blocks a practical substitute for MLPs. Because the design is modular, established ViT architectures could adopt the replacement without redesigning the attention mechanism.

What carries the argument

The load-bearing mathematical basis is the Kolmogorov-Arnold representation theorem, which expresses a multivariate function as a sum of univariate functions; KAN layers implement that idea with learnable univariate basis functions. Eff-KAN uses B-spline basis functions on each connection, with trainable coefficients and a base linear weight, so spline knot locations can adapt to sharp image features like edges. Wav-KAN applies a fast wavelet transform to separate the input into frequency bands, multiplies the coefficients by trainable scale scalars, prunes low-magnitude high-frequency bands, and reconstructs with the inverse transform, using one of three wavelet families: Derivative of Gaussian, Mexican Hat, or Morlet. The framework's central move is treating these blocks as drop-in replacements for the MLP in both the transformer encoder and the classification head, and combining them as Hybrid-1, a Wav-KAN encoder plus an Eff-KAN head. The derivative-of-Gaussian wavelet is the one that empirically carries the classification and detection gains.

What would settle it

Symbolically or automatically differentiate the three wavelet formulas in Eqs. (5.1), (6.1), and (7.1) with respect to $\sigma$, and with respect to $w_0$ for Morlet, and compare the results with Eqs. (5.2), (6.2), (7.2), and (7.3) at sample values of $(x, \sigma, \tau, w_0)$; any discrepancy beyond floating-point error would invalidate the stated training procedure and require the benchmark claims to be rechecked.

Watch

Extended reading notes

Core claim

Hyb-KAN ViT is a Vision Transformer whose encoder MLPs are switched for Wav-KAN blocks and whose classification head uses Eff-KAN blocks. The paper's core claim is that this combination captures both high-frequency edge and texture structure, through wavelet decomposition, and smooth adaptive decision boundaries, through B-spline activations, in a way fixed MLPs cannot. The reported numbers are the evidence: 84.5% top-1 accuracy for the Small model and 86.3% for the Base model on ImageNet-1K, box and mask average precision of 50.6 and 44.9 for the Base hybrid on COCO, and 52.3 mIoU for the standalone Wav-KAN ViT-B on ADE20K. The authors interpret these results as showing a dual-path spectral encoding: wavelet-guided attention extracts edges in early layers, while spline heads refine spatial-semantic fusion later. They also identify a scaling limitation, noting that Eff-KAN ViT-B grows to 162.4 million parameters for a smaller accuracy gain, which is why the hybrid configuration is the recommended one.

Load-bearing premise

The load-bearing premise is that the handwritten derivatives of the three wavelet basis functions with respect to scale and center frequency, Eqs. (5.2), (6.2), (7.2), and (7.3), are mathematically correct; the paper provides no proof or code for them, and training the scale parameters depends on them.

Editorial extensions

If this is right

  • If the reported results are correct, KAN-based nonlinearities are a practical replacement for MLPs inside Vision Transformers, not just an isolated module, and can lift top-1 accuracy above comparable MLP-based ViTs at Base scale.
  • The 86.3% top-1 figure for Hyb-KAN ViT-B would put it ahead of the listed ViT-B, DeiT-B, and PVTv2-B5 baselines, implying that wavelet-KAN encoders provide a stronger inductive bias for ImageNet-scale training than standard fixed activations.
  • The ADE20K result implies that wavelet frequency decomposition is especially valuable for dense prediction tasks requiring precise boundaries, since Wav-KAN ViT-B reports 52.3 mIoU, above the listed Swin-B and ConvNeXt-B scores.
  • The ablation parameter counts, with Hyb-KAN ViT-S at 29.7 million parameters versus 42.5 million for Eff-KAN ViT-S, imply that the wavelet encoder is the parameter-efficient part of the design while the spline head adds accuracy without the full KAN parameter cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to freeze the wavelet scale parameters at their initial values and retrain; if accuracy stays roughly the same, the wavelet family matters less than the added representation capacity, and if it drops substantially, the trainable scale parameter is the hidden engine of the gains.
  • The reported DoG advantage over Morlet and Mexican Hat suggests that first-derivative edge-like wavelets match patch-based image statistics; testing higher-order derivatives of Gaussians would show whether the derivative order can be pushed further.
  • The paper's conclusion proposes sharing activation weights across neuron groups to cut parameters; a cheap check is whether such weight sharing preserves the accuracy margin at Tiny scale, which would separate the KAN architecture's benefit from its parameter count.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Hyb-KAN ViT, a family of Vision Transformer variants in which MLP blocks are replaced by two KAN-based modules: Efficient-KAN (spline-based) and Wavelet-KAN (wavelet-based). The central claim is state-of-the-art performance on ImageNet-1K (86.3% top-1 for Hyb-KAN ViT-B, Table 4), COCO object detection and instance segmentation (Table 5), and ADE20K semantic segmentation (52.3 mIoU for Wav-KAN ViT-B, Table 6). The paper gives algorithmic pseudocode, derivative formulas for wavelet scale parameters, ablations, and comparisons to prior CNN/ViT backbones. No code, checkpoints, training logs, or compute budget are provided.

Significance. If the reported results were reproducible, the hybrid KAN/wavelet design would be an interesting contribution to the accuracy-efficiency trade-off of Vision Transformers, and the modular encoder/head decomposition is a sensible organizing principle. The paper deserves credit for explicitly comparing several wavelet families, reporting parameter and FLOP counts, and including ablation experiments. However, the headline claims are empirical and the paper provides no artifacts for verification; in addition, some of the derivative formulas used in the described training procedure are mathematically incorrect. As submitted, the central empirical claims are unsupported.

major comments (4)
  1. [Section 3.3, Eqs. (5.2), (6.2), (7.3)] The gradient formulas are not correct. For the DoG wavelet with m=0, evaluating both sides of Eq. (5.2) at x=τ gives ∂Ψ/∂σ = 0 on the left and -1 on the right. For the Mexican-hat wavelet at x=τ, σ=1, the left side of Eq. (6.2) equals -1/2 while the right side equals -3/2. For the Morlet wavelet, Eq. (7.2) is correct, but Eq. (7.3) has a sign error: differentiating cos(ω0(x-τ)/σ) with respect to ω0 gives a negative term. Since these derivatives are the mechanism by which scale and frequency parameters are optimized in Algorithm 3, the training procedure described in Section IV cannot be relied upon to produce the reported results.
  2. [Section V, Tables 5 and 6] The model labels are internally inconsistent. In the Base segment of Table 5, rows are labeled 'Eff-KAN ViT-S (Ours)', 'Wav-KAN ViT-S (Ours)', and 'Hyb-KAN ViT-S (Ours)' with 191M, 136M, and 140M parameters, whereas Table 4 reports the corresponding Base variants as 162.4M, 109.4M, and 113.6M. The Base column of Table 6 similarly lists 'Eff-KAN ViT-S (Ours)' and 'Wav-KAN ViT-S (Ours)' rows. Because the parameter counts do not match either the 'S' or 'B' labels from Table 4, the claimed margins over Swin-B and ViT-Adapter-B in the text cannot be verified.
  3. [Section V, Tables 3-6, and Section 4] No code, checkpoints, training logs, or external evaluation protocols are provided, so the state-of-the-art accuracy (86.3% on ImageNet), detection AP, and segmentation mIoU values are not independently verifiable. The paper also omits the compute budget (GPU-days) and the absolute throughput/memory numbers behind Fig. 3, leaving the central efficiency claims unsupported.
  4. [Section 3.3, Fig. 3] The text after Algorithm 3 claims 'up to 4× FLOPs reduction compared to vanilla Wav-KANs', but no FLOPs measurement protocol or comparison table is given, and Fig. 3 gives no numerical axis labels or values. This efficiency claim is therefore not substantiated.
minor comments (6)
  1. [Abstract] The sentence 'prior work has failed to focus on the prebuilt modularity of the ViT architecture and integration of edge detection capabilities of Wavelet functions' is a sentence fragment and should be revised.
  2. [Section II] There are two subsections numbered 2.1 ('Convolutional Neural Network Backbones' and 'Vision Transformer Backbones'); the second should be numbered 2.2.
  3. [References] Reference [19] duplicates reference [24] (both are Touvron et al. on training data-efficient image transformers).
  4. [Equation (2)] The summation in Eq. (2), written as ∑ ... k, has a misplaced summation index; it should be explicit in the notation.
  5. [Fig. 3] The axes of Fig. 3 are labeled only as 'Input Dimensions Sizes' without units or tick values, and the figure caption does not state what is being measured.
  6. [Section 5.2] The opening sentence 'Table. 4. Demonstrates that, o ur KAN-augmented Vision Transformers demonstrate compelling trade-offs' contains spacing and capitalization errors.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical benchmark measurements, not derivations that reduce to their own inputs.

full rationale

The paper makes no first-principles prediction whose outcome is built into the inputs. Its central claims are measured accuracies, AP scores, and mIoU values reported in Tables 3 through 6, produced by training the proposed architectures on ImageNet-1K, COCO, and ADE20K under stated protocols. The only elements that could resemble circularity are the reliance on prior KAN implementations [4], [6], [34] and the wavelet gradient formulas in Eqs. (5.2), (6.2), (7.2), and (7.3). The prior works are external building blocks, not self-citations by the present authors, and they are used as components to be benchmarked rather than as authority for the reported results. The gradient formulas are mathematical claims about the training procedure; even if they are wrong or unverified, that is a correctness or reproducibility defect, not a reduction of the reported outcomes to the paper's own definitions. The ablation study compares model variants empirically, and selecting a better-performing variant afterward is model selection, not circular reasoning. No fitted parameter is relabeled as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation. Therefore no specific circular step can be quoted, and the circularity score is 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method depends on a small set of hand-chosen hyperparameters and on the questionable assumption that continuous wavelets behave as orthogonal bases. The most fragile point is the set of derivative formulas, which the paper states without proof and which appear to contain errors (e.g., the spurious -1 term in Eq. 5.2).

free parameters (5)
  • Wavelet pruning ratio rho = 0.4
    Chosen in Table 1; directly controls how many high-frequency wavelet coefficients are removed in Wav-KAN.
  • Number of wavelet scales = 6
    Sets multi-resolution bandwidths in Wav-KAN (Table 1).
  • Decomposition levels = 4
    Depth of wavelet decomposition used in Algorithm 3 (Table 1).
  • Grid size for Eff-KAN = 5
    Number of knots for B-spline approximations in Eff-KAN (Table 1).
  • Spline order = 3
    Polynomial smoothness of the splines in Eff-KAN (Table 1).
assumptions (3)
  • standard math B-spline basis functions and wavelet transforms are well-defined and differentiable as background mathematics.
    Used in Eqs. (2)-(4) and Algorithms 2 and 3.
  • domain assumption The wavelets DoG, Mexican Hat, and Morlet provide orthogonal multi-resolution decompositions.
    The abstract and Section 3.3 claim orthogonal wavelet transforms, but these continuous wavelets are not orthogonal bases, so the assumption is questionable.
  • ad hoc to paper The derivative formulas in Eqs. (5.2), (6.2), (7.2), and (7.3) correctly compute gradients for scale parameters.
    The paper asserts these formulas are essential for gradient-based training but provides no derivation, and they appear to contain errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hyb-KAN ViT: Hybrid Kolmogorov-Arnold Networks Augmented Vision Transformer." pith.science (2026). https://pith.science/paper/67XZXAD7

@misc{pith2026250504740,
  author       = {Pith},
  title        = {Pith review of: Hyb-KAN ViT: Hybrid Kolmogorov-Arnold Networks Augmented Vision Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/67XZXAD7}},
  note         = {Machine review of arXiv:2505.04740}
}
read the original abstract

This study addresses the inherent limitations of Multi-Layer Perceptrons (MLPs) in Vision Transformers (ViTs) by introducing Hybrid Kolmogorov-Arnold Network (KAN)-ViT (Hyb-KAN ViT), a novel framework that integrates wavelet-based spectral decomposition and spline-optimized activation functions, prior work has failed to focus on the prebuilt modularity of the ViT architecture and integration of edge detection capabilities of Wavelet functions. We propose two key modules: Efficient-KAN (Eff-KAN), which replaces MLP layers with spline functions and Wavelet-KAN (Wav-KAN), leveraging orthogonal wavelet transforms for multi-resolution feature extraction. These modules are systematically integrated in ViT encoder layers and classification heads to enhance spatial-frequency modeling while mitigating computational bottlenecks. Experiments on ImageNet-1K (Image Recognition), COCO (Object Detection and Instance Segmentation), and ADE20K (Semantic Segmentation) demonstrate state-of-the-art performance with Hyb-KAN ViT. Ablation studies validate the efficacy of wavelet-driven spectral priors in segmentation and spline-based efficiency in detection tasks. The framework establishes a new paradigm for balancing parameter efficiency and multi-scale representation in vision architectures.

Figures

Figures reproduced from arXiv: 2505.04740 by the authors.

Figure 1
Figure 1. The illustration of the detailed architecture of our proposed Vision Transformer 𝑦 = 𝑊2 × 𝐺𝑒𝐿𝑈(𝑊1𝑥 + 𝑏1) +𝑏2 (1) The first layer expands the feature dimension, while the second reduces it back, enabling the model to capture complex patterns. However, MLPs have limitations. Existing MLP-based methods, [2], [20], [21], [23], [24], and [25] increase the patch size to reduce the number of tokens, which results in lower￾… view at source ↗
Figure 2
Figure 2. The illustration of the detailed architecture of the (a) Eff-KAN Block and (b) Wav-KAN Block In Algorithm 2, 𝐶 (𝑙) , a 3D tensor storing k spline coefficients per connection to d nonlinear curves within the grid range [a,b], 𝑆 (𝑙) , a scaling matrix to dynamically adjust spline outputs, and 𝑏 (𝑙) , a bias vector [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 6
Figure 6. Accuracy vs Params vs GFLOPs compared across model variants [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 45 canonical work pages

  1. [1]

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73

    work page 2017

  2. [2]

    & Houlsby, N

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    arXiv 2020

  3. [3]

    T., Pham, H., & Manning, C

    Luong, M. T., Pham, H., & Manning, C. D. (2015). Effective approaches to attention -based neural machine translation. arXiv preprint arXiv:1508.04025

    arXiv 2015

  4. [4]

    (2024, September)

    Yang, X., & Wang, X. (2024, September). Kolmogorov -arnold transformer. In The Thirteenth International Conference on Learning Representations

    work page 2024

  5. [5]

    & Tegmark, M

    Liu, Z., Wang, Y., Vaidya, S., Ruehle, F., Halverson, J., Soljačić, M., ... & Tegmark, M. (2024). Kan: Kolmogorov -arnold networks. arXiv preprint arXiv:2404.19756

    arXiv 2024

  6. [6]

    Wav -kan: Wavelet kolmogorov -arnold networks, 2024

    Bozorgasl, Z., & Chen, H. Wav -kan: Wavelet kolmogorov -arnold networks, 2024. arXiv preprint arXiv:2405.12832

    arXiv 2024

  7. [7]

    ImageNet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105, 2012

    work page 2012

  8. [8]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in International Conference on Learning Representations, 2015

    work page 2015

Show all 57 references
  1. [9]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016

    2016

  2. [10]

    Going deeper with convolutions,

    C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 1–9, 2015

    2015

  3. [11]

    Rethinking the inception architecture for computer vision,

    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 2818–2826, 2016

    2016

  4. [12]

    Aggregated residual transformations for deep neural networks,

    S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, "Aggregated residual transformations for deep neural networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500, 2017

    2017

  5. [13]

    Densely connected convolutional networks,

    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708, 2017

    2017

  6. [14]

    Sparsely aggregated convolutional networks,

    L. Zhu, R. Deng, M. Maire, Z. Deng, G. Mori, and P. Tan, "Sparsely aggregated convolutional networks," in European Conference on Computer Vision, pp. 186–201, 2018

    2018

  7. [15]

    ShuffleNet: An extremely efficient convolutional neural network for mobile devices,

    X. Zhang, X. Zhou, M. Lin, and J. Sun, "ShuffleNet: An extremely efficient convolutional neural network for mobile devices," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856, 2018

    2018

  8. [16]

    MobileNets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard et al., "MobileNets: Efficient convolutional neural networks for mobile vision applications," arXiv:1704.04861, 2017

    2017 arXiv

  9. [17]

    EfficientNet: Rethinking model scaling for convolutional neural networks,

    M. Tan and Q. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks," in International Conference on Machine Learning, pp. 6105–6114, 2019

    2019

  10. [18]

    CSPNet: A new backbone that can enhance learning capability of CNN,

    C.-Y. Wang, H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I.- H. Yeh, "CSPNet: A new backbone that can enhance learning capability of CNN," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391, 2020

    2020

  11. [19]

    Training data -efficient image transformers & distillation through attention,

    H. Touvron et al., "Training data -efficient image transformers & distillation through attention," in International Conference on Machine Learning, pp. 10347–10357, 2021

    2021

  12. [20]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu et al., "Swin transformer: Hierarchical vision transformer using shifted windows," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022, 2021

    2021

  13. [21]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang et al., "Pyramid vision transformer: A versatile backbone for dense prediction without convolutions," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578, 2021

    2021

  14. [22]

    CvT: Introducing convolutions to vision transformers,

    H. Wu et al., "CvT: Introducing convolutions to vision transformers," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22–31, 2021

    2021

  15. [23]

    PVTv2: Improved baselines with pyramid vision transformer,

    W. Wang et al., "PVTv2: Improved baselines with pyramid vision transformer," Computational Visual Media , vol. 8, no. 3, pp. 415 –424, 2022

    2022

  16. [24]

    DeiT: Data -efficient image transformers,

    H. Touvron et al., "DeiT: Data -efficient image transformers," in International Conference on Machine Learning , pp. 10347 –10357, 2021

    2021

  17. [25]

    ConViT: Improving vision transformers with soft convolutional inductive biases,

    S. d'Ascoli et al., "ConViT: Improving vision transformers with soft convolutional inductive biases," in International Conference on Machine Learning, pp. 2286–2297, 2021

    2021

  18. [26]

    Going deeper with image transformers,

    H. Touvron et al., "Going deeper with image transformers," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 32–42, 2021

    2021

  19. [27]

    DeepViT: Towards deeper vision transformer,

    D. Zhou et al., "DeepViT: Towards deeper vision transformer," arXiv:2103.11886, 2021

    2021 arXiv

  20. [28]

    Tokens-to-token ViT: Training vision transformers from scratch on ImageNet,

    L. Yuan et al., "Tokens-to-token ViT: Training vision transformers from scratch on ImageNet," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 558–567, 2021

    2021

  21. [29]

    RegionViT: Regional -to-local attention for vision transformers,

    Q. Chen et al., "RegionViT: Regional -to-local attention for vision transformers," arXiv:2106.02689, 2021

    2021 arXiv

  22. [30]

    Dual vision transformer,

    Y. Xu et al., "Dual vision transformer," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  23. [31]

    MicroViT: A vision transformer with low complexity self attention for edge device,

    R. Mehta, "MicroViT: A vision transformer with low complexity self attention for edge device," arXiv:2502.05800, 2024

    2024 arXiv

  24. [32]

    MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer,

    S. Mehta and M. Rastegari, "MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer," in International Conference on Learning Representations, 2022

    2022

  25. [33]

    Le, T. X. H., Tran, T. D., Pham, H. L., Le, V. T. D., Vu, T. H., Nguyen, V. T., & Nakashima, Y. (2024, November). Exploring the limitations of kolmogorov-arnold networks in classification: Insights to software training and hardware implementation. In 2024 Twelfth International...

    2024

  26. [34]

    Blealtan. (2023). Efficient -KAN [Computer software]. GitHub. https://github.com/Blealtan/efficient-kan

    2023

  27. [35]

    D., Palafox, A., & Guerrero, E

    Meshir, J. D., Palafox, A., & Guerrero, E. A. (2025). On the study of frequency control and spectral bias in Wavelet -Based Kolmogorov Arnold networks: A path to physics -informed KANs. arXiv preprint arXiv:2502.00280

    2025 arXiv

  28. [36]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L. -J. Li, K. Li, and L. Fei -Fei, “Imagenet: A large-scale hierarchical image database,” in CVPR, 2009

    2009

  29. [37]

    Microsoft coco: Common objects in ´ context,

    T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, “Microsoft coco: Common objects in ´ context,” in ECCV, 2014

    2014

  30. [38]

    Semantic understanding of scenes through the ade20k dataset,

    B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,” IJCV, 2019

    2019

  31. [39]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  32. [40]

    Volo: Vision outlooker for visual recognition,

    L. Yuan, Q. Hou, Z. Jiang, J. Feng, and S. Yan, “Volo: Vision outlooker for visual recognition,” arXiv preprint arXiv:2106.13112, 2021

    2021 arXiv

  33. [41]

    Randaugment: Practical automated data augmentation with a reduced search space,

    E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, “Randaugment: Practical automated data augmentation with a reduced search space,” in CVPR Workshops, 2020

    2020

  34. [42]

    J., Chun, S., Choe, J., & Yoo, Y

    Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., & Yoo, Y. (2019). CutMix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2019

  35. [43]

    N., & Lopez -Paz, D

    Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez -Paz, D. (2018). mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICLR)

    2018

  36. [44]

    Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020). Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 13001-13008

    2020

  37. [45]

    Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016). Deep networks with stochastic depth. In Computer Vision – ECCV 2016 (pp. 646–661). Springer International Publishing

    2016

  38. [46]

    T., & Juditsky, A

    Polyak, B. T., & Juditsky, A. B. (1992). Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4), 838–855

    1992

  39. [47]

    He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    2017

  40. [48]

    Li, Y., Mao, H., Girshick, R., & He, K. (2022). Exploring plain vision transformer backbones for object detection. In Computer Vision – ECCV 2022 (pp. 280–296). Springer Nature Switzerland

    2022

  41. [49]

    Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018). Unified perceptual parsing for scene understanding. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 418–434)

    2018

  42. [50]

    Loshchilov, I., & Hutter, F. (2017). SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR)

    2017

  43. [51]

    Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., ... & Wu, H. (2018). Mixed precision training. In International Conference on Learning Representations (ICLR)

    2018

  44. [52]

    Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., & Hu, X. (2020). Score -CAM: Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  45. [53]

    Heo, B., Yun, S., Han, D., Chun, S., Choe, J., & Oh, S. J. (2021). Rethinking spatial dimensions of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2021

  46. [54]

    Xia, C., Wang, X., Lv, F., Hao, X., & Shi, Y. (2024). Vit -comer: Vision transformer with convolutional multi -scale feature interaction for dense predictions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 5493-5502)

    2024

  47. [55]

    Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., & Qiao, Y. (2022). Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534

    2022 arXiv

  48. [56]

    S., & Xie, S

    Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I. S., & Xie, S. (2023). Convnext v2: Co -designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16133-16142)

    2023

  49. [57]

    Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., & Shen, C. (2021). Twins: Revisiting spatial attention design in vision transformers. In Advances in Neural Information Processing Systems (NeurIPS) (Vol. 34, pp. 9355–9366)

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.