{"id":"d4a95e44-a2a3-499a-bfcd-c2b3426e2b68","arxiv_id":"2411.18224","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"On MNIST, CIFAR-10, and Fashion-MNIST, KANs match MLP accuracy while using significantly more parameters and showing higher sensitivity to grid and order settings.","lead":"This paper measures how Kolmogorov-Arnold Networks (KANs) perform on image classification benchmarks against MLPs. It finds KANs give similar accuracy but with far more parameters and greater sensitivity to hyperparameters, suggesting they need architectural changes for computer vision.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 contradicts the paper's blanket claim that KAN-based hybrids suffer higher parameter counts, so the central conclusion is overgeneralized and needs correction.","rationale":"The paper has two evidential pillars for its central claim: standalone KANs vs MLPs and KANs combined with CNNs. The standalone pillar is reasonably supported by Table 1/Figure 3 and the O(N^2L(G+K)) parameter formula. The hybrid pillar, however, is undermined by the paper's own Table 2, where a Convolutional KAN hybrid is listed with 7.4k parameters and 98.53% accuracy, beating a 34k-parameter standard CNN. This is not a disagreement with external consensus; it is an internal inconsistency between the stated conclusion and the reported results. The surrounding text even mischaracterizes the table's ranking. Since no code, seeds, or MLP+CNN baselines at matched parameter counts are provided, one cannot tell whether Table 2 is reliable or an outlier. A conditional verdict remains appropriate because the standalone KAN conclusion is plausible and the paper is an empirical study that could be corrected by narrowing the claim and adding matched baselines, rather than a non-reproducible method that should be rejected outright. The reader's weakest_assumption identified external implementations and missing seeds, which compounds this issue, but the load-bearing problem is the internal contradiction itself, so I mark agreement as partial. My recommendation is unchanged from the reader's conditional verdict.","tokens_in":8229,"tokens_out":7868,"duration_ms":66713,"concrete_test":"Re-run Table 2's 'Convolutional KAN with 1 layer MLP' configuration on MNIST using the exact code from ref [24], with the same training protocol (epochs, learning rate, seeds) and record the parameter count and test accuracy. Then train a standard CNN feature extractor with a 1-layer MLP classifier head matched to roughly 7.4k parameters. If the 7.4k/98.53 result reproduces and the matched hybrid is not similarly accurate at equal parameter count, the paper's blanket claim about KAN-hybrid parameter overhead is false; if it does not reproduce, the contested table entry is an unverified single-run artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the paper's headline parameter-cost claim is contradicted by its own Table 2 in Section 3.4.2. The Introduction says KANs 'suffer from a significantly higher number of parameters, both when applied alone or combined with other architectures like CNNs.' Table 2 reports 'Convolutional KAN with 1 layer MLP' at 7.4k parameters and 98.53% accuracy versus 'Standard CNN (Small)' at 34k parameters and 97.9%, and 'Standard CNN (Medium)' at 157k parameters and 99.10%. That reported hybrid is more accurate with about 4.6x fewer parameters than the smaller CNN and within 0.57 points of the larger CNN with about 21x fewer parameters. No matched MLP+CNN baseline at 7.4k parameters is shown, so the claim that combining KANs with CNNs incurs a parameter penalty is unsupported, and the one reported data point appears to refute it. The text above the table also misreads the table by saying Convolutional KANs 'achieve higher accuracy compared to standard CNNs,' since Standard CNN (Medium) is actually highest at 99.10%. This matters because the abstract and conclusion generalize the parameter-cost conclusion to hybrid KAN/CNN models, making the central claim broader than the evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an experimental comparison of Kolmogorov-Arnold Networks (KANs) against MLPs and hybrid KAN/CNN variants on MNIST, CIFAR-10, and Fashion-MNIST. The authors report that standalone KANs achieve accuracy comparable to MLPs but with substantially higher parameter counts, and that KANs are more sensitive to the grid and order hyperparameters. They also experiment with EfficientKAN and Convolutional KAN hybrids, concluding that KANs are not yet a strong alternative for image classification but may be promising in combination with other architectures.","tokens_in":8450,"tokens_out":5446,"duration_ms":43258,"significance":"If the findings were robust, the paper would provide a useful cautionary benchmark for the KAN community, and the parameter-scaling argument O(N^2 L(G+K)) versus O(N^2 L) in Section 3.4.1 is a correct and relevant observation. The heatmap analysis of grid/order sensitivity in Section 3.3 is a helpful visualization that supports the hyperparameter sensitivity claim. However, the paper's central conclusion is broader than its evidence: the hybrid KAN/CNN results in Table 2 contradict the blanket 'higher parameter count' claim for hybrids, and the absence of error bars, repeated runs, or a validation-based model selection weakens the accuracy comparisons. With targeted revisions, the paper could serve as a reference point for practitioners, but in its current form the headline claims are only partially supported.","major_comments":[{"comment":"The paper's central claim that KANs 'suffer from a significantly higher number of parameters, both when applied alone or combined with other architectures like CNNs' is contradicted by Table 2 in Section 3.4.2. The row 'Convolutional KAN with 1 layer MLP' reports 7.4k parameters and 98.53% accuracy, whereas 'Standard CNN (Small)' has 34k parameters and 97.9% accuracy and 'Standard CNN (Medium)' has 157k parameters and 99.10% accuracy; the hybrid thus uses 4.6x and 21x fewer parameters than the two CNN baselines while matching or nearly matching their accuracy. Moreover, the text above Table 2 states that Convolutional KANs 'achieve higher accuracy compared to standard CNNs,' but the table shows Standard CNN (Medium) is the most accurate model at 99.10%. The hybrid parameter-cost claim needs either to be removed or supported by matched baselines at the same parameter count.","section":"Abstract/Introduction/Conclusion vs. Section 3.4.2, Table 2"},{"comment":"The accuracy comparisons in Figure 3 and Table 1 are based on single runs without error bars or multiple random seeds. Section 3.2 states that the displayed values are 'the best accuracy achieved with these optimizations,' meaning the grid and order were selected on the test set via the sweep in Section 3.3, which risks overfitting to the test set and exaggerating the reported accuracy differences. Without variance estimates or a validation split, the conclusion that KANs achieve 'slightly better or equivalent performance compared to MLPs' is not statistically supported.","section":"Sections 3.2 and 3.3"},{"comment":"The experimental protocol is under-specified. Section 3.2 reports a fixed learning rate of 0.05 for all experiments, while Section 3.4.1 states the EfficientKAN experiments used a constant learning rate of 0.01 for 10 epochs; the number of epochs, optimizer, batch size, weight initialization, and hardware are not given for the other experiments, and no code is released. This makes the results difficult to reproduce and leaves open the possibility that differences in training budgets, not architectural properties, drive the observed accuracy gaps.","section":"Sections 3.2 and 3.4.1"},{"comment":"The definition of 'order' in Section 3.3 is internally inconsistent with Section 2.2: Section 2.2 correctly defines order as the polynomial degree of the B-spline segments, but Section 3.3 says 'Order: Denotes the depth of the network, corresponding to the number of layers composing these univariate functions.' This misdefinition obscures the heatmap analysis in Figure 4, where order is varied as a hyperparameter, and should be corrected.","section":"Section 3.3"}],"minor_comments":[{"comment":"The phrase 'specific neural work like Multi-Layer Perceptrons' should be 'specific neural networks such as Multi-Layer Perceptrons.'","section":"Abstract"},{"comment":"The caption says 'Comparison of EfficientKAN and MLP architectures for image classification on the MNIST and CIFAR-10 datasets,' but the table only contains MNIST results; either add the CIFAR-10 column or correct the caption.","section":"Table 1"},{"comment":"The parameter count formula uses K for grid in 'O(N^2 L(G + K))' while Section 3.4.2 uses K for the convolution kernel; use a consistent symbol (e.g., G for grid and k for spline order) to avoid confusion.","section":"Section 3.4.1"},{"comment":"The illustrated outputs of standard and KAN convolution are hard to follow and appear to show element-wise multiplication rather than a sliding-window convolution; rewriting with standard tensor notation would improve clarity.","section":"Section 3.4.2"},{"comment":"References [5] and [29] are identical (Krizhevsky et al., 2012), reference [22] duplicates reference [2], and reference [20] contains an embedded duplicate citation to Fashion-MNIST.","section":"References"},{"comment":"The statement that KANs 'offer parameter efficiency by replacing large weight matrices with smaller, learnable univariate functions' is at odds with the paper's own empirical finding of significantly higher parameter counts; the paper should clarify that this is a theoretical advantage that does not materialize in the tested settings.","section":"Section 2.3"},{"comment":"The conclusion claims 'excessive computational overhead' but the paper reports no runtime or memory measurements beyond parameter counts; computational cost and parameter count are not equivalent, so this statement should be qualified.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper is a lightweight empirical benchmark with limited methodological novelty, but it addresses a timely question about KANs. The most serious issue is that Table 2 contradicts a central claim in the abstract and conclusion; this needs to be resolved through either narrowing the claim or adding matched baselines. I would also encourage the editor to ask for code release and full experimental details, as the current level of reporting is insufficient for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the standalone KAN vs MLP parameter-efficiency result is real and cleanly measured, but the paper overgeneralizes it to hybrids, and its own Table 2 shows the opposite for Convolutional KANs. The experimental reporting also lacks the seeds and error bars you'd want before trusting single-run accuracies.\n\nWhat's genuinely useful here: the parameter count comparison is exact (KAN O(N^2 L(G+K)) vs MLP O(N^2 L)), and the heatmaps in Figure 4 give a concrete picture of grid/order sensitivity on MNIST. Those are worth keeping. The conclusion that vanilla KANs are not parameter-competitive for standalone MLP-style image classification is consistent with prior work like [25] and [11], and this paper adds a couple of data points on Fashion-MNIST and CIFAR-10.\n\nThe soft spots are structural. The Introduction and Abstract claim KANs suffer higher parameter counts 'when applied alone or combined with other architectures like CNNs.' Table 2 reports a Convolutional KAN + 1-layer MLP at 7.4k parameters and 98.53% accuracy, versus 34k parameters for the small CNN at 97.9%. That's a 4.6x parameter advantage for the hybrid. The text above the table also misreads it, saying Convolutional KANs beat standard CNNs, when the medium CNN is highest at 99.10%. So the central claim needs to be narrowed to standalone KANs, or the hybrid data explained. If there's a reason that 7.4k model is not comparable (e.g., fewer channels, less capacity in the CNN front-end), say so, but as reported it refutes the blanket claim.\n\nSecond, all accuracies are single-run, best-accuracy selected from a grid/order sweep on the test set, and no code is released. That makes the numerical results hard to verify and the hyperparameter-sensitivity conclusion partly a test-set artifact. A validation split or multiple seeds would fix most of this.\n\nWho is this for? Practitioners deciding whether to try KANs in small-scale vision pipelines, and researchers working on KAN variants. It's not a methods paper, and the novelty is low, but the corrected dataset would be a reasonable benchmark reference. With the claims narrowed and the experimental detail added, it deserves a serious referee. I'd send it out, but with a strong request to fix the overgeneralization.","headline":"A useful but under-scoped KAN benchmark whose headline parameter-cost claim is contradicted by its own Table 2; worth a referee after the claims are narrowed and experimentation is tightened.","tokens_in":9029,"tokens_out":2325,"would_cite":false,"duration_ms":20599,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Kolmogorov-Arnold Networks match MLP accuracy on small image benchmarks but need far more parameters.","keywords":["Kolmogorov-Arnold Networks","image classification","MLP comparison","B-spline activations","hyperparameter sensitivity","Convolutional KAN","EfficientKAN","parameter efficiency"],"falsifier":"Run the same KAN-versus-MLP comparison with parameter-matched model sizes, a grid search over grid and order for every configuration, and at least five random seeds per setting, reporting mean and standard deviation; if KANs then show a clear, reproducible accuracy advantage on any of the three datasets, or if an independent reimplementation of EfficientKAN yields materially different parameter counts, the paper's central negative claim would be refuted.","tokens_in":7982,"feed_emoji":"🤖","tokens_out":5076,"duration_ms":39699,"temperature":0.7,"pith_summary":"Kolmogorov-Arnold Networks (KANs), which replace fixed neuron activations with learnable B-spline functions on connections, have shown promise in symbolic regression but remain untested on large-scale vision. This paper runs a systematic comparison on MNIST, CIFAR-10, and Fashion-MNIST and finds that standalone KANs deliver accuracy on par with MLPs while using substantially more parameters and requiring extra hyperparameter tuning of grid and order. The same cost disadvantage appears when KAN layers are combined with convolutions. The study concludes that in their current form KANs are not a practical substitute for MLP layers in image classification, and that hybrid architectures such as Convolutional KANs are the more promising direction to explore.","feed_headline":"KANs match MLP accuracy but cost far more parameters","feed_subtitle":"The experimental study on MNIST, CIFAR-10, and Fashion-MNIST says vanilla KANs don't beat MLPs enough to justify the cost.","key_machinery":"The central object is the KAN layer, where each connection between neurons is a learnable univariate function, typically a B-spline, instead of a fixed activation at the node. The argument is carried by the parameter-scaling formula $O(N^2 L (G+K))$ for a KAN versus $O(N^2 L)$ for an MLP, and by the extra hyperparameters grid and order that control spline resolution and polynomial degree. These two facts explain both the accuracy parity and the parameter/computation gap observed in the experiments.","core_discovery":"On three standard image classification benchmarks, KANs achieve test accuracy that is only slightly better than or equal to MLPs of the same width and depth, while requiring significantly more parameters because each edge carries a spline parameterized by grid intervals and polynomial order. This holds both when KANs replace MLP layers outright and when KAN-based layers are attached to convolutional feature extractors. The paper further shows that KAN accuracy is sensitive to the grid and order hyperparameters, with the best results in a narrow region around grid 2–3 and low order, and that parameter counts grow linearly with both. The intended conclusion is that, at least on small-scale image classification, the flexibility of learnable activations does not translate into a practical advantage, and that future work should focus on adapting KANs into hybrid convolutional designs.","pith_inferences":["The paper's flattening of images into 1D vectors may stack the deck against KANs, since raw pixels are not the compositional input structure KANs are designed to exploit; testing KANs on convolutional feature maps or other latent representations is the natural follow-up.","With no repeated seeds or confidence intervals reported, the sub-1% accuracy gaps between KANs and MLPs could be within run-to-run noise; a re-run with error bars would sharpen or weaken the equivalence claim.","The parameter-count comparison uses small custom CNNs; a check against modern, well-tuned baselines (such as residual networks) at matched compute would show whether the hybrid advantage survives outside this narrow setting."],"forward_implications":["On MNIST, CIFAR-10, and Fashion-MNIST, KANs and MLPs with identical architectures land within about a point of accuracy, so the learnable activations do not provide a meaningful edge for small-scale image classification.","Because KAN parameter counts scale linearly with grid and order, even moderate spline settings multiply the computational cost relative to an MLP of the same width and depth.","The extra grid and order hyperparameters add a tuning burden that MLPs do not have, and the best accuracy appears only in a narrow range (grid 2–3, low order).","Hybrid designs such as Convolutional KANs can beat a small standard CNN on MNIST, suggesting that integrating KANs with convolutional feature extraction is more promising than using them as standalone layers."],"supporting_citations":[{"why":"Defines the KAN architecture with learnable edge functions (B-splines) that the study benchmarks against MLPs.","marker":"[16]"},{"why":"Provides the EfficientKAN implementation used for the efficient KAN experiments on flattened images.","marker":"[21]"},{"why":"Introduces Convolutional KAN layers, which the paper evaluates in hybrid models with CNNs.","marker":"[24]"},{"why":"Supplies the MNIST dataset, one of the three image classification benchmarks.","marker":"[17]"},{"why":"Supplies the CIFAR-10 dataset, the color-image benchmark in the comparisons.","marker":"[18]"},{"why":"Supplies the Fashion-MNIST dataset, the third benchmark used for evaluation.","marker":"[19]"},{"why":"Prior evidence of KAN hyperparameter sensitivity that the paper's own grid/order analysis extends.","marker":"[11]"},{"why":"A prior KAN-versus-MLP comparison that the paper cites when discussing continual learning and fair comparison.","marker":"[25]"}],"fun_headline_variants":["Vision test: KANs match MLPs, at higher cost","KANs on images: equal accuracy, more params","Study: KANs don't beat MLPs on vision tasks","KANs in CV: no practical edge over MLPs","Experiment: KANs underperform on image benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison rests on the assumptions that the third-party EfficientKAN and Convolutional-KAN implementations faithfully represent the KAN architecture, and that the reported single-run accuracy numbers are stable; if an implementation is buggy or a run is an outlier, the measured gaps would not generalize to KANs as a class.","fun_headline_variants_meta":{"raw":{"variants":["Vision test: KANs match MLPs, at higher cost","KANs on images: equal accuracy, more params","Study: KANs don't beat MLPs on vision tasks","KANs in CV: no practical edge over MLPs","Experiment: KANs underperform on image benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1256,"prompt_tokens":911,"completion_tokens":345,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":260}},"tokens_in":527,"tokens_out":345,"duration_ms":3771,"temperature":1.0,"reasoning_tokens":260,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:23:15.990875+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same KAN-versus-MLP comparison with parameter-matched model sizes, a grid search over grid and order for every configuration, and at least five random seeds per setting, reporting mean and standard deviation; if KANs then show a clear, reproducible accuracy advantage on any of the three datasets, or if an independent reimplementation of EfficientKAN yields materially different parameter counts, the paper's central negative claim would be refuted.","supporting_citations":[{"cited_title":"https://github.com/Blealtan/efficient-kan.git, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the EfficientKAN implementation used for the efficient KAN experiments on flattened images."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MNIST dataset, one of the three image classification benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CIFAR-10 dataset, the color-image benchmark in the comparisons."}],"review_version":1}