REVIEW 3 major objections 6 minor 13 references
Kolmogorov-Arnold network training can be parallelized by training copies on disjoint data subsets and merging their parameters, yielding near-linear speedups and a fixed-point FPGA implementation that trains 1,024 records in about 0.06 ms.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 14:50 UTC pith:KVTCTND5
load-bearing objection Solid engineering results on parallelizing NK-KAN training, but the FPGA 'training' claim is unproven and the headline speedup is inflated. the 3 major comments →
Concurrent training methods for Kolmogorov-Arnold networks: Disjoint datasets and FPGA implementation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that Newton-Kaczmarz training of KANs, although formulated as a sequence of dependent updates, can be parallelized effectively. Training several copies of the same model on disjoint subsets of the data and then averaging every parameter across copies preserves predictive accuracy closely enough that the total work scales almost linearly with the number of threads. A new pre-training step splits the model's addends into groups, trains those groups concurrently, and assembles them with rescaling, which the authors report gives a further substantial speedup. For FPGAs, rescaling the inputs and choosing the node spacing as a power of two replaces division with binary shifts
What carries the argument
The Newton-Kaczmarz layer update rule: for each training record, the layer parameters are updated by damped gradient steps such as G_new = G_old + μ(z_i - \hat z_i)(1-f) and G_new = G_old + μ(z_i - \hat z_i)f, with residuals propagated to the preceding layer via the chain rule. This update is linear in the parameters and local to each basis function, which makes three mechanisms natural: pre-training on groups of addends, concurrent training on disjoint batches with merging by averaging, and fixed-point arithmetic where power-of-two scalings turn divisions into shifts. The merging step carries the argument: averaging all parameters of independently trained copies is asserted to approximate t
Load-bearing premise
The entire speedup story rests on the assumption that averaging the parameters of models trained on different data subsets closely reproduces the model that would have been trained sequentially; the paper reports this holds only approximately, with accuracy dropping as the number of threads increases, and provides no theoretical guarantee.
What would settle it
Run the same KAN on a fixed dataset both sequentially and with, say, six threads using disjoint batches and the same total number of records, then compare validation Pearson correlation; if the merged model's correlation falls clearly below the sequential model's and additional rounds cannot recover it within the sequential run time, the near-linear speedup claim collapses. For the FPGA part, measure the cycle count for a model with more layers: if the number of cycles per record grows with layer size rather than remaining a small fixed constant, the stated 'small fixed number of sequential st
If this is right
- If the claims hold, KAN training on a laptop CPU becomes faster than GPU-accelerated training of conventional neural networks on the same examples, by up to roughly seven times when concurrency is used.
- The pre-training and disjoint-merge strategies are described as independent of the basis-function choice, so the speedups should transfer to spline-based or other KAN variants; only the FPGA variant is specific to piecewise-linear functions.
- The reported accuracy loss from merging is modest (about one to two percent Pearson correlation with six threads) and can be offset by more rounds, at the cost of reducing speedup, giving a tunable trade-off.
- FPGA training at about 0.06 ms per 1,024 records would make on-device training practical for small models, not merely inference, and the fixed-point design avoids division entirely.
Where Pith is reading between the lines
- The parameter-averaging merge is the load-bearing simplification; an editorial extension would be to test whether the accuracy loss with thread count follows a predictable curve from the loss landscape's curvature, so the optimal batch count could be chosen without exhaustive runs.
- The pre-training idea is a form of model parallelism and could be combined with the data-parallel disjoint training to scale further, a combination the paper does not test.
- The fixed-point FPGA training suggests a two-stage pipeline—train on the device in fixed point, then refine in floating point—which the paper leaves implicit and which could be benchmarked directly.
- Because the disjoint batches are random splits of one dataset, the method's behavior under non-identically-distributed splits (as in federated settings) remains untested; the merge-by-averaging mechanism may behave differently there.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three concurrency-oriented enhancements to the Newton-Kaczmarz (NK) training algorithm for Kolmogorov-Arnold networks (KANs). The first is a pre-training procedure that trains groups of addends separately and then assembles them with scaling. The second is training on disjoint subsets of the data in parallel, followed by merging the resulting models by averaging all parameters. The third is a fixed-point, division-free FPGA implementation of the training loop, claimed to be the first on-device KAN training implementation. The authors report CPU/GPU comparisons against MATLAB, FastKAN, and Keras on a determinant-prediction task (Det4) and a tetrahedron-area task (Tetra), strong and weak scaling tests on a laptop and an HPC cluster, respectively, and an FPGA run on two norm-prediction datasets. The central claims are substantial training speedups (up to ~30× over baseline CPU tests and ~7× over GPU baselines) and near-linear parallel scaling, with source code made available.
Significance. If the claims are fully validated, the paper would make a useful contribution to the growing KAN literature: the pre-training and disjoint-merge strategies are simple and potentially portable to other basis choices, and an actual FPGA training implementation would be a notable step beyond the existing FPGA-inference work. The strong scaling results on an HPC node (Table 4) are encouraging and, together with the released C++ and RTL code, give the paper a reproducible-engineering character that is valuable. However, the significance is bounded by two gaps. First, the FPGA section reports only cycle counts and wall-clock time, with no accuracy, loss, or comparison to a fixed-point software reference, so the paper's most distinctive claim—on-device training—is not actually demonstrated. Second, the merge-by-averaging premise shows a measurable accuracy drop that is acknowledged but not explained; the paper's own Table 3 indicates that restoring accuracy requires roughly doubling the compute, reducing the real-world speedup to about 2.4×. These issues are local and fixable with additional experiments or recalibrated claims, but they currently prevent the paper from supporting its headl
major comments (3)
- [§4.5] The FPGA validation is missing the essential evidence for 'training'. The section reports only that 1024 records were 'processed' in 3 cycles per record over 2 epochs, totaling 6144 cycles at 100 MHz, giving 0.06144 ms. No validation accuracy, loss, or any error metric is reported, and no comparison to a fixed-point software reference is made. The text also states that 'the same number of latches is used for both the 4-function and 16-function models', which suggests a structural claim but does not show that parameter updates occur or that the residual decreases. Consequently, the Abstract's 'implemented and tested directly on the device' and the Introduction's 'first implementation of KANs' training on FPGAs' are not supported by the reported results. Please add at least (a) a convergence/accuracy measure on the two norm datasets, (b) a comparison with a fixed-point C++ or MATLAB implem
- [§4.1, Table 1] The 'approximately 30 times' speedup summary is not supported by the authors' own baseline. From Table 1, the C++ sequential time is 6.57 s and the disjoints+pre-training Linux time is 0.70 s, which is a 9.4× speedup, not ~30×. The ~30× figure appears to be relative to the MATLAB CPU time (32.7 s), which is a different baseline and is not labelled as such in the Abstract or Section 4.1. The Conclusion's 'up to a 7 times speedup' refers to yet another comparison (GPU time). Please state explicitly for each speedup claim which baseline is used and avoid conflating CPU/GPU or MATLAB/FastKAN/Keras baselines. The underlying speedup over the sequential C++ implementation is still substantial, but the current wording is misleading.
- [§3.2 and Table 3] The central hypothesis that disjoint-batch training with averaging gives near-ideal parallel speedup is only partially supported. Table 3 shows Pearson correlation dropping from 96.8% (1 thread) to 94.5% (6 threads) at the same total workload, and the text concedes that 20 rounds with 6 threads restores 97.0% but requires twice the computational work, yielding a 2.4× speedup over 1 thread. This discrepancy is not merely a scaling artifact; it reflects the fact that parallel threads update from a stale model, as the authors note. The paper currently provides no quantitative model or theoretical bound for this accuracy drop, nor a principled rule for choosing the number of rounds/batches. Given that the speedup claim is the main practical contribution of strategies (ii), this gap should be addressed, at least with a more careful discussion of the accuracy/compute trade-off and a statement
minor comments (6)
- [§3.2] Typo: 'model coping/merging' should read 'model copying/merging'.
- [§4.5] The phrase 'same number of latches' is unclear—do you mean the same number of hardware resources/registers? Please clarify the resource utilization and the meaning of 'latches' in this context.
- [Table 2] The header row appears to have an extra entry: it lists 'Pearson (%) time (s)' followed by four accuracy columns and one time column. Please align the columns and label the four outputs (e.g., Face 1–4).
- [§4.4, Table 4] The definition of efficiency E = t(1)/t(n) is non-standard for weak scaling, where ideal efficiency is 100% when time is constant. The text explains this correctly, but the formula should be stated as E = t(1)/t(n) with t(n) the time for n times the work, not 't(n)' as currently implied; please make the notation explicit in the text.
- [§3.1] The pre-training procedure is described for the two-layer and three-layer cases, but the scaling factor p/v is introduced without a derivation. Since this is a central component of strategy (i), a short derivation or reference to a prior derivation would improve reproducibility.
- [§4.1] The discussion of FastKAN's default configuration is lengthy and somewhat editorial ('Even first-year undergraduate students are taught...'). While the parameter-count mismatch is a valid concern, the tone could be more neutral and the point more concise.
Circularity Check
No circularity found: the paper's contributions are empirical engineering results, not quantities derived from fitted inputs or from a self-citation chain.
full rationale
The paper's core claims are measured outcomes: training-time comparisons (Tables 1 and 2), strong/weak scaling measurements (Tables 3 and 4), and FPGA cycle counts (Section 4.5). None of these are derived by construction from fitted parameters or from equations equivalent to their inputs. The merge-by-averaging strategy in Section 3.2 is explicitly called a hypothesis and is tested experimentally, with the accuracy loss openly reported in Table 3. The pre-training procedure in Section 3.1 is a training-scheme proposal, not a prediction derived from a fitted quantity. The self-citations to the authors' prior work [3,5] motivate the underlying NK-KAN training method, but the paper re-establishes the baseline independently against external packages (MATLAB, FastKAN, Keras) in Table 1, so the efficiency premise does not rest solely on self-citation. The FPGA section reports cycle counts but no validation accuracy; this is a missing-validation/correctness concern, not a circularity, since no claimed equation reduces to a fitted or self-cited quantity.
Axiom & Free-Parameter Ledger
free parameters (3)
- learning rate / scaling factor µ =
1/64 on FPGA (Section 3.3.1); not tabulated for CPU runs
- batch size and number of rounds (threads) =
50K down to 3K records; 10-40 rounds (Table 3); 50K-100K records, 10-200 batches (Table 4)
- network architecture (blocks per layer, points per function) =
e.g. 70 blocks, 3 pts/function -> 5,460 params (Det4); 160/3 + 1/30 -> 16,800 (Det5); 70/3, 30/18, 4/22 (Tetra)
axioms (6)
- standard math Kolmogorov-Arnold representability: the target function can be expressed as Eq. (1) and a layered KAN of form (2) with piecewise-linear g_i,j.
- domain assumption Newton-Kaczmarz iterative updates (Eqs. 5-7) converge, and the chain-rule residual backpropagation (Eq. 7) yields correct layer targets.
- domain assumption Parameter averaging of models trained on disjoint batches yields a merged model with accuracy close to the sequential model.
- domain assumption Each disjoint batch is 'sufficiently representative of the modelled system' so that averaging works.
- domain assumption Fixed-point integer arithmetic with power-of-2 domain scaling preserves training dynamics close enough to floating point.
- ad hoc to paper Pre-trained sub-models assembled with scaling by p/v approximate the original model (Eq. 8).
read the original abstract
The present paper introduces concurrency-driven enhancements to the training algorithm for the Kolmogorov-Arnold networks (KANs) that is based on the Newton-Kaczmarz (NK) method. Prior research shows that KANs trained using the NK-based approach outperform classical neural networks (multilayer perceptrons - MLPs) both in terms of accuracy and training time. Up to now, the fundamental limitation of the algorithm has been the sequential computation of the updates - each update depends on the results of the previous step, obstructing parallelisation; even though parallelisation of some parts of the algorithm, such as the evaluation of the basis functions, has already been proposed and tested. However, substantial acceleration is achievable. Three complementary concurrency-driven novel strategies are proposed in the present paper: (i) a pre-training procedure tailored to the NK updates' structure, (ii) training on disjoint subsets of data together with models' merging, and (iii) a division-free customisation of the algorithm for field-programmable gate arrays (FPGAs), which is implemented and tested directly on the device. Computational experiments are used to assess the scalability and to compare the authors' KAN implementation with popular MLP packages. All presented experimental results are fully reproducible, with the complete source codes available online.
Reference graph
Works this paper leans on
-
[1]
Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaˇ ci´ c, T. Y. Hou, and M. Tegmark. KAN: Kolmogorov-Arnold Networks. arXiv:2404.19756, 2024
Pith/arXiv arXiv 2024
-
[2]
Igelnik and N
B. Igelnik and N. Parikh. Kolmogorov’s spline network.IEEE Transactions on Neural Networks, 14(4):725–733, 2003
2003
-
[3]
Poluektov and A
M. Poluektov and A. Polar. Construction of the Kolmogorov-Arnold networks using the Newton- Kaczmarz method.Machine Learning, 114(8):185, 2025
2025
-
[4]
Montanelli and H
H. Montanelli and H. Yang. Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem.Neural Networks, 129:1–6, 2020
2020
-
[5]
Polar and M
A. Polar and M. Poluektov. A deep machine learning algorithm for construction of the Kolmogorov- Arnold representation.Engineering Applications of Artificial Intelligence, 99:104137, 2021
2021
-
[6]
Kolmogorov–arnold networks are radial basis function networks
Ziyao Li. Kolmogorov–arnold networks are radial basis function networks. 2024
2024
-
[7]
Vaca-Rubio, Luis Blanco, Roberto Pereira, Marius Caus, and Abdullah Aydeger
Engin Zeydan, Cristian J. Vaca-Rubio, Luis Blanco, Roberto Pereira, Marius Caus, and Abdullah Aydeger. F-kans: Federated kolmogorov-arnold networks.arXiv preprint, abs/2407.20100, 2024. arXiv:2407.20100
Pith/arXiv arXiv 2024
-
[8]
Evaluating federated kolmogorov-arnold networks on non-iid data.arXiv preprint, abs/2410.08961, 2024
Arthur Mendonca Sasse and Claudio Miceli de Farias. Evaluating federated kolmogorov-arnold networks on non-iid data.arXiv preprint, abs/2410.08961, 2024. arXiv:2410.08961
Pith/arXiv arXiv 2024
-
[9]
Yizhou Ma, Zhuoqin Yang, and Luis-Daniel Ibanez. Enhancing federated learning with kolmogorov- arnold networks: A comparative study across diverse aggregation strategies.arXiv preprint, abs/2505.07629, 2025. arXiv:2505.07629
Pith/arXiv arXiv 2025
-
[10]
Youngjoon Lee, Jinu Gong, and Joonhyuk Kang. A unified benchmark of federated learning with kolmogorov-arnold networks for medical imaging.arXiv preprint, abs/2504.19639, 2025. arXiv:2504.19639
arXiv 2025
-
[11]
Duc Hoang, Aarush Gupta, and Philip Harris. KANELE: Kolmogorov-Arnold Networks for Efficient LUT-based Evaluation.arXiv preprint arXiv:2512.12850, 2025
Pith/arXiv arXiv 2025
-
[12]
V. V. Krylov. Models of sampled data systems having an infinite-dimensional state space.Automation and Remote Control, 40(5):694–699, 1979
1979
-
[13]
T. J. Hastie and R. J. Tibshirani.Generalized additive models. Chapman and Hall/CRC, 1990. 1http://openkan.org/downloadskan/matlabscripts.zip 2https://github.com/andrewpolar/kankan-8 3https://github.com/andrewpolar/meanmodel 4https://github.com/andrewpolar/fpga 5http://openkan.org 9
1990
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.