Pith. sign in

REVIEW 3 major objections 6 minor 24 references

Local Control Networks (LCNs): Optimizing Flexibility in Neural Network Data Pattern Capture

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes Local Control Networks, which give each neuron its own learnable B-spline activation curve, and argues this matches or beats both fixed-activation MLPs and the more complex Kolmogorov-Arnold Networks while using less…

desk verdict A plausible but not novel architecture is undermined by an incorrect backprop derivation and under-reported experiments. read the letter →

arxiv 2501.14000 v2 pith:G4OQGAXY submitted 2025-01-23 cs.LG cs.AI

Hy Nguyen , Duy Khoa Pham , Srikanth Thudumu , Hung Du , Rajesh Vasa , Kon Mouzakis This is my paper · ORCID
classification cs.LGcs.AI
keywords LocalControlNetworksB-splineactivationslearnableactivationfunctionsdiversityKolmogorov-Arnoldsparsegradientupdatessupportneuralnetworkefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the standard practice of using one fixed activation function at every neuron is unnecessarily restrictive. It proposes Local Control Networks (LCNs), which give each neuron its own learnable B-spline activation curve, allowing different regions of the network to respond to data with different shapes. The authors claim that on standard benchmarks LCNs match or slightly beat MLPs, outperform KANs by about 5% on vision tasks and about 0.6% on basic tasks, and are cheaper to run than KANs because the local support of B-splines makes gradient updates sparse and localized. If correct, this means the extra architectural machinery of KANs is not always necessary and a standard feed-forward network with per-node spline activations can capture the same patterns at lower cost.

What carries the argument

The machinery is the B-spline basis function, a piecewise polynomial that is nonzero only on a bounded knot interval. LCNs build each neuron's activation as $h_i^{(l)} = \sum_{n} w_{l,i,n} B_{N_l,p_l,n}(z_i^{(l)})$: the same standard linear transformation $z$ followed by a spline curve whose shape is set by learnable coefficients $w$. The load-bearing property is local support: for any input only a few basis functions are active, which the paper uses to justify sparse gradients, selective updates, and a built-in dropout effect, while the smoothness of B-splines is invoked for stable optimization.

What would settle it

Train an LCN with standard backpropagation and count, for each minibatch, what fraction of B-spline coefficients receive an exactly nonzero gradient. If the fraction is close to 1, or if exact backpropagation gives hidden-layer gradients different from the paper's formula, the central efficiency claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that replacing a shared fixed activation with a per-node learnable B-spline activation, while keeping the ordinary MLP feed-forward structure, gives a network that captures data patterns more flexibly and updates more locally. Concretely, each neuron's activation is a linear combination of B-spline basis functions with trainable coefficients, and because each basis function is nonzero only on a bounded interval, each neuron responds only to localized input regions. The paper reports that this design improves accuracy over MLPs on all tested benchmarks, outperforms KANs by about 5% on MNIST and Fashion-MNIST and by about 0.6% on basic tabular tasks, matches KANs on symbolic regression, and does so with fewer FLOPs than KANs.

Load-bearing premise

The proof of sparse, localized updates assumes that for every neuron the error signal is simply the difference between prediction and true value, but that is only true for output neurons; for hidden neurons the error must be propagated backwards, so the claimed sparsity is not established.

Editorial extensions

If this is right

  • On MNIST and Fashion-MNIST, LCNs outperform KANs by about 5% and slightly beat MLPs, so per-node B-spline activations can replace KAN-style edge functions in these vision tasks.
  • On the basic tabular benchmarks, LCNs improve accuracy over MLPs by about 1% and over KANs by about 0.6%, while converging faster in the first epochs.
  • Because B-splines have local support, LCN updates are sparse and localized, which the paper argues gives faster convergence, lower compute, and a natural regularization effect.
  • On symbolic regression, LCNs match KANs and both beat MLPs, suggesting edge-wise nonlinearity is not required for these function-representation tasks.
  • Each neuron's learned spline shape can be inspected, which the paper claims makes LCNs more interpretable than fixed-activation networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the hidden-layer gradient error is corrected, the qualitative claim of local updates may still hold, but the specific proof of sparsity would need to be re-derived from the exact chain rule.
  • A sharper test of the per-node-diversity hypothesis would be an ablation that keeps the architecture fixed and only toggles shared versus per-node spline parameters; the paper's MLP and KAN comparisons mix architectural and activation differences.
  • The fixed knot grid and spline degree are untested design choices; making them learnable or adaptive could enlarge the accuracy gap on the high-dimensional tasks where the paper expects LCNs to excel.
  • The claimed margin over KANs may be dataset-sensitive, so retesting on larger image sets such as CIFAR-10 would show whether the advantage scales.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes Local Control Networks (LCNs), in which each neuron has its own learnable B-spline activation function, as a middle ground between fixed-activation MLPs and Kolmogorov-Arnold Networks. The authors derive gradient formulas, argue that B-spline local support yields localized and sparse updates and hence efficiency advantages over KANs, and report experiments on four tabular benchmarks, MNIST/FMNIST, and symbolic regression tasks. The central thesis is that per-node B-spline activations achieve comparable or better accuracy than MLPs and KANs at lower computational cost.

Significance. If the efficiency claims were established, LCNs would be a useful contribution: the architecture is simple, retains the standard network structure, and B-spline local support is a principled way to obtain node-specific activations. The paper also addresses a timely comparison to KANs and evaluates on standard benchmarks, including symbolic regression. However, the theoretical efficiency argument, which is a central contribution, currently rests on an incorrect backpropagation derivation, and the empirical comparisons lack error bars and exact configuration details. With corrected derivations and stronger empirical reporting, the idea could be valuable, but as written the main novelty is not substantiated.

major comments (3)
  1. [Section 3.3.2, Eqs. (5)-(8), (13), (17)] Equation (13) asserts that ∂L/∂h_i^(l) = 2/m(ŷ_i - y_i) for every hidden layer l. This identity holds only for the output layer when the loss is MSE. For l < L the correct recursion is ∂L/∂h_i^(l) = Σ_k ∂L/∂h_k^(l+1) · φ'_k(z_k^(l+1)) · W_{ki}^{(l+1)}, which couples all downstream neurons through the weight matrices. Because Eqs. (6) and (8) drop this coupling, the claimed localized weight updates and sparse updates in §3.3.3 and the efficiency comparison in §4.2 do not follow from B-spline local support. The B-spline factor only limits updates for an individual neuron whose pre-activation is outside its support; it does not decouple layers. This is the load-bearing error for the paper's central efficiency claim.
  2. [Appendix A.2.4, Eq. (16)] Equation (16) states that ∂z_i^(l)/∂x_d = W_id^(1) for every layer l, but this is only valid for l = 1. For l > 1 the pre-activation depends on x_d through h^(l-1), so the correct expression is ∂z_i^(l)/∂x_d = Σ_j W_ij^(l) ∂h_j^(l-1)/∂x_d. This compounds the error in Eq. (17), so the input-gradient formula is not a correct chain-rule application.
  3. [Section 5, Figures 3-5] The empirical claims are not supported with uncertainty estimates: Figures 3-5 show single accuracy curves without error bars, and the text reports improvements of 'approximately 5%' (MNIST/FMNIST) and '1%'/'0.6%' without standard deviations or significance tests. Without repeated seeds and variance reporting, these margins may be within noise. In addition, the claim that parameter counts are 'standardized' (§5.1.2) is not backed by a table of model configurations or a description of the grid search, so the fairness of the MLP/KAN/LCN comparison is hard to verify.
minor comments (6)
  1. [Section 3.2.2, Eq. (3)] The notation in Eq. (3) is unclear: the B-spline basis B_{N_l,p_l,n} depends on a knot sequence, but no per-neuron knot sequence is defined in Section 3.2.2. Please specify the knots for each neuron or state explicitly that knots are shared and learned.
  2. [Section 3.2.1] The input layer assumes x ∈ [0,1]^D, but the experiments on Bank, Bean, Spam, Telescope, MNIST, and FMNIST presumably require rescaling; please describe the preprocessing applied to each dataset.
  3. [Section 5.2.4] The text says LCN showed faster learning and convergence, but no learning curves, epoch counts, or convergence thresholds are reported; please either add the curves or temper the claim.
  4. [Appendix A.3] Appendix A.3 defines FLOPs as 'floating-point operations per second', which is a rate, while the figures plot accuracy versus FLOPs; please clarify whether FLOPs means total floating-point operations.
  5. [Section 4.2 and Appendix A.3] The claim that LCNs are 'more computationally efficient than KANs' conflates theoretical sparsity with measured runtime; the figures in Appendix A.3 show accuracy versus FLOPs but do not include training time or memory measurements.
  6. [Appendix A.1] Appendix A.1 on the Consciousness Prior appears unrelated to LCNs and is not referenced anywhere in the main derivation or experiments; it should be removed or integrated into the related-work discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: LCNs are tested against external benchmarks, B-spline properties come from standard references, and the central claims do not reduce to their inputs by construction.

full rationale

I walked the paper's derivation chain. The central empirical claim—that per-node learnable B-spline activations achieve accuracy comparable to or better than MLPs and KANs—is evaluated on external benchmark datasets (Bank, Bean, Spam, Telescope, MNIST, FMNIST) and symbolic regression tasks; no fitted parameter is renamed as a prediction, and no result is defined in terms of the target claim. The B-spline properties used (local support, non-negativity, partition of unity, derivative formula) are imported from standard references (De Boor 1978; Lyche et al. 2018; Bohra et al. 2020) rather than from a self-citation chain, and the paper contains no load-bearing self-citations by the present authors. The theoretical efficiency argument relies on the local support of B-splines, which is a genuine mathematical property with independent external support. I also noted the mathematical issue in Section 3.3: Eq. (13) uses the output-layer residual form for hidden-layer gradients, and Eq. (16) treats ∂z_i^(l)/∂x_d as W_id^(1) for every layer. That is a correctness or derivation flaw, not a circular reduction—the efficiency and accuracy claims do not assume the conclusion they are trying to establish. The authors' own FLOPs measurements are not externally reproduced, which affects verifiability and reproducibility, but it does not make the derivation circular. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim is supported by a generic B-spline activation definition plus grid-searched hyperparameters. The ledger lists the B-spline degree, basis count, and knot sequence as hand-chosen free parameters whose values are not reported. The only non-standard axiom is the questionable gradient assumption in Section 3.3, which if removed changes the sparsity argument. No new physical or conceptual entities are introduced.

free parameters (3)
  • B-spline degree p_l = not reported
    Chosen per layer but not listed in the paper; the degree controls smoothness and support width of each neuron's activation.
  • Number of B-spline basis functions N_l = not reported
    Controls the capacity of each neuron's activation; determined by grid search but values are not provided, which affects any reproducibility attempt.
  • B-spline knot sequence E = not reported
    Knot placement defines where each B-spline basis function is active; the paper does not specify the knot vectors used in experiments.
assumptions (3)
  • standard math B-spline basis functions are non-negative, form a partition of unity, have local support, and are C^{p-1} continuous (Appendix A.2.1)
    Used throughout Sections 3 and 4 to argue smooth gradients, sparse updates, and localized data capture.
  • ad hoc to paper The gradient of the loss with respect to every hidden activation is 2/m(ŷ_i - y_i) (Eq. 13)
    This is only true for the output layer; the paper applies it to all hidden layers, so the sparsity and efficiency conclusions in Section 3.3 rest on this unsupported simplification.
  • domain assumption KAN is overparameterized and computationally expensive in large-scale settings (Section 4.2)
    Used to motivate the efficiency advantage of LCNs; no independent benchmark or theoretical cost model is provided, and the only evidence is the authors' own FLOPs figures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Local Control Networks (LCNs): Optimizing Flexibility in Neural Network Data Pattern Capture." pith.science (2026). https://pith.science/paper/G4OQGAXY

@misc{pith2026250114000,
  author       = {Pith},
  title        = {Pith review of: Local Control Networks (LCNs): Optimizing Flexibility in Neural Network Data Pattern Capture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4OQGAXY}},
  note         = {Machine review of arXiv:2501.14000}
}
read the original abstract

The widespread use of Multi-layer perceptrons (MLPs) often relies on a fixed activation function (e.g., ReLU, Sigmoid, Tanh) for all nodes within the hidden layers. While effective in many scenarios, this uniformity may limit the networks ability to capture complex data patterns. We argue that employing the same activation function at every node is suboptimal and propose leveraging different activation functions at each node to increase flexibility and adaptability. To achieve this, we introduce Local Control Networks (LCNs), which leverage B-spline functions to enable distinct activation curves at each node. Our mathematical analysis demonstrates the properties and benefits of LCNs over conventional MLPs. In addition, we demonstrate that more complex architectures, such as Kolmogorov-Arnold Networks (KANs), are unnecessary in certain scenarios, and LCNs can be a more efficient alternative. Empirical experiments on various benchmarks and datasets validate our theoretical findings. In computer vision tasks, LCNs achieve marginal improvements over MLPs and outperform KANs by approximately 5\%, while also being more computationally efficient than KANs. In basic machine learning tasks, LCNs show a 1\% improvement over MLPs and a 0.6\% improvement over KANs. For symbolic formula representation tasks, LCNs perform on par with KANs, with both architectures outperforming MLPs. Our findings suggest that diverse activations at the node level can lead to improved performance and efficiency.

Figures

Figures reproduced from arXiv: 2501.14000 by the authors.

Figure 1
Figure 1. Limitations of conventional DNNs and the corresponding improvements from LCNs. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Local Control Network Architecture with B-spline Activation Functions. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of accuracy over the number of parameters for MLP, KAN, and LCN models [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of accuracy over the number of parameters for MLP, KAN, and LCN models [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of accuracy over the number of parameters for MLP, KAN, and LCN models [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of accuracy versus FLOPs for MLP, KAN, and LCN models on Bank, Bean, [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Comparison of accuracy versus FLOPs for MLP, KAN, and LCN models on MNIST and [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages

  1. [1]

    Understanding deep neural networks with rectified linear units

    Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=B1J_rgWRW

    work page 2018

  2. [2]

    Reconciling modern machine-learning practice and the classical bias–variance trade-off

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116 0 (32): 0 15849--15854, 2019. doi:10.1073/pnas.1903070116. URL https://www.pnas.org/doi/abs/10.1073/pnas.1903070116

    doi:10.1073/pnas.1903070116 2019

  3. [3]

    The consciousness prior

    Yoshua Bengio. The consciousness prior. arXiv preprint arXiv:1709.08568, 2017

    arXiv 2017

  4. [4]

    Learning activation functions in deep (spline) neural networks

    Pakshal Bohra, Joaquim Campos, Harshit Gupta, Shayan Aziznejad, and Michael Unser. Learning activation functions in deep (spline) neural networks. IEEE Open Journal of Signal Processing, 1: 0 295--309, 2020

    work page 2020

  5. [5]

    A practical guide to splines

    C De Boor. A practical guide to splines. Springer-Verlag google schola, 2: 0 4135--4195, 1978

    work page 1978

  6. [6]

    Activation functions in deep learning: A comprehensive survey and benchmark

    Shiv Ram Dubey, Satish Kumar Singh, and Bidyut Baran Chaudhuri. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503: 0 92--108, 2022

    work page 2022

  7. [7]

    Adaptive activation functions for deep networks

    Michael Dushkoff and Raymond Ptucha. Adaptive activation functions for deep networks. Electronic Imaging, 28: 0 1--5, 2016

    work page 2016

  8. [8]

    Exsplinet: An interpretable and expressive spline-based neural network

    Daniele Fakhoury, Emanuele Fakhoury, and Hendrik Speleers. Exsplinet: An interpretable and expressive spline-based neural network. Neural Networks, 152: 0 332--346, 2022. ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2022.04.029. URL https://www.sciencedirect.com/science/article/pii/S0893608022001617

    work page doi:10.1016/j.neunet.2022.04.029 2022

Show all 24 references
  1. [9]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJl-b3RcF7

    2019

  2. [10]

    Evolving parsimonious networks by mixing activation functions

    Alexander Hagg, Maximilian Mensing, and Alexander Asteroth. Evolving parsimonious networks by mixing activation functions. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '17, pp.\ 425–432, New York, NY, USA, 2017. Association for Computing Machine...

    2017

  3. [11]

    Deep linear networks with arbitrary loss: All local minima are global

    Thomas Laurent and James Brecht. Deep linear networks with arbitrary loss: All local minima are global. In International conference on machine learning, pp.\ 2902--2907. PMLR, 2018

    2018

  4. [12]

    Measuring the intrinsic dimension of objective landscapes

    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In ICLR (Poster), 2018. URL https://openreview.net/forum?id=ryup8-WCW

    2018

  5. [13]

    Kan: Kolmogorov-arnold networks

    Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Solja c i \'c , Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756, 2024

    2024 arXiv

  6. [14]

    Foundations of Spline Theory: B-Splines, Spline Approximation, and Hierarchical Refinement, pp.\ 1--76

    Tom Lyche, Carla Manni, and Hendrik Speleers. Foundations of Spline Theory: B-Splines, Spline Approximation, and Hierarchical Refinement, pp.\ 1--76. Springer International Publishing, Cham, 2018. ISBN 978-3-319-94911-6. doi:10.1007/978-3-319-94911-6_1. URL https://doi.org/10....

    2018 doi

  7. [15]

    Mish: A self regularized non-monotonic activation function

    Diganta Misra. Mish: A self regularized non-monotonic activation function. arXiv preprint arXiv:1908.08681, 2019

    1908 arXiv

  8. [16]

    Optimal approximation of piecewise smooth functions using deep relu neural networks

    Philipp Petersen and Felix Voigtlaender. Optimal approximation of piecewise smooth functions using deep relu neural networks. Neural Networks, 108: 0 296--330, 2018

    2018

  9. [17]

    Searching for activation functions

    Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017

    2017 arXiv

  10. [18]

    The kolmogorov–arnold representation theorem revisited

    Johannes Schmidt-Hieber. The kolmogorov–arnold representation theorem revisited. Neural Networks, 137: 0 119--126, 2021. ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2021.01.020. URL https://www.sciencedirect.com/science/article/pii/S0893608021000289

    2021 doi

  11. [19]

    Kan or mlp: A fairer comparison

    Runpeng Yu, Weihao Yu, and Xinchao Wang. Kan or mlp: A fairer comparison. arXiv preprint arXiv:2407.16674, 2024

    2024 arXiv

  12. [20]

    Understanding deep learning requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Sy8gdB9xx

    2017

  13. [21]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  14. [22]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  15. [23]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  16. [24]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.