Low-power analogue neural networks with trainable nonlinear connections for continuous control
Pith reviewed 2026-06-26 10:18 UTC · model grok-4.3
The pith
Placing trainable nonlinear functions on analogue connections lets networks represent smooth continuous targets with far fewer nodes than multilayer perceptrons.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Realising trainable nonlinear functions as analogue band-pass filters on field-programmable analogue arrays allows the networks to represent smooth, continuously valued targets, including robotic kinematics, continuous control, and photovoltaic maximum-power-point tracking, with far fewer nodes and connections than multilayer perceptrons, while offering no parameter-efficiency advantage on classification-like decision boundaries; the same behaviour appears in a memristive simulation, indicating the advantage stems from trainable nonlinearity on connections.
What carries the argument
Trainable nonlinear connections realised as analogue band-pass filters on field-programmable analogue arrays
If this is right
- Networks achieve the same task performance on robotic kinematics and photovoltaic tracking using substantially fewer nodes and connections than multilayer perceptrons.
- Trained networks map to physical hardware across approximately 35,000 connections with quantified fidelity.
- A dedicated CMOS implementation is projected to operate at approximately 30 microwatts.
- A memristive version reproduces the identical parameter-efficiency pattern, showing the gain is independent of any single device technology.
Where Pith is reading between the lines
- The same connection-wise nonlinearity could be ported to other physical substrates whose native device responses already supply smooth nonlinearities, potentially extending the efficiency gain beyond field-programmable analogue arrays.
- For sensor-driven continuous control loops that must remain always on, the projected microwatt power budget opens the possibility of embedding the entire controller inside the sensor package without external digital processors.
- Because the advantage vanishes on non-smooth decision boundaries, hybrid systems could route smooth sub-tasks to the analogue nonlinear network and discrete sub-tasks to conventional digital layers.
Load-bearing premise
The computational benefit appears only for tasks whose targets are smooth and follows directly from the smoothness of the underlying physical basis.
What would settle it
A direct comparison on a smooth continuous-control benchmark in which the nonlinear-connection network requires equal or greater total parameters than a multilayer perceptron of comparable accuracy, or a hardware transfer whose output fidelity falls below the level needed for closed-loop stability.
Figures
read the original abstract
Physical neural networks promise low-power machine learning by computing directly with analogue device physics, but most architectures force nonlinear device responses to act as scalar weights. Inspired by Kolmogorov-Arnold networks, we place trainable nonlinear functions on the connections, making each physical connection a learnable computational element. Realising these functions as analogue band-pass filters on field-programmable analogue arrays, we find that the benefit is task-dependent and follows from the smoothness of the physical basis: the networks represent smooth, continuously valued targets, including robotic kinematics, continuous control, and photovoltaic maximum-power-point tracking, with far fewer nodes and connections than multilayer perceptrons, but offer no parameter-efficiency advantage on classification-like decision boundaries. Trained networks transfer to hardware across approximately 35,000 connections with quantified fidelity, and a dedicated CMOS implementation is projected to operate at approximately 30 microwatts. A memristive realisation reproduces the same behaviour in simulation, indicating that the advantage comes from placing trainable nonlinearity on connections, rather than from a particular device.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces analogue neural networks that place trainable nonlinear functions (implemented as band-pass filters on field-programmable analogue arrays) on the connections rather than using scalar weights, drawing inspiration from Kolmogorov-Arnold networks. It claims that this yields parameter-efficiency gains specifically for smooth, continuously valued targets such as robotic kinematics, continuous control, and photovoltaic maximum-power-point tracking—using far fewer nodes and connections than multilayer perceptrons—while showing no such advantage on classification-like tasks. The work reports successful hardware transfer of trained networks across ~35,000 connections with quantified fidelity, projects ~30 µW operation for a dedicated CMOS implementation, and reproduces the behavior in memristor simulations to argue that the advantage arises from the placement of trainable nonlinearity on connections.
Significance. If the empirical hardware results and task-dependent comparisons hold, the architecture offers a concrete route to low-power physical neural networks for continuous-control domains by exploiting device physics directly for edge nonlinearities. The explicit hardware transfer across tens of thousands of connections and the memristive simulation check are strengths that distinguish the contribution from purely theoretical proposals.
minor comments (2)
- [Abstract] Abstract: the central empirical claims (hardware transfer fidelity, node/connection counts versus MLPs, task-dependent advantage, power projection) are stated without accompanying metrics, baselines, error bars, or experimental protocol summaries, which reduces verifiability of the headline results even though the full manuscript presumably contains these details.
- The manuscript should clarify whether the reported node/connection savings are measured at equivalent task performance (e.g., same mean-squared error or tracking accuracy) or at a fixed architecture size; this distinction is load-bearing for the parameter-efficiency claim.
Simulated Author's Rebuttal
We thank the referee for the positive assessment of our work on analogue neural networks with trainable nonlinear connections and for recommending minor revision. No major comments were provided in the report.
Circularity Check
No significant circularity detected
full rationale
The paper's central claims rest on empirical hardware transfer experiments across ~35,000 connections and separate memristor simulations, with performance advantages presented as observed, task-dependent outcomes for smooth continuous targets rather than derived necessities. No equations, fitted parameters, or self-citations are invoked in a load-bearing way that reduces predictions to inputs by construction; the Kolmogorov-Arnold inspiration is external and the advantage is explicitly attributed to the placement of nonlinearity on connections, not to any internal self-referential step.
Axiom & Free-Parameter Ledger
free parameters (1)
- band-pass filter parameters
axioms (1)
- domain assumption The analogue band-pass filters provide a smooth physical basis suitable for continuous valued targets
Reference graph
Works this paper leans on
-
[1]
Physics for neuro- morphic computing.Nature Reviews Physics, 2(9):499–510, September 2020
Danijela Marković, Alice Mizrahi, Damien Querlioz, and Julie Grollier. Physics for neuro- morphic computing.Nature Reviews Physics, 2(9):499–510, September 2020. doi: 10.1038/ s42254-020-0208-2
2020
-
[2]
Michiel Hermans, Michaël Burm, Thomas Van Vaerenbergh, Joni Dambre, and Peter Bien- stman. Trainable hardware for dynamical computing using error backpropagation through physical media.Nature Communications, 6(1):6729, March 2015. doi: 10.1038/ncomms7729
-
[3]
Mitsumasa Nakajima, Katsuma Inoue, Kenji Tanaka, Yasuo Kuniyoshi, Toshikazu Hashimoto, and Kohei Nakajima. Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware.Nature Communications, 13(1):7847, 2022. doi: 10.1038/s41467-022-35216-2
-
[4]
Toward a formal theory for computing machines made out of whatever physics offers.Nature Communications, 14(1):4911,
Herbert Jaeger, Beatriz Noheda, and Wilfred G Van Der Wiel. Toward a formal theory for computing machines made out of whatever physics offers.Nature Communications, 14(1):4911,
-
[5]
doi: 10.1038/s41467-023-40533-1
-
[6]
Doo Seok Jeong, Kyung Min Kim, Sungho Kim, Byung Joon Choi, and Cheol Seong Hwang. Memristors for energy-efficient new computing paradigms.Advanced Electronic Materials, 2 (9):1600090, 2016. doi: 10.1002/aelm.201600090. 25
-
[7]
Abu Sebastian, Manuel Le Gallo, Riduan Khaddam-Aljameh, and Evangelos Eleftheriou. Mem- ory devices and applications for in-memory computing.Nature Nanotechnology, 15(7):529–544, July 2020. doi: 10.1038/s41565-020-0655-z
-
[8]
Fernando Aguirre, Abu Sebastian, Manuel Le Gallo, Wenhao Song, Tong Wang, J Joshua Yang, Wei Lu, Meng-Fan Chang, Daniele Ielmini, Yuchao Yang, Adnan Mehonic, Anthony Kenyon, et al. Hardware implementation of memristor-based artificial neural networks.Nature Communications, 15:1974, 2024. doi: 10.1038/s41467-024-45670-9
-
[9]
Dan A Allwood, Matthew OA Ellis, David Griffin, Thomas J Hayward, Luca Manneschi, Mohammad F Musameh, Simon O’Keefe, Susan Stepney, Charles Swindells, Martin A Trefzer, et al. A perspective on physical reservoir computing with nanomagnetic devices.Applied Physics Letters, 122(4):040501, 2023. doi: 10.1063/5.0119040
-
[10]
Kilian D. Stenning, Jack C. Gartside, ..., Eleni Vasilaki, and Will R. Branford. Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks.Nature Communications, 15(1):7377, 2024. doi: 10.1038/s41467-024-50633-1
-
[11]
Nikolay L Kazanskiy, Muhammad A Butt, and Svetlana N Khonina. Optical computing: Status and perspectives.Nanomaterials, 12(13):2171, 2022. doi: 10.3390/nano12132171
-
[12]
Physical reservoir computing with emerging electronics.Nature Electronics, 7(3):193–206, 2024
Xiangpeng Liang, Jianshi Tang, Yanan Zhong, Bin Gao, He Qian, and Huaqiang Wu. Physical reservoir computing with emerging electronics.Nature Electronics, 7(3):193–206, 2024. doi: 10.1038/s41928-024-01133-z
-
[13]
KAN: Kolmogorov–arnold networks.International Con- ference on Learning Representations, 2025
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y Hou, and Max Tegmark. KAN: Kolmogorov–arnold networks.International Con- ference on Learning Representations, 2025
2025
-
[14]
Logan G Wright, Tatsuhiro Onodera, Martin M Stein, Tianyu Wang, Darren T Schachter, Zoey Hu, and Peter L McMahon. Deep physical neural networks trained with backpropagation. Nature, 601(7894):549–555, 2022. doi: 10.1038/s41586-021-04223-6
-
[15]
Vidamour, ..., and Eleni Vasilaki
Luca Manneschi, Ian T. Vidamour, ..., and Eleni Vasilaki. Noise-aware training of neuro- morphic dynamic device networks.Nature Communications, 16:9192, 2025. doi: 10.1038/ s41467-025-64232-1
2025
-
[16]
Backpropagation-free training of deep physical neural networks.Science, 382(6676):1297–1303,
Ali Momeni, Babak Rahmani, Matthieu Malléjac, Philipp del Hougne, and Romain Fleury. Backpropagation-free training of deep physical neural networks.Science, 382(6676):1297–1303,
-
[17]
doi: 10.1126/science.adi8474
-
[18]
Physical neural networks using sharpness-aware training.Nature Com- munications, 17:1766, 2026
Tengji Xu, Zeyu Luo, Shaojie Liu, Li Fan, Qiarong Xiao, Benshan Wang, Dongliang Wang, and Chaoran Huang. Physical neural networks using sharpness-aware training.Nature Com- munications, 17:1766, 2026. doi: 10.1038/s41467-026-68470-9
-
[19]
Yiwei Peng, Sean Hooten, Xinling Yu, Thomas Van Vaerenbergh, Yuan Yuan, Xian Xiao, Bassem Tossoun, Stanley Cheung, Marco Fiorentino, and Raymond Beausoleil. Photonic KAN: a kolmogorov–arnold network inspired efficient neuromorphic accelerator.Optical Fiber Com- munication Conference, page W2A.40, 2025. doi: 10.1364/OFC.2025.W2A.40
-
[20]
Small-scale photonic kolmogorov-arnold networks using standard telecom nonlinear modules, 2026
Luca Nogueira Calçado, Sergei K Turitsyn, and Egor Manuylovich. Small-scale photonic kolmogorov-arnold networks using standard telecom nonlinear modules, 2026. 26
2026
-
[21]
Zhixing Wen, Qirui Zhang, Jiangang Chen, Tianhua Yang, Fan Yang, Xuemei Wang, Qing Liu, Xiao Luo, Peng Lin, Liang-Jian Deng, et al. Computing-in-memory architecture for kolmogorov-arnold networks based on tunable gaussian-like memory cells.Nature Communi- cations, 17:3496, 2026. doi: 10.1038/s41467-026-69592-w
-
[22]
Hay- ward, Marco Fanciulli, and Jack C
Fabiana Taglietti, Andrea Pulici, Maxwell Roxburgh, Gabriele Seguini, Ian Vidamour, Stephan Menzel, Edoardo Franco, Michele Laus, Eleni Vasilaki, Michele Perego, Thomas J. Hay- ward, Marco Fanciulli, and Jack C. Gartside. Learning nonlinear heterogeneity in physical kolmogorov-arnold networks, 2026
2026
-
[23]
Physical analog kolmogorov-arnold networks based on reconfigurable nonlinear-processing units, 2026
Manuel Escudero, Mohamadreza Zolfagharinejad, Sjoerd van den Belt, Nikolaos Alachiotis, and Wilfred G van der Wiel. Physical analog kolmogorov-arnold networks based on reconfigurable nonlinear-processing units, 2026
2026
-
[24]
Science Advances 6(16), eaay2631 (2020)
Silviu-Marian Udrescu and Max Tegmark. Ai feynman: A physics-inspired method for symbolic regression.Science Advances, 6(16):eaay2631, 2020. doi: 10.1126/sciadv.aay2631
-
[25]
Sandi Baressi Šegota, Nikola Anđelić, Vedran Mrzljak, Ivan Lorencin, Ivan Kuric, and Zla- tan Car. Utilization of multilayer perceptron for determining the inverse kinematics of an industrial robotic manipulator.International Journal of Advanced Robotic Systems, 18(4): 1729881420925283, 2021. doi: 10.1177/1729881420925283
-
[26]
Sutton, Qinghua Zhou, Alexander N
Oliver J. Sutton, Qinghua Zhou, Alexander N. Gorban, and Ivan Y. Tyukin. Relative intrinsic dimensionality is intrinsic to learning. In Lazaros Iliadis, Antonios Papaleonidas, Plamen Angelov, and Chrisina Jayne, editors,Artificial Neural Networks and Machine Learning – ICANN 2023, pages 516–529, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031- 44207-0
2023
-
[27]
Alivarani Mohapatra, Byamakesh Nayak, Priti Das, and Kanungo Barada Mohanty. A review on mppt techniques of pv system under partial shading condition.Renewable and Sustainable Energy Reviews, 80:854–867, 2017. doi: 10.1016/j.rser.2017.05.083
-
[28]
L. O. Chua. Memristor—the missing circuit element.IEEE Transactions on Circuit Theory, 18(5):507–519, 1971. doi: 10.1109/TCT.1971.1083337
-
[29]
Dmitri B. Strukov, Gregory S. Snider, Duncan R. Stewart, and R. Stanley Williams. The missing memristor found.Nature, 453(7191):80–83, May 2008. doi: 10.1038/nature06932
-
[30]
Waser.Nanoelectronics and Information Technology
R. Waser.Nanoelectronics and Information Technology. Wiley-VCH, 2009
2009
-
[31]
Ilia Valov, Eike Linn, Stefan Tappertzhofen, Sebastian Schmelzer, Jan van den Hurk, Florian Lentz, and Rainer Waser. Nanobatteries in redox-based resistive switches require extension of memristor theory.Nature Communications, 4:1771, 2013. doi: 10.1038/ncomms2784
-
[32]
S. M. Sze and K. K. Ng.Physics of Semiconductor Devices. Wiley, 2006
2006
-
[33]
Backpropagation through time: what it does and how to do it.Proceedings of the IEEE, 78(10):1550–1560, 1990
Paul J Werbos. Backpropagation through time: what it does and how to do it.Proceedings of the IEEE, 78(10):1550–1560, 1990
1990
-
[34]
A solution to the learning dilemma for recurrent networks of spiking neurons.Nature Communications, 11(1):3625, July 2020
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Leg- enstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons.Nature Communications, 11(1):3625, July 2020. ISSN 2041-1723. doi: 10. 27 1038/s41467-020-17236-y. URLhttps://www.nature.com/articles/s41467-020-17236-y. Number: ...
2020
-
[35]
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013
Pith/arXiv arXiv 2013
-
[36]
Deterministic policy gradient algorithms
David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. In Eric P. Xing and Tony Jebara, editors,Proceedings of the 31st International Conference on Machine Learning, volume 32 ofProceedings of Machine Learning Research, pages 387–395, Beijing, China, 2014. PMLR
2014
-
[37]
Lillicrap, Jonathan J
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. International Conference on Learning Representations, 2016
2016
-
[38]
Neuronlike adaptive elements that can solve difficult learning control problems,
Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson. Neuronlike adaptive elements that can solve difficult learning control problems.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5):834–846, 1983. doi: 10.1109/TSMC.1983.6313077
-
[39]
Openai gym, 2016
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016
2016
-
[40]
Pv string partial shading model
Rodney Tan. Pv string partial shading model. MATLAB Central File Ex- change, 2025. URLhttps://uk.mathworks.com/matlabcentral/fileexchange/ 62743-pv-string-partial-shading-model. Accessed 22 October 2025
2025
-
[41]
A deep reinforcement learning-based mppt control for pv systems under partial shading condition.Sensors, 20(11):3039, 2020
Bao Chau Phan, Ying-Chih Lai, and Chin E Lin. A deep reinforcement learning-based mppt control for pv systems under partial shading condition.Sensors, 20(11):3039, 2020. doi: 10. 3390/s20113039
2020
-
[42]
Spice implementation of the dynamic memdiode model for bipolar resistive switching devices.Micromachines, 13(2):330,
Fernando Leonel Aguirre, Jordi Suñé, and Enrique Miranda. Spice implementation of the dynamic memdiode model for bipolar resistive switching devices.Micromachines, 13(2):330,
-
[43]
doi: 10.3390/mi13020330
-
[44]
A power-efficient reconfigurable ota-c filter for low-frequency biomedical applications.IEEE Transactions on Circuits and Systems I: Regular Papers, 65(2):543–555,
Sheng-Yu Peng, Yu-Hsien Lee, Tzu-Yun Wang, Hui-Chun Huang, Min-Rui Lai, Chiang-Hsi Lee, and Li-Han Liu. A power-efficient reconfigurable ota-c filter for low-frequency biomedical applications.IEEE Transactions on Circuits and Systems I: Regular Papers, 65(2):543–555,
-
[45]
doi: 10.1109/TCSI.2017.2728809
-
[46]
Shuenn-Yuh Lee, Cheng-Pin Wang, and Yuan-Sun Chu. Low-voltage ota–c filter with an area- and power-efficient ota for biosignal sensor applications.IEEE Transactions on Biomedical Circuits and Systems, 13(1):56–67, 2019. doi: 10.1109/TBCAS.2018.2882521
-
[47]
Real-Time Depth From Focus on a Programmable Focal Plane Processor
Urs Denier. Analysis and design of an ultralow-power cmos relaxation oscillator.IEEE Trans- actions on Circuits and Systems I: Regular Papers, 57(8):1973–1982, 2010. doi: 10.1109/TCSI. 2010.2041504
-
[48]
Anand Savanth, Alex S. Weddell, James Myers, David Flynn, and Bashir M. Al-Hashimi. A sub-nw/khz relaxation oscillator with ratioed reference and sub-clock power gated compara- tor.IEEE Journal of Solid-State Circuits, 54(11):3097–3106, 2019. doi: 10.1109/JSSC.2019. 2930360. 28 Supplementary Material:Trainable nonlinear connections for low-power control S...
work page internal anchor Pith review Pith/arXiv arXiv doi:10.1109/jssc.2019 2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.