REVIEW 4 major objections 5 minor 23 references
Full Integer Arithmetic Online Training for Spiking Neural Networks
T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read An all-integer, online learning algorithm matches or beats floating-point accuracy on spiking neural networks while cutting memory by more than 60 percent.
desk verdict A genuinely integer-only online SNN training method that ships code and shows competitive results, but the 'without sacrificing accuracy' claim is undercut by an untuned FP32 baseline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the mixed-precision weight pair: shadow weights stored as high-precision integers accumulate the online gradient updates, while low-precision weights are used for every forward and backward computation and are regenerated from the shadow weights by bit-shifting each training step. Around this pair, the algorithm wraps an eligibility-trace-based local gradient approximation (presynaptic and correlation traces), an integer surrogate gradient, and a simplified integer loss that removes the exponential softmax and approximates the simulation-time factor by the nearest power of two. These pieces together let voltage decay, learning rate, and weight decay all be imple
What would settle it
Run the same mixed-precision integer training pipeline twice on SHD, once with the paper's simplified integer loss and once with a fixed-point approximation of the full softmax cross-entropy whose gradients are known to match the floating-point ones. If the simplified loss is substantially worse on held-out test accuracy, the claim that it is a sufficient learning signal fails; separately, re-running the FP32 baseline with the same gradient clipping and weight decay as the integer models would show whether the reported parity comes from regularization rather than from integer arithmetic.
Extended reading notes
Core claim
The paper's central claim is that an SNN can be trained from scratch with no floating-point operations and still match floating-point accuracy. The mechanism is a mixed-precision weight pair: high-precision integer shadow weights accumulate online updates, low-precision integer weights serve all forward and backward computations, and bit-shifting regenerates one from the other each iteration. On SHD the authors report 62.06% accuracy for the integer-only SNN with 16-bit shadow and 12-bit inference weights, versus 55.27% for their FP32 baseline and 48.10% for a BPTT-trained reference; the 16-bit/8-bit configuration uses 36.95% as much memory as FP32. They claim this recipe generalizes across
Load-bearing premise
The simplified integer scoring rule, which drops the exponential calculation and approximates the time-step constant by a power of two, must still guide the weight updates toward correct classification; the paper assumes this rather than proving it.
Editorial extensions
If this is right
- A spiking network can be trained on-chip: the online, trace-based rule has constant memory in time, so no computation graph needs to be stored or replayed.
- A digital neuromorphic processor without a floating-point unit could run both learning and inference, using only integer adds, multiplies, and shifts.
- The SHD SNN memory footprint drops to about 37 percent of the FP32 model, with further savings possible when hardware stores arbitrary bit widths instead of padded 32-bit words.
- Low inference precision (4- or 8-bit) remains usable on static tasks like MNIST, but temporal tasks like SHD need 12- or 16-bit inference weights to retain accuracy; recurrent models are more sensitive to low precision than feedforward ones.
- Integer-only training appears to act as an implicit regularizer, which can give mixed-precision models a slight edge over an unregularized floating-point baseline.
Reading between the lines
- Beyond the paper, the power-of-two bit-shift recipe suggests a general strategy: any scale factor in a local learning rule that is a ratio of constants could be rounded to a bit shift, turning multiplications into register shifts and shrinking the integer range.
- Beyond the paper, because the method reduces training to integer multiply-accumulates and shifts, the same loss approximation could be applied to other local learning rules without redesigning the quantization scheme.
- Beyond the paper, the accuracy gap between 8-bit and 12-bit inference weights on temporal tasks hints that a dynamic bit-width scheduler—growing inference precision during training—might combine speed with final accuracy; the paper does not test this.
- Beyond the paper, the 36.95 percent memory figure uses standard 8/16/32-bit storage; on neuromorphic hardware with arbitrary bit widths, storing only the needed bits could cut memory further than the reported number.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a fully integer-arithmetic online training algorithm for spiking neural networks. A mixed-precision scheme keeps high-precision integer shadow weights for gradient updates while using low-precision integer weights for forward and backward computations; voltage decay, learning rate, and weight decay are implemented by bit shifts; and a simplified integer loss removes softmax exponentials and approximates the simulation horizon by a power of two. The method is evaluated on fully connected, convolutional, and recurrent SNNs on MNIST and SHD, reporting accuracy comparable to or better than FP32 baselines and a memory reduction of about 63%.
Significance. If the accuracy-equivalence claim survives controlled comparisons, the paper is a meaningful step toward integer-only online learning on neuromorphic hardware. Strengths include the availability of code, evaluation across three architectures, 10-seed statistics, and a concrete memory-accounting table. However, the central claim is currently confounded by the baseline setup: the FP32 model omits gradient clipping and weight decay that the mixed-precision models use. The contribution is therefore promising but not yet established at the level claimed in the abstract and conclusion.
major comments (4)
- [§3.2, Tables 1 & 3, Eqs. (9)–(10)] The claim that integer-only mixed-precision training reaches accuracy 'comparable to or better than' FP32 is not supported by the baselines as reported. The manuscript itself states that gradient clipping and weight decay applied to MP configurations were not applied to the FP32 model. Both are standard regularizers, and they are part of the MP update in Eq. (9) (weight decay) and Eq. (10) (clipping). Thus the SHD result where MP-16-12 reaches 62.06% vs. FP32's 55.27%, and the MNIST result where MP-16-8 beats FP32, may be due to the missing regularizers or to quantization acting as implicit regularization, not to the integer-only design. Please rerun the FP32 baseline with the same gradient clipping, weight decay, and hyperparameter tuning protocol. If this is not done, the abstract's 'without sacrificing accuracy' claim should be weakened accordingly.
- [§2.2, Eq. (6)] The simplified integer loss is load-bearing because every reported accuracy is obtained by training with it. The paper says it 'simplifies the softmax function' but provides no derivation, no gradient analysis, and no ablation comparing this proxy loss with a standard softmax cross-entropy under otherwise identical conditions. The approximation of ts to a nearby power of two changes the weighting of evidence over time and is not analyzed. Please add an ablation (e.g., FP32 with standard loss vs. FP32 with Eq. 6, and MP with Eq. 6 vs. a higher-precision loss) and/or a formal argument that this loss supplies a usable training signal. Without this, the reader cannot tell whether the method works because of or despite the loss modification.
- [§2.4, Eq. (13)] The recurrent update as written is not integer-only: it multiplies by β·V_s(t−1), whereas feedforward layers use the bit-shift decay in Eq. (2). Unless β is explicitly replaced by a bit-shift operation (which is not stated), the RSNN experiments violate the paper's 'full integer arithmetic' claim. Please either rewrite Eq. (13) to use a bit-shift decay, define V_s and B16, and state the integer widths, or restrict the integer-only claim to the non-recurrent configurations.
- [§2.2 and §3.1] Hyperparameter selection is not documented with a validation protocol. The global clipping parameter Δmax is 'assigned after hyperparameter search' on the same benchmarks, but no search ranges, validation splits, or number of trials are reported. Combined with the baseline asymmetry, this makes it impossible to exclude tuning on the test set. Please describe the train/validation/test protocol and report the selected hyperparameters (Δmax, learning-rate bit shifts, weight-decay bit shifts, α, and β̂) for each dataset and architecture.
minor comments (5)
- [Eq. (1)] For typical β close to 1, floor(log2(1/β)) is 0, so the bit-shift decay in Eq. (2) may be a very coarse approximation. Please report the β and β̂ values used in each experiment and discuss the approximation error introduced by the floor operation.
- [Eq. (6)] The quantity 'prediction' is not precisely defined: is it the total spike count of each output neuron over the whole trial, or the count at each time step? Clarify at which time steps the loss is evaluated and how the aggregation interacts with the online weight updates.
- [§2.4, Eq. (13)] The notation V_s and the operation B16 are not defined. Please specify the bit width of the voltage and how the 16-bit truncation is applied.
- [Table 1] The table header is difficult to parse: the placement of 'Shadow weights' and 'FP32' is ambiguous, and the row/column organization does not match the text. Please reformat so the shadow-precision and inference-precision axes are unambiguous.
- [§3.4, Table 4] The memory comparison counts dynamic memory needed during training. Clarify whether the memory-reduction claim also applies at inference time, when the shadow weights would not be needed.
Circularity Check
No significant circularity: the integer-only accuracy and memory claims are empirical benchmark results, not derived from their inputs; a minor self-citation to the authors' prior BioGrad work is present but not load-bearing.
full rationale
The paper's central claims—accuracy comparable to or better than FP32 and memory reduction over 60%—are supported by running the proposed integer-only algorithm on MNIST and SHD and comparing measured test accuracy and byte counts, not by fitting a parameter and re-reporting it as a prediction. Equation (6) introduces an integer surrogate loss, but its adequacy is an empirical design choice tested by the later accuracy tables; such an assumption is a correctness/robustness risk, not a self-referential reduction. The global clipping parameter, learning rates, and weight decays are tuned on the same benchmarks without a described validation split; this can inflate reported numbers but is a benchmarking-discipline concern rather than circularity. The paper itself flags in §3.2 that gradient clipping and weight decay were not applied to the FP32 baseline, potentially advantaging MP configurations; this is a fairness confound for the 'without sacrificing accuracy' comparison, not a circular step, because the MP accuracy is still independently measured. The only self-citation of note is [13] (BioGrad), a prior work by co-author Tang, cited in §2.2 as the eligibility-trace base ('Built on top of existing gradient-based online learning method for SNN [13]') and used as an additional MNIST baseline. That citation is not load-bearing: the new integer-only contribution is not proved by BioGrad, and the reported results stand on external benchmarks and direct memory/operation accounting. Accordingly, no circular step meets the quoted-reduction standard.
Assumptions & free parameters
free parameters (4)
- Gradient clipping threshold Delta_max =
not reported (chosen by hyperparameter search)
- Loss precision alpha =
not reported
- Mixed-precision bit widths (shadow/inference) =
16/8 for MNIST, 16/12 for SHD best configurations
- Voltage decay rate via bit shift beta_hat =
not reported
assumptions (5)
- domain assumption Eligibility traces from BioGrad/E-prop provide a sufficient gradient approximation for online SNN learning.
- domain assumption The surrogate gradient g∇V in Eq. 3 is a valid replacement for the non-differentiable spike function.
- ad hoc to paper The simplified integer loss in Eq. 6 provides a usable training signal for classification.
- domain assumption Transposed low-precision weights as feedback (Eq. 7) approximate true error propagation.
- domain assumption Random recurrent feedback matrices are sufficient for RSNN training.
Cite this review
Pith. "Pith review of Full Integer Arithmetic Online Training for Spiking Neural Networks." pith.science (2026). https://pith.science/paper/QKAJXZ5X
@misc{pith2026250906636,
author = {Pith},
title = {Pith review of: Full Integer Arithmetic Online Training for Spiking Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/QKAJXZ5X}},
note = {Machine review of arXiv:2509.06636}
}
read the original abstract
Spiking Neural Networks (SNNs) are promising for neuromorphic computing due to their biological plausibility and energy efficiency. However, training methods like Backpropagation Through Time (BPTT) and Real Time Recurrent Learning (RTRL) remain computationally intensive. This work introduces an integer-only, online training algorithm using a mixed-precision approach to improve efficiency and reduce memory usage by over 60%. The method replaces floating-point operations with integer arithmetic to enable hardware-friendly implementation. It generalizes to Convolutional and Recurrent SNNs (CSNNs, RSNNs), showing versatility across architectures. Evaluations on MNIST and the Spiking Heidelberg Digits (SHD) dataset demonstrate that mixed-precision models achieve accuracy comparable to or better than full-precision baselines using 16-bit shadow and 8- or 12-bit inference weights. Despite some limitations in low-precision and deeper models, performance remains robust. In conclusion, the proposed integer-only online learning algorithm presents an effective solution for efficiently training SNNs, enabling deployment on resource-constrained neuromorphic hardware without sacrificing accuracy.
Figures
Reference graph
Works this paper leans on
-
[1]
Nguyen,D.A.,Tran,X.T.,Iacopi,F.:AReviewofAlgorithmsandHardwareImple- mentationsforSpikingNeuralNetworks.JournalofLowPowerElectronicsandAp- plications11(2), 23 (Jun 2021). https://doi.org/10.3390/jlpea11020023, https:// www.mdpi.com/2079-9268/11/2/23, number: 2 Publisher: Multidisciplinary Digi- tal Publishing Institute
-
[2]
https://doi.org/10.48550/arXiv.1806.08342, http: //arxiv.org/abs/1806.08342, arXiv:1806.08342
Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient infer- ence: A whitepaper (Jun 2018). https://doi.org/10.48550/arXiv.1806.08342, http: //arxiv.org/abs/1806.08342, arXiv:1806.08342
-
[3]
2023 26th Euromicro Con- ference on Digital System Design (DSD) pp
Schnöll, D., Wess, M., Bittner, M., Götzinger, M., Jantsch, A.: Fast, Quantization Aware DNN Training for Efficient HW Implementation. 2023 26th Euromicro Con- ference on Digital System Design (DSD) pp. 700–707 (Sep 2023). https://doi.org/ 10.1109/DSD60849.2023.00100, https://ieeexplore.ieee.org/document/10456774/, conference Name: 2023 26th Euromicro Con...
arXiv 2023
-
[4]
Frontiers in Neuro- science12, 331 (May 2018)
Wu, Y., Deng, L., Li, G., Zhu, J., Shi, L.: Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks. Frontiers in Neuro- science12, 331 (May 2018). https://doi.org/10.3389/fnins.2018.00331, https: //www.frontiersin.org/article/10.3389/fnins.2018.00331/full
arXiv 2018
-
[5]
Xiao, M., Meng, Q., Zhang, Z., He, D., Lin, Z.: Online Training Through Time for Spiking Neural Networks (Dec 2022), http://arxiv.org/abs/2210.04195, arXiv:2210.04195
work page Pith review arXiv 2022
-
[6]
Nature Communications11(1), 3625 (Jul 2020)
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., Maass, W.: A solution to the learning dilemma for recurrent networks of spiking neu- rons. Nature Communications11(1), 3625 (Jul 2020). https://doi.org/10.1038/ s41467-020-17236-y, https://www.nature.com/articles/s41467-020-17236-y
work page 2020
-
[7]
Horowitz, M.: 1.1 Computing’s energy problem (and what we can do about it). In: 2014 IEEE International Solid-State Circuits Conference Digest of Technical Pa- pers(ISSCC).pp.10–14(Feb2014).https://doi.org/10.1109/ISSCC.2014.6757323, https://ieeexplore.ieee.org/document/6757323, iSSN: 2376-8606
arXiv 2014
-
[8]
Is Integer Arithmetic Enough for Deep Learning Training?
Ghaffari, A., Tahaei, M.S., Tayaranian, M., Asgharian, M., Nia, V.P.: Is Integer Arithmetic Enough for Deep Learning Training? (Jan 2023). https://doi.org/10. 48550/arXiv.2207.08822, http://arxiv.org/abs/2207.08822, arXiv:2207.08822
work page Pith review arXiv doi:10.48550/arxiv.2207.08822 2023
Show all 23 references
-
[9]
ArXiv (Jan 2022), https://www.semanticscholar.org/ paper/PocketNN%3A-Integer-only-Training-and-Inference-of-in-Song-Lin/ 81fcadee5f9fc3b57a35df38d4ce3eb9e868b426
Song, J.S., Lin, F.: PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activa- tions in Pure C++. ArXiv (Jan 2022), https://www.semanticscholar.org/ paper/PocketNN%3A-Integer-only-Training-and-Inference-of-in-Song-Lin/ 81...
2022
-
[10]
arXiv preprint arXiv:2407.11698 (2024) 12 Ismael Gomez and Guangzhi Tang
Pirillo, A., Colombo, L., Roveri, M.: Nitro-d: Native integer-only training of deep convolutional neural networks. arXiv preprint arXiv:2407.11698 (2024) 12 Ismael Gomez and Guangzhi Tang
2024
-
[11]
Frontiers in Neuroscience18(Dec 2024)
Zou, C., Cui, X., Feng, S., Chen, G., Zhong, Y., Dai, Z., Wang, Y.: An all integer-based spiking neural network with dynamic thresh- old adaptation. Frontiers in Neuroscience18(Dec 2024). https://doi.org/ 10.3389/fnins.2024.1449020, https://www.frontiersin.org/journals/neurosc...
2024
-
[12]
https://doi.org/10.48550/arXiv.2407.20708, http://arxiv.org/abs/2407.20708, arXiv:2407.20708
Luo, X., Yao, M., Chou, Y., Xu, B., Li, G.: Integer-Valued Training and Spike- Driven Inference Spiking Neural Network for High-performance and Energy- efficient Object Detection (Aug 2024). https://doi.org/10.48550/arXiv.2407.20708, http://arxiv.org/abs/2407.20708, arXiv:2407.20708
-
[13]
Tang, G., Kumar, N., Polykretis, I., Michmizos, K.P.: BioGrad: Biologically Plau- sible Gradient-Based Learning for Spiking Neural Networks (Oct 2021), http: //arxiv.org/abs/2110.14092, arXiv:2110.14092
2021 arXiv
-
[14]
Elife8, e43299 (2019)
Murray, J.M.: Local online learning in recurrent networks with random feedback. Elife8, e43299 (2019)
2019
- [15]
- [16]
-
[17]
Frontiers in Neuroscience18, 1439155 (Jul 2024)
Fernandez, J.G., Keemink, S., Gerven, M.v.: Gradient-Free Training of Recur- rent Neural Networks using Random Perturbations. Frontiers in Neuroscience18, 1439155 (Jul 2024). https://doi.org/10.3389/fnins.2024.1439155, http://arxiv.org/ abs/2405.08967, arXiv:2405.08967
2024
-
[18]
Proceedings of the IEEE86(11), 2278–2324 (Nov 1998)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE86(11), 2278–2324 (Nov 1998). https://doi.org/10.1109/5.726791, https://ieeexplore.ieee.org/document/ 726791, conference Name: Proceedings of the IEEE
1998 doi
-
[19]
IEEE Transactions on Neural Networks and Learning Systems33(7), 2744–2757 (Jul 2022)
Cramer, B., Stradmann, Y., Schemmel, J., Zenke, F.: The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems33(7), 2744–2757 (Jul 2022). https://doi.org/10.1109/TNNLS.2020.3044364, http...
2022
-
[20]
https://doi.org/10.48550/arXiv.1805.11046, http: //arxiv.org/abs/1805.11046, arXiv:1805.11046
Banner, R., Hubara, I., Hoffer, E., Soudry, D.: Scalable Methods for 8-bit Training of Neural Networks (Jun 2018). https://doi.org/10.48550/arXiv.1805.11046, http: //arxiv.org/abs/1805.11046, arXiv:1805.11046
- [21]
-
[22]
112130359, https://ieeexplore.ieee.org/document/8259423/
Davies, M., Srinivasa, N., Lin, T.H., Chinya, G., Cao, Y., Choday, S.H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C.K., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y.H., Wild, A., Yang, Y., Wang, H.: Loihi: A Neu...
2018
-
[23]
arXiv preprint arXiv:2401.04491 (2024)
Gonzalez, H.A., Huang, J., Kelber, F., Nazeer, K.K., Langer, T., Liu, C., Lohrmann, M., Rostami, A., Schöne, M., Vogginger, B., et al.: Spinnaker2: A large- scale neuromorphic system for event-based and asynchronous machine learning. arXiv preprint arXiv:2401.04491 (2024)
2024 arXiv
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.