Pith. sign in

REVIEW 4 major objections 5 minor 22 references

Average-Over-Time Spiking Neural Networks for Uncertainty Estimation in Regression

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spiking neural networks can estimate regression uncertainty on par with deep nets.

desk verdict A legitimate AOT-SNN regression extension whose headline comparison is undercut by an undefined RAC NLL metric and a misreported toy result. read the letter →

arxiv 2412.00278 v1 pith:SS6T5OVP submitted 2024-11-29 cs.LG cs.AIcs.NE

Tao Sun , Sander Bohté This is my paper · ORCID
classification cs.LGcs.AIcs.NE
keywords spikingneuralnetworksuncertaintyestimationregressionaverage-over-timeheteroscedasticGaussianregression-as-classificationMC-dropoutenergyefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to give spiking neural networks (SNNs) the same ability to estimate prediction uncertainty in regression that deep neural networks (DNNs) have, without paying the cost of repeated forward passes. It adapts the Average-Over-Time SNN (AOT-SNN) framework to two regression setups: one where each time step predicts a mean and variance under a heteroscedastic Gaussian assumption, and one based on Regression-as-Classification (RAC), where the target range is binned and probabilities over bins are averaged over time. On a toy dataset and eight UCI benchmarks, the reported experiments show both methods reaching error and negative-log-likelihood values comparable to state-of-the-art DNN baselines (MC-dropout and deep ensembles), with the RAC variant improving RMSE notably on Concrete Strength, Energy Efficiency, and Protein Structure datasets. The efficiency gain comes from replacing many stochastic forward passes with a single forward pass whose time steps supply the samples to average.

What carries the argument

The central mechanism is the Average-Over-Time trick: an SNN whose output layer is a set of non-spiking readout integrator neurons, so the membrane potential at each time step $t$ is treated as one stochastic prediction, and the predictive distribution is the average over the $T$ time steps of a single forward pass. Two output parameterizations plug into this machinery: (i) a two-neuron readout emitting $\mu_t(x)$ and $\sigma_t^2(x)$ at every time step, combined through the deep-ensemble formula of Eq. 5, and (ii) a $K$-neuron readout for binned targets, whose time-averaged softmax probabilities become a continuous density through the uniform-within-bin assumption. The training objectives are the negative log-likelihood for the Gaussian variant and, for the RAC variant, a distance loss $L_{dis} = \sum_k |k-j|^q p_k$ with entropy regularization $\tau H$ that keeps the predictive distribution neither overconfident nor misplaced.

What would settle it

Take a trained AOT-SNN, feed the same input many times, and compute the autocorrelation of the per-time-step outputs; then compare the predictive variance from the mixture formula with the variance obtained from truly independent forward passes that reset membrane potentials each time. If the independent-pass version gives systematically different NLL or reliability-diagram coverage, the time-average-as-MC-dropout equivalence is not valid.

Watch

Extended reading notes

Core claim

The core claim is that uncertainty for regression can be estimated efficiently in event-driven spiking networks by reusing their inherent time-step structure: instead of running many forward passes, one forward pass with $T$ time steps produces $T$ predictions whose average approximates what MC-dropout obtains with multiple passes. In the Gaussian variant, each time step $t$ emits a mean $\mu_t(x)$ and a variance $\sigma_t^2(x)$, and the predictive distribution is assembled as $\mu_*(x) = \frac{1}{T}\sum_t \mu_t(x)$ and $\sigma_*^2(x) = \frac{1}{T}\sum_t (\sigma_t^2(x)+\mu_t^2(x)) - \mu_*^2(x)$, the standard ensemble combination rule (Eq. 5) applied across time steps. In the RAC variant, the target range is discretized into $K$ bins, each time step produces logits over bins, the averaged softmax probabilities $p_k$ are converted into a continuous conditional density by assuming uniformity within each bin ($f_k = p_k/(b_{k+1}-b_k)$), and the predictive value is $\sum_k p_k m_k$; training uses a distance loss $\sum_k |k-j|^q p_k$ plus entropy regularization $\tau H$. The paper reports that on eight UCI regression datasets, both AOT-SNN variants achieve RMSE and NLL in the same range as MC-dropout and deep ensembles, and that the RAC variant is better in RMSE on three of the datasets.

Load-bearing premise

The load-bearing premise is that the outputs at different time steps can be treated as independent MC-dropout samples when combining mean and variance, even though the membrane-potential dynamics make those time steps correlated within one forward pass.

Editorial extensions

If this is right

  • AOT-SNN regression needs only one forward pass for uncertainty estimates, so event-driven networks become practical for regression with confidence bounds.
  • The RAC variant with distance loss beats MC-dropout and deep ensembles in RMSE on Concrete Strength, Energy Efficiency, and Protein Structure, suggesting classification-style losses can improve accuracy in spiking regression.
  • NLL values reported on most UCI benchmarks are in the DNN baseline range, indicating calibrated uncertainty does not require abandoning spiking hardware.
  • Because time averaging emulates ensemble averaging, improvements to MC-dropout or deep ensembles can likely be ported to SNNs through the same temporal averaging mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The independence assumption can be probed directly by measuring the autocorrelation of per-time-step outputs for a fixed input; strong correlation would mean the reported variances understate true uncertainty.
  • If time-step outputs are close to independent, the same trick could extend to mixture density networks by treating each time step as one mixture component, yielding richer densities at negligible extra cost.
  • The uniform-within-bin density could be swapped for a smooth kernel (triangular or Gaussian) per bin, giving continuous predictive densities and possibly better NLL without changing the training objective.
  • In streaming settings, time-averaged mean and variance can be updated incrementally as time steps arrive, yielding anytime uncertainty that deep ensembles cannot supply without retraining.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper adapts the Average-Over-Time SNN (AOT-SNN) framework to regression tasks for uncertainty estimation. Two methods are proposed: a heteroscedastic Gaussian approach, where each time step outputs a mean and variance that are combined across time steps, and a Regression-as-Classification (RAC) approach, where the target is discretized into bins and a piecewise-uniform predictive density is formed from softmax probabilities. The methods are evaluated on a toy dataset and eight UCI benchmark datasets, with claims that the proposed SNN models achieve uncertainty performance comparable to or better than MC-dropout and deep ensemble DNN baselines. The paper also describes a distance loss with entropy regularization for training the RAC models.

Significance. If the reported results are reliable, the paper would provide a principled and efficient way to obtain predictive uncertainty from event-driven spiking networks in regression settings, an area with comparatively few existing solutions. The conceptual extension of AOT-SNN to regression and the use of RAC with a continuous piecewise-uniform density are reasonable and align with recent work on regression-as-classification. The experimental evaluation covers standard benchmarks and includes both accuracy (RMSE) and uncertainty quality (NLL). However, the central claim is currently hampered by a missing definition of the NLL metric for RAC, an internal inconsistency in the toy experiment, and unspecified hyperparameters, which together prevent verification of the headline result. The derivations of the predictive mean and variance are straightforward applications of the mixture variance formula and are not circular.

major comments (4)
  1. [Section III-C and Table II] The manuscript never defines how NLL is computed for the RAC model. Equation (8) defines the predictive density as piecewise uniform, f_k = p_k/(b_{k+1}-b_k); the continuous NLL for a point y in bin k is therefore -log f_k = -log p_k + log(b_{k+1}-b_k). If instead the discrete cross-entropy -log p_k was used, those values are not continuous densities and are not comparable to the Gaussian NLL values reported for MC-dropout and Deep Ensembles. Please state the exact formula used to compute every NLL value in Table II, and correct the table if necessary.
  2. [Section IV-A and Figure 3] The text states that the Gaussian-based AOT-SNN 'significantly outperforms its DNN counterpart in both RMSE and NLL', but Figure 3 reports SNN Gaussian NLL = 2.59 versus DNN Gaussian NLL = 2.48, so the SNN is worse on NLL. The figure caption also claims the Gaussian-based AOT-SNN 'shows improved RMSE and NLL', which is contradicted by the displayed numbers. This internal inconsistency in a core illustrative experiment undermines confidence in the reported comparisons; please correct the text or the figure and recheck the benchmark tables for analogous errors.
  3. [Section III-C.2 and Section IV] The distance loss in Eq. (14) depends on the hyperparameter q>0, but its value is never stated for any experiment. The toy-experiment paragraph mentions grid search and the benchmark paragraph specifies tau=1, yet q is omitted entirely. Without q, the RAC training loss is not reproducible. Please report the chosen q values or the selection procedure for both the toy and benchmark experiments.
  4. [Abstract, Section V, and Tables I-II] The abstract and conclusion claim that both AOT-SNN methods achieve uncertainty performance 'on par with' or 'often surpassing' state-of-the-art DNN approaches, but Table II shows the Gaussian AOT-SNN NLL is substantially worse than MC-dropout on Boston housing (2.74 vs 2.46), Kin8nm (-1.13 vs -0.95), and Power plant (3.19 vs 2.80), and the RAC NLL is worse than MC-dropout on Kin8nm (-0.75 vs -0.95). The paper does not provide a quantitative criterion for 'on par' nor statistical significance tests. Please qualify the claim with respect to the datasets that actually support it, or report error bars and tests that substantiate the stated level of performance.
minor comments (5)
  1. [Section III-B] The variance combination formula is presented as analogous to MC-dropout and deep ensembles; please clarify that it is a mixture variance across time steps, not a variance over independent samples, since membrane potentials in an SNN are temporally correlated and the independence assumption is not stated or tested.
  2. [Section IV-A] The sentence 'Both SNN models set their dropout rate as 0.05' and 'optimized via the grid search' is ambiguous: please specify which hyperparameters (q, K, tau, dropout rate) were grid-searched and which were fixed for the toy experiment.
  3. [Throughout] There are several typos and grammatical errors, e.g., 'the the mean' in the Figure 1d caption, 'can often outperforms' in Section II-C, and 'solutions' should be 'solution' in Section V. Please proofread the manuscript.
  4. [Section IV-B] The Gaussian models are trained for 600 epochs while the RAC models are trained for 200 epochs; this difference could affect the comparison and should be justified or the training lengths should be matched.
  5. [Table II] The caption of Table II does not state the units or the exact definition of NLL; given that Gaussian NLL can be negative and RAC NLL depends on bin width, please add a note specifying how each NLL value was computed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the variance and RAC derivations are standard identities, and the benchmark claims rest on external comparisons rather than fitted inputs or self-citation chains.

full rationale

The paper's two methodological derivations are self-contained and are not circular. For the Gaussian AOT-SNN, the predictive variance sigma2_*(x) = (1/T) sum_t (sigma2_t(x) + mu2_t(x)) - mu2_*(x) is the law of total variance applied to T per-time-step Gaussian components; it is an algebraic identity, not a quantity fitted to the evaluation data. For the RAC method, the density f_k = p_k/(b_{k+1} - b_k) is a stated modeling assumption (uniform within each bin), and the derivation E[y] = sum_k p_k m_k follows directly from integrating that density; the result is by construction the definition of the predictive expectation, but this is a legitimate modeling choice rather than the paper presenting a fitted parameter as a prediction. The empirical claims in Tables I and II are comparisons against external DNN baselines (MC-dropout and Deep Ensembles) on UCI benchmarks, so the central result does not reduce to fitted constants. The paper does cite the authors' prior AOT-SNN framework [6] and readout-integrator design [21], but these citations supply the network architecture and averaging scheme, not the regression-specific uncertainty results; the uncertainty performance is re-evaluated independently on benchmark data. Therefore the self-citations are not load-bearing for the headline claim. Separate transparency issues exist but are not circularity: the NLL computation for the RAC model is never specified (Eq. 8 implies a continuous NLL -log p_k + log(b_{k+1}-b_k), while the tables may have used discrete cross-entropy), the distance-loss exponent q in Eq. 14 is left unspecified, and the text in Section IV-A says the Gaussian AOT-SNN 'significantly outperforms' its DNN counterpart while Fig. 3 reports SNN NLL 2.59 versus DNN NLL 2.48. These are reproducibility and consistency concerns, not instances of a derivation reducing to its own inputs, so they do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central methods rest on standard probabilistic assumptions (Gaussian CPD, uniform bin density) and on the authors' earlier AOT-SNN framework. No new entities are introduced. The main free parameters are experimental hyperparameters, some of which are unreported.

free parameters (5)
  • dropout rate = per dataset from [0.005, 0.01, 0.05, 0.1]
    Selected by validation NLL per dataset (Section IV-B), affecting uncertainty estimates.
  • bin count K = 150 (toy), 50 (benchmarks)
    Set following [14] for benchmarks, chosen by hand/grid search; defines the RAC discretization.
  • entropy weight tau = 1
    Set to 1; for toy selected by grid search, for benchmarks fixed. Controls distance loss regularization.
  • distance exponent q = not reported
    Hyperparameter in Eq. 14 is never specified in experiments, so the exact loss function used is undefined.
  • time steps T = 8 (toy), unspecified (benchmarks)
    Number of AOT-SNN time steps; affects the variance combination.
assumptions (4)
  • domain assumption Heteroscedastic Gaussian assumption for target CPD
    Eq. 2 assumes p(y|x) is Gaussian with input-dependent variance; standard in deep ensembles.
  • domain assumption Uniform distribution within RAC bins
    Eq. 8 assumes uniform density within each bin to construct a continuous CPD from bin probabilities.
  • standard math Law of total variance for aggregating time steps
    The variance formula in Section III-B decomposes predictive variance into mean and variance components, as in Eq. 5.
  • domain assumption Time steps are exchangeable MC-dropout samples
    The AOT-SNN framework from [6] treats time-step outputs as dropout evaluations; this paper inherits that assumption without testing it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Average-Over-Time Spiking Neural Networks for Uncertainty Estimation in Regression." pith.science (2026). https://pith.science/paper/SS6T5OVP

@misc{pith2026241200278,
  author       = {Pith},
  title        = {Pith review of: Average-Over-Time Spiking Neural Networks for Uncertainty Estimation in Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SS6T5OVP}},
  note         = {Machine review of arXiv:2412.00278}
}
read the original abstract

Uncertainty estimation is a standard tool to quantify the reliability of modern deep learning models, and crucial for many real-world applications. However, efficient uncertainty estimation methods for spiking neural networks, particularly for regression models, have been lacking. Here, we introduce two methods that adapt the Average-Over-Time Spiking Neural Network (AOT-SNN) framework to regression tasks, enhancing uncertainty estimation in event-driven models. The first method uses the heteroscedastic Gaussian approach, where SNNs predict both the mean and variance at each time step, thereby generating a conditional probability distribution of the target variable. The second method leverages the Regression-as-Classification (RAC) approach, reformulating regression as a classification problem to facilitate uncertainty estimation. We evaluate our approaches on both a toy dataset and several benchmark datasets, demonstrating that the proposed AOT-SNN models achieve performance comparable to or better than state-of-the-art deep neural network methods, particularly in uncertainty estimation. Our findings highlight the potential of SNNs for uncertainty estimation in regression tasks, providing an efficient and biologically inspired alternative for applications requiring both accuracy and energy efficiency.

Figures

Figures reproduced from arXiv: 2412.00278 by the authors.

Figure 1
Figure 1. a, The MC-dropout method estimates uncertainty by averaging the outputs of multiple forward passes through a dropout-enabled DNN. b, Deep ensembles train an ensemble of DNNs with identical architectures. For regression tasks, each member in an ensemble assumes a heteroscedastic Gaussian distribution and outputs two variables per input: the mean µm(x) and variance σ 2m(x). The predictive mean µ∗(x) and variance σ∗(x)… view at source ↗
Figure 2
Figure 2. Illustration of the heteroscedastic Gaussian and Regression-as-Classification (RAC) approaches for uncertainty estimation in regression tasks. (a) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of RMSE and NLL metrics for AOT-SNN and DNN [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 14 canonical work pages

  1. [1]

    Uncertainty in deep learning,

    Y . Gal, “Uncertainty in deep learning,” Ph.D. dissertation, Department of Engineering, University of Cambridge, 2016

    work page 2016

  2. [2]

    Probabilistic machine learning and artificial intelli- gence,

    Z. Ghahramani, “Probabilistic machine learning and artificial intelli- gence,” Nature, vol. 521, no. 7553, pp. 452–459, 2015

    2015

  3. [3]

    Simple and scalable predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” NIPS, vol. 30, 2017

    work page 2017

  4. [4]

    A survey of uncertainty in deep neural networks,

    J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscheret al., “A survey of uncertainty in deep neural networks,” Artificial Intelligence Review , vol. 56, no. Suppl 1, pp. 1513–1589, 2023

    2023

  5. [5]

    A review of uncertainty quantification in deep learning: Techniques, applications and challenges,

    M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information fusion, vol. 76, pp. 243–297, 2021

    work page 2021

  6. [6]

    Efficient uncertainty estimation in spiking neural networks via mc-dropout,

    T. Sun, B. Yin, and S. Boht ´e, “Efficient uncertainty estimation in spiking neural networks via mc-dropout,” in International Conference on Artificial Neural Networks . Springer, 2023, pp. 393–406

    work page 2023

  7. [7]

    Opportunities for neuromorphic computing algorithms and applications,

    C. D. Schuman, S. R. Kulkarni, M. Parsa, J. P. Mitchell, P. Date, and B. Kay, “Opportunities for neuromorphic computing algorithms and applications,” Nature Computational Science , vol. 2, no. 1, pp. 10–19, 2022

    2022

  8. [8]

    Dpsnn: Spiking neural network for low- latency streaming speech enhancement,

    T. Sun and S. M. Bohte, “Dpsnn: Spiking neural network for low- latency streaming speech enhancement,” Neuromorphic Computing and Engineering, 2024

    work page 2024

Show all 22 references
  1. [9]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in ICML. PMLR, 2016, pp. 1050–1059

    2016

  2. [10]

    Bayesian deep learning and a proba- bilistic perspective of generalization,

    A. G. Wilson and P. Izmailov, “Bayesian deep learning and a proba- bilistic perspective of generalization,” NIPS, vol. 33, pp. 4697–4708, 2020

    2020

  3. [11]

    Estimating the mean and variance of the target probability distribution,

    D. A. Nix and A. S. Weigend, “Estimating the mean and variance of the target probability distribution,” in Proceedings of 1994 ieee international conference on neural networks (ICNN’94), vol. 1. IEEE, 1994, pp. 55– 60

    1994

  4. [12]

    Mixture Density Networks,

    C. M. Bishop, “Mixture Density Networks,” Aston University, Tech. Rep., 1994. [Online]. Available: http://www.ncrg.aston.ac.uk/

    1994

  5. [13]

    Regression as clas- sification: Influence of task formulation on neural network features,

    L. Stewart, F. Bach, Q. Berthet, and J.-P. Vert, “Regression as clas- sification: Influence of task formulation on neural network features,” in International Conference on Artificial Intelligence and Statistics . PMLR, 2023, pp. 11 563–11 582

    2023

  6. [14]

    Conformal prediction via regression-as-classification,

    E. Guha, S. Natarajan, T. M ¨ollenhoff, M. E. Khan, and E. Ndiaye, “Conformal prediction via regression-as-classification,” arXiv preprint arXiv:2404.08168, 2024

    2024 arXiv

  7. [15]

    Colorful image colorization,

    R. Zhang, P. Isola, and A. A. Efros, “Colorful image colorization,” in Computer Vision–ECCV 2016: 14th European Conference, Amster- dam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14 . Springer, 2016, pp. 649–666

    2016

  8. [16]

    Deep ordinal regression network for monocular depth estimation,

    H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, “Deep ordinal regression network for monocular depth estimation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2002–2011

    2018

  9. [17]

    Dex: Deep expectation of apparent age from a single image,

    R. Rothe, R. Timofte, and L. Van Gool, “Dex: Deep expectation of apparent age from a single image,” in Proceedings of the IEEE international conference on computer vision workshops , 2015, pp. 10– 15

    2015

  10. [18]

    Lcr-net: Localization- classification-regression for human pose,

    G. Rogez, P. Weinzaepfel, and C. Schmid, “Lcr-net: Localization- classification-regression for human pose,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3433– 3441

    2017

  11. [19]

    On monocular depth estimation and uncertainty quantification using classification approaches for regression,

    X. Yu, G. Franchi, and E. Aldea, “On monocular depth estimation and uncertainty quantification using classification approaches for regression,” in 2022 IEEE International Conference on Image Processing (ICIP) . IEEE, 2022, pp. 1481–1485

    2022

  12. [20]

    Strictly proper scoring rules, prediction, and estimation,

    T. Gneiting and A. E. Raftery, “Strictly proper scoring rules, prediction, and estimation,” Journal of the American statistical Association , vol. 102, no. 477, pp. 359–378, 2007

    2007

  13. [21]

    Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks,

    B. Yin, F. Corradi, and S. M. Boht ´e, “Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks,” Nature Machine Intelligence, vol. 3, no. 10, pp. 905–913, 2021

    2021

  14. [22]

    Incorporating learnable membrane time constant to enhance learning of spiking neural networks,

    W. Fang, Z. Yu, Y . Chen, T. Masquelier, T. Huang, and Y . Tian, “Incorporating learnable membrane time constant to enhance learning of spiking neural networks,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 2661–2671

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.