Pith. sign in

REVIEW 3 major objections 4 minor 47 references

Training of Spiking Neural Networks with Expectation-Propagation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Expectation-propagation can train spiking neural networks without gradients, learning distributions over discrete or continuous weights in mini-batches.

desk verdict Solid EP-extension paper whose mini-batch innovation is real, but the load-bearing mean-substitution approximation in Eq. (17) needs quantification and the experiments need strengthening. read the letter →

arxiv 2506.23757 v1 pith:HSRVAFOK submitted 2025-06-30 cs.LG stat.MEstat.ML

Dan Yao , Steve McLaughlin , Yoann Altmann This is my paper · ORCID
classification cs.LGstat.MEstat.ML
keywords spikingneuralnetworksexpectation-propagationBayesianinferencegradient-freetrainingdiscreteweightsstochasticneuronsuncertaintyquantificationmini-batchlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that Expectation-Propagation can train spiking neural networks by message passing instead of gradient descent, and that this is the first framework to cover all four settings previously handled separately: continuous or discrete weights and deterministic (Heaviside) or stochastic (Bernoulli-sigmoid) spiking neurons. The method, called EPSNN, returns approximate posterior distributions over the weights while integrating out the hidden layers' spike outputs, so uncertainty estimates come as a by-product. Because it combines stochastic and average EP updates, it can learn from mini-batches and make multiple passes through the data. The experiments on MNIST classification and 1D regression suggest the algorithm converges faster than gradient-based training, needs fewer passes through the data, and gives predictive intervals that widen in regions where training data are missing.

What carries the argument

The central machinery is a factor graph in which each network layer is split into a mixing block, enforcing the linear constraint $u = W v$, and an activation block, enforcing the spiking likelihood $f(v|u)$. Every variable group carries a two-factor approximation $q(W,\Theta)=q_0 q_1$, and updates are done by KL-based moment matching between a cavity distribution (the current approximation with the factor being updated removed) and the tilted distribution (the cavity times that factor). The mixing-block update handles each delta constraint $\delta(u - w^T v)$ with closed-form Gaussian moment matching; the backward pass becomes tractable by fixing the presynaptic spike $v$ in the denominator of the weight update to its cavity mean. Mini-batch learning is handled by stochastic EP, which treats the weight factor as an average over batches and creates a cavity $q(W)/q_1(W)$ for each batch.

What would settle it

For a small two-layer SNN with Bernoulli-sigmoid activations and low firing rates, compare the weight-update moments produced by replacing each presynaptic spike with its current average (the rule in Section IV-B) against exact moments computed by numerical integration or Monte Carlo from the same approximate distribution; a systematic divergence as firing rate drops would show where that approximation breaks.

Watch

Extended reading notes

Core claim

The paper's central claim is that the intractable Bayesian posterior over SNN weights and latent activities can be approximated by a factorized message-passing distribution, updated by Expectation-Propagation, in a way that preserves the familiar forward/backward pass structure of neural network training. A forward pass propagates approximate moments of membrane potentials and spike outputs through mixing and activation blocks; a backward pass refines the weight and activity factors by matching tilted moments. The result is a training algorithm that estimates marginal weight distributions, supports discrete and continuous weight families, handles deterministic and stochastic spiking activations in one framework, and produces predictive distributions at inference time from a single forward pass without Monte Carlo sampling.

Load-bearing premise

The backward pass assumes each presynaptic spike value can be replaced by its current average when computing weight updates, and if spike activity is sparse or highly variable, that substitution may bias the learned weight distributions.

Editorial extensions

If this is right

  • SNNs with binary weights can be trained directly, without surrogate gradients or relaxation tricks, and the trained model carries a posterior probability for every weight.
  • One implementation covers both Heaviside and Bernoulli-sigmoid neurons, so switching between deterministic and stochastic spiking models does not require a different training loop.
  • Predictive means and variances at test time come from one forward pass through the factor graph, which makes Bayesian uncertainty practical on hardware where repeated sampling is too expensive.
  • Sparse priors such as Bernoulli-Gaussian produce per-weight activation probabilities, giving feature-relevance maps that are not available from standard point-estimate training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the mean-replacement approximation in the backward pass should be stress-tested in the low-firing-rate regime; if its bias grows as spike variance increases, the uncertainty estimates would be least reliable precisely in sparse networks where they matter most.
  • The factor-graph block structure suggests a path to convolutional and recurrent spiking layers, but shared weights and temporal dependencies would require new cavity derivations rather than direct reuse of the present equations.
  • Because predictive entropy is cheap to compute, a natural extension is out-of-distribution detection for neuromorphic sensors, where input spike patterns that fall outside the training manifold should inflate the predictive variance; the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes EPSNN, a message-passing framework based on Expectation-Propagation (EP) for training spiking neural networks (SNNs). The method combines average EP and stochastic EP to handle mini-batches, and it is designed to estimate marginal posterior distributions of network weights, including discrete or continuous weights, for deterministic or stochastic spiking activations. The authors present the Bayesian model, derive the local EP updates for mixing and activation blocks, discuss algorithmic considerations such as damping and convergence stabilization, and report experiments on MNIST classification and a 1D regression task. The paper claims that this is the first framework enabling training of discrete and continuous weights for deterministic and stochastic SNNs using batches of training samples, while also providing uncertainty estimates without Monte Carlo sampling.

Significance. If the method is correct, it constitutes a useful contribution to Bayesian training of SNNs: it is gradient-free, provides marginal weight uncertainties, handles discrete weights, and supports mini-batch training through SEP. The paper gives a clear modular derivation and includes pseudo-code for the main algorithms, which aids reproducibility. The experimental results, while limited to small architectures and datasets, are plausible and show competitive performance with an MLE baseline. However, the central algorithmic approximation is not quantitatively validated, and several details needed to reproduce the algorithm are omitted, which tempers the strength of the claims.

major comments (3)
  1. [Section V-B, Convergence paragraph] The approximation that replaces the Bernoulli presynaptic spike v_{n,j}^{(ell-1)} by its mean under the cavity distribution q_c is a load-bearing step in the derivation of the weight updates. For a Bernoulli variable, conditioning on v=1 versus v=0 yields qualitatively different constraints on the weight: when v=0, the delta factor does not constrain w_{i,j} at all. Substituting the mean collapses this mixture and discards the zero-spike branch, which can systematically bias both the mean and the variance of the estimated weight posterior, particularly for sparse or highly variable activations. This bias propagates through every hidden-layer weight update and affects all four claimed variants of the method (discrete/continuous weights, deterministic/stochastic activations). The paper provides no ablation, diagnostic, or theoretical bound quantifying this bias, and no fallback for the case where the mean of v is near zero, which would make the denominator in Eq. (17) unstable. Please provide an analysis of the bias or an alternative update that marginalizes over the Bernoulli variable, and demonstrate its effect in a controlled experiment with varied spike rates.
  2. [Section V-B, Convergence paragraph] The manuscript states that stability is achieved by bounding the natural parameters of the Gaussian approximating factors and by using constrained KL minimization for categorical distributions, but the derivation of these constrained updates is omitted, with the text saying it is 'similar to that in [37]'. Since these constraints are necessary to prevent negative variances and to keep the algorithm stable in the reported experiments, their omission prevents full reproduction of the method. Please include the derivations in a supplementary document or provide a precise specification of the constraints and the optimization procedure used, as this is essential for the paper's claims to be independently verified.
  3. [Section VI, Experiments] The abstract's 'first time' claim is supported by only a narrow experimental evaluation: MNIST with up to 10^4 training samples, a one-layer and a two-layer architecture, and a single 1D regression problem. Moreover, the text in Section VI says the regression experiment allows visual comparison with the BiSNN method, but no BiSNN results are shown; the comparison is only against a basic MLE baseline. There is also no wall-clock time comparison, despite the claim of faster convergence than gradient-based methods. Please either broaden the experimental evaluation (e.g., compare with existing Bayesian SNN methods such as BiSNN, report runtimes, include a deeper architecture or a more realistic dataset) or moderate the 'first time' and 'faster than gradient-based methods' claims to match the evidence presented.
minor comments (4)
  1. [Tables I, II, III captions] The phrase 'The first (resp. second) values are the metrics computed on the test (resp. test) data' appears to contain a typo; presumably the first value is on the training data and the second on the test data. Please correct the captions.
  2. [Section IV-B, Eq. (16)] In Eq. (16), the numerator uses u_{i,j}, but the membrane potential notation elsewhere is u_{n,i} (or u_{i,n} is defined for sample n and neuron i). Please standardize the subscripts to avoid confusion.
  3. [Section VI-B] There is a grammatical error: 'The training sets consists of up to 10^4 frames' should be 'The training set consists of up to 10^4 frames'.
  4. [Throughout] The paper uses q_c(v_n^{(l)}) and q_c(v_n^{(l-1)}) interchangeably in the cavity definitions; please ensure notation is consistent, especially in Eqs. (13) and (20).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EP/SEP training derivation is self-contained and its predictions are genuine posterior computations, not fitted inputs.

full rationale

The paper's derivation chain is a standard EP construction: it defines a posterior p(W,Theta|D_N) proportional to a prior times a factorized likelihood, introduces a factorized surrogate q, and obtains q by moment matching of tilted distributions (Eqs. (10)-(20)). The mixing-block update replaces the presynaptic Bernoulli spike by its cavity mean in Eq. (17); this is an uncontrolled approximation and a correctness/robustness risk, but it is not circular because the update is not defined in terms of the quantity being predicted. The SEP mini-batch treatment (Eqs. (22)-(24)) uses the factorization of [34] and is also not circular: q1(W) is updated from batch-local tilted distributions and combined with the cavity, with the data accounting handled by the (B-1)/B exponent. Self-citations to [31] and [38] are contextual (related prior EP uses) and are not load-bearing: the central updates cite [32] and [34] (external) plus the new approximation in Eq. (17). Inference-time predictive moments are computed from the trained approximate posterior by a forward pass, not from constants fitted to the test outputs. The reported experiments compare EPSNN against MLE and BiSNN on held-out data, providing external checkpoints. The paper's own stated limitations (non-guaranteed convergence, omitted constrained-optimization derivation, unquantified bias of Eq. (17)) are honest caveats rather than circularity. I therefore find no circular step and assign score 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard EP factorization assumptions, the SEP mini-batch approximation, and an ad-hoc mean-fixing approximation in the weight update. These are reasonable but not fully justified; the method's convergence depends on damping heuristics.

free parameters (2)
  • AEP damping coefficient = 0.7
    Set by hand to stabilize the inner AEP updates (Section V-B); convergence is not guaranteed and this value is architecture-dependent.
  • Gaussian natural parameter bounds = not specified
    The paper proposes bounding the two natural parameters of Gaussian factors to prevent negative variances, but the actual bounds are not reported (Section V-B).
assumptions (5)
  • domain assumption The approximate posterior q(W,Theta) factorizes as q(W)q(Theta) with separable exponential-family marginals (Gaussian for membrane potentials and continuous weights, Bernoulli/categorical for spikes and discrete weights).
    Section IV-A, Eq. (10); standard mean-field EP assumption.
  • domain assumption The SEP representation q(W) proportional to (q1(W))^B q0(W) holds for mini-batch training, where q1 is the average batch contribution.
    Section V-A, Eq. (22), from [34]; assumes batch contributions are exchangeable.
  • ad hoc to paper The presynaptic spike v_{n,j}^{(ell-1)} can be fixed to its mean q_c when deriving weight updates (Eq. 17), making the moments of eta*_w tractable.
    Section IV-B, Eq. (17); introduced without justification of its accuracy.
  • standard math The KL minimizations can be solved by moment matching within the exponential family, even when the tilted distributions are not in the family.
    Standard EP assumption; used in Sections IV-B and IV-C.
  • domain assumption Damping (geometric averaging) with coefficient 0.7 in AEP and 1/B in SEP is sufficient to stabilize convergence, although global convergence is not guaranteed.
    Section V-B; empirical heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training of Spiking Neural Networks with Expectation-Propagation." pith.science (2026). https://pith.science/paper/HSRVAFOK

@misc{pith2026250623757,
  author       = {Pith},
  title        = {Pith review of: Training of Spiking Neural Networks with Expectation-Propagation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSRVAFOK}},
  note         = {Machine review of arXiv:2506.23757}
}
read the original abstract

In this paper, we propose a unifying message-passing framework for training spiking neural networks (SNNs) using Expectation-Propagation. Our gradient-free method is capable of learning the marginal distributions of network parameters and simultaneously marginalizes nuisance parameters, such as the outputs of hidden layers. This framework allows for the first time, training of discrete and continuous weights, for deterministic and stochastic spiking networks, using batches of training samples. Although its convergence is not ensured, the algorithm converges in practice faster than gradient-based methods, without requiring a large number of passes through the training data. The classification and regression results presented pave the way for new efficient training methods for deep Bayesian networks.

Figures

Figures reproduced from arXiv: 2506.23757 by the authors.

Figure 1
Figure 1. Factor graph used to compute approximating distributions in layered SNNs (single timestamp). The orange (resp. blue) rectangles correspond to factor [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Convergence analysis of MLE-based optimization and EPSNN [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Estimated weights associated with the output of the ’0’ digit (left) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Estimated weight distributions for the MNIST classification problem [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Top: posterior probability of activation of the 120 features of the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Example of population encoding for the regression problem. For each [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Example of 1D regression with population encoding using MLE (top) [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 41 canonical work pages

  1. [37]

    Expectation propagation in linear regression models with spike-and- slab priors,

    J. M. Hernández-Lobato, D. Hernández-Lobato, and A. Suárez, “Expectation propagation in linear regression models with spike-and- slab priors,” Machine Learning, vol. 99, no. 3, pp. 437–487, 2015

    work page 2015

  2. [1]

    Networks of spiking neurons: the third generation of neural network models,

    W. Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural networks, vol. 10, no. 9, pp. 1659–1671, 1997

    1997

  3. [2]

    Deep learning in spiking neural networks,

    A. Tavanaei, M. Ghodrati, S. R. Kheradpisheh, T. Masquelier, and A. Maida, “Deep learning in spiking neural networks,” Neural networks, vol. 111, pp. 47–63, 2019

    work page 2019

  4. [3]

    A review of learning in biologically plausible spiking neural networks,

    A. Taherkhani, A. Belatreche, Y . Li, G. Cosma, L. P. Maguire, and T. M. McGinnity, “A review of learning in biologically plausible spiking neural networks,” Neural Networks, vol. 122, pp. 253–272, 2020. 10

    work page 2020

  5. [4]

    Memory and information processing in neuromorphic systems,

    G. Indiveri and S.-C. Liu, “Memory and information processing in neuromorphic systems,” Proceedings of the IEEE , vol. 103, no. 8, pp. 1379–1397, 2015

    work page 2015

  6. [5]

    N. K. Kasabov, Time-space, spiking neural networks and brain-inspired artificial intelligence. Springer, 2019

    work page 2019

  7. [6]

    Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks,

    M. Mozafari, M. Ganjtabesh, A. Nowzari-Dalini, S. J. Thorpe, and T. Masquelier, “Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks,” Pattern recognition, vol. 94, pp. 87–95, 2019

    work page 2019

  8. [7]

    Spiking-yolo: spiking neural network for energy-efficient object detection,

    S. Kim, S. Park, B. Na, and S. Yoon, “Spiking-yolo: spiking neural network for energy-efficient object detection,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 07, 2020, pp. 11 270–11 277

    work page 2020

Show all 47 references
  1. [8]

    Training a spiking neural network to control a 4-dof robotic arm based on spike timing-dependent plasticity,

    A. Bouganis and M. Shanahan, “Training a spiking neural network to control a 4-dof robotic arm based on spike timing-dependent plasticity,” in The 2010 International Joint Conference on Neural Networks (IJCNN). IEEE, 2010, pp. 1–8

    2010

  2. [9]

    Pose estimation and map formation with spiking neural networks: towards neuromorphic slam,

    R. Kreiser, A. Renner, Y . Sandamirskaya, and P. Pienroj, “Pose estimation and map formation with spiking neural networks: towards neuromorphic slam,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 2159–2166

    2018

  3. [10]

    Simultaneous localization and mapping for event-based vision systems,

    D. Weikersdorfer, R. Hoffmann, and J. Conradt, “Simultaneous localization and mapping for event-based vision systems,” in Computer Vision Systems: 9th International Conference, ICVS 2013, St. Petersburg, Russia, July 16-18, 2013. Proceedings 9 . Springer, 2013, pp. 133–142

    2013

  4. [11]

    Conversion of continuous-valued deep networks to efficient event- driven networks for image classification,

    B. Rueckauer, I.-A. Lungu, Y . Hu, M. Pfeiffer, and S.-C. Liu, “Conversion of continuous-valued deep networks to efficient event- driven networks for image classification,” Frontiers in neuroscience , vol. 11, p. 682, 2017

    2017

  5. [12]

    Real- time classification and sensor fusion with a spiking deep belief network,

    P. O’Connor, D. Neil, S.-C. Liu, T. Delbruck, and M. Pfeiffer, “Real- time classification and sensor fusion with a spiking deep belief network,” Frontiers in neuroscience, vol. 7, p. 178, 2013

    2013

  6. [13]

    Fast- classifying, high-accuracy spiking deep networks through weight and threshold balancing,

    P. U. Diehl, D. Neil, J. Binas, M. Cook, S.-C. Liu, and M. Pfeiffer, “Fast- classifying, high-accuracy spiking deep networks through weight and threshold balancing,” in 2015 International joint conference on neural networks (IJCNN). ieee, 2015, pp. 1–8

    2015

  7. [14]

    Going deeper in spiking neural networks: Vgg and residual architectures,

    A. Sengupta, Y . Ye, R. Wang, C. Liu, and K. Roy, “Going deeper in spiking neural networks: Vgg and residual architectures,” Frontiers in neuroscience, vol. 13, p. 95, 2019

    2019

  8. [15]

    Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,

    E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,” IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 51–63, 2019

    2019

  9. [16]

    The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks,

    F. Zenke and T. P. V ogels, “The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks,” Neural computation, vol. 33, no. 4, pp. 899–925, 2021

    2021

  10. [17]

    An introduction to probabilistic spiking neural networks: Probabilistic models, learning rules, and applications,

    H. Jang, O. Simeone, B. Gardner, and A. Gruning, “An introduction to probabilistic spiking neural networks: Probabilistic models, learning rules, and applications,” IEEE Signal Processing Magazine , vol. 36, no. 6, pp. 64–77, 2019

    2019

  11. [18]

    Matching recall and storage in sequence learning with spiking neural networks,

    J. Brea, W. Senn, and J.-P. Pfister, “Matching recall and storage in sequence learning with spiking neural networks,” Journal of neuroscience, vol. 33, no. 23, pp. 9565–9575, 2013

    2013

  12. [19]

    Noise as a resource for computation and learning in networks of spiking neurons,

    W. Maass, “Noise as a resource for computation and learning in networks of spiking neurons,” Proceedings of the IEEE , vol. 102, no. 5, pp. 860– 880, 2014

    2014

  13. [20]

    Lightweight probabilistic deep networks,

    J. Gast and S. Roth, “Lightweight probabilistic deep networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3369–3378

    2018

  14. [21]

    Bayesian continual learning via spiking neural networks,

    N. Skatchkovsky, H. Jang, and O. Simeone, “Bayesian continual learning via spiking neural networks,” Frontiers in Computational Neuroscience, vol. 16, p. 1037976, 2022

    2022

  15. [22]

    What uncertainties do we need in bayesian deep learning for computer vision?

    A. Kendall and Y . Gal, “What uncertainties do we need in bayesian deep learning for computer vision?” Advances in neural information processing systems, vol. 30, 2017

    2017

  16. [23]

    A survey of uncertainty in deep neural networks,

    J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscheret al., “A survey of uncertainty in deep neural networks,” Artificial Intelligence Review , vol. 56, no. Suppl 1, pp. 1513–1589, 2023

    2023

  17. [24]

    Stochastic dynamics as a principle of brain function,

    G. Deco, E. T. Rolls, and R. Romo, “Stochastic dynamics as a principle of brain function,” Progress in neurobiology , vol. 88, no. 1, pp. 1–16, 2009

    2009

  18. [25]

    Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons,

    L. Buesing, J. Bill, B. Nessler, and W. Maass, “Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons,” PLoS computational biology , vol. 7, no. 11, p. e1002211, 2011

    2011

  19. [26]

    Bisnn: training spiking neural networks with binary weights via bayesian learning,

    H. Jang, N. Skatchkovsky, and O. Simeone, “Bisnn: training spiking neural networks with binary weights via bayesian learning,” in 2021 IEEE Data Science and Learning Workshop (DSLW) . IEEE, 2021, pp. 1–6

    2021

  20. [27]

    Expectation propagation for approximate Bayesian inference,

    T. P. Minka, “Expectation propagation for approximate Bayesian inference,” in Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, ser. UAI’01. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001, p. 362–369

    2001

  21. [28]

    M. J. Wainwright and M. I. Jordan, Graphical Models, Exponential Families, and Variational Inference . Now Publishers Inc, 2008

    2008

  22. [29]

    Expectation propagation for neural networks with sparsity-promoting priors,

    P. Jylänki, A. Nummenmaa, and A. Vehtari, “Expectation propagation for neural networks with sparsity-promoting priors,” The Journal of Machine Learning Research , vol. 15, no. 1, pp. 1849–1901, 2014

    1901

  23. [30]

    Assumed density filtering methods for learning bayesian neural networks,

    S. Ghosh, F. Delle Fave, and J. Yedidia, “Assumed density filtering methods for learning bayesian neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 30, no. 1, 2016

    2016

  24. [31]

    Probabilistic spiking neural networks training with expectation-propagation,

    D. Shen, D. Yao, S. McLaughlin, and Y . Altmann, “Probabilistic spiking neural networks training with expectation-propagation,” in 2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) . IEEE, 2023, pp. 41–45

    2023

  25. [32]

    Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights,

    D. Soudry, I. Hubara, and R. Meir, “Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights,” Advances in neural information processing systems , vol. 27, 2014

    2014

  26. [33]

    Probabilistic backpropagation for scalable learning of bayesian neural networks,

    J. M. Hernández-Lobato and R. Adams, “Probabilistic backpropagation for scalable learning of bayesian neural networks,” in International conference on machine learning . PMLR, 2015, pp. 1861–1869

    2015

  27. [34]

    Stochastic expectation propagation,

    Y . Li, J. M. Hernández-Lobato, and R. E. Turner, “Stochastic expectation propagation,” Advances in neural information processing systems , vol. 28, 2015

    2015

  28. [35]

    Expectation propagation in the large data limit,

    G. Dehaene and S. Barthelmé, “Expectation propagation in the large data limit,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 80, no. 1, pp. 199–217, 2018

    2018

  29. [36]

    Time structure of the activity in neural network models,

    W. Gerstner, “Time structure of the activity in neural network models,” Physical review E , vol. 51, no. 1, p. 738, 1995

    1995

  30. [38]

    Fast scalable image restoration using total variation priors and expectation propagation,

    D. Yao, S. McLaughlin, and Y . Altmann, “Fast scalable image restoration using total variation priors and expectation propagation,” IEEE Trans. Image Processing, vol. 31, pp. 5762–5773, 2022

    2022

  31. [39]

    On expectation propagation for generalised, linear and mixed models,

    A. S. Kim and M. P. Wand, “On expectation propagation for generalised, linear and mixed models,” Australian & New Zealand Journal of Statistics, vol. 60, no. 1, pp. 75–102, 2018

    2018

  32. [40]

    Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables,

    J. Daunizeau, “Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables,” arXiv preprint arXiv:1703.00091, 2017

    2017 arXiv

  33. [41]

    A family of algorithms for approximate bayesian inference,

    T. Minka, “A family of algorithms for approximate bayesian inference,” PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2001, available at https://tminka.github.io/papers/ep/

    2001

  34. [42]

    Training spiking neural networks using lessons from deep learning,

    J. K. Eshraghian, M. Ward, E. Neftci, X. Wang, G. Lenz, G. Dwivedi, M. Bennamoun, D. S. Jeong, and W. D. Lu, “Training spiking neural networks using lessons from deep learning,” Proceedings of the IEEE , vol. 111, no. 9, pp. 1016–1054, 2023

    2023

  35. [43]

    The mnist database of handwritten digit images for machine learning research,

    L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine , vol. 29, no. 6, pp. 141–142, 2012

    2012

  36. [44]

    Eliasmith and C

    C. Eliasmith and C. H. Anderson, Neural Engineering (Computational Neuroscience Series): Computational, Representation, and Dynamics in Neurobiological Systems. Cambridge, MA, USA: MIT Press, 2002

    2002

  37. [45]

    Synaptic plasticity dynamics for deep continuous local learning (decolle),

    J. Kaiser, H. Mostafa, and E. Neftci, “Synaptic plasticity dynamics for deep continuous local learning (decolle),” Frontiers in Neuroscience , vol. 14, p. 424, 2020

    2020

  38. [46]

    Computation with spikes in a winner-take-all network,

    M. Oster, R. Douglas, and S.-C. Liu, “Computation with spikes in a winner-take-all network,” Neural computation, vol. 21, no. 9, pp. 2437– 2465, 2009

    2009

  39. [47]

    V owel: A local online learning rule for recurrent networks of probabilistic spiking winner- take-all circuits,

    H. Jang, N. Skatchkovsky, and O. Simeone, “V owel: A local online learning rule for recurrent networks of probabilistic spiking winner- take-all circuits,” in 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021, pp. 4597–4604

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.