Pith. sign in

REVIEW 3 major objections 6 minor 42 references

This paper argues that a biologically inspired sparse learning rule, combining competitive Hebbian plasticity with weight perturbation, stores more usable information per synapse than backpropagation on MNIST while retaining far fewer conne

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 19:51 UTC pith:NKBMNFJI

load-bearing objection The headline efficiency claim is not established: the mutual information 'estimate' in Table 2 is actually an upper bound, and with β=0 it can be made arbitrarily loose, so the C_S ranking may be an artifact. the 3 major comments →

arxiv 2603.00588 v1 pith:NKBMNFJI submitted 2026-02-28 cs.LG cs.NE

Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning

Patrick Stricker , Florian R\"ohrbein , Andreas Knoblauch This is my paper
classification cs.LG cs.NE
keywords biologically inspired learningcompetitive Hebbian plasticityweight perturbationsynaptic capacitymutual informationMNIST classificationsparse neural networksassociative memory
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that its biologically inspired learning rule—built from competitive Hebbian plasticity and weight perturbation—uses synaptic resources far more efficiently than standard or constrained backpropagation on an MNIST classification task. By framing MNIST as a heteroassociative memory and measuring mutual information between input and a latent encoding, the authors compute a 'synaptic capacity' in bits per nonsilent synapse. Their central claim is that this rule achieves the highest synaptic capacity across every tested hidden-layer size, because it keeps only essential synapses and stores fewer total bits while preserving near-BP accuracy at larger widths. If true, this suggests training rules that mimic the brain's structural plasticity could produce smaller, more energy-efficient networks with less redundant storage.

Core claim

The paper's central claim is that a sparse, locally-updated learning rule can outperform gradient-based backpropagation in representation efficiency: on the 1/2/6 subset of MNIST, the rule achieves the highest synaptic capacity C_S = I(X;Z)/(#nonsilent synapses) at 10, 30, 100, and 200 hidden units, even though it stores fewer total bits of estimated mutual information than BP. The authors attribute this to the rule's competitive Hebbian updates, which drive weights toward a sparse solution determined mainly by the input covariance, and to weight perturbation in the classification layer, which prunes redundant connections. They interpret this as evidence that structural plasticity—rather tha

What carries the argument

The load-bearing machinery is the synaptic capacity metric C_S (bits of stored mutual information per nonsilent synapse) plus three interacting update rules: competitive Hebbian plasticity in the hidden layer, which promotes sparse, nonnegative weights; weight perturbation in the classification layer, which adjusts weights based on local perturbations and error differences; and a homeostatic bias rule. Mutual information I(X;Z) is estimated by fitting a variational encoder to the frozen hidden representation with beta=0. The metric and the estimate together do the work: they convert 'efficient use of synapses' into a single comparable number.

Load-bearing premise

The mutual-information estimate obtained by fitting a stochastic encoder with beta=0 to the frozen hidden layer is accurate and comparable across training rules, so that the synaptic-capacity ranking reflects real differences in stored information.

What would settle it

Recompute C_S for the same trained networks using a tighter or ground-truth mutual-information estimator (e.g., a nonparametric estimate on the hidden codes, or repeated variational fits with different seeds); if the sparse rule no longer has the highest bits-per-synapse at all hidden sizes, the paper's central efficiency claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, biologically inspired sparse rules offer a path to networks that need far fewer stored parameters to reach comparable accuracy on classification tasks.
  • The method eliminates the need for explicit network architecture pre-optimization, because sparsity emerges from local competition rather than from pruning after training.
  • The measured advantage grows with hidden-layer width (C_S rises to 0.475 bits/synapse at 200 units, far above backpropagation), indicating the efficiency benefit is most pronounced in larger networks.
  • Because the rule stores fewer bits overall while keeping only essential synapses, it points toward lower energy consumption per bit of stored information, relevant to large-scale model deployment.
  • The approach reframes classification as heteroassociative memory, suggesting that associative-memory metrics can serve as a design objective for energy-efficient learning systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the variational estimate is tight, a natural extension is to use the same C_S analysis to compare other sparse learning rules and see whether the efficiency ranking persists under tighter mutual-information estimators.
  • The paper's efficiency gains appear at the cost of lower accuracy at small hidden-layer sizes (64% vs 99% at 10 units); a reader might infer that the sweet spot for the rule is wide, overparameterized regimes, where backpropagation stores the most redundancy.
  • The beta=0 variational encoder is fit to the frozen hidden representation, not the trained network's true conditional; an immediate testable extension is to verify the C_S ranking with a nonparametric or ground-truth information estimate on the same hidden codes.
  • The paper's framing suggests that 'reserving space for new memories' could be operationalized as maintaining low total stored information after task learning, which might transfer to continual learning settings if the rule's sparsity carries over to multiple tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper analyzes a biologically inspired learning rule (competitive Hebbian plasticity plus weight perturbation) on an MNIST subset containing digits 1, 2, and 6. It frames classification as a heteroassociative memory task, extracts a deterministic hidden representation H from a pretrained network, fits a stochastic encoder q(Z|H) with a Variational Information Bottleneck-style setup, and estimates the mutual information I(X;Z). It then defines synaptic capacity C_S = I(X;Z) / number of nonsilent synapses and compares the rule with backpropagation and Chorowski's constrained BP. For hidden-layer sizes 10, 30, 100, and 200, the paper reports that the proposed rule stores fewer bits than the benchmarks and achieves the highest C_S in every configuration, concluding that the rule is more energy-efficient, prevents overparameterization, and retains only essential synapses.

Significance. If the central claim were supported, the paper would provide a useful demonstration that a sparse Hebbian/weight-perturbation rule can use synaptic resources more efficiently than BP on a small vision task, with potential implications for neuromorphic and sustainable AI. The paper reports raw accuracy and mutual-information values, which is helpful for assessing the claim, and it uses a capacity metric from prior associative-memory work. However, the main numerical conclusion rests on an unvalidated variational estimate of mutual information and on a characterization of the accuracy gap as 'comparable' that is not supported by the Table 2 numbers. These issues are load-bearing and need to be addressed before the efficiency claim can be accepted.

major comments (3)
  1. [§2.3, Eq. (5), Table 2] The mutual-information estimate is unvalidated and is an upper bound, not an estimate. For any fixed variational marginal r(z), E_x D_KL[q(z|x)||r(z)] = I(X;Z) + D_KL[q_marginal(z)||r(z)] >= I(X;Z). Setting beta=0 removes the KL penalty that would keep the variational marginal close to the prior, so the reported 'I(X,Z)' can be arbitrarily inflated by the fitted encoder's variance. Because BP, constrained BP, and the proposed rule produce very different hidden-activation statistics (dense continuous vs sparse nonnegative), the looseness is likely method-dependent, which would directly bias the C_S ranking. No ground-truth MI, no alternative nonparametric estimator, no repeated seeds, and no posterior variances are provided. The statement in §2.3 that beta=0 'still allow[s] accurate mutual information estimation' is unsupported. Every C_S in Table 2 divides by this numerator, so this is t
  2. [Table 2, all rows] The accuracy claims are internally inconsistent. The Abstract, Introduction, and Discussion describe the performance as 'comparable' to BP and 'slightly inferior', but Table 2 shows 64.29%, 86.21%, 89.79%, and 95.55% for the proposed rule versus 99.01-99.23% for BP and 98.34-99.10% for Chorowski. The gap is up to 35 percentage points for the 10-hidden-unit case, which is not 'comparable' or 'slight'; at 10 units the model is only about 31 points above chance for a 3-class task. This undermines the practical efficiency claim, because a near-trivial model can have high bits-per-nonsilent-synapse while being of little classification value. Either report results at matched accuracy or revise the strength of the 'comparable' claim.
  3. [§3.1, Eq. (5), Table 2] The denominator of C_S is never reported. No values are given for the number of nonsilent synapses, no definition is provided of which weights count as nonsilent (hidden layer only? classification layer? bias weights?), and no code, seeds, or standard deviations are supplied. Since the paper's main empirical conclusion is that the proposed rule achieves the highest synaptic capacity, the reader cannot verify the sparsity advantage or recalculate C_S without these counts. Reporting the raw synapse counts and a precise counting rule is essential for reproducibility.
minor comments (6)
  1. [Table 3] Table 3 is a dangling entry: it is labeled 'Table 3: *' and has a caption describing metrics, but no table body and no footnote explaining the asterisk. Either complete or remove it.
  2. [References] Reference [16] is cited as the source of the 'Variational Information Bottleneck (VIB) layer', but the reference is Hoffmann and Mäder on synaptic scaling. The correct VIB reference appears to be Alemi et al. [8]. Please correct the citation.
  3. [§2.3 and §3.1] The encoder parameterization is ambiguous. §2.3 says the encoding layer is parameterized by 2K parameters where K is the size of the layer, while §3.1 says 2K=M, where M is the number of hidden units. Please define K and M explicitly and consistently.
  4. [Abstract and Introduction] The claim that networks can become 'overparameterized by factors of up to 13' is not supported by any citation in this manuscript. Please provide a source or qualify the statement.
  5. [§3.1] The modified sigmoid activation is described as transforming outputs from [0.5,1.0] to [0.0,1.0]. The exact affine transformation should be stated explicitly, since it changes the distribution of hidden activations.
  6. [§2.3] The Markov-chain claim says the property is 'naturally satisfied by our Hebbian learning rule, which updates weights based on local activity'. The Markov property concerns the sequence of representations, not the learning rule. Please rephrase.

Circularity Check

2 steps flagged

Highest-synaptic-capacity claim rests on a self-cited metric plus a fitted, unvalidated variational MI 'estimate' with beta=0.

specific steps
  1. fitted input called prediction [§2.3, Eq. (5), Table 2]
    "In this study, we depart from the conventional methodology by incorporating a deterministic bottleneck in the form of a pretrained hidden layer preceding the variational encoder, while setting the β parameter to zero. This adjustment removes the typical information bottleneck, ensuring that compression is solely induced by the frozen hidden representations while still allowing for accurate mutual information estimation. ... We then estimate the mutual information I(X;Z) using the Kullback–Leibler divergence D_KL [p(Z|X)‖p(Z)]."

    The reported I(X;Z) is not a direct measurement of information stored by each learning rule; it is the KL term of a stochastic encoder q(Z|H) fitted to the frozen hidden representation H with β=0. For any variational marginal r(z), E_x KL(q(z|x)||r(z)) = I_q(X;Z) + KL(q_marg(z)||r(z)), so this quantity is an upper bound, not an estimate. With β=0, the encoder can collapse the posterior variance and inflate the bound arbitrarily. Table 2 feeds this fitted quantity into C_S, and the central claim 'highest synaptic capacity' is then read off an unvalidated bound that can bias sparse versus dense rules differently. No ground-truth MI, repeated seeds, posterior-variance reporting, or nonparametric check is provided.

  2. self citation load bearing [§2.4, Eq. (5); §3.3]
    "we evaluate neural network performance in terms of associative memory capabilities using the synaptic capacity C_S metric introduced by Knoblauch et al. [6, 13]. Synaptic capacity C_S is defined as: C_S = I(Z;X)/Number of nonsilent synapses... Our algorithm achieves the highest synaptic capacity across all settings, efficiently retaining only essential synapses while storing fewer bits compared to other methods."

    The paper's efficiency metric C_S is imported from the authors' own prior work, and the algorithm is explicitly designed to 'retain only the essential number of synapses'. The denominator is therefore small by the algorithm's design goal, and the headline conclusion that the method is 'most efficient' is largely a restatement of that design goal under a self-cited metric. The paper does report independent raw MI and accuracy values, so the result is not purely definitional, but the load-bearing efficiency verdict is scored with a ruler made by the same authors rather than an externally validated benchmark.

full rationale

The central result of the paper is that the biologically inspired rule achieves the highest synaptic capacity C_S = I(Z;X)/nonsilent synapses. Two load-bearing choices make this conclusion substantially self-supporting rather than independently established. First, C_S is a metric introduced in prior work co-authored by the present authors; it normalizes by nonsilent synapses, which is precisely the quantity the proposed algorithm is designed to minimize. Second, the numerator I(X;Z) is not measured from the trained networks but obtained by fitting a variational encoder with β=0 and calling the resulting KL divergence an 'accurate mutual information estimation'. Mathematically this KL is an upper bound that can be made arbitrarily loose when β=0, and no validation against ground-truth MI or any nonparametric estimator is given. Because the hidden activations of BP, constrained BP, and the authors' rule are very different in density and distribution, the bound's looseness can systematically favor the sparse rule. The paper does report raw MI and accuracy, so the result is not completely forced by definition; however, the headline 'highest synaptic capacity' is derived from a self-cited metric and a fitted, unvalidated estimate, which is a partial but significant circularity in the central claim.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claims rest on hand-set hyperparameters, a variational MI estimator whose accuracy is unvalidated, and the nonsilent-synapse normalization from co-authored prior work. No new physical or architectural entities are introduced.

free parameters (6)
  • η = 0.000158
    Learning rate in Eqs. (1)–(3); fixed value with no tuning or sensitivity analysis reported.
  • α = 0.1
    Weight for the Hebbian component in Eq. (2); hand-set.
  • β = 446.25
    Weight for the weight-perturbation component in Eq. (2); the large value suggests WP dominates the update.
  • γ = 0.1
    Homeostatic bias contribution in Eq. (4); hand-set.
  • σ² = 0.0157
    Perturbation strength in Eq. (3); hand-set, no sensitivity analysis.
  • initial weight range = U[0.01, 0.1]
    Weight initialization described in §3.1; affects which synapses survive competitive Hebbian learning.
axioms (5)
  • domain assumption The layers of the trained network form a Markov chain, allowing layerwise mutual-information analysis.
    Invoked in §2.3 following Tishby & Zaslavsky; standard in the information-bottleneck literature but an idealization of feedforward dynamics.
  • ad hoc to paper The variational encoder p(Z|H) with β=0 gives an accurate estimate of I(X;Z).
    In §2.3 and §3.1 no tightness check is performed; the stochastic layer is not trained under an IB objective, so the estimated MI may be loose or biased.
  • domain assumption Training minibatches contain equally distributed labels, so the mean target activation simplifies to 1/K.
    Stated in §2.2 before Eq. (4); if label distribution within minibatches is not balanced, the bias update rule changes.
  • domain assumption Nonsilent synapses are the correct cost model for energy and storage efficiency.
    Justifies the C_S metric in §2.4, Eq. (5), based on co-authored prior work [6,13,31-33]; the entire efficiency ranking depends on this normalization.
  • domain assumption Competitive Hebbian learning converges to weights determined by the input covariance, thereby learning minimal sufficient statistics.
    Asserted in §4 citing Zhou [35]; not derived or demonstrated in this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 8946 in / 18258 out tokens · 173435 ms · 2026-08-02T19:51:08.297028+00:00 · methodology

0 comments
read the original abstract

Efficient representation learning is essential for optimal information storage and classification. However, it is frequently overlooked in artificial neural networks (ANNs). This neglect results in networks that can become overparameterized by factors of up to 13, increasing redundancy and energy consumption. As the demand for large language models (LLMs) and their scale increase, these issues are further highlighted, raising significant ethical and environmental concerns. We analyze our previously developed biologically inspired learning rule using information-theoretic concepts, evaluating its efficiency on the MNIST classification task. The proposed rule, which emulates the brain's structural plasticity, naturally prevents overparameterization by optimizing synaptic usage and retaining only the essential number of synapses. Furthermore, it outperforms backpropagation (BP) in terms of efficiency and storage capacity. It also eliminates the need for pre-optimization of network architecture, enhances adaptability, and reflects the brain's ability to reserve 'space' for new memories. This approach advances scalable and energy-efficient AI and provides a promising framework for developing brain-inspired models that optimize resource allocation and adaptability.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 5 canonical work pages

  1. [1]

    To compress or not to compress–self-supervised learning and information theory: A review,

    R. Shwartz-Ziv and Y. LeCun, “To compress or not to compress–self-supervised learning and information theory: A review,” 2023. [Online]. Available:https: //arxiv.org/abs/2304.09355

    Pith/arXiv arXiv 2023

  2. [2]

    Survey on deep neural networks in speech and vision systems,

    M. Alam, M. D. Samad, L. Vidyaratne, A. Glandon, and K. M. Iftekharuddin, “Survey on deep neural networks in speech and vision systems,”Neurocomput- ing, vol. 417, pp. 302–321, 2020

    2020

  3. [3]

    Deep residual learning for image recog- nition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recog- nition,”CoRR, vol. abs/1512.03385, 2015. [Online]. Available:https://arxiv. org/abs/1512.03385

    Pith/arXiv arXiv 2015

  4. [4]

    Deep learning,

    Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”Nature, vol. 521, pp. 436– 444, 2015

    2015

  5. [5]

    Weight perturbation and competi- tive Hebbian plasticity for training sparse excitatory neural networks,

    P. Stricker, F. Röhrbein, and A. Knoblauch, “Weight perturbation and competi- tive Hebbian plasticity for training sparse excitatory neural networks,” inProc. 2024 Int. Joint Conf. Neural Networks (IJCNN), Yokohama, Japan, 2024, pp. 1–8, doi:10.1109/IJCNN60899.2024.10650478

    arXiv 2024

  6. [6]

    Structural plasticity, effectual connectivity, and memory in cortex,

    A. Knoblauch and F. T. Sommer, “Structural plasticity, effectual connectivity, and memory in cortex,”Front. Neuroanat., vol. 10, 2016

    2016

  7. [7]

    A theoretical framework for learning through structural plasticity,

    G. Tiddia, L. Sergi, and B. Golosio, “A theoretical framework for learning through structural plasticity,” 2024. [Online]. Available:https://arxiv.org/abs/2307. 11735

    2024

  8. [8]

    Deep variational in- formation bottleneck,

    A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy, “Deep variational in- formation bottleneck,” inProc. Int. Conf. Learn. Representations (ICLR), 2017, arXiv:1612.00410.https://doi.org/10.48550/arXiv.1612.00410 11

    Pith/arXiv arXiv doi:10.48550/arxiv.1612.00410 2017

  9. [9]

    Learning both weights and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” inAdv. Neural Inf. Process. Syst., 2015, pp. 1135–1143

    2015

  10. [10]

    LLM pruning and distillation in practice: The Minitron approach,

    S. T. Sreenivaset al., “LLM pruning and distillation in practice: The Minitron approach,” 2024. [Online]. Available:https://arxiv.org/abs/2408.11796

    Pith/arXiv arXiv 2024

  11. [11]

    Brain structural plasticity: From adult neurogenesis to immature neurons,

    C. L. Rosa, R. Parolisi, and L. Bonfanti, “Brain structural plasticity: From adult neurogenesis to immature neurons,”Front. Neurosci., vol. 14, p. 75, 2020. [On- line]. Available:https://www.frontiersin.org/articles/10.3389/fnins. 2020.00075/full

    arXiv 2020

  12. [12]

    Structural plasticity for neuromorphic networks with elec- tropolymerized dendritic PEDOT connections,

    K. Janzakovaet al., “Structural plasticity for neuromorphic networks with elec- tropolymerized dendritic PEDOT connections,”Nat. Commun., vol. 14, no. 1, p. 8143, 2023, doi:10.1038/s41467-023-43887-8

    work page doi:10.1038/s41467-023-43887-8 2023

  13. [13]

    Memory capacities for synaptic and structural plasticity,

    A. Knoblauch, G. Palm, and F. T. Sommer, “Memory capacities for synaptic and structural plasticity,”Neural Comput., vol. 22, no. 2, pp. 289–341, 2009

    2009

  14. [14]

    Deep learning and the information bottleneck prin- ciple,

    N. Tishby and N. Zaslavsky, “Deep learning and the information bottleneck prin- ciple,”CoRR, 2015. [Online]. Available:http://arxiv.org/abs/1503.02406

    Pith/arXiv arXiv 2015

  15. [15]

    Opening the black box of deep neural networks via information,

    R. Shwartz-Ziv and N. Tishby, “Opening the black box of deep neural networks via information,”CoRR, vol. abs/1703.00810, 2017. [Online]. Available:https: //arxiv.org/abs/1703.00810

    Pith/arXiv arXiv 2017

  16. [16]

    Synaptic scaling—an artificial neural network reg- ularization inspired by nature,

    M. Hoffmann and P. Mäder, “Synaptic scaling—an artificial neural network reg- ularization inspired by nature,”IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 7, pp. 3094–3108, 2022

    2022

  17. [17]

    Learning understandable neural networks with nonnegative weight constraints,

    J. Chorowski and J. M. Zurada, “Learning understandable neural networks with nonnegative weight constraints,”IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 1, pp. 62–69, 2015, doi:10.1109/TNNLS.2014.2310059

    arXiv 2015

  18. [18]

    Hebb,The Organization of Behavior: A Neuropsychological Theory, NewYork, NY: John Wiley and Sons, 1949

    D.O. Hebb,The Organization of Behavior: A Neuropsychological Theory, NewYork, NY: John Wiley and Sons, 1949

    1949

  19. [19]

    Spike timing-dependent plasticity: a Heb- bian learning rule,

    N. Caporale and Y. Dan, “Spike timing-dependent plasticity: a Heb- bian learning rule,”Ann. Rev. Neurosci., vol. 31, pp. 25–46, 2008, doi:10.1146/annurev.neuro.31.060407.125639

    arXiv 2008

  20. [20]

    Connectivity reflects coding: a model of voltage-based STDP with homeostasis,

    C. Clopath, L. Büsing, E. Vasilaki, and W. Gerstner, “Connectivity reflects coding: a model of voltage-based STDP with homeostasis,”Nat. Neurosci., vol. 13, pp. 344–352, 2010, doi:10.1038/nn.2479

    doi:10.1038/nn.2479 2010

  21. [21]

    Does spike-timing-dependent synaptic plasticity couple or decouple neurons firing in synchrony?,

    A. Knoblauch, F. Hauser, M.-O. Gewaltig, E. Körner, and G. Palm, “Does spike-timing-dependent synaptic plasticity couple or decouple neurons firing in synchrony?,”Front. Comput. Neurosci., vol. 6, p. 55, 2012, doi:10.3389/fncom.2012.00055. 12

    Pith/arXiv arXiv 2012

  22. [22]

    Cell assemblies in the cerebral cortex,

    G. Palm, A. Knoblauch, F. Hauser, and A. Schüz, “Cell assemblies in the cerebral cortex,”Biol. Cybernet., vol. 108, pp. 559–572, 2014, doi:10.1007/s00422-014- 0596-4

    work page doi:10.1007/s00422-014- 2014

  23. [23]

    Die Lernmatrix,

    K. Steinbuch, “Die Lernmatrix,”Kybernetik, vol. 1, pp. 36–45, 1961

    1961

  24. [24]

    Kohonen,Associative Memory: A System Theoretic Approach, Springer, Berlin, 1977

    T. Kohonen,Associative Memory: A System Theoretic Approach, Springer, Berlin, 1977

    1977

  25. [25]

    Bindinginhippocampal- entorhinal circuits enables compositionality in cognitive maps,

    C.J.Kymn,S.Mazelet,A.Thomas,andB.A.Olshausen,“Bindinginhippocampal- entorhinal circuits enables compositionality in cognitive maps,”J. Neurosci., June

  26. [26]

    A fast stochastic error-descent algorithm for supervised learn- ingandoptimization,

    G. Cauwenberghs, “A fast stochastic error-descent algorithm for supervised learn- ingandoptimization,” inAdv. Neural Inf. Process. Syst., vol.5, pp.244–251, 1993

    1993

  27. [27]

    Model-free distributed learning,

    A. Dembo and T. Kailath, “Model-free distributed learning,”IEEE Trans. Neural Netw. Learn. Syst., vol. 1, no. 1, pp. 58–70, 1990

    1990

  28. [28]

    Weight versus node perturbation learning in temporally extended tasks: Weight perturbation often performs simi- larly or better,

    P. Züge, C. Klos, and R. M. Memmesheimer, “Weight versus node perturbation learning in temporally extended tasks: Weight perturbation often performs simi- larly or better,”Phys. Rev., vol. 13, April 2023

    2023

  29. [29]

    Scalable layer-parallel readouttrainingruleforself-organizingrecurrentneuralnetworks,

    S. Nambusubramaniyan, A. Knoblauch, and F. Röhrbein, “Scalable layer-parallel readouttrainingruleforself-organizingrecurrentneuralnetworks,”inProc. Bern- stein Conf., 2022, doi:10.12751/nncn.bc2022.165

    work page doi:10.12751/nncn.bc2022.165 2022

  30. [30]

    On variational bounds of mutual information,

    B. Poole, S. Ozair, A. van den Oord, A. A. Alemi, and G. Tucker, “On variational bounds of mutual information,”arXiv preprint, 2019. [Online]. Available:https: //arxiv.org/abs/1905.06922

    Pith/arXiv arXiv 2019

  31. [31]

    The cost of cortical computation,

    P. Lennie, “The cost of cortical computation,”Curr . Biol., vol. 13, pp. 493–497, 2003, doi:10.1016/S0960-9822(03)00135-0

    doi:10.1016/s0960-9822(03)00135-0 2003

  32. [32]

    Communication in neuronal networks,

    S. Laughlin and T. Sejnowski, “Communication in neuronal networks,”Science, vol. 301, pp. 1870–1874, 2003, doi:10.1126/science.1089662

    work page doi:10.1126/science.1089662 2003

  33. [33]

    An energy budget for signaling in the grey mat- ter of the brain,

    D. Attwell and S. Laughlin, “An energy budget for signaling in the grey mat- ter of the brain,”J. Cereb. Blood Flow Metabol., vol. 21, pp. 1133–1145, 2001, doi:10.1097/00004647-200110000-00001

    doi:10.1097/00004647-200110000-00001 2001

  34. [34]

    MNIST Handwritten Digit Database,

    Y. LeCun, C. Cortes, and C. Burges, “MNIST Handwritten Digit Database,” Ac- cessed: Feb. 28, 2026. [Online]. Available:http://yann.lecun.com/exdb/ mnist

    2026

  35. [35]

    Activation learning by local competitions,

    H. Zhou, “Activation learning by local competitions,”arXiv, vol. 2209.13400v2, 2022

    Pith/arXiv arXiv 2022

  36. [36]

    Asynapticmodelofmemory: Long-termpotentiation in the hippocampus,

    T.BlissandG.Collingridge,“Asynapticmodelofmemory: Long-termpotentiation in the hippocampus,”Nature, vol. 361, pp. 31–39, 1993. 13

    1993

  37. [37]

    Natural patterns of activity and long-term synaptic plasticity,

    O. Paulsen and T. Sejnowski, “Natural patterns of activity and long-term synaptic plasticity,”Curr . Opin. Neurobiol., vol. 10, pp. 172–179, 2000, doi:10.1016/S0959-4388(00)00076-3

    work page doi:10.1016/s0959-4388(00)00076-3 2000

  38. [38]

    Competitive Hebbian learning through spike- timing-dependent synaptic plasticity,

    S. Song, K. Miller, and L. Abbott, “Competitive Hebbian learning through spike- timing-dependent synaptic plasticity,”Nat. Neurosci., vol. 3, pp. 919–926, 2000, doi:10.1038/78829

    doi:10.1038/78829 2000

  39. [39]

    Decreasing-rate pruning optimizes theconstructionofefficientandrobustdistributednetworks,

    S. Navlakha, A. L. Barth, and Z. Bar-Joseph, “Decreasing-rate pruning optimizes theconstructionofefficientandrobustdistributednetworks,”PLOS Comput. Biol., vol. 11, pp. 1–23, 2015

    2015

  40. [40]

    Activity-dependent synaptogenesis in the adult mam- malian cortex,

    K. Zito and K. Svoboda, “Activity-dependent synaptogenesis in the adult mam- malian cortex,”Neuron, vol. 35, pp. 1015–1017, 2002

    2002

  41. [41]

    Impact of structural plasticity on memory formation and decline,

    A. Knoblauch, “Impact of structural plasticity on memory formation and decline,” inRewiring the Brain: A Computational Approach to Structural Plasticity in the Adult Brain, A. van Ooyen and M. Butz, Eds. Elsevier, 2017, pp. 361–386. 14

    2017

  42. [2024]

    (License: CC BY-NC-SA 4.0)