Pith. sign in

REVIEW 2 major objections 6 minor 59 references

Binding threshold units with artificial oscillatory neurons

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that threshold and oscillatory neuron models can be coupled in a single associative memory whose joint energy never increases, so the network converges to stored patterns.

desk verdict A genuinely useful Lyapunov coupling between Hopfield and Kuramoto units, with a caveat that the theorem's shared-frequency assumption may not match the trainable Omega in the experiments. read the letter →

arxiv 2505.03648 v1 pith:R4YW5DGM submitted 2025-05-06 q-bio.NC cs.AIcs.LG

Vladimir Fanaskov , Ivan Oseledets This is my paper · ORCID
classification q-bio.NCcs.AIcs.LG MSC 92B2034C1534D06
keywords HopfieldnetworksKuramotooscillatorsassociativememoryLyapunovfunctionlow-rankadaptationHebbianlearningfrequencymodulationneuralcoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that the two standard abstractions of a neuron can be combined in one dynamical system without losing the guarantee that the system settles down. Threshold units, which track average firing intensity and form Hopfield associative memories, and oscillatory units, which encode information in phase relationships through a generalized Kuramoto equation, are coupled with interaction terms derived from a single joint energy function. Provided every oscillator shares the same natural-frequency matrix, this energy is a Lyapunov function, so trajectories converge to stored patterns. The coupling terms act as a state-dependent low-rank correction to the synaptic weight matrix, which the authors connect to Hebbian learning, fast weights, and the LoRA fine-tuning method. Small experiments on handwritten-digit recognition show the oscillatory units can switch or edit the memories of a fixed threshold network.

What carries the argument

The load-bearing object is the joint Lyapunov energy $E_{HK}$ together with the requirement that all oscillators share one frequency matrix $\Omega$. Here a Lyapunov function is a function that never increases along trajectories, so the system must settle at a fixed point. The proof works by writing the coupled oscillatory dynamics so that the rotation term $\Omega_i\mu_i$ cancels against the skew-symmetry of the scalar-product terms when $\Omega_i=\Omega$, reducing the oscillator part to constrained gradient flow on a sphere; the threshold part inherits the standard Hopfield energy decrease. The interaction terms (13) are then constructed so that the derivative of the cross-energy cancels the new terms in the equations of motion, leaving a sum of a negative quadratic form in $\dot{x}$ and a sum of negative spherical-gradient terms. This Lyapunov-function constraint is what turns the otherwise heterogeneous coupling into a principled binding mechanism.

What would settle it

Theorem 4.1 is a mathematical proof, so the sharp falsifier is a counterexample within its stated assumptions: integrate (11) with identical $\Omega$, symmetric $G$ and $\chi$, and positive-semidefinite Hessian, and check Eq. (14) at fine time resolution; any positive $\dot E_{HK}$ above solver tolerance would refute the theorem. A less destructive check is to integrate with $\Omega_1\neq\Omega_2$ and exhibit a trajectory on which $E_{HK}$ increases, which would show the identical-frequency condition is doing essential work.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.1: for symmetric smooth functions $G_{ij}$ and $\chi_{ij}$, a Lagrange function with positive-semidefinite Hessian, symmetric Hopfield weights, and identical natural-frequency matrices $\Omega_i=\Omega$, the coupled dynamics (11) admit the Lyapunov function $$E_{HK}(x,\mu)=\kappa_H E_H(x)+\kappa_K E_K(\mu)-\frac{1}{2}\sum_{ij}G_{ij}(g_i g_j)\chi_{ij}(\$mu_i^{{\top}}$\mu_j),$$ with time derivative given by Eq. (14), always non-positive. This makes the Hopfield-Kuramoto system an associative memory with guaranteed convergence. The coupling terms specified by Eq. (13) are not ad hoc: they are exactly the terms that make the cross-terms in the energy derivative cancel. The paper further claims that a natural choice of these terms yields low-rank updates $W\leftarrow W + A(\mu)A(\mu)^{\top}$ and $R\leftarrow R + g(x)g(x)^{\top}$, so oscillatory units implement a time-dependent low-rank correction to the threshold-unit weights, interpretable as fast weights, LoRA, or Hebbian learning. Experiments with a pretrained threshold network confirm that oscillatory dynamics can alter stored labels and recover most of the original accuracy on modified handwritten-digit tasks.

Load-bearing premise

Every oscillator must have the same natural-frequency matrix $\Omega$; if frequencies differ, the rotation term no longer cancels and the guaranteed decrease of the joint energy is lost.

Editorial extensions

If this is right

  • With the theorem's conditions, a network mixing both unit types can be used as an associative memory: initial states converge to fixed points, and those fixed points are the stored patterns.
  • Oscillatory units can act on a frozen threshold network as a state-dependent weight correction; in the paper's experiments this restores near-baseline accuracy on label-swap and label-conflation tasks without retraining the threshold weights.
  • Because the correction factors have rank at most $D+1$ for the threshold weights and rank one for the oscillator weights, the coupling provides a dynamic, biologically motivated analogue of LoRA-style low-rank adaptation.
  • The construction extends to layered architectures with ReLU, convolution, softmax, and attention-type couplings while preserving a global non-increasing energy for the whole network.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the authors do not pursue is to use slowly varying or slightly different $\Omega_i$ as a controlled way to inject energy and escape local minima; the paper itself hints at this by noting that sufficiently different frequencies prevent synchronization.
  • The worst-case encoding bound in Appendix C suggests that scalar products alone have limited capacity; a testable consequence is that hybrid networks gain capacity precisely because threshold units carry most patterns while oscillators provide low-rank modulation.
  • Equation (15)'s gating when $\mu_i^{\top}\mu_j=0$ predicts a refractory-like effect: two highly active threshold units can be effectively disconnected by oscillator geometry, a prediction that could be tested in a spiking or phase model.
  • Because the low-rank factors are functions of the network state, one could ask whether training learns task-specific oscillator fixed points; the paper's associative-swap experiments with learnable initial conditions already move in this direction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes a coupled Hopfield–Kuramoto associative memory in which threshold units evolve by a Hopfield-type equation and oscillatory units by a generalized Kuramoto equation. The interaction terms are chosen so that a joint energy E_HK(x,mu) is a Lyapunov function; this is stated as Theorem 4.1, with coupling terms in Eq. (13) and an energy-derivative identity in Eq. (14). The assumptions are symmetric G and chi, positive-semidefinite Hessian of L, and Omega_i = Omega. The authors interpret the coupling as a low-rank weight correction resembling LoRA, fast weights, and Hebbian learning, discuss a multiplexing scenario in Example 4, and report MNIST toy experiments in which a pretrained frozen Hopfield network is fine-tuned by learned oscillatory weights to implement label swap or conflation, in both non-associative and associative versions.

Significance. If Theorem 4.1 is correct under its stated and natural additional hypotheses, the paper gives a clean Lyapunov-based mechanism for binding threshold and oscillatory units, with a concrete low-rank-correction interpretation that connects to LoRA and Hebbian learning. The proof is self-contained and the energy decrease is verified by direct differentiation rather than by reusing the target result; no fitted constant is used in the Lyapunov claim. The paper is also transparent about the key assumption Omega_i = Omega and about the scalar-product dependence of the Kuramoto energy, and it ships code and trained models. The main weaknesses are two places where the manuscript overstates the scope of the guarantee: the theorem statement omits a necessary symmetry condition on W, and the experimental section trains Omega without specifying the skew-symmetric shared parameterization required by the theorem. These are fixable, but they are load-bearing for the central claim.

major comments (2)
  1. [Section 4, Theorem 4.1 and Appendix D, Eq. (40)] The statement of Theorem 4.1 does not include the condition W^T = W, but this condition is necessary for the result. The Hopfield energy E_H in Eq. (12) is a Lyapunov function only when W is symmetric (Theorem 3.1), and the proof in Appendix D uses W^T = W when the two terms in parentheses in Eq. (40) are combined into \dot{x}_j. As written, the theorem is false for asymmetric W. Please add W^T = W to the hypotheses of Theorem 4.1 (and state it explicitly in the theorem statement rather than leaving it implicit in the notation E_H).
  2. [Section 6, Eq. (18) and Appendix F.4] Theorem 4.1 and its proof require \Omega_i = \Omega with a skew-symmetric matrix \Omega, as defined in Eq. (3); the cancellation in Eq. (44) is exactly what makes the frequency term vanish. In the fine-tuning experiments, Section 6 uses model (18) with a trainable \Omega, and Appendix F.4 states only that associative training uses a trainable \Omega. The text does not say whether \Omega is a single shared matrix or whether optimization enforces \Omega^T = -\Omega. If \Omega is trained without this constraint, the trained model can fall outside the hypotheses of Theorem 4.1, so the reported accuracy improvements would not be covered by the paper's Lyapunov guarantee. Please specify the shared, skew-symmetric parameterization of \Omega and state that it is preserved during training; alternatively, if per-oscillator frequency matrices are learned, the theorem does not apply.
minor comments (6)
  1. [Section 6, Tables 2 and 3] The text says models with D=2 and D=3 are trained, but Tables 2 and 3 report results for D=4 and D=6; please correct the text or the tables.
  2. [Table 3] The associative-task rows combine D=4 and D=6 into a single entry, so the statement that there is no difference between D=4 and D=6 is not supported by the reported data; please report the runs separately or remove the claim.
  3. [Section 4, Example 4] The existence claim that 'as long as inputs A and B are sufficiently distinct it is possible to construct such energy function' is asserted without a construction or proof. As written this is a conjecture, and the phrase 'E_K is not constrained' overlooks the scalar-product-only condition and the shared-frequency condition required by Theorem 3.2.
  4. [Eq. (18) and Theorem 4.1] The empirical model uses ReLU in the Kuramoto potential (the S-term), while Theorem 4.1 assumes smooth functions G and chi. Please state explicitly whether the smoothed variant or ReLU-squared potential is used, or relax the smoothness assumption to the differentiability actually required by the proof.
  5. [Theorems 3.2 and 4.1] The skew-symmetry of \Omega is stated in Eq. (3) but not repeated in the theorem statements, which only say \Omega_i = \Omega. Since the proofs rely on orthogonality of exp(\Omega t) and on skew-symmetry of \mu_\beta^T \Omega \mu_i, please state explicitly in each theorem that \Omega is skew-symmetric.
  6. [Appendix C] The capacity argument is informal. If the bound 'N oscillators can encode at most N-1 scalars' is intended as a rigorous result, the step 'each time we pick a novel pair we lose D degrees of freedom' needs a formal proof; if it is a heuristic, please label it as such.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: The Lyapunov theorems are proved directly from stated assumptions, and the coupling terms are constructed, not fitted, so the central claim is self-contained.

full rationale

The paper's central derivation is a constructive Lyapunov argument, not a reuse of its target conclusion. Theorem 3.1 is imported from Krotov and Hopfield [35, 34] and independently reproduced in Appendix A. Theorem 3.2 is proved in Appendix B by two routes: a rotating-frame reduction to constrained gradient flow, and a direct derivative computation that uses only the scalar-product dependence of E_K and the condition Ω_i = Ω. Theorem 4.1 is proved in Appendix D by differentiating the proposed energy E_HK and choosing the coupling terms (13) so that the cross terms cancel; the proof explicitly verifies that the bracket in Eq. (44) is symmetric and μ_β^T Ω μ_i is skew-symmetric, making the contraction vanish. The coupling mechanism is therefore a designed, mathematically verified construction rather than a prediction fitted to data. The numerical section is an empirical fine-tuning study: the Hopfield subnetwork is pretrained, then the oscillatory parameters and scalar κ are trained on modified labels, and the reported accuracies are measurements, not claims that the theorem predicts them. The paper also states its key assumption honestly in Section 3.2 ('The result we used here relies heavily on the assumption Ω_i = Ω') and in Theorem 4.1. The later use of trainable Ω in Appendix F.4 may place the trained model outside the theorem's scope, but that is a correctness or applicability concern, not a circularity: the theorem itself does not presuppose its conclusion. No load-bearing self-citation, no uniqueness argument imported from the authors' prior work, and no renamed empirical pattern masquerading as a derivation were found.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The Lyapunov theorem rests on structural symmetry assumptions rather than fitted numbers. The paper introduces no new physical entity, conserved quantity, or field; the artificial oscillatory neuron is borrowed from prior work, and the Hopfield-Kuramoto model is a mathematical construction. The experimental low-rank claim additionally uses a learned coupling constant kappa and a trainable Omega.

free parameters (3)
  • kappa_H, kappa_K coupling constants in the joint model = positive; set to 1 in the Section 5 examples
    Theorem 4.1 requires positive constants; the low-rank example sets both to 1. They control the relative strength of Hopfield and Kuramoto terms and are not derived from data.
  • kappa (coupling strength in experiments) = learned during fine-tuning; value not reported
    In Eq. (18) kappa multiplies the low-rank coupling terms and is the only learnable scalar in the frozen-Hopfield fine-tuning stage, so the empirical demonstration depends on it.
  • Omega (oscillator frequency matrix in associative experiments) = trainable; values not reported
    Appendix F.4 says associative training uses a trainable Omega. The theorem requires Omega_i=Omega, so if the trained Omega is per-neuron the Lyapunov guarantee is not preserved.
assumptions (6)
  • domain assumption Weight matrix W is symmetric (W^T=W).
    Required in Theorem 3.1 for E_H to be a Lyapunov function; carried into Theorem 4.1 via the Hopfield term (Section 3.1, Eqs. 6-8).
  • domain assumption Hessian of the Lagrange function is positive semidefinite, d^2 L/dx^2 >= 0.
    Used in Eqs. (8) and (14) to make the x-part of the energy derivative non-positive; restricts admissible activation functions.
  • domain assumption Kuramoto energy E_K depends only on scalar products mu_i^T mu_j.
    Needed for the rotating-frame reduction in Appendix B.1 and for the direct derivative identity in Eq. (33); stated in Theorem 3.2.
  • domain assumption All oscillators share the same rotation matrix, Omega_i=Omega.
    Makes the Omega term vanish in the energy derivative via skew-symmetry; without it the Lyapunov conclusion fails (Theorems 3.2 and 4.1, Appendix B.2).
  • domain assumption Coupling functions G_ij and chi_ij are symmetric and smooth.
    Symmetry of G_ij(g_i g_j) and chi_ij(mu_i^T mu_j) is used to cancel the rotation term in Appendix D, Eq. (44).
  • standard math Oscillator states stay on the unit sphere, mu_i^T mu_i=1.
    Follows from the generalized Kuramoto equation (3); makes I-mu_i mu_i^T an orthogonal projector, giving non-positivity in Eqs. (10) and (14).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Binding threshold units with artificial oscillatory neurons." pith.science (2026). https://pith.science/paper/R4YW5DGM

@misc{pith2026250503648,
  author       = {Pith},
  title        = {Pith review of: Binding threshold units with artificial oscillatory neurons},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R4YW5DGM}},
  note         = {Machine review of arXiv:2505.03648}
}
read the original abstract

Artificial Kuramoto oscillatory neurons were recently introduced as an alternative to threshold units. Empirical evidence suggests that oscillatory units outperform threshold units in several tasks including unsupervised object discovery and certain reasoning problems. The proposed coupling mechanism for these oscillatory neurons is heterogeneous, combining a generalized Kuramoto equation with standard coupling methods used for threshold units. In this research note, we present a theoretical framework that clearly distinguishes oscillatory neurons from threshold units and establishes a coupling mechanism between them. We argue that, from a biological standpoint, oscillatory and threshold units realise distinct aspects of neural coding: roughly, threshold units model intensity of neuron firing, while oscillatory units facilitate information exchange by frequency modulation. To derive interaction between these two types of units, we constrain their dynamics by focusing on dynamical systems that admit Lyapunov functions. For threshold units, this leads to Hopfield associative memory model, and for oscillatory units it yields a specific form of generalized Kuramoto model. The resulting dynamical systems can be naturally coupled to form a Hopfield-Kuramoto associative memory model, which also admits a Lyapunov function. Various forms of coupling are possible. Notably, oscillatory neurons can be employed to implement a low-rank correction to the weight matrix of a Hopfield network. This correction can be viewed either as a form of Hebbian learning or as a popular LoRA method used for fine-tuning of large language models. We demonstrate the practical realization of this particular coupling through illustrative toy experiments.

Figures

Figures reproduced from arXiv: 2505.03648 by the authors.

Figure 1
Figure 1. In the middle: membrane potential of three Izhikevich neurons under constant injected dc-current. Two first neurons fire at the same frequency but have different phases. The third neuron spikes at a higher frequency. On the left: threshold unit with smooth activation function σ(x); interaction term of additive model. Threshold unit is a simplified description of neuron’s interaction that only models time-averaged in… view at source ↗
Figure 2
Figure 2. Empirical validation of interaction between threshold and oscillatory units. [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 39 canonical work pages

  1. [1]

    Hebbian learning from first principles

    Linda Albanese, Adriano Barra, Pierluigi Bianco, Fabrizio Durante, and Diego Pallara. Hebbian learning from first principles. Journal of Mathematical Physics , 65(11), 2024

    work page 2024

  2. [2]

    In search of dispersed memories: Generative diffusion models are associative memory networks

    Luca Ambrogioni. In search of dispersed memories: Generative diffusion models are associative memory networks. arXiv preprint arXiv:2309.17290 , 2023

    arXiv 2023

  3. [3]

    Using fast weights to attend to the recent past

    Jimmy Ba, Geoffrey E Hinton, Volodymyr Mnih, Joel Z Leibo, and Catalin Ionescu. Using fast weights to attend to the recent past. Advances in neural information processing systems , 29, 2016

    work page 2016

  4. [4]

    Deep equilibrium models

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in neural information processing systems, 32, 2019

    2019

  5. [5]

    Universal approximation bounds for superpositions of a sigmoidal function

    Andrew R Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory , 39(3):930–945, 1993

    1993

  6. [6]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018

    2018

  7. [7]

    Neuronal oscillations in cortical networks.science, 304(5679):1926– 1929, 2004

    Gyorgy Buzsaki and Andreas Draguhn. Neuronal oscillations in cortical networks.science, 304(5679):1926– 1929, 2004

    work page 1926

  8. [8]

    Symbolic discovery of optimization algorithms

    Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, et al. Symbolic discovery of optimization algorithms. Advances in neural information processing systems , 36:49205–49233, 2023

    work page 2023

Show all 59 references
  1. [9]

    On the properties of neural machine translation: Encoder-decoder approaches

    Kyunghyun Cho, Bart Van Merri¨ enboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 , 2014

    2014 arXiv

  2. [10]

    Approximation by superpositions of a sigmoidal function

    George Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems , 2(4):303–314, 1989

    1989

  3. [11]

    The DeepMind JAX Ecosystem, 2020

    DeepMind, Igor Babuschkin, Kate Baumli, Alison Bell, Surya Bhupatiraju, Jake Bruce, Peter Buchlovsky, David Budden, Trevor Cai, Aidan Clark, Ivo Danihelka, Antoine Dedieu, Claudio Fantacci, Jonathan Godwin, Chris Jones, Ross Hemsley, Tom Hennigan, Matteo Hessel, Shaobo Hou, St...

    2020

  4. [12]

    Synchronization in complex networks of phase oscillators: A survey

    Florian D¨ orfler and Francesco Bullo. Synchronization in complex networks of phase oscillators: A survey. Automatica, 50(6):1539–1564, 2014

    2014

  5. [13]

    Brain oscillations and memory

    Emrah D¨ uzel, Will D Penny, and Neil Burgess. Brain oscillations and memory. Current opinion in neurobiology, 20(2):143–149, 2010

    2010

  6. [14]

    Training spiking neural networks using lessons from deep learning

    Jason K Eshraghian, Max Ward, Emre O Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D Lu. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9):1016–1054, 2023. 13

    2023

  7. [15]

    Nonlinear neural networks: Principles, mechanisms, and architectures

    Stephen Grossberg. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural networks, 1(1):17–61, 1988

    1988

  8. [16]

    Hagberg, Daniel A

    Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart. Exploring network structure, dynamics, and function using networkx. In Ga¨ el Varoquaux, Travis Vaught, and Jarrod Millman, editors,Proceedings of the 7th Python in Science Conference , pages 11 – 15, Pasadena, CA USA, 2008

    2008

  9. [17]

    Common oscillatory mechanisms across multiple memory systems

    Drew B Headley and Denis Par´ e. Common oscillatory mechanisms across multiple memory systems. npj Science of Learning, 2(1):1, 2017

    2017

  10. [18]

    Long short-term memory.Neural computation, 9(8):1735–1780, 1997

    Sepp Hochreiter and J¨ urgen Schmidhuber. Long short-term memory.Neural computation, 9(8):1735–1780, 1997

    1997

  11. [19]

    A universal abstraction for hierarchical hopfield networks

    Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt, and Dmitry Krotov. A universal abstraction for hierarchical hopfield networks. In The Symbiosis of Deep Learning and Differential Equations II , 2022

    2022

  12. [20]

    Energy transformer

    Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobelt, Duen Horng Chau, Mohammed Zaki, and Dmitry Krotov. Energy transformer. Advances in Neural Information Processing Systems, 36, 2024

    2024

  13. [21]

    Neural networks and physical systems with emergent collective computational abilities

    John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences , 79(8):2554–2558, 1982

    1982

  14. [22]

    Neurons with graded response have collective computational properties like those of two-state neurons

    John J Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the national academy of sciences , 81(10):3088–3092, 1984

    1984

  15. [23]

    Oscillatory neurocomputers with dynamic connectivity

    Frank C Hoppensteadt and Eugene M Izhikevich. Oscillatory neurocomputers with dynamic connectivity. Physical Review Letters, 82(14):2983, 1999

    1999

  16. [24]

    Weakly connected neural networks , volume 126

    Frank C Hoppensteadt and Eugene M Izhikevich. Weakly connected neural networks , volume 126. Springer Science & Business Media, 2012

    2012

  17. [25]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  18. [26]

    Exploring weight symmetry in deep neural networks

    Shell Xu Hu, Sergey Zagoruyko, and Nikos Komodakis. Exploring weight symmetry in deep neural networks. Computer Vision and Image Understanding , 187:102786, 2019

    2019

  19. [27]

    Weakly pulse-coupled oscillators, fm interactions, synchronization, and oscillatory associative memory

    Eugene M Izhikevich. Weakly pulse-coupled oscillators, fm interactions, synchronization, and oscillatory associative memory. IEEE Transactions on Neural Networks , 10(3):508–526, 1999

    1999

  20. [28]

    Simple model of spiking neurons

    Eugene M Izhikevich. Simple model of spiking neurons. IEEE Transactions on neural networks , 14(6):1569–1572, 2003

    2003

  21. [29]

    A new hybrid routing protocol using a modified k-means clustering algorithm and continuous hopfield network for vanet

    Khalid Kandali, Lamyae Bennis, and Hamid Bennis. A new hybrid routing protocol using a modified k-means clustering algorithm and continuous hopfield network for vanet. IEEE Access, 9:47169–47183, 2021

    2021

  22. [30]

    A spacetime perspective on dynamical computation in neural information processing systems

    T Anderson Keller, Lyle Muller, Terrence J Sejnowski, and Max Welling. A spacetime perspective on dynamical computation in neural information processing systems. arXiv preprint arXiv:2409.13669 , 2024

    2024

  23. [31]

    On Neural Differential Equations

    Patrick Kidger. On Neural Differential Equations . PhD thesis, University of Oxford, 2021

    2021

  24. [32]

    Equinox: neural networks in JAX via callable PyTrees and filtered transformations

    Patrick Kidger and Cristian Garcia. Equinox: neural networks in JAX via callable PyTrees and filtered transformations. Differentiable Programming workshop at Neural Information Processing Systems 2021 , 2021. 14

    2021

  25. [33]

    Biological computations: limitations of attractor-based formalisms and the need for transients

    Daniel Koch, Akhilesh Nandan, Gayathri Ramesan, and Aneta Koseska. Biological computations: limitations of attractor-based formalisms and the need for transients. Biochemical and Biophysical Research Communications, 720:150069, 2024

    2024

  26. [34]

    Hierarchical associative memory

    Dmitry Krotov. Hierarchical associative memory. arXiv preprint arXiv:2107.06446 , 2021

    2021 arXiv

  27. [36]

    Dense associative memory for pattern recognition

    Dmitry Krotov and John J Hopfield. Dense associative memory for pattern recognition. Advances in neural information processing systems, 29, 2016

    2016

  28. [37]

    Chemical Oscillations, Waves, and Turbulence

    Yoshiki Kuramoto. Chemical Oscillations, Waves, and Turbulence . Springer, 1984

    1984

  29. [38]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015

    2015

  30. [39]

    Mnist handwritten digit database

    Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010

    2010

  31. [40]

    Image segmentation with traveling waves in an exactly solvable recurrent neural network

    Luisa HB Liboni, Roberto C Budzinski, Alexandra N Busch, Sindy L¨ owe, Thomas A Keller, Max Welling, and Lyle E Muller. Image segmentation with traveling waves in an exactly solvable recurrent neural network. Proceedings of the National Academy of Sciences , 122(1):e2321319121, 2025

    2025

  32. [41]

    Convolutional neural networks as a model of the visual system: Past, present, and future

    Grace W Lindsay. Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of cognitive neuroscience, 33(10):2017–2031, 2021

    2017

  33. [42]

    Non-Abelian Kuramoto models and synchronization

    MA2539317 Lohe. Non-Abelian Kuramoto models and synchronization. Journal of Physics A: Mathe- matical and Theoretical, 42(39):395101, 2009

    2009

  34. [43]

    Rotating features for object discovery

    Sindy L¨ owe, Phillip Lippe, Francesco Locatello, and Max Welling. Rotating features for object discovery. Advances in Neural Information Processing Systems , 36, 2024

    2024

  35. [44]

    Networks of spiking neurons: the third generation of neural network models

    Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):1659–1671, 1997

    1997

  36. [45]

    Characterizing cancer subtypes as attractors of hopfield networks

    Stefan R Maetschke and Mark A Ragan. Characterizing cancer subtypes as attractors of hopfield networks. Bioinformatics, 30(9):1273–1279, 2014

    2014

  37. [46]

    Oscillator array models for associative memory and pattern recognition

    Paolo Maffezzoni, Bichoy Bahr, Zheng Zhang, and Luca Daniel. Oscillator array models for associative memory and pattern recognition. IEEE Transactions on Circuits and Systems I: Regular Papers , 62(6):1591–1598, 2015

    2015

  38. [47]

    Almost global convergence to practical synchronization in the generalized kuramoto model on networks over the n-sphere

    Johan Markdahl, Daniele Proverbio, La Mi, and Jorge Goncalves. Almost global convergence to practical synchronization in the generalized kuramoto model on networks over the n-sphere. Communications Physics, 4(1):187, 2021

    2021

  39. [48]

    A logical calculus of the ideas immanent in nervous activity

    Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics , 5:115–133, 1943

    1943

  40. [49]

    Artificial Kuramoto Oscillatory Neurons

    Takeru Miyato, Sindy L¨ owe, Andreas Geiger, and Max Welling. Artificial Kuramoto Oscillatory Neurons. arXiv preprint arXiv:2410.13821 , 2024

    2024 arXiv

  41. [50]

    Memorization to generalization: The emergence of diffusion models from associative memory

    Bao Pham, Gabriel Raya, Matteo Negri, Mohammed J Zaki, Luca Ambrogioni, and Dmitry Krotov. Memorization to generalization: The emergence of diffusion models from associative memory. 2024

    2024

  42. [51]

    Hopfield networks is all you need

    Hubert Ramsauer, Bernhard Sch¨ afl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlovi´ c, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217 , 2020. 15

    2008 arXiv

  43. [52]

    End-to-end differentiable clustering with associative memories

    Bishwajit Saha, Dmitry Krotov, Mohammed J Zaki, and Parikshit Ram. End-to-end differentiable clustering with associative memories. In International Conference on Machine Learning , pages 29649– 29670. PMLR, 2023

    2023

  44. [53]

    Associative memories via predictive coding

    Tommaso Salvatori, Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bogacz, and Thomas Lukasiewicz. Associative memories via predictive coding. Advances in Neural Information Processing Systems, 34:3874–3886, 2021

    2021

  45. [54]

    Reducing the ratio between learning complexity and number of time varying variables in fully recurrent nets

    J¨ urgen Schmidhuber. Reducing the ratio between learning complexity and number of time varying variables in fully recurrent nets. In ICANN’93: Proceedings of the International Conference on Artificial Neural Networks Amsterdam, The Netherlands 13–16 September 1993 3 , pages 4...

    1993

  46. [55]

    Deep learning in neural networks: An overview

    J¨ urgen Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85–117, 2015

    2015

  47. [56]

    A remark on a paper of krotov and hopfield [arxiv: 2008.06996]

    Fei Tang and Michael Kopp. A remark on a paper of krotov and hopfield [arxiv: 2008.06996]. arXiv preprint arXiv:2105.15034, 2021

    2008 arXiv

  48. [57]

    Processes and measurements: a framework for understanding neural oscillations in field potentials

    Sander van Bree, Daniel Levenstein, Matthew R Krause, Bradley Voytek, and Richard Gao. Processes and measurements: a framework for understanding neural oscillations in field potentials. Trends in Cognitive Sciences, 2025

    2025

  49. [58]

    Collective dynamics of ‘small-world’networks

    Duncan J Watts and Steven H Strogatz. Collective dynamics of ‘small-world’networks. nature, 393(6684):440–442, 1998

    1998

  50. [59]

    Modern hopfield networks and attention for immune repertoire classification

    Michael Widrich, Bernhard Sch¨ afl, Milena Pavlovi´ c, Hubert Ramsauer, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter, Geir Kjetil Sandve, Victor Greiff, Sepp Hochreiter, et al. Modern hopfield networks and attention for immune repertoire classification. Advances in ...

    2020

  51. [60]

    encoding capacity

    Dmitry Yarotsky. Error bounds for approximations with deep relu networks. Neural networks, 94:103–114, 2017. 16 A Lyapunov function for Hopfield associative memory As we mentioned in the main text the proof of this statement is available in [ 35], [34]. Nonetheless we find it ...

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.