pith. sign in

arxiv: 2605.24611 · v1 · pith:6J2AWHHBnew · submitted 2026-05-23 · 💻 cs.LG

Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks

Aakash Kumar , Anatoly Khina , Frederik Mallmann-Trenn , Emanuele Natale This is my paper

Pith reviewed 2026-06-30 13:52 UTC · model grok-4.3

classification 💻 cs.LG
keywords Hopfield networksasymmetric networkslimit cyclessequence memorynetwork capacityopinion dynamicscombinatorial constructions
0
0 comments X

The pith

Asymmetric Hopfield networks with n neurons support exp(Ω(n/(log n)^2)) distinct robust limit-cycle attractors of length exp(Ω(√n/log n)).

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that asymmetric Hopfield networks can store a superpolynomial number of long sequences as limit cycles. Classical symmetric networks are limited to static patterns, but asymmetry allows temporal sequences via cycles. The construction yields exp(Ω(n/(log n)^2)) cycles each of length exp(Ω(√n/log n)), robust to noise flips up to nearly 1/2. This shows simple asymmetric connections suffice for high-capacity sequence memory in the classical synchronous model.

Core claim

In the classical asymmetric Hopfield model with binary neurons and synchronous updates, n neurons support exp(Ω(n/(log n)^2)) distinct limit-cycle attractors, each with period exp(Ω(√n/log n)) and robust to random noise with flip probability up to 1/2-o(1). This capacity is achieved by combining results from combinatorics, number theory and the analysis of opinion dynamics, demonstrating that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized.

What carries the argument

A construction that combines combinatorial objects, number-theoretic structures, and opinion dynamics analysis to produce non-interfering robust limit cycles under the asymmetric synchronous update rule.

If this is right

  • Synchronous asymmetric Hopfield networks can store superpolynomially many sequences.
  • Each stored sequence can have length exponential in √n / log n.
  • The limit cycles remain stable against random neuron flips up to probability nearly 1/2.
  • Robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Asymmetry in weights alone may enable efficient temporal sequence storage in recurrent networks without additional mechanisms.
  • The approach could be simulated for moderate n to check whether the claimed cycles form and remain distinct.
  • Similar combinatorial constructions might extend sequence capacity in other recurrent models used for memory or prediction.
  • High noise robustness suggests potential applications in fault-tolerant sequence processing hardware.

Load-bearing premise

The combinatorial and number-theoretic objects invoked in the construction exist and can be combined with the synchronous asymmetric update rule to produce non-interfering robust cycles.

What would settle it

An explicit counterexample for some n where the number of distinct robust limit cycles falls to polynomial in n, or where constructed cycles interfere or lose stability under noise with flip probability 1/2-o(1), would falsify the superpolynomial claim.

Figures

Figures reproduced from arXiv: 2605.24611 by Aakash Kumar, Anatoly Khina, Emanuele Natale, Frederik Mallmann-Trenn.

Figure 1
Figure 1. Figure 1: Revival probability ± standard error over 50 trials. (a) Single sparse block cycle of length ℓ = 200, where each block contains d neurons and each neuron receives h = 3 random incoming edges from the preceding block. The x-axis varies d, and curves correspond to the independent initial flip probability p. (b) Sparse block-cyclic architecture with two block cycles of lengths 150 and 200, block size d = 1000… view at source ↗
Figure 2
Figure 2. Figure 2: A block cycle of (fully-)connected blocks. [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) Recovery probability on a sparse block cycle versus number of adversarial neurons, [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Spectral embeddings of two sparse block-cyclic architectures with adversarial connections: [PITH_FULL_IMAGE:figures/full_fig_p024_4.png] view at source ↗
read the original abstract

Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows $n$ neurons to support $\exp\!\big(\Omega(n/(\log n)^2)\big)$ distinct limit-cycle attractors, each with period $\exp\!\big(\Omega(\sqrt n/\log n)\big)$ and robust to random noise with flip probability up to $\frac12-o(1)$, yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that classical synchronous asymmetric Hopfield networks with binary neurons can achieve superpolynomial capacity: n neurons support exp(Ω(n/(log n)^2)) distinct limit-cycle attractors, each of period exp(Ω(√n/log n)) and robust to independent bit-flip noise of probability 1/2-o(1). The construction is obtained by combining results from combinatorics, number theory, and opinion dynamics to produce non-interfering robust cycles under the synchronous sign-update rule.

Significance. If the explicit construction and embedding argument hold, the result would substantially advance the known capacity of asymmetric Hopfield networks for sequence memory, moving beyond polynomial bounds on fixed points or short cycles and showing that simple asymmetric weights suffice for high-capacity, noise-robust temporal attractors. This has potential implications for both theoretical models of biological sequence memory and practical recurrent network design.

major comments (2)
  1. [Abstract and §3] Abstract and §3 (construction): the central claim requires that specific combinatorial and number-theoretic objects can be jointly embedded into a single asymmetric weight matrix W such that the synchronous update map produces at least exp(Ω(n/(log n)^2)) distinct cycles of the stated length with no cross-talk (one cycle's trajectory mapping into another). The abstract asserts this follows from combining external results, but no explicit W or argument that the separate robustness and non-interference properties survive the joint embedding is supplied; this is load-bearing for the capacity bound.
  2. [§4] §4 (robustness analysis): the noise margin of 1/2-o(1) must be shown to hold simultaneously for all cycles after embedding. If the opinion-dynamics component supplies the margin only for isolated cycles, an additional argument is needed that the combined dynamics do not reduce the basin size below the claimed threshold under the synchronous rule.
minor comments (2)
  1. [Abstract] Notation for the period and capacity exponents should be made uniform (e.g., consistently using Ω(·) with explicit log factors) and cross-referenced to the combinatorial lemmas invoked.
  2. [Introduction] The manuscript should include a short table or diagram contrasting the new bounds with prior polynomial or sub-exponential results for asymmetric Hopfield networks.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful and constructive review. The two major comments correctly identify that the joint embedding argument and simultaneous robustness require explicit verification in the text. We address both points below and will incorporate clarifications in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (construction): the central claim requires that specific combinatorial and number-theoretic objects can be jointly embedded into a single asymmetric weight matrix W such that the synchronous update map produces at least exp(Ω(n/(log n)^2)) distinct cycles of the stated length with no cross-talk (one cycle's trajectory mapping into another). The abstract asserts this follows from combining external results, but no explicit W or argument that the separate robustness and non-interference properties survive the joint embedding is supplied; this is load-bearing for the capacity bound.

    Authors: We agree that an explicit argument for the joint embedding is necessary. The construction selects cycle supports via a combinatorial design ensuring pairwise separation larger than the interaction range of the synchronous update; weights are then assigned block-wise using the number-theoretic sequences on each support and the opinion-dynamics rule for the local fields. Because the supports are disjoint and the separation bound exceeds the maximum degree of any neuron, the update map on each cycle is identical to the isolated case, eliminating cross-talk. We will add a short subsection in §3 that spells out this modular construction, the separation lemma, and the resulting invariance of the cycle set. revision: yes

  2. Referee: [§4] §4 (robustness analysis): the noise margin of 1/2-o(1) must be shown to hold simultaneously for all cycles after embedding. If the opinion-dynamics component supplies the margin only for isolated cycles, an additional argument is needed that the combined dynamics do not reduce the basin size below the claimed threshold under the synchronous rule.

    Authors: The opinion-dynamics margin is preserved because the effective local field for each neuron remains dominated by its own cycle after embedding; the combinatorial separation ensures that cross terms from other cycles are o(1) relative to the margin threshold, uniformly over all cycles. Consequently the basin-size lower bound applies simultaneously. We will insert a clarifying paragraph in §4 that invokes the separation lemma from the revised §3 to justify the simultaneous application. revision: yes

Circularity Check

0 steps flagged

No circularity: construction combines external combinatorics, number theory, and opinion dynamics results

full rationale

The paper states its result is obtained by combining results from combinatorics, number theory and the analysis of opinion dynamics to produce the claimed limit-cycle attractors in the asymmetric Hopfield model. No equations, fitted parameters, or self-referential definitions appear in the provided text. The central claim is an existence construction relying on external theorems rather than any reduction of outputs to inputs by construction, self-citation chains, or renaming of known results. This is the most common honest finding for construction-based papers that invoke independent external results.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review; the construction is asserted to rest on external results from combinatorics and number theory whose details are not supplied.

axioms (1)
  • standard math Existence of suitable combinatorial objects from number theory and combinatorics that can be wired into the asymmetric Hopfield dynamics
    Abstract states the capacity is obtained by combining these results with the model.

pith-pipeline@v0.9.1-grok · 5745 in / 1125 out tokens · 36483 ms · 2026-06-30T13:52:55.490660+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 10 canonical work pages

  1. [1]

    doi: 10.1088/0305-4470/ 30/16/007

    ISSN 0305-4470. doi: 10.1088/0305-4470/ 30/16/007. L. Becchetti, A. Clementi, E. Natale, F. Pasquale, and L. Trevisan. Stabilizing consensus with many opinions. InProceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algo- rithms, page 620–635. Society for Industrial and Applied Mathematics, December

    work page doi:10.1088/0305-4470/

  2. [2]

    URLhttp://dx.doi.org/10.1137/1.9781611974331.ch46

    1137/1.9781611974331.ch46. URLhttp://dx.doi.org/10.1137/1.9781611974331.ch46. Petra Berenbrink, Amin Coja-Oghlan, Oliver Gebhard, Max Hahn-Klimroth, Dominik Kaaser, and Malin Rau. On the hierarchy of distributed majority protocols. InInternational Symposium on Distributed Computing (DISC). Schloss Dagstuhl – Leibniz-Zentrum für Informatik,

    work page doi:10.1137/1.9781611974331.ch46

  3. [3]

    URL https://drops.dagstuhl.de/entities/document/ 10.4230/LIPIcs.OPODIS.2022.23

    4230/LIPICS.OPODIS.2022.23. URL https://drops.dagstuhl.de/entities/document/ 10.4230/LIPIcs.OPODIS.2022.23. Hamza Tahir Chaudhry, Jacob A Zavatone-Veth, Dmitry Krotov, and Cengiz Pehlevan. Long sequence hopfield memory. InThirty-seventh Conference on Neural Information Processing Systems,

    work page doi:10.4230/lipics.opodis.2022.23 2022

  4. [4]

    doi: 10.1016/j.jnt.2014.04.011

    ISSN 0022-314X. doi: 10.1016/j.jnt.2014.04.011. URL http://dx.doi.org/10.1016/j.jnt. 2014.04.011. Andrea Clementi, Mohsen Ghaffari, Luciano Gualà, Emanuele Natale, Francesco Pasquale, and Giacomo Scornavacca. A tight analysis of the parallel undecided-state dynamics with two colors. InInternational Symposium on Mathematical Foundations of Computer Science...

    work page doi:10.1016/j.jnt.2014.04.011 2014

  5. [5]

    URL https: //drops.dagstuhl.de/entities/document/10.4230/LIPIcs.MFCS.2018.28

    doi: 10.4230/LIPICS.MFCS.2018.28. URL https: //drops.dagstuhl.de/entities/document/10.4230/LIPIcs.MFCS.2018.28. Parikshit Gopalan, Cheng Huang, Huseyin Simitci, and Sergey Yekhanin. On the locality of codeword symbols.IEEE Transactions on Information Theory, 58(11):6925–6934, November

    work page doi:10.4230/lipics.mfcs.2018.28 2018

  6. [6]

    J J Hopfield

    doi: 10.1109/TIT.2012.2208937. J J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8):2554–2558, April

    work page doi:10.1109/tit.2012.2208937 2012

  7. [7]

    doi: 10.1073/pnas.79.8.2554

    ISSN 1091-6490. doi: 10.1073/pnas.79.8.2554. URLhttp://dx.doi.org/10.1073/pnas.79.8.2554. 10 Sungmin Hwang, Viola Folli, Enrico Lanza, Giorgio Parisi, Giancarlo Ruocco, and Francesco Zamponi. On the number of limit cycles in asymmetric neural networks.Journal of Statistical Mechanics: Theory and Experiment, 2019(5):053402, May

    work page doi:10.1073/pnas.79.8.2554 2019

  8. [8]

    doi: 10.1016/0025-5564(74)90031-5

    ISSN 0025-5564. doi: 10.1016/0025-5564(74)90031-5. URL https://doi.org/10.1016/0025-5564(74)90031-5. Samuel P. Muscinelli, Wulfram Gerstner, and Johanni Brea. Exponentially long orbits in hopfield neural networks.Neural Computation, 29(2):458–484, February

    work page doi:10.1016/0025-5564(74)90031-5

  9. [9]

    doi: 10.1162/neco_a_00919

    ISSN 1530-888X. doi: 10.1162/neco_a_00919. URLhttp://dx.doi.org/10.1162/NECO_a_00919. Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gru- ber, Markus Holzleitner, Thomas Adler, David Kreil, Michael K Kopp, Günter Klambauer, Jo- hannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. InInternat...

    work page doi:10.1162/neco_a_00919

  10. [10]

    doi: 10.1016/j.neunet.2013.06.008

    ISSN 0893-6080. doi: 10.1016/j.neunet.2013.06.008. URL http: //dx.doi.org/10.1016/j.neunet.2013.06.008. A Aperiodic Binary Sequences Definition 11(Möbius function).TheMöbius functionµ:N→ {−1,0,1}is defined by µ(n) =    1ifn= 1, 0ifnis divisible by the square of a prime, (−1)r ifnis the product ofrdistinct primes. Theorem 6.Let k≥1 and ℓ≥1 be intege...

    work page doi:10.1016/j.neunet.2013.06.008 2013

  11. [11]

    The same union bound over the n/d blocks again gives total failure probability o(1)

    rounds each block reaches consensus on some sign up to at most k=⌈C adv √ d/logd⌉ exceptional neurons, with failure probability at most d−ξ. The same union bound over the n/d blocks again gives total failure probability o(1). Once every block is monochromatic up to those k neurons, the regular part of the network again evolves by cyclically shifting the r...

    2023

  12. [12]

    Theorem 9(Convergence of j-Majority in the Gossip Model).Let j≥3 be a fixed integer, and let (Ct)t≥0 be the stochastic process defined above

    on the convergence time ofjmajority. Theorem 9(Convergence of j-Majority in the Gossip Model).Let j≥3 be a fixed integer, and let (Ct)t≥0 be the stochastic process defined above. Then there exists a constant c=c(j)>0 such that, for any initial configuration C0 ∈ {a, b}N , the process reaches consensus within O(logN) rounds with high probability. More prec...

    2018

  13. [13]

    Notation Meaning ntotal number of neurons in the network

    20 F Additional Related-Work Comparison and Experimental Details F.1 Notation Summary Table 1: Summary of the recurring notation used throughout the paper. Notation Meaning ntotal number of neurons in the network. x(t)∈ {−1,+1} n network state at timet. W= (w ij)asymmetric weight matrix;w ij is the edge weight from neuronjto neuroni. Fsynchronous update m...

    1997