arxiv: 2604.25481 · v1 · submitted 2026-04-28 · ⚛️ physics.data-an · cs.LG· nlin.AO· physics.soc-ph

Recognition: unknown

Emergent Self-Attention from Astrocyte-Gated Associative Memory Dynamics

Arnau Vivet , Alex Arenas

Authors on Pith no claims yet

Pith reviewed 2026-05-07 13:37 UTC · model grok-4.3

classification ⚛️ physics.data-an cs.LGnlin.AOphysics.soc-ph

keywords Hopfield networkastrocytesself-attentionreplicator dynamicsassociative memoryLyapunov functionsoftmax

0 comments

The pith

Astrocytic gains in a Hopfield network dynamically realize self-attention via softmax allocation at equilibrium.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a model combining Hopfield associative memory with astrocytic modulation. Astrocytic gains evolve according to an entropy-regularized replicator equation while the neurons update their states. The coupled system has a Lyapunov function that guarantees global convergence to stable points. At these points, the gains allocate connectivity weights as a softmax over pattern similarities, which is how self-attention works in modern AI models. This emergent mechanism leads to better memory retrieval when many patterns compete for recall.

Core claim

At the fixed points of the coupled dynamics, the astrocytic gains implement a softmax-normalized allocation over pattern similarity scores, providing a mechanistic realization of self-attention as emergent routing on the gain simplex.

What carries the argument

The entropy-regularized replicator equation for astrocytic gains coupled to the Hopfield neuron dynamics, which at equilibrium produces the softmax attention weights.

Load-bearing premise

The coupled neuron-astrocyte dynamics admit a Lyapunov function that ensures convergence to fixed points where the gains exactly match the softmax of similarities.

What would settle it

Simulate the model with stored patterns and check whether the steady-state astrocytic gains equal the softmax of the similarities between the query state and each pattern; mismatch or failure to converge would disprove the claim.

Figures

Figures reproduced from arXiv: 2604.25481 by Alex Arenas, Arnau Vivet.

**Figure 1.** Figure 1: FIG. 1. End-of-run retrieval error view at source ↗

**Figure 4.** Figure 4: To enable a dense scan over memory load and cor view at source ↗

**Figure 2.** Figure 2: FIG. 2. Final-time retrieval error view at source ↗

**Figure 4.** Figure 4: FIG. 4. Retrieval benchmark across models. Each heat map reports the mean Hamming retrieval error view at source ↗

read the original abstract

We introduce a Hopfield-type associative memory in which effective connectivity is multiplicatively modulated by astrocytic gains evolving under an entropy-regularized replicator equation. The coupled neuron-astrocyte dynamics admit a Lyapunov function, ensuring global convergence. At fixed points, astrocytic gains implement a softmax-normalized allocation over pattern similarity scores, yielding a mechanistic realization of self-attention as emergent routing on the gain simplex. In regimes of high memory load and interference, the model significantly improves retrieval accuracy relative to classical Hopfield dynamics and recent neuron-astrocyte baselines. These results establish a dynamical systems framework linking glial modulation, competitive resource allocation, and attention-like computation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper builds a Hopfield model with astrocytic gains evolving under entropy-regularized replicator dynamics to produce emergent softmax attention at equilibrium, but the Lyapunov claim and exact fixed-point derivation need the equations to confirm they are not constructed to fit the outcome.

read the letter

The core contribution is a coupled neuron-astrocyte system where multiplicative gains on the Hopfield weights follow an entropy-regularized replicator equation. At equilibrium the gains reportedly allocate according to a softmax over pattern overlaps, and the joint dynamics are said to have a Lyapunov function that drives global convergence. This gives a mechanistic story for self-attention arising from competitive resource allocation rather than being inserted by hand. The abstract also reports better retrieval accuracy than plain Hopfield or prior neuron-astrocyte models under high load, which is the practical payoff they emphasize. That framing is straightforward and connects glial modulation to attention-like routing in a way that could interest both theoretical neuroscientists and people building dynamical AI architectures. The replicator-plus-entropy setup is a reasonable choice for producing normalized positive gains, and claiming a Lyapunov function is the right way to argue stability. Still, the abstract gives no explicit ODEs, no candidate Lyapunov function, and no derivation showing that the fixed-point gains are exactly the softmax rather than a close variant that depends on ordering or extra terms. Without those steps it is impossible to tell whether the emergence is genuine or whether the update rules were tuned to land on softmax. The accuracy improvement is stated but not quantified here, so its size and robustness are unclear. The work is aimed at readers who already think about associative memory or bio-plausible attention mechanisms; a serious referee should see it because the modeling approach is coherent and the claimed link is specific enough to be checked or falsified. If the algebra holds, it is worth publishing after the derivations are written out clearly.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces a Hopfield-type associative memory in which effective connectivity is multiplicatively modulated by astrocytic gains that evolve according to an entropy-regularized replicator equation. The authors claim that the coupled neuron-astrocyte ODEs admit a Lyapunov function guaranteeing global convergence from arbitrary initial conditions. At the resulting fixed points, the astrocytic gains are asserted to equal a softmax over pattern similarity scores, providing a mechanistic realization of self-attention as emergent routing on the gain simplex. Numerical experiments are reported to show significantly improved retrieval accuracy relative to classical Hopfield networks and recent neuron-astrocyte baselines under high memory load and interference.

Significance. If the Lyapunov function and exact softmax fixed-point relation are rigorously established without hidden assumptions or post-hoc choices, the work would supply a concrete dynamical-systems bridge between glial modulation and attention-like computation, with potential implications for both computational neuroscience and memory models in AI. The reported accuracy gains under interference constitute a concrete empirical contribution that could be tested in follow-up studies.

major comments (2)

[Model derivation and fixed-point analysis] The abstract and model section assert that the entropy-regularized replicator dynamics on astrocytic gains, when coupled to the Hopfield neuron equations, converge to equilibria where the gains exactly equal the softmax over pattern similarities. However, the algebraic steps establishing this fixed-point relation are not shown; it remains unclear whether the softmax emerges independently or is built into the update rule by construction (e.g., via the entropy term and the definition of the similarity scores). This is load-bearing for the central claim of mechanistic self-attention emergence.
[Lyapunov analysis] The claim that the joint neuron-astrocyte system admits a Lyapunov function ensuring global convergence is stated but not supported by an explicit candidate function or a demonstration that its time derivative is non-positive everywhere. Without this, it cannot be verified that the system reaches the claimed softmax fixed points from arbitrary initial conditions rather than only from specially chosen ones.

minor comments (2)

The abstract refers to 'recent neuron-astrocyte baselines' without naming the specific models or citing the corresponding papers; these should be listed explicitly in the methods or results section for reproducibility.
The entropy regularization strength is listed as a free parameter; the manuscript should report its value(s) used in the experiments and include a brief sensitivity analysis showing that the accuracy gains persist across a reasonable range.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments that help clarify the presentation of our central claims. We address each point below and will revise the manuscript to include the requested explicit derivations.

read point-by-point responses

Referee: [Model derivation and fixed-point analysis] The abstract and model section assert that the entropy-regularized replicator dynamics on astrocytic gains, when coupled to the Hopfield neuron equations, converge to equilibria where the gains exactly equal the softmax over pattern similarities. However, the algebraic steps establishing this fixed-point relation are not shown; it remains unclear whether the softmax emerges independently or is built into the update rule by construction (e.g., via the entropy term and the definition of the similarity scores). This is load-bearing for the central claim of mechanistic self-attention emergence.

Authors: We thank the referee for noting the missing algebraic steps. Setting the astrocyte derivative to zero in the entropy-regularized replicator equation directly yields gains proportional to exp(similarity scores), normalized to unity, which is the softmax. The similarity scores are defined from the instantaneous neuron activations via the Hopfield weights, so the relation arises from the coupling between the two subsystems rather than being imposed by fiat in the update rule. We will insert a new subsection in the model section that walks through these steps from the equilibrium conditions. revision: yes
Referee: [Lyapunov analysis] The claim that the joint neuron-astrocyte system admits a Lyapunov function ensuring global convergence is stated but not supported by an explicit candidate function or a demonstration that its time derivative is non-positive everywhere. Without this, it cannot be verified that the system reaches the claimed softmax fixed points from arbitrary initial conditions rather than only from specially chosen ones.

Authors: We agree that an explicit Lyapunov function and its derivative are required to substantiate global convergence. The candidate function is the sum of the classical Hopfield energy and the negative entropy of the gain vector. Its time derivative along the coupled trajectories is non-positive, with equality only at the fixed points. We will add the explicit expression and the full proof that dV/dt ≤ 0 in the revised analysis section, thereby confirming global asymptotic stability from arbitrary initial conditions. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained from ODE fixed-point analysis.

full rationale

The abstract and skeptic summary describe a model whose core claims rest on explicit coupled ODEs (Hopfield neurons plus entropy-regularized replicator for astrocytic gains) together with an asserted Lyapunov function. The softmax allocation is presented as the algebraic fixed-point solution of those dynamics rather than an input parameter or post-hoc fit. No quoted step reduces the target result to a self-definition, a fitted subset renamed as prediction, or a load-bearing self-citation whose own justification is internal. Standard replicator equilibria yielding softmax under entropy regularization are mathematically independent of the present paper's target claim once the update rule is stated; the Lyapunov argument, if supplied with an explicit candidate whose derivative is shown non-positive, likewise constitutes independent verification rather than circular renaming. The paper therefore remains within the default non-circular regime.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The central claim rests on the existence of a Lyapunov function for the coupled system (a standard dynamical-systems assumption) and on the replicator equation producing softmax allocation without extra parameters. Astrocytic gains are introduced as a new modulating entity whose independent biological grounding is not supplied in the abstract.

free parameters (1)

entropy regularization strength
Controls the sharpness of the gain allocation in the replicator dynamics; its value is required for the softmax-like behavior but not specified as derived from data or first principles in the abstract.

axioms (1)

domain assumption The coupled neuron-astrocyte system admits a Lyapunov function ensuring global convergence to fixed points.
Invoked to guarantee that the system reaches the states where the softmax allocation occurs.

invented entities (1)

astrocytic gains no independent evidence
purpose: Multiplicative modulation of effective connectivity that evolves to implement attention-like routing.
New postulated variable introduced to link glial biology to computation; no independent falsifiable prediction outside the model is given in the abstract.

pith-pipeline@v0.9.0 · 5413 in / 1418 out tokens · 38720 ms · 2026-05-07T13:37:42.193077+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

42 extracted references · 3 canonical work pages

[1]

The system is a global gradient flow A key property of the classical Hopfield network is the existence of an energy (Lyapunov) function that de- creases along the trajectories thus guaranteeing conver- gence of the dynamics. As shown in the Appendix 1, our system can also be written as a gradient flow whose energy is given by L(x, p) =KT X µ pµ log pµ πµ ...
[2]

Fixed point analysis The equilibrium (x ∗, p∗), satisfies the coupled fixed point condition x∗ =W(p ∗)ϕ(x ∗) (10) p∗ µ = exp(fµ/T)P ν exp(fν/T) = softmaxµ fµ(x∗) T .(11) This can be interpreted as a mixture of (rank 1) experts
[3]

In this interpretation, astrocytic modulation provides the routing mechanism that selects the relevant memory patterns in a context- dependent and dynamical manner

commonly expressed asx ∗ =P µ gµ(x∗)Eµ(ϕ(x∗)), where the self-attention like routingg µ(x)≡p µ emerges naturally from the s-replicator, and the experts are rank- 1 matricesE µ(ϕ(x))≡ξ µξ⊺ µϕ(x). In this interpretation, astrocytic modulation provides the routing mechanism that selects the relevant memory patterns in a context- dependent and dynamical manne...
[4]

For the first case we take 4 the gain timescale to be infiniteτ p → ∞such that the astrocytic influence remains constant for all time ˙p µ = 0 (see Appendix 4)

Recovering the classical Hopfield model Our model recovers the classical associative memory in two dynamical regimes. For the first case we take 4 the gain timescale to be infiniteτ p → ∞such that the astrocytic influence remains constant for all time ˙p µ = 0 (see Appendix 4). Since the modulation is initialized uniformlyp µ = 1/K, we recover the classic...
[5]

To that end, we define two observables, one representative of the neuron component and the other for the astrocytic gains

Dynamical analysis In this first analysis, the goal is to understand the long term behavior of the system. To that end, we define two observables, one representative of the neuron component and the other for the astrocytic gains. Our focus is on how two control knobs shape retrieval: (i) the relative timescales of neuronal relaxation and astrocytic routin...
[6]

Comparative retrieval performance We benchmark retrieval performance against (i) the classical Hopfield network and (ii) the neuron–astrocyte associative-memory model of Kozachkovet al.[20]; see Fig. 4. To enable a dense scan over memory load and cor- ruption, we setN= 20 and vary the number of stored random binary patterns fromK= 2 toK= 200. For each (K,...
[7]

and the neuron–astrocyte associative-memory base- line [20] in the tested regimes (Fig. 4). Intuitively, adap- tive gain allocation reshapes the effective connectivity (and hence the attractor structure) during recall by am- plifying task-relevant patterns and suppressing competi- tors, in line with recent evidence that time-dependent modulation can impro...
[8]

(4,6) admit a Lyapunov function and can be written as a (pro- jected) gradient flow

Gradient-flow structure and Lyapunov function Here we show that the coupled dynamics in Eqs. (4,6) admit a Lyapunov function and can be written as a (pro- jected) gradient flow. We divide the discussion into the astrocyte domain and the neuron domain, where we de- rive their respective potentials. Note that the coupled dynamics occur inx, p∈R N ×∆ K−1 and...
[9]

Dynamical analysis of the system: Now that we have defined the evolution of the system, we can explore both its symmetries as well as its long- term behavior in different parameter regimes to better understand its dynamics
[10]

Consequently, the gain dy- namics are invariant under the sign flip⟨ξ µ, ϕ(x)⟩ 7→ 10 −⟨ξµ, ϕ(x)⟩

Symmetry remarks a.Z 2 invariance of the squared-overlap score In the main model the pattern score is a squared over- lap,f µ(x)∝ ⟨ξ µ, ϕ(x)⟩2. Consequently, the gain dy- namics are invariant under the sign flip⟨ξ µ, ϕ(x)⟩ 7→ 10 −⟨ξµ, ϕ(x)⟩. Equivalently, definingm µ(x)≡ ⟨ξ µ, ϕ(x)⟩, the gain update depends only onm 2 µ and cannot distin- guishm µ from−m ...
[11]

Plugging the second equation into the first, it can be rewritten as a single equation on thexdomain

Fixed points of the system Given the system dynamics of Eq.28 fixed point con- ditions: x∗ =W(p ∗)ϕ(x∗) (31) p∗ = softmax f(x ∗) T ,(32) where the second equation comes from Eq.21. Plugging the second equation into the first, it can be rewritten as a single equation on thexdomain. In this general regime, there are no analytic solutions, but we can see how...
[12]

(6), we use an explicit Euler method

Simulations details To integrate Eq. (6), we use an explicit Euler method. Trajectories are computed for 10/dttime steps, with dt= 0.001, using smaller steps for small parameter value regimes (i.e. ifαis the parameter thendt=α·0.05 if α≤0.01). For statistically significant results, we run 50 simulations with different random pattern matrices Ξ for every d...
[13]

J. J. Hopfield, Proceedings of the National Academy of Sciences79, 2554 (1982)

1982
[14]

D. J. Amit,Modeling Brain Function: The World of At- tractor Neural Networks(Cambridge University Press, 1989)

1989
[15]

Krotov and J

D. Krotov and J. J. Hopfield, inAdvances in Neural In- formation Processing Systems, Vol. 29 (2016) pp. 1172– 1180

2016
[17]

Vaswani, N

A. Vaswani, N. Shazeer,et al., inAdvances in Neural Information Processing Systems(2017)

2017
[18]

Hopfield networks is all you need.arXiv preprint arXiv:2008.02217, 2020

H. Ramsauer, B. Sch¨ afl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlovi´ c, G. K. Sandve, V. Greiff, D. Kreil, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, Hopfield networks is all you need (2021), arXiv:2008.02217 [cs.NE]

work page arXiv 2021
[19]

Araque, R

A. Araque, R. P. Sanzgiri, V. Parpura, and P. G. Haydon, Canadian Journal of Physiology and Pharmacology77, 699 (1999)

1999
[20]

Perea, M

G. Perea, M. Navarrete, and A. Araque, Trends in Neu- rosciences32, 421 (2009)

2009
[21]

Letellier, Y

M. Letellier, Y. K. Park, T. E. Chater, P. H. Chipman, S. G. Gautam, T. Oshima-Takago, and Y. Goda, Pro- ceedings of the National Academy of Sciences of the USA 113, E2685 (2016)

2016
[22]

De Pitt` a, N

M. De Pitt` a, N. Brunel, and A. Volterra, Neuroscience 323, 43 (2016)

2016
[23]

Giaume, A

C. Giaume, A. Koulakoff, L. Roux, D. Holcman, and N. Rouach, Nature Reviews Neuroscience11, 87 (2010)

2010
[24]

Verkhratsky and M

A. Verkhratsky and M. Nedergaard, Physiological Re- views98, 239 (2018)

2018
[25]

S. S. Purushotham and Y. Buskila, Frontiers in Network Physiology3, 1205544 (2023)

2023
[26]

Makovkin, E

S. Makovkin, E. Kozinov, M. Ivanchenko, and S. Gordleeva, Scientific Reports12, 6970 (2022)

2022
[27]

Thompson, J

L. Thompson, J. Khuc, M. S. Saccani, N. Zokaei, and M. Cappelletti, Experimental Brain Research239, 2711 (2021)

2021
[28]

K.-I. Dewa, K. Kaseda, A. Kuwahara, H. Kubotera, A. Yamasaki, N. Awata, A. Komori, M. A. Holtz, A. Ka- sai, H. Skibbe, N. Takata, T. Yokoyama, M. Tsuda, G. Numata, S. Nakamura, E. Takimoto, M. Sakamoto, M. Ito, T. Masuda, and J. Nagai, Nature 10.1038/s41586- 025-09619-2 (2025), online ahead of print

work page doi:10.1038/s41586- 2025
[29]

Zbaranska and S

S. Zbaranska and S. A. Josselyn, Cell Research35, 241 (2025)

2025
[30]

S. A. Josselyn and S. Tonegawa, Science367, eaaw4325 (2020)

2020
[31]

Kozachkov, K

L. Kozachkov, K. V. Kastanenka, and D. Krotov, Pro- ceedings of the National Academy of Sciences120, e2219150120 (2023)

2023
[32]

Kozachkov, J.-J

L. Kozachkov, J.-J. Slotine, and D. Krotov, Proceedings of the National Academy of Sciences122, e2417788122 (2025)

2025
[33]

M. A. Di Castro, J. Chuquet, N. Liaudet, K. Bhaukau- rally, M. Santello, D. Bouvier, P. Tiret, and A. Volterra, Nature Neuroscience14, 1276 (2011)

2011
[34]

Perea and A

G. Perea and A. Araque, Science317, 1083 (2007)

2007
[35]

Agulhon, T

C. Agulhon, T. A. Fiacco, and K. D. McCarthy, Science 327, 1250 (2010)

2010
[36]

Shigetomi, S

E. Shigetomi, S. Patel, and B. S. Khakh, Trends in Cell Biology26, 300 (2016)

2016
[37]

Krotov and J

D. Krotov and J. Hopfield, arXiv preprint arXiv:2008.06996 (2020)

work page arXiv 2008
[38]

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, Neural computation3, 79 (1991)

1991
[39]

Betteti, G

S. Betteti, G. Baggio, F. Bullo, and S. Zampieri, Science Advances11, eadu6991 (2025)

2025
[40]

M. K. Benna and S. Fusi, Nature Neuroscience19, 1697 (2016)

2016
[41]

Mertikopoulos and W

P. Mertikopoulos and W. H. Sandholm, Journal of Eco- nomic Theory177, 315 (2018)

2018
[42]

Halder, K

A. Halder, K. F. Caluya, B. Travacca, and S. J. Moura, IEEE Transactions on Neural Networks and Learning Systems31, 4869 (2020)

2020
[43]

Vivet and A

A. Vivet and A. Arenas, Astrocyte-gated as- sociative memory: code repository, GitHub repository (2026), accessed 10 Feb 2026. https://github.com/arnauvivett/Astrocyte-gated- associative-memory-

2026