arxiv: 2604.02789 · v1 · submitted 2026-04-03 · ❄️ cond-mat.dis-nn

Recognition: no theorem link

Dense Associative Memory with biased patterns: a Replica Symmetric analysis

Linda Albanese , Andrea Alessandrelli , Federico Carella

Authors on Pith no claims yet

Pith reviewed 2026-05-13 19:03 UTC · model grok-4.3

classification ❄️ cond-mat.dis-nn

keywords dense associative memorybiased patternsreplica symmetric analysishigh storage regimesignal-to-noise analysisstorage capacityGuerra interpolation

0 comments

The pith

Bias in stored patterns reduces the storage capacity of dense higher-order associative memories by the factor (1-b squared) to the power P while the superlinear scaling with network size remains intact.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines dense higher-order associative memories in the regime where many patterns are stored relative to the network size, but now with biased patterns whose entries are not balanced symmetrically around zero. The standard learning rule is adjusted by recentering and rescaling the patterns, and an extra term is added to the Hamiltonian to keep the network activity consistent with the pattern statistics. A zero-temperature signal-to-noise calculation shows that this bias multiplies the maximum storable load by (1-b²)^P. The same factor is recovered exactly from the replica-symmetric free energy derived via Guerra interpolation, where it appears as a renormalization of the cross-talk noise variance. This result clarifies how realistic uneven pattern statistics affect retrieval without eliminating the models' favorable scaling properties.

Core claim

The central claim is that the effective storage capacity α_c of the model with biased patterns equals the unbiased capacity multiplied by the factor (1-b²)^P, where b quantifies the pattern bias and P is the interaction order. This follows from both a heuristic signal-to-noise analysis at zero temperature and the full replica-symmetric treatment of the quenched statistical pressure, which produces self-consistency equations for the order parameters in which the bias enters solely through the reduced noise variance.

What carries the argument

The bias-dependent multiplicative renormalization (1-b²)^P of the cross-talk noise variance that appears in the effective local field of each unit.

If this is right

Retrieval remains possible provided the renormalized load stays below the critical value.
The location of the retrieval-to-spin-glass transition shifts exactly by the factor (1-b²)^P.
The superlinear dependence of capacity on network size N is unchanged by the bias.
The self-consistency equations for the overlaps incorporate the same noise renormalization derived heuristically.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Preprocessing real data to reduce bias before storage could restore the full unbiased capacity in practical implementations.
The same renormalization structure may appear in other associative memory variants when input statistics deviate from zero mean.
Finite-temperature extensions would be needed to assess how thermal fluctuations combine with the bias correction in retrieval dynamics.

Load-bearing premise

The replica-symmetric ansatz correctly describes the saddle-point equations and phase structure for biased patterns in the high-storage regime.

What would settle it

Large-scale numerical simulations of the network at loads near the predicted capacity threshold for several values of b and P that fail to exhibit the expected multiplicative reduction would falsify the central claim.

read the original abstract

We investigate dense higher-order associative memories in the high storage regime when the stored patterns are biased, namely when the entries of the patterns are not symmetrically distributed around zero. In this setting, the standard Hebbian prescription must be modified by recentering and rescaling the pattern entries, and an additional term must be introduced in the Hamiltonian to enforce consistency between the average activity of the network and that of the stored patterns. As a first step, we perform a signal-to-noise analysis in the zero-temperature limit and show that the bias reduces the effective storage capacity through a multiplicative correction factor (1-b^2)^P, while preserving the superlinear scaling with the system size. We then derive the quenched statistical pressure within the Replica Symmetric framework by means of Guerra's interpolation method and obtain the corresponding self consistency equations for the relevant order parameters. The analytical treatment confirms the heuristic prediction of the signal-to-noise argument, showing that the same bias dependent renormalization naturally emerges in the variance of the cross-talk noise. Finally, we discuss the resulting phase behavior of the model and its implications for retrieval performance in the model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper derives an explicit multiplicative capacity correction (1-b²)^P for biased patterns in dense higher-order associative memories and confirms it via both signal-to-noise and Guerra interpolation under the RS ansatz.

read the letter

The main point is that bias in the stored patterns reduces effective storage capacity by the factor (1-b²)^P while the superlinear scaling with system size remains. They first show this with a zero-temperature signal-to-noise argument after recentering the patterns and adding an activity term to the Hamiltonian, then recover the same renormalization from the cross-talk noise variance in the replica-symmetric quenched pressure derived by Guerra interpolation. The self-consistency equations for the order parameters follow directly and the phase discussion links back to retrieval performance. This is a clean extension of the symmetric-pattern literature, and the agreement between the heuristic and analytic routes is a real strength. The renormalization is not inserted by hand but comes out of the variance calculation, which keeps the logic tight. The main limitation is that they do not check replica-symmetric stability. No replicon eigenvalue or de Almeida-Thouless line is computed, even though RSB often appears near capacity in higher-order models. Without that, it is unclear whether the reported factor still governs the threshold once fluctuations are accounted for. The zero-temperature argument is heuristic, so the finite-temperature claim rests entirely on the RS saddle point. This work is aimed at researchers already familiar with spin-glass treatments of associative memory. A reader who knows the unbiased case will see the extension immediately and can use the correction factor in their own calculations. The derivations are standard enough to be checked and the result is concrete enough to test numerically. It deserves a serious referee to verify the algebra, run stability checks, and see whether simulations support the RS prediction near the storage limit.

Referee Report

2 major / 2 minor

Summary. The manuscript analyzes dense higher-order associative memories in the high-storage regime when patterns are biased (non-zero mean entries). The Hebbian rule is modified by recentering and rescaling, and an activity-enforcing term is added to the Hamiltonian. A zero-temperature signal-to-noise analysis shows that bias reduces effective capacity by the multiplicative factor (1-b²)^P while preserving superlinear scaling with system size. The quenched statistical pressure is then derived under the replica-symmetric ansatz via Guerra interpolation, yielding self-consistency equations in which the same bias-dependent renormalization emerges naturally from the variance of the cross-talk noise. The resulting phase behavior and retrieval implications are discussed.

Significance. If the central results hold, the work supplies an analytic confirmation of the bias-induced capacity renormalization in dense associative memories, extending heuristic signal-to-noise arguments with an independent Guerra-interpolation derivation. The consistency between the two approaches and the preservation of superlinear scaling are notable strengths. The analysis bears on retrieval performance in high-order networks with non-symmetric patterns.

major comments (2)

[Guerra interpolation derivation of the quenched pressure and self-consistency equations] The replica-symmetric ansatz is adopted for the quenched-pressure derivation and self-consistency equations without a stability check. No replicon eigenvalue or de Almeida–Thouless line is computed, even though RSB is known to appear near the storage threshold in higher-order dense models. This leaves open whether the (1-b²)^P renormalization governs the phase boundary in the high-storage regime.
[signal-to-noise analysis and its relation to the RS equations] The signal-to-noise argument is performed at zero temperature and is heuristic; the RS confirmation is presented as analytic support, yet the finite-temperature validity of the factor rests entirely on the unverified RS saddle point. A direct comparison of the zero-T limit of the RS equations with the signal-to-noise result would strengthen the claim.

minor comments (2)

[Abstract and Introduction] The order P of the dense memory is used in the factor (1-b²)^P but is not defined in the abstract or early introduction; a brief statement of the Hamiltonian form (e.g., the P-body interaction term) would improve readability.
[Model definition] Notation for the rescaled and recentered patterns should be introduced once and used consistently; occasional reuse of the original pattern symbols after the modification step creates minor ambiguity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading of the manuscript and the constructive comments. We address each major point below and indicate the revisions we will make.

read point-by-point responses

Referee: [Guerra interpolation derivation of the quenched pressure and self-consistency equations] The replica-symmetric ansatz is adopted for the quenched-pressure derivation and self-consistency equations without a stability check. No replicon eigenvalue or de Almeida–Thouless line is computed, even though RSB is known to appear near the storage threshold in higher-order dense models. This leaves open whether the (1-b²)^P renormalization governs the phase boundary in the high-storage regime.

Authors: We acknowledge that the manuscript does not include a stability analysis of the replica-symmetric saddle point (e.g., replicon eigenvalue or de Almeida–Thouless line). This is a valid observation, as RSB is known to appear near capacity thresholds in related higher-order models. The consistency between the zero-temperature signal-to-noise analysis and the RS equations provides supporting evidence that the (1-b²)^P renormalization remains relevant in the high-storage regime, but without an explicit stability check the precise location of the phase boundary under possible RSB remains open. In the revised manuscript we will add a dedicated paragraph in the discussion section noting this limitation and identifying a full replicon analysis as an important direction for future work. revision: partial
Referee: [signal-to-noise analysis and its relation to the RS equations] The signal-to-noise argument is performed at zero temperature and is heuristic; the RS confirmation is presented as analytic support, yet the finite-temperature validity of the factor rests entirely on the unverified RS saddle point. A direct comparison of the zero-T limit of the RS equations with the signal-to-noise result would strengthen the claim.

Authors: We agree that an explicit zero-temperature limit of the RS self-consistency equations would strengthen the connection to the signal-to-noise analysis. In the revised manuscript we will derive this limit and show that it recovers the same multiplicative factor (1-b²)^P for the effective capacity, thereby providing a direct analytic bridge between the two approaches and confirming consistency at T=0. revision: yes

Circularity Check

0 steps flagged

No significant circularity; renormalization arises from explicit noise variance calculation

full rationale

The paper derives the (1-b²)^P factor first via zero-temperature signal-to-noise analysis on the recentered patterns and activity term, then recovers the identical factor from the variance of the cross-talk term inside the Guerra-interpolated RS free energy. Both steps are explicit algebraic reductions from the modified Hamiltonian and pattern statistics; no parameter is fitted to data and then relabeled as a prediction, no self-citation supplies a uniqueness theorem, and the RS saddle-point equations are obtained directly rather than by redefinition. The derivation chain is therefore self-contained against the model's own equations.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the replica-symmetric ansatz and the validity of Guerra interpolation for the quenched pressure in the high-storage regime; no new free parameters or invented entities are introduced beyond the bias parameter b that is given as input.

axioms (2)

domain assumption Replica symmetric ansatz holds for the order parameters in the biased high-storage regime
Invoked to obtain the self-consistency equations from the quenched statistical pressure.
standard math Guerra interpolation method applies directly to the modified Hamiltonian with recentering and extra consistency term
Used to derive the analytical expression that confirms the signal-to-noise prediction.

pith-pipeline@v0.9.0 · 5494 in / 1264 out tokens · 45190 ms · 2026-05-13T19:03:01.850943+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

[1]

Agliari, L

E. Agliari, L. Albanese, F. Alemanno, A. Alessandrelli, A. Barra, F. Giannotti, D. Lotito, and D. Pedreschi. Dense hebbian neural networks: a replica symmetric picture of supervised learning. Physica A: Statistical Mechanics and its Applications, 626:129076, 2023

work page 2023
[2]

Agliari, L

E. Agliari, L. Albanese, F. Alemanno, A. Alessandrelli, A. Barra, F. Giannotti, D. Lotito, and D. Pedreschi. Dense hebbian neural networks: A replica symmetric picture of unsupervised learning. Physica A: Statistical Mechanics and its Applications, 627:129143, 2023

work page 2023
[3]

Agliari, F

E. Agliari, F. Alemanno, A. Barra, and A. Fachechi. Dreaming neural networks: rigorous results. Journal of Statistical Mechanics: Theory and Experiment, 2019(8):083503, 2019

work page 2019
[4]

Agliari, A

E. Agliari, A. Alessandrelli, A. Barra, M. S. Centonze, and F. Ricci-Tersenghi. Generalized hetero-associative neural networks.Journal of Statistical Mechanics: Theory and Experiment, 2025(1):013302, 2025

work page 2025
[5]

Agliari and G

E. Agliari and G. De Marzo. Tolerance versus synaptic noise in dense associative memories.The European Physical Journal Plus, 135(11):1–22, 2020

work page 2020
[6]

Agliari, F

E. Agliari, F. E. Leonelli, and C. Marullo. Storing, learning and retrieving biased patterns.Applied Mathematics and Computation, 415:126716, 2022

work page 2022
[7]

Albanese, F

L. Albanese, F. Alemanno, A. Alessandrelli, and A. Barra. Replica symmetry breaking in dense hebbian neural networks.Journal of Statistical Physics, 189(2):1–41, 2022

work page 2022
[8]

Albanese, A

L. Albanese, A. Alessandrelli, A. Annibale, and A. Barra. Replica symmetry breaking in supervised and unsupervised hebbian networks.Journal of Physics A: Mathematical and Theoretical, 57(16):165003, 2024. – 14 –

work page 2024
[9]

Albanese, A

L. Albanese, A. Alessandrelli, A. Barra, and P. Sollich. Yet another exponential Hopfield model. Physica A: Statistical Mechanics and its Applications, page 131223, 2025

work page 2025
[10]

D. J. Amit.Modeling brain function: The world of attractor neural networks. Cambridge university press, 1989

work page 1989
[11]

D. J. Amit, H. Gutfreund, and H. Sompolinsky. Storing infinite numbers of patterns in a spin-glass model of neural networks.Physical Review Letters, 55:1530–1533, 1985

work page 1985
[12]

D. J. Amit, H. Gutfreund, and H. Sompolinsky. Information storage in neural networks with low levels of activity.Physical Review A, 35(5):2293, 1987

work page 1987
[13]

Baldi and S

P. Baldi and S. S. Venkatesh. Number of stable points for spin-glasses and neural networks of higher orders.Physical Review Letters, 58, 1987

work page 1987
[14]

Bovier and B

A. Bovier and B. Niederhauser. The spin-glass phase-transition in the Hopfield model with p-spin interactions.Advances in Theoretical and Mathematical Physics, 5:1001–1046, 8 2001

work page 2001
[15]

Fachechi, E

A. Fachechi, E. Agliari, and A. Barra. Dreaming neural networks: forgetting spurious memories and reinforcing pure ones.Neural Networks, 112:24–40, 2019

work page 2019
[16]

E. Gardner. Multiconnected neural network models.Journal of Physics A: General Physics, 20, 1987

work page 1987
[17]

F. Guerra. Broken replica symmetry bounds in the mean field spin glass model.Communications in Mathematical Physics, 233:1–12, 2003

work page 2003
[18]

J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79:2554–2558, 1982

work page 1982
[19]

Krotov and J

D. Krotov and J. Hopfield. Dense associative memory is robust to adversarial inputs.Neural Computation, 30:3151–3167, 2018

work page 2018
[20]

Krotov and J

D. Krotov and J. J. Hopfield. Dense associative memory for pattern recognition.Advances in Neural Information Processing Systems, pages 1180–1188, 2016

work page 2016
[21]

M. Lowe. On the storage capacity of the Hopfield model with biased patterns.IEEE Transactions on Information Theory, 45(1):314–318, 1999

work page 1999
[22]

Lucibello and M

C. Lucibello and M. M´ ezard. Exponential capacity of dense associative memories.Physical Review Letters, 132(7):077301, 2024

work page 2024
[23]

M´ ezard, G

M. M´ ezard, G. Parisi, and M. A. Virasoro.Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications, volume 9. World Scientific Publishing Company, 1987

work page 1987
[24]

Nishimori.Statistical physics of spin glasses and information processing: an introduction

H. Nishimori.Statistical physics of spin glasses and information processing: an introduction. Number

work page
[25]

Clarendon Press, 2001

work page 2001
[26]

P. Peretto. Collective properties of neural networks: a statistical physics approach.Biological cybernetics, 50(1):51–62, 1984

work page 1984
[27]

Hopfield networks is all you need.arXiv preprint arXiv:2008.02217, 2020

H. Ramsauer, B. Sch¨ afl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlovi´ c, G. K. Sandve, et al. Hopfield networks is all you need.arXiv preprint arXiv:2008.02217, 2020

work page arXiv 2008
[28]

1 Zg,N (β|ξ1,λ,J;t) X σ Bg,N (β|ξ1,λ,J;t)λ i1···iP σi1 · · ·σ iP # – 16 – = β √ K 2 √ tN P NX i1,···,i P =1 Eξ1,λ

M. V. Tsodyks and M. V. Feigel’man. The enhanced storage capacity in neural networks with low activity level.EPL (Europhysics Letters), 6(2):101–105, 1988. – 15 – A Computation oft−derivatives and one body terms of the quenched sta- tistical pressure Let us start from the computation of the derivative with respect totof the quenched statistical pressure (...

work page 1988