Stationary covariance spectra of discrete-time non-normal random recurrent dynamics

Jacob A. Zavatone-Veth

arxiv: 2606.31944 · v1 · pith:25WKVIQ3new · submitted 2026-06-30 · 🧬 q-bio.NC · cond-mat.dis-nn

Stationary covariance spectra of discrete-time non-normal random recurrent dynamics

Jacob A. Zavatone-Veth This is my paper

Pith reviewed 2026-07-01 02:03 UTC · model grok-4.3

classification 🧬 q-bio.NC cond-mat.dis-nn

keywords stationary covariance spectrumnon-normal dynamicsfree probabilityrecurrent neural networksmoment generating functiondiscrete-time dynamicscritical regimeSchwinger-Dyson equations

0 comments

The pith

A free-probability approach produces a closed functional equation for the moment generating function of the limiting stationary covariance spectrum in discrete-time non-normal random recurrent dynamics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper derives a closed functional equation for the moment generating function of the limiting stationary covariance spectrum using free probability. This equation applies to discrete-time dynamics driven by stationary noise with random non-normal Gaussian weights and permits analysis of tail eigenvalues near criticality. The same method applied to continuous-time dynamics instead produces an infinite hierarchy of Schwinger-Dyson equations. The result is positioned as a tool for comparing models of non-normal dynamics against the variance distribution seen in principal-component analyses of neural recordings.

Core claim

We use a free-probability approach to formally derive a closed functional equation for the moment generating function of the limiting stationary covariance spectrum of discrete-time dynamics with random non-normal Gaussian weights. This characterization allows us to analyze the behavior of tail eigenvalues in the critical regime. In contrast, applying the same approach to the analogous continuous-time dynamics leads to an infinite hierarchy of Schwinger-Dyson equations, rather than a closed scalar equation.

What carries the argument

The closed scalar functional equation for the moment generating function of the limiting stationary covariance spectrum, obtained via free probability.

Load-bearing premise

The synaptic weight matrix is drawn from a random non-normal Gaussian ensemble and the dynamics are stationary and noise-driven.

What would settle it

Direct numerical computation of the eigenvalue distribution of the stationary covariance matrix for large finite-size realizations of the discrete-time non-normal Gaussian ensemble, compared against the spectrum predicted by the functional equation.

Figures

Figures reproduced from arXiv: 2606.31944 by Jacob A. Zavatone-Veth.

**Figure 2.** Figure 2: FIG. 2. Scaling of eigenvalues in the critical tail. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Figure 3: FIG. 3. Comparison of empirical and predicted spectral mo [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

read the original abstract

Principal component analysis is widely used to characterize structure in the dynamics of recurrent neural networks. For stationary noise-driven dynamics, the distribution of variance among the principal components is determined by the spectrum of the stationary covariance matrix. While the spectral properties of this matrix are well-understood for linear networks with normal synaptic weight matrices, our understanding of the stationary covariance spectrum for random non-normal dynamics remains incomplete. In this note, we use a free-probability approach to formally derive a closed functional equation for the moment generating function of the limiting stationary covariance spectrum of discrete-time dynamics with random non-normal Gaussian weights. This characterization allows us to analyze the behavior of tail eigenvalues in the critical regime. In contrast, applying the same approach to the analogous continuous-time dynamics leads to an infinite hierarchy of Schwinger-Dyson equations, rather than a closed scalar equation. We conclude with some comments regarding the relevance of these results to comparisons of models of non-normal dynamics to neural data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Derives closed functional equation for discrete-time non-normal covariance spectra via free probability, but critical regime analysis rests on stationarity that may not hold at g=1.

read the letter

Hey colleague,

The main thing here is a closed scalar functional equation for the moment generating function of the limiting stationary covariance spectrum in discrete-time dynamics with random non-normal Gaussian weights. That is new; prior results were limited to normal matrices or produced an infinite hierarchy of equations in the continuous-time case.

The paper does a clean job setting up the free-probability derivation and spelling out the discrete-versus-continuous contrast. The approach stays within standard tools from the literature, avoids self-referential fitting, and gives a concrete handle on tail eigenvalues that could help model-to-data comparisons in neuroscience.

The soft spot is the critical-regime claim. The derivation assumes the spectral radius of W is strictly less than 1 so that the dynamics remain stationary and the covariance stays finite. For the i.i.d. Gaussian ensemble the circular law puts the radius at g, so at g=1 the largest Lyapunov exponent is zero and variance grows linearly with time. The abstract says the equation is used to analyze tail eigenvalues in that regime, yet the provided text does not describe an extra limiting step or regularization. If the full manuscript supplies one that works, fine; if not, that part needs tightening.

This is for specialists working on random-matrix treatments of recurrent networks who want analytic covariance spectra rather than simulations alone. A reader already using free probability or comparing non-normal models to neural PCA data would get direct value.

It should go to peer review. The core derivation is a verifiable result that the field can build on, even if the critical-regime section needs clarification.

Referee Report

1 major / 0 minor

Summary. The paper claims that a free-probability approach yields a closed functional equation for the moment generating function of the limiting stationary covariance spectrum of discrete-time linear dynamics driven by noise and with i.i.d. non-normal Gaussian weights; the same approach applied to the continuous-time analog produces an infinite hierarchy of Schwinger-Dyson equations instead. The derivation is used to examine the tail eigenvalues of the covariance spectrum in the critical regime (spectral radius approaching 1).

Significance. If the derivation is valid and the stationarity assumption can be justified at criticality, the closed scalar equation would supply an analytic handle on the eigenvalue distribution of the stationary covariance that is currently unavailable for non-normal random matrices; this would be directly relevant to PCA-based analyses of recurrent network models and neural recordings. The contrast between discrete- and continuous-time cases is a clear technical contribution.

major comments (1)

[Abstract / derivation section] Abstract and the section deriving the functional equation: the stationarity assumption (spectral radius of W strictly less than 1) is required for the covariance to be finite and for the free-probability derivation to apply, yet the paper invokes the same equation to analyze tail eigenvalues in the 'critical regime' where the circular law places the radius at g=1. At g=1 the largest Lyapunov exponent vanishes and variance grows linearly with time, so the limiting covariance diverges; no explicit limiting procedure or regularization is supplied to justify passage to the critical point.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful reading of the manuscript and for highlighting an important point regarding the applicability of the derived functional equation. We address the major comment below and indicate the revisions we will make to strengthen the presentation.

read point-by-point responses

Referee: [Abstract / derivation section] Abstract and the section deriving the functional equation: the stationarity assumption (spectral radius of W strictly less than 1) is required for the covariance to be finite and for the free-probability derivation to apply, yet the paper invokes the same equation to analyze tail eigenvalues in the 'critical regime' where the circular law places the radius at g=1. At g=1 the largest Lyapunov exponent vanishes and variance grows linearly with time, so the limiting covariance diverges; no explicit limiting procedure or regularization is supplied to justify passage to the critical point.

Authors: We agree that the free-probability derivation of the closed functional equation is valid under the strict stationarity condition (spectral radius <1). The manuscript analyzes the critical regime by taking the limit g o 1^-, in which the covariance diverges but the shape of the normalized spectrum (including tail eigenvalues) remains well-defined and can be extracted from the functional equation. However, we acknowledge that the current text does not supply an explicit description of this limiting procedure. We will revise the derivation section and the critical-regime analysis to include a clear statement of the limiting process, the normalization used, and any auxiliary regularization (e.g., a small additive damping term taken to zero after the limit) needed to justify passage to g=1. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on external free-probability methods applied to stated assumptions

full rationale

The paper claims to derive a closed functional equation for the MGF of the stationary covariance spectrum via a free-probability approach applied to the random non-normal Gaussian weight ensemble under the explicit stationarity assumption (spectral radius of W < 1). No equations or steps in the provided text reduce the target result to a fitted parameter defined inside the paper, a self-citation chain, or an ansatz smuggled from prior author work. The derivation is presented as importing standard free-probability tools from the external literature rather than constructing the spectrum by renaming or re-fitting its own inputs. The stationarity assumption is stated upfront as a precondition, so the functional equation is conditional on it rather than tautological. This is the most common honest non-finding for papers that invoke established external mathematical machinery.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The derivation rests on standard free-probability results for non-Hermitian random matrices and the assumption of Gaussian i.i.d. entries; no free parameters or new entities are introduced in the abstract.

axioms (1)

domain assumption Free probability theory applies to the non-normal random weight matrices and yields a closed scalar equation for the moment generating function
Invoked to obtain the functional equation for the discrete-time case

pith-pipeline@v0.9.1-grok · 5693 in / 1206 out tokens · 32788 ms · 2026-07-01T02:03:06.802270+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages

[1]

Then, we can solve for f(ζ) = 1− p 1−2ζ.(56) This branch ceases to be real atζ + = 1/2, hence we conclude that asδ↓0 we have λ+ ∼ 1 δ2ζ+ = 2 δ2 .(57) We therefore consider the tail density of eigenvalues on the critical scale λ= x δ2 (58) for 0< x <2. Writing the infinitesimal quantity appear- ing in the Stieltjes inversion formula asi0, we have G x δ2 −i...

work page
[2]

Sompolinsky, A

H. Sompolinsky, A. Crisanti, and H. J. Sommers, Chaos in random neural networks, Physical Review Letters61, 259 (1988)

work page 1988
[3]

Hu and H

Y. Hu and H. Sompolinsky, The spectrum of covariance matrices of randomly connected recurrent neuronal net- works with linear dynamics, PLOS Computational Biol- ogy18, 1 (2022)

work page 2022
[4]

Hennequin, T

G. Hennequin, T. P. Vogels, and W. Gerstner, Non- normal amplification in random balanced neuronal net- works, Physical Review E86, 011909 (2012)

work page 2012
[5]

Toyoizumi and L

T. Toyoizumi and L. F. Abbott, Beyond the edge of chaos: Amplification and temporal integration by recur- rent networks in the chaotic regime, Physical Review E 84, 051908 (2011)

work page 2011
[6]

Mastrogiuseppe, J

F. Mastrogiuseppe, J. Carmona, and C. K. Machens, Stochastic activity in low-rank recurrent neural networks, PLOS Computational Biology21, 1 (2025)

work page 2025
[7]

Bordelon, J

B. Bordelon, J. Cotler, C. Pehlevan, and J. A. Zavatone- Veth, Dynamics of learning to integrate in linear recur- rent neural networks, arXiv (2026)

work page 2026
[8]

J. T. Chalker and B. Mehlig, Eigenvector statistics in non-Hermitian random matrix ensembles, Physical Re- view Letters81, 3367 (1998)

work page 1998
[9]

G. J. Tian, O. Zhu, V. Shirhatti, C. M. Greenspon, J. E. Downey, D. J. Freedman, and B. Doiron, Firing rate di- versity lowers the dimension of population covariability in neuronal networks, bioRxiv 10.1101/2024.08.30.610535 (2026)

work page doi:10.1101/2024.08.30.610535 2024
[10]

Couillet, G

R. Couillet, G. Wainrib, H. T. Ali, and H. Sevi, A ran- dom matrix approach to echo-state neural networks, in Proceedings of The 33rd International Conference on Ma- chine Learning, Proceedings of Machine Learning Re- search, Vol. 48, edited by M. F. Balcan and K. Q. Wein- berger (PMLR, New York, New York, USA, 2016) pp. 517–525

work page 2016
[11]

C. W. Gardiner,Handbook of stochastic methods, Vol. 3 (Springer Berlin, 1985)

work page 1985
[12]

Pachitariu, L

M. Pachitariu, L. Zhong, A. Gracias, A. Minisi, C. Lopez, and C. Stringer, A critical initialization for biologi- cal neural networks, Nature 10.1038/s41586-026-10528-1 (2026)

work page doi:10.1038/s41586-026-10528-1 2026
[13]

Chen and W

X. Chen and W. Bialek, Searching for long timescales without fine tuning, Physical Review E110, 034407 (2024)

work page 2024
[14]

Stringer, M

C. Stringer, M. Pachitariu, N. Steinmetz, M. Carandini, and K. D. Harris, High-dimensional geometry of popula- tion responses in visual cortex, Nature571, 361 (2019)

work page 2019
[15]

V. M. Preciado and M. A. Rahimian, Controllability Gramian spectra of random networks, in2016 American Control Conference (ACC)(2016) pp. 3874–3879

work page 2016
[16]

Godr` eche and J.-M

C. Godr` eche and J.-M. Luck, Characterising the nonequi- librium stationary states of Ornstein–Uhlenbeck pro- cesses, Journal of Physics A: Mathematical and Theo- retical52, 035002 (2018)

work page 2018
[17]

Y. V. Fyodorov, E. Gudowska-Nowak, M. A. Nowak, and W. Tarnowski, Nonorthogonal eigenvectors, fluctuation- dissipation relations, and entropy production, Physical Review Letters134, 087102 (2025)

work page 2025
[18]

L. S. Ferreira, F. L. Metz, and P. Barucca, Random matrix ensemble for the covariance matrix of ornstein- uhlenbeck processes with heterogeneous temperatures, Physical Review E111, 014151 (2025)

work page 2025
[19]

Shen and Y

X. Shen and Y. Hu, Covariance spectrum in nonlinear recurrent neural networks, arXiv (2025)

work page 2025
[20]

J. A. Mingo and R. Speicher,Free Probability and Ran- dom Matrices(Springer New York, New York, NY, 2017)

work page 2017
[21]

Guionnet, Free analysis and random matrices, Japanese Journal of Mathematics11, 33 (2016)

A. Guionnet, Free analysis and random matrices, Japanese Journal of Mathematics11, 33 (2016)

work page 2016
[22]

Byun and P

S.-S. Byun and P. J. Forrester, Progress on the study of the Ginibre ensembles II: GinOE and GinSE, arXiv (2023)

work page 2023
[23]

Collins, T

B. Collins, T. Mai, A. Miyagawa, F. Parraud, and S. Yin, Convergence for noncommutative rational functions eval- uated in random matrices, Mathematische Annalen388, 543 (2024)

work page 2024
[24]

Mart´ ı, N

D. Mart´ ı, N. Brunel, and S. Ostojic, Correlations be- tween synapses in pairs of neurons slow down dynamics in randomly connected neural networks, Physical Review E97, 062314 (2018)

work page 2018
[25]

Belinschi, B

S. Belinschi, B. Ko lodziejek, and K. Szpojankowski, Free perpetuities I: Existence, subordination and tail asymp- totics, arXiv (2025)

work page 2025
[26]

Ko lodziejek and K

B. Ko lodziejek and K. Szpojankowski, On the empir- ical spectral distribution of matrix perpetuities, arXiv (2026)

work page 2026
[27]

D. G. Clark, Linear equivalence of nonlinear recurrent neural networks, arXiv (2026)

work page 2026
[28]

A. J. Wakhloo, Solution of a large nonlinear recurrent neural network at fixed connectivity, arXiv (2026)

work page 2026
[29]

Y. M. Lu and H.-T. Yau, An equivalence principle for the spectrum of random inner-product kernel matrices with polynomial scalings, The Annals of Applied Probability 35, 2411 (2025)

work page 2025
[30]

Atanasov, J

A. Atanasov, J. A. Zavatone-Veth, and C. Pehlevan, Scal- ing and renormalization in high-dimensional regression, Journal of Statistical Mechanics: Theory and Experi- ment2026, 043404 (2026)

work page 2026
[31]

Virtanen, R

P. Virtanen, R. Gommers, T. E. Oliphant, M. Haber- land, T. Reddy, D. Cournapeau, E. Burovski, P. Pe- terson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Po- lat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henr...

work page 2020

[1] [1]

Then, we can solve for f(ζ) = 1− p 1−2ζ.(56) This branch ceases to be real atζ + = 1/2, hence we conclude that asδ↓0 we have λ+ ∼ 1 δ2ζ+ = 2 δ2 .(57) We therefore consider the tail density of eigenvalues on the critical scale λ= x δ2 (58) for 0< x <2. Writing the infinitesimal quantity appear- ing in the Stieltjes inversion formula asi0, we have G x δ2 −i...

work page

[2] [2]

Sompolinsky, A

H. Sompolinsky, A. Crisanti, and H. J. Sommers, Chaos in random neural networks, Physical Review Letters61, 259 (1988)

work page 1988

[3] [3]

Hu and H

Y. Hu and H. Sompolinsky, The spectrum of covariance matrices of randomly connected recurrent neuronal net- works with linear dynamics, PLOS Computational Biol- ogy18, 1 (2022)

work page 2022

[4] [4]

Hennequin, T

G. Hennequin, T. P. Vogels, and W. Gerstner, Non- normal amplification in random balanced neuronal net- works, Physical Review E86, 011909 (2012)

work page 2012

[5] [5]

Toyoizumi and L

T. Toyoizumi and L. F. Abbott, Beyond the edge of chaos: Amplification and temporal integration by recur- rent networks in the chaotic regime, Physical Review E 84, 051908 (2011)

work page 2011

[6] [6]

Mastrogiuseppe, J

F. Mastrogiuseppe, J. Carmona, and C. K. Machens, Stochastic activity in low-rank recurrent neural networks, PLOS Computational Biology21, 1 (2025)

work page 2025

[7] [7]

Bordelon, J

B. Bordelon, J. Cotler, C. Pehlevan, and J. A. Zavatone- Veth, Dynamics of learning to integrate in linear recur- rent neural networks, arXiv (2026)

work page 2026

[8] [8]

J. T. Chalker and B. Mehlig, Eigenvector statistics in non-Hermitian random matrix ensembles, Physical Re- view Letters81, 3367 (1998)

work page 1998

[9] [9]

G. J. Tian, O. Zhu, V. Shirhatti, C. M. Greenspon, J. E. Downey, D. J. Freedman, and B. Doiron, Firing rate di- versity lowers the dimension of population covariability in neuronal networks, bioRxiv 10.1101/2024.08.30.610535 (2026)

work page doi:10.1101/2024.08.30.610535 2024

[10] [10]

Couillet, G

R. Couillet, G. Wainrib, H. T. Ali, and H. Sevi, A ran- dom matrix approach to echo-state neural networks, in Proceedings of The 33rd International Conference on Ma- chine Learning, Proceedings of Machine Learning Re- search, Vol. 48, edited by M. F. Balcan and K. Q. Wein- berger (PMLR, New York, New York, USA, 2016) pp. 517–525

work page 2016

[11] [11]

C. W. Gardiner,Handbook of stochastic methods, Vol. 3 (Springer Berlin, 1985)

work page 1985

[12] [12]

Pachitariu, L

M. Pachitariu, L. Zhong, A. Gracias, A. Minisi, C. Lopez, and C. Stringer, A critical initialization for biologi- cal neural networks, Nature 10.1038/s41586-026-10528-1 (2026)

work page doi:10.1038/s41586-026-10528-1 2026

[13] [13]

Chen and W

X. Chen and W. Bialek, Searching for long timescales without fine tuning, Physical Review E110, 034407 (2024)

work page 2024

[14] [14]

Stringer, M

C. Stringer, M. Pachitariu, N. Steinmetz, M. Carandini, and K. D. Harris, High-dimensional geometry of popula- tion responses in visual cortex, Nature571, 361 (2019)

work page 2019

[15] [15]

V. M. Preciado and M. A. Rahimian, Controllability Gramian spectra of random networks, in2016 American Control Conference (ACC)(2016) pp. 3874–3879

work page 2016

[16] [16]

Godr` eche and J.-M

C. Godr` eche and J.-M. Luck, Characterising the nonequi- librium stationary states of Ornstein–Uhlenbeck pro- cesses, Journal of Physics A: Mathematical and Theo- retical52, 035002 (2018)

work page 2018

[17] [17]

Y. V. Fyodorov, E. Gudowska-Nowak, M. A. Nowak, and W. Tarnowski, Nonorthogonal eigenvectors, fluctuation- dissipation relations, and entropy production, Physical Review Letters134, 087102 (2025)

work page 2025

[18] [18]

L. S. Ferreira, F. L. Metz, and P. Barucca, Random matrix ensemble for the covariance matrix of ornstein- uhlenbeck processes with heterogeneous temperatures, Physical Review E111, 014151 (2025)

work page 2025

[19] [19]

Shen and Y

X. Shen and Y. Hu, Covariance spectrum in nonlinear recurrent neural networks, arXiv (2025)

work page 2025

[20] [20]

J. A. Mingo and R. Speicher,Free Probability and Ran- dom Matrices(Springer New York, New York, NY, 2017)

work page 2017

[21] [21]

Guionnet, Free analysis and random matrices, Japanese Journal of Mathematics11, 33 (2016)

A. Guionnet, Free analysis and random matrices, Japanese Journal of Mathematics11, 33 (2016)

work page 2016

[22] [22]

Byun and P

S.-S. Byun and P. J. Forrester, Progress on the study of the Ginibre ensembles II: GinOE and GinSE, arXiv (2023)

work page 2023

[23] [23]

Collins, T

B. Collins, T. Mai, A. Miyagawa, F. Parraud, and S. Yin, Convergence for noncommutative rational functions eval- uated in random matrices, Mathematische Annalen388, 543 (2024)

work page 2024

[24] [24]

Mart´ ı, N

D. Mart´ ı, N. Brunel, and S. Ostojic, Correlations be- tween synapses in pairs of neurons slow down dynamics in randomly connected neural networks, Physical Review E97, 062314 (2018)

work page 2018

[25] [25]

Belinschi, B

S. Belinschi, B. Ko lodziejek, and K. Szpojankowski, Free perpetuities I: Existence, subordination and tail asymp- totics, arXiv (2025)

work page 2025

[26] [26]

Ko lodziejek and K

B. Ko lodziejek and K. Szpojankowski, On the empir- ical spectral distribution of matrix perpetuities, arXiv (2026)

work page 2026

[27] [27]

D. G. Clark, Linear equivalence of nonlinear recurrent neural networks, arXiv (2026)

work page 2026

[28] [28]

A. J. Wakhloo, Solution of a large nonlinear recurrent neural network at fixed connectivity, arXiv (2026)

work page 2026

[29] [29]

Y. M. Lu and H.-T. Yau, An equivalence principle for the spectrum of random inner-product kernel matrices with polynomial scalings, The Annals of Applied Probability 35, 2411 (2025)

work page 2025

[30] [30]

Atanasov, J

A. Atanasov, J. A. Zavatone-Veth, and C. Pehlevan, Scal- ing and renormalization in high-dimensional regression, Journal of Statistical Mechanics: Theory and Experi- ment2026, 043404 (2026)

work page 2026

[31] [31]

Virtanen, R

P. Virtanen, R. Gommers, T. E. Oliphant, M. Haber- land, T. Reddy, D. Cournapeau, E. Burovski, P. Pe- terson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Po- lat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henr...

work page 2020