Discontinuous phase transition of feature detection in lateral predictive coding

Hai-Jun Zhou; Weikang Wang; Zhen-Ye Huang

REVIEW 3 major objections 6 minor 48 references

Discontinuous phase transition of feature detection in lateral predictive coding

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Minimizing a free energy in lateral predictive coding makes feature detection appear and disappear through two discontinuous phase transitions.

desk verdict A solid analytical framework for energy-entropy tradeoff in lateral predictive coding, but the exact discontinuity temperatures rest on an annealing search that demonstrably misses global minima. read the letter →

arxiv 2501.12139 v2 pith:EDZSXKZG submitted 2025-01-21 q-bio.NC cond-mat.dis-nncond-mat.stat-mech

Zhen-Ye Huang , Weikang Wang , Hai-Jun Zhou This is my paper · ORCID

classification q-bio.NCcond-mat.dis-nncond-mat.stat-mech MSC 82B2692B20

keywords lateralpredictivecodingphasetransitionfeaturedetectionenergy-informationtradeoffL1-normenergynon-Gaussianfeaturesfreeminimizationentropy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single optimization principle — minimizing a free energy that balances transmission cost against information robustness — is enough to make a lateral predictive coding (LPC) network detect non-Gaussian features that are invisible to principal component analysis. The cost is the mean absolute ($L_1$-norm) output of the units, and information robustness is measured by the entropy difference $S = -\ln \det(I+W)$ of the output distribution. For one hidden non-Gaussian feature among Gaussian backgrounds with the same variance, the optimal weight matrix comes in three distinct types, and the switches between them at two temperatures are discontinuous. At both low and high temperatures one unit snaps into selective response to the feature (overlap $Q \approx 0.83$–$0.97$), while at intermediate temperatures the optimal matrix is weak and homogeneous with no selectivity. The same mechanism separates two non-Gaussian features into two distinct units, even when the two features are not orthogonal.

What carries the argument

The load-bearing object is the thermodynamic free energy $F = E - T S$, with $S = -\ln \det(I+W)$ measuring how much the linear map $\vec{x} = (I+W)^{-1}\vec{s}$ expands output entropy (information robustness) and $E$ the mean $L_1$-norm of internal states (metabolic cost). The argument is carried by the order parameter $Q = \max_i |\mu_i| / \sqrt{\sum_j \mu_j^2}$, where $\mu_i$ is the mean response of unit $i$ to the non-Gaussian feature; $Q \approx 1$ means one unit is selective and $Q = 1/\sqrt{N}$ means all units respond equally and weakly. Because the energy curve $E(S)$ is non-convex in the middle entropy range, the free energy has two coexisting minima at the transition temperatures. The $L_1$ norm is essential: it couples the energy to higher moments of the input, which lets the non-Gaussian feature be detected even though the input correlation matrix is the identity and carries no directional information.

What would settle it

Evaluate the same $E(S)$ optimization on a small system ($N=10$) with a certified global-search method that cannot be trapped, and compare the resulting free-energy crossings to $T=0.8320$ and $T=1.1283$; if the global minimum at intermediate $S$ is not the weak homogeneous solution with $S\approx0$ and $Q=1/\sqrt{N}$, or if the low-energy feature-detecting branch near $S=-1.12$ is only a local minimum, the discontinuous transitions are artifacts of the annealing algorithm. A concrete spot to check is the paper's own $S=-1.5$ data, where 71% of runs return $E\approx27.55$ instead of the claimed global $E\approx27.4955$.

Watch

Extended reading notes

Core claim

The central claim is that in the linear LPC model with steady-state output $\vec{x} = (I+W)^{-1}\vec{s}$ and $L_1$-norm energy $E$, the global minimum of $F = E - T S$ undergoes two discontinuous phase transitions in the temperature-like parameter $T$. At $T = 1.1283$ (for $N=36$, $p_0=0.7$) the free-energy landscape has two degenerate minima, one at $S \approx 7.10$ with $Q \approx 0.97$ and one at $S = 0$ with $Q = 0.1667 = 1/\sqrt{36}$; below $T = 0.8320$ another degenerate pair appears near $S \approx -1.12$ with $Q \approx 0.83$. Feature detection is therefore feasible at high and low temperature but impossible in between. The paper extends the same optimization to two non-orthogonal non-Gaussian features and reports three discontinuous transitions at $T \approx 0.8316$, $0.9935$, and $1.2612$, after which the two features are represented by two separate single units. The same qualitative scenario is reported for Laplace and power-law feature distributions and for random feature directions.

Load-bearing premise

The load-bearing premise is that the annealing search finds the true global energy minimum at every fixed entropy $S$; since roughly 71% of runs at $S=-1.5$ land in a local minimum with $E\approx27.55$ rather than the claimed global $27.4955$, the phase-transition temperatures depend on an optimizer whose global optimality is not certified.

Editorial extensions

If this is right

At $T<0.8320$ and $T>1.1283$ for the $N=36$, $p_0=0.7$ ensemble, the global minimum of $F=E-TS$ is a matrix in which one unit responds selectively to the non-Gaussian feature, with overlap $Q\approx0.83$–$0.97$.
Between those temperatures, the optimal matrix has very weak, homogeneous lateral weights, $S\approx0$, and $Q=1/\sqrt{N}$, so feature detection fails; the switches at $T=0.8320$ and $T=1.1283$ are discontinuous.
For two non-Gaussian features with angle $\theta=\pi/4$, $N=16$, and $p_0=0.6$, the same tradeoff produces discontinuous transitions at $T\approx0.8316$, $0.9935$, and $1.2612$, after which two different units each detect one feature — even when the features are not orthogonal.
The $L_1$-norm energy, rather than the usual $L_2$-norm, is what enables detection: an $L_2$-norm energy depends only on the input correlation matrix, which is exactly the same for the Gaussian background and the non-Gaussian feature.
The qualitative result persists for the continuous Laplace distribution, a power-law feature distribution, and randomly oriented feature directions, so the mechanism is not an artifact of the discrete three-point prior used in the main runs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If a biologically plausible local learning rule can be shown to descend the same free energy, the model predicts sudden, insight-like jumps in single-neuron selectivity during learning, because the feature-detecting and feature-blind solutions are separated by an energy barrier rather than connected continuously.
A natural next experiment is to scan the fraction $p_g$ of Gaussian trials in the coefficient $a_1$; the paper notes this direction but does not explore it, and one would expect a critical $p_g$ beyond which the $L_1$ energy can no longer discriminate the non-Gaussian feature.
For artificial recurrent networks with lateral connections, replacing the usual squared internal states by an $L_1$ cost on activations should reproduce the same discontinuous assignment of hidden units to non-Gaussian latent sources, giving a practical signature of the transition outside the biological setting.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, here's the read.

The genuinely new thing is the L1-norm energy term. With L2 energy, the cost only sees the correlation matrix, so non-Gaussian features are invisible. Replacing it with L1 makes the objective sensitive to higher-order moments, and the paper shows that minimizing E - T S spontaneously produces a single unit that detects the hidden non-Gaussian feature at both low and high T, with an undetecting phase in between. That is a real result and a clean way to connect predictive coding to sparse coding and ICA.

The analytical core is in good shape. The energy expression Eq. (15), the conditional output statistics, and the free-energy arithmetic at the stated crossings are internally consistent. The paper also reports local-minima bands honestly, which is more than most optimization papers do. Self-citation to Ref. [13] is appropriate: the entropy definition and annealing method come from there. The qualitative picture is robust across different feature directions, p0 values, Laplace and power-law priors, and the two-feature case.

Now the soft spot. The quantitative claim lives entirely in the E(S) curve from entropy-clamped annealing. The paper's own Fig. 1 shows the algorithm frequently landing in local minima whose energy gaps are 0.05 or more—at S=0, the global minimum is 28.72 but 40% of runs sit at 29.15 with a qualitatively different matrix. The free-energy crossings are balanced to four decimal places. A miss of a few hundredths on one branch, smaller than the observed local-minimum gaps, could shift those transition temperatures and potentially shrink or eliminate the intermediate undetecting phase. There are no error bars on E(S) and no independent optimization (exhaustive small-N enumeration, basin hopping, or similar). So the exact values T=0.8320 and T=1.1283 are not established; the existence of the two transitions is plausible but not proven.

Is that load-bearing? For the paper's central claim, partially. If the annealing is missing a lower-energy detecting branch at S=0, the middle phase could vanish. The authors argue the branches are genuine because they see consistent bands and because E(S) is monotonic, but that is not a certificate. This makes the paper a solid conditional result rather than a definitive one.

Who this is for: statistical physicists working on neural representation, and anyone interested in energy-information tradeoffs as an optimization principle. A serious referee can push on the global optimality question and the paper will survive or not depending on that.

Recommendation: send it to peer review. The framework and the qualitative phenomenon are worth referee time, and the numerical gap is addressable in revision.

Referee Report

3 major / 6 minor

Summary. This paper studies a linear lateral predictive coding (LPC) model with an L1-norm energetic cost E (Eq. 3) and an entropy measure S = -ln det(I+W) (Eq. 4), optimized through the free energy F = E - T S (Eq. 5). For inputs consisting of one non-Gaussian feature embedded in Gaussian background with identity covariance, the authors derive conditional output statistics and the analytical energy expression Eq. (15), then use entropy-clamped annealing to construct the minimum-energy curve E(S). From this curve they infer two discontinuous phase transitions at T = 0.8320 and T = 1.1283 for N = 36, p0 = 0.7: a low-temperature selective phase, an intermediate S ≈ 0 non-selective phase, and a high-temperature selective phase. The same construction is applied to two non-Gaussian features with angle θ = π/4 (N = 16, p0 = 0.6), yielding three reported transitions at T = 0.8316, 0.9935, and 1.2612. The paper also reports a spectrum analysis showing that the complex eigenvalues of the optimal (I+W) lie approximately on a semicircle.

Significance. If the numerical determination is reliable, the result is significant: it demonstrates in a transparent linear model how non-Gaussian structure can be detected through L1-norm energy minimization, with feature detection emerging spontaneously from an energy-information tradeoff rather than from an imposed sparsity penalty. The analytical parts—the conditional single-unit statistics, the energy expression Eq. (15), and the entropy calculation—are internally consistent, and I found no error in the free-energy arithmetic at the reported crossings. The paper is also commendably candid about the existence of local minima. The main weakness is that the central quantitative claim is built on an E(S) curve whose global optimality is not certified; the paper's own numbers show that the annealing frequently stops in structurally different local minima, and the reported transition temperatures are obtained by comparing free-energy branches whose differences are comparable to the observed local-minimum gaps.

major comments (3)

[Sec. III.B, Fig. 1, Fig. 3] The global optimality of the E(S) curve is not established, and this is load-bearing because the transition temperatures are read from crossings of minima of F = E - T S. The paper's own data show that the annealing frequently terminates in local minima that differ in exactly the order parameter used to define phases: at S = 0 about 40% of runs find high-Q matrices with E ≈ 29.15 instead of the claimed global E = 28.7235, and at S = -1.5 about 71% of runs find E ≈ 27.55 with Q ≈ 0.565 instead of E = 27.4955 with Q ≈ 0.8387. A systematic miss of a few hundredths in E on one branch—an order of magnitude smaller than the gaps visible in Fig. 1—could shift the crossings at T = 0.8320 and T = 1.1283 or even eliminate the intermediate non-detecting phase. The manuscript should provide independent certification of the global minima, for example exact enumeration or branch-and-bound for a small N, multi-start basin hopping with random initial matrices, and/or explicit error bars or bounds on the E(S) values used in Fig. 3.
[Sec. III.C, Fig. 3(d); Sec. IV, Fig. 6] The transition temperatures are not accompanied by any sensitivity or error analysis. The two degenerate free-energy minima at T = 1.1283 (S = 7.10 and S = 0) and at T = 0.8320 (S = -1.12 and S = 0) are claimed to be global minima of F, but Fig. 3(d) shows no uncertainty, and the text gives no stopping criterion or optimization tolerance beyond the local-minimum counts of Fig. 1. Because the competing branches are balanced to a precision comparable to the observed local-minimum gaps, the authors should report how T_c changes when E(S) on one branch is perturbed by, say, ±0.01, and should state explicitly whether the reported transitions survive under such perturbations. The same requirement applies to the two-feature transitions reported in Fig. 6, which inherit the same annealing-based E(S) construction.
[Sec. IV, Eq. (16)] For the two-feature setup with θ ≠ π/2, the input covariance implied by Eq. (16) is not the identity: the feature directions contribute variance 2cos^2(θ/2) along φ1 and 2sin^2(θ/2) along φ2, while the background contributes unit variance. The paper does not discuss whether this makes the task partially accessible to second-order statistics. This does not invalidate the energy-information tradeoff argument, but the text should state the covariance explicitly and clarify what aspect of the two-feature task is genuinely not solvable by PCA when the features are non-orthogonal.

minor comments (6)

[Abstract] The phrase 'Energy–information tradeoff induce' should be 'Energy–information tradeoff induces' for subject-verb agreement.
[Sec. III.B] The text says 'typically 10 6 trials'; this should be typeset as 10^6.
[Appendix C, after Eq. (C12)] The statement that ⟨x_i^2⟩ = (π/2)^{1/2}T appears dimensionally inconsistent with E = N T and with the preceding derivation, which gives ⟨x_i^2⟩ = π T^2 / 2. Please check and correct this expression.
[Fig. 5(d) caption] The phrase 'magnitudes sqrt(|λ|^2)' should be simply 'magnitudes |λ|'.
[Fig. 1 and Fig. 2] The label 'Rank' is used for the sorted run index in Fig. 1 and for the unit index in Fig. 2; consider using distinct labels such as 'run index' and 'unit index' to avoid confusion.
[References] The supplementary material is cited as Ref. [30] without a formal title or identifier; please provide a stable citation or DOI for the supplement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the free-energy objective is defined from the generative model and the order parameters are post-optimization diagnostics, not fitted targets.

full rationale

Walking the derivation chain: the energy E is computed from the assumed input statistics (Eq. 15, derived in Appendix E), the entropy S is derived from the change-of-variables formula for the output density (Eq. 4 and Appendix A/S1), and the free energy F = E - T S is a stated modeling choice (Eq. 5). The phase transitions are obtained by minimizing F over the numerically computed E(S) curve (Sec. III C), and the overlap order parameters Q, Q(1), Q(2) are defined (Eqs. 12-13, 21) and evaluated only after the optimizing matrix is found; they are not part of the objective being minimized. The only self-citations, chiefly Ref. [13], supply the earlier energy-information framework and the entropy-clamped annealing technique, but the present paper re-derives S independently and reports its own 600-run optimizations with explicit local-minimum analysis (Fig. 1), so the central claim does not reduce to a self-citation chain. The main caveat, that global optimality of E(S) is not certified (e.g., 71% of runs at S = -1.5 land at E ~ 27.55 rather than the claimed 27.4955), is a quantitative correctness risk rather than a circularity: the transition temperatures are balanced to small energy differences, but no equation defines the transitions in terms of the order parameters or fits them to the claimed detection phases.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No entities are invented beyond the modeling choices of L1 energy and the non-Gaussian input prior. The 'feature detection' is an emergent property measured by an order parameter, not an input to the optimization.

free parameters (4)

p0 (sparsity fraction of non-Gaussian coefficient) = 0.7 (single feature), 0.6 (two features)
Chosen by hand as a representative deviation from Gaussian; phase diagram in Fig. S1 scans it. The location of the transitions depends on p0, but the qualitative picture does not.
theta (angle between two non-Gaussian features) = pi/4 (main text), pi/2 (supplementary)
Represents degree of alignment between the two features; results qualitatively similar for theta in {pi/4, pi/3, pi/2}.
N (network size) = 10, 16, 36, 100
Finite system sizes; onset of detection shifts with N, but qualitative behavior remains.
Temperature T = varied 0.7 to 1.6
Control parameter of the tradeoff, not fitted to data; phase transitions located at T = 0.8320 and T = 1.1283 (N = 36, p0 = 0.7).

assumptions (4)

domain assumption The brain minimizes the free energy F = E - T S with E the mean L1 norm of internal states and S the output-entropy difference.
Central modeling assumption, stated in Sec. II; the optimal LPC matrix is defined as the global minimum of F.
domain assumption The input consists of one (or two) non-Gaussian feature directions plus Gaussian background with the same covariance matrix (Eq. (7), Eq. (16)).
Defines the task; covariance is identity so PCA cannot find the feature, motivating L1 energy.
domain assumption The steady-state output of the linear dynamics is x = (I+W)^{-1} s, requiring all eigenvalues of I+W to have positive real parts.
Used to derive the entropy S = -ln det(I+W) and the output statistics in Secs. II and III.
ad hoc to paper The annealing algorithm with fixed entropy S and the Metropolis acceptance rule converges to the global minimum energy E(S).
Underlies the E(S) curve and the phase transition locations in Sec. III C; the paper reports local minima, so this is an assumption rather than a proven property.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Discontinuous phase transition of feature detection in lateral predictive coding." pith.science (2026). https://pith.science/paper/EDZSXKZG

@misc{pith2026250112139,
  author       = {Pith},
  title        = {Pith review of: Discontinuous phase transition of feature detection in lateral predictive coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EDZSXKZG}},
  note         = {Machine review of arXiv:2501.12139}
}

abstract

The brain may adopt the strategy of lateral predictive coding (LPC) to construct optimal internal representations for salient features in input sensory signals, reducing the energetic cost of information transmission. Here we first consider the task of detecting one non-Gaussian signal by LPC from Gaussian background signals of the same magnitude, which is intractable by principal component decomposition. We study the emergence of feature detection function from the perspective of tradeoff between energetic cost $E$ and information robustness, and implement this tradeoff by a thermodynamic free energy. We define $E$ as the mean $L_1$-norm of the internal state vectors, and quantify the level of information robustness by an entropy measure $S$. There are at least three types of optimal LPC matrices, one type with very weak synaptic weights and $S \approx 0$, and two functional types either with low energy $E$ or with high entropy $S$ in which one single unit selectively responds to the non-Gaussian input feature. Energy--information tradeoff induce two discontinuous phase transitions between these three types of optimal LPC networks. We then extend the discussion to detecting and distinguishing between two non-Gaussian input features and observe similar discontinuous phase transitions.

Figures

Figures reproduced from arXiv: 2501.12139 by the authors.

**Figure 1.** (a) plots in ascending order the obtained minimal energies E and the corresponding overlaps Q from 600 independent runs of the matrix annealing algorithm at fixed S = 0, all starting from the same initial weight matrix. The minimal energies form several bands, indicating the existence of many local minimal energies. The global minimum energy is E = 28.7235, and the corresponding overlap Q = 0.1667 is equal to the… view at source ↗

**Figure 2.** FIG. 2. Rank plots of the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** FIG. 3. Thermodynamic quantities versus entropy [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: FIG. 4. Example optimal weight matrices for the system of [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: FIG. 5. Spectrum analysis for system size [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: FIG. 6. Thermodynamic quantities of optimal lateral predictive coding for input vectors (16) containing two non-orthogonal [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: FIG. 7. Four representative optimal LPC matrices of Fig. 6, with [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

**Figure 8.** Figure 8: FIG. 8. The output signals [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

**Figure 9.** Figure 9: FIG. 9. Continuous phase transition phenomenon for correlated Gaussian input signals (C1) with [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages

[13]

Co-emergence of multi-scale cortical activities of irregular firing, oscil- lations and avalanches achieves cost-efficient information capacity,

D.-P. Yang, H.-J. Zhou, and C. Zhou, “Co-emergence of multi-scale cortical activities of irregular firing, oscil- lations and avalanches achieves cost-efficient information capacity,” PLoS Comput. Biol. 13, e1005384 (2017)

work page 2017
[1]

Given an input signal ⃗s, the conditional distribution of the output signal ⃗x is then pout(⃗x|⃗s) = 1 (2πσ 2 0)N/2 exp h − ⃗x − (I + W )−1⃗s 2 2σ2 0 i . (B2) The mutual information between output ⃗x and input ⃗s is given by I ⃗x; ⃗s ≡ H pout(⃗x) − H ⃗x|⃗s , (B3) 13 where H ⃗x|⃗s is the conditional entropy of the output ⃗x given the input ⃗s: H ⃗x|⃗s ≡ − ...

work page
[2]

When σ2 0 is small, we may assume H pout(⃗x) to be a smooth function of σ2

work page
[3]

As a zeroth-order ap- proximation, we approximate the value of H pout(⃗x) by its limiting value at σ2 0 = 0, which is Eq. (S11). The W -dependent part of the mutual information I ⃗x; ⃗s is therefore approximated by I ⃗x; ⃗s ≈ −ln h det(I + W ) i = S . (B5) When the output noise ⃗ϵ of Eq. (B1) is not Gaussian, Eq. (B4) will no longer hold exactly, and the ...

work page
[4]

Pre- dictive coding: a fresh view of inhibition in the retina,

M. V. Srinivasan, S. B. Laughlin, and A. Dubs, “Pre- dictive coding: a fresh view of inhibition in the retina,” Proc. R. Soc. Lond. B 216, 427–459 (1982)

work page 1982
[5]

Predictive coding in the visual cortex: a functional interpretation of some extra- classical receptive-field effects,

R. P. N. Rao and D. H. Ballard, “Predictive coding in the visual cortex: a functional interpretation of some extra- classical receptive-field effects,” Nature Neurosci. 2, 79– 87 (1999)

work page 1999
[6]

Predictive coding,

Y. Huang and R. P. N. Rao, “Predictive coding,” WIREs Cogn. Sci. 2, 580–593 (2011)

work page 2011
[7]

Predictive coding is a consequence of energy efficiency in recurrent neural networks,

A. Ali, N. Ahmad, E. de Groot, M. A. J. van Gerven, and T. C. Kietzmann, “Predictive coding is a consequence of energy efficiency in recurrent neural networks,” Patterns 3, 100639 (2022)

work page 2022

Show all 48 references

[8]

Pre- dictive coding networks and inference learning: Tutorial and survey,

B. van Zwol, R. Jefferson, and E. L. van den Broek, “Pre- dictive coding networks and inference learning: Tutorial and survey,” eprint arXiv:2407.04117 [cs.LG] (2024)

2024
[9]

Predictive coding: Towards a future of deep learning beyond backpropagation?

B. Millidge, T. Salvatori, Y. Song, R. Bogacz, and T. Lukasiewicz, “Predictive coding: Towards a future of deep learning beyond backpropagation?” in Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI), Vienna, Austria (2022) pp. 5538– 5545

2022
[10]

Lat- eral predictive coding revisited: internal model, symme- try breaking, and response time,

Z.-Y. Huang, X.-Y. Fan, J. Zhou, and H.-J. Zhou, “Lat- eral predictive coding revisited: internal model, symme- try breaking, and response time,” Commun. Theor. Phys. 74, 095601 (2022)

2022
[11]

Sparse coding via theresholding and local competition in neural circuits,

C. J. Rozell, D. H. Johnson, R. G. Baraniuk, and B. A. Olshausen, “Sparse coding via theresholding and local competition in neural circuits,” Neural Comput. 20, 2526–2563 (2008)

2008
[12]

Efficient coding and energy efficiency are promoted by balanced excita- tory and inhibitory synaptic currents in neuronal net- work,

L. Yu, Z. Shen, C. Wang, and Y. Yu, “Efficient coding and energy efficiency are promoted by balanced excita- tory and inhibitory synaptic currents in neuronal net- work,” Front. Cell. Neurosci. 12, 123 (2018)

2018
[14]

Recurrent predictive cod- ing models for associative memory employing covariance learning,

M. Tang, T. Salvatori, B. Millidge, Y. Song, T. Lukasiewicz, and R. Bogacz, “Recurrent predictive cod- ing models for associative memory employing covariance learning,” PLOS Comput. Biol. 19 (4), e1010719 (2023)

2023
[15]

Hyv¨ arinen, J

A. Hyv¨ arinen, J. Hurri, and P. O. Hoyer,Natural Image Statistics: A Probabilistic Approach to Early Computa- tional Vision (Springer, London, UK, 2009)

2009
[16]

Energy–information trade-off induces continuous and discontinuous phase transitions in lateral predictive cod- ing,

Z.-Y. Huang, R. Zhou, M. Huang, and H.-J. Zhou, “Energy–information trade-off induces continuous and discontinuous phase transitions in lateral predictive cod- ing,” Science China: Phys. Mech. Astron. 67, 260511 (2024)

2024
[17]

Single units and sensation: A neuron doc- trine for perceptual psychology?

H. B. Barlow, “Single units and sensation: A neuron doc- trine for perceptual psychology?” Perception 1, 371–394 (1972)

1972
[18]

Ambitions for theory in the physics of life,

W. Bialek, “Ambitions for theory in the physics of life,” SciPost Phys. Lect. Notes , 84 (2024)

2024
[19]

Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture,

C. Jutten and J. Herault, “Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture,” Signal Processing 24, 1–10 (1991)

1991
[20]

Indicators of discontinuous change in the devel- opment of analogical reasoning,

B. Hosenfeld, H. L. J. van den Maas, and D. C. van den Boom, “Indicators of discontinuous change in the devel- opment of analogical reasoning,” J. Exper. Child Psychol. 64, 367–395 (1997)

1997
[21]

Does practice make perfect? a slow and discontinuous process,

H. P. A. Boshuizen, “Does practice make perfect? a slow and discontinuous process,” in Professional Learn- ing: Gaps and Transitions on the Way from Novice to Expert, edited by H. P. A. Boshuizen, R. Bromme, and H. Gruber (Kluwer Academic Publishers, New York, 2004) Chap. 5, pp. 73–96

2004
[22]

Measuring cognitive load and insight: A methodology exemplified in a virtual reality learning context,

J. Collins, H. Regenbrecht, T. Langlotz, Y. S. Can, C. Ersoy, and R. Butson, “Measuring cognitive load and insight: A methodology exemplified in a virtual reality learning context,” in Proceedings of 2019 IEEE Interna- tional Symposium on Mixed and Augmented Reality (IS- MAR),...

2019
[23]

Mice in a labyrinth show rapid learning, sudden insight, and efficient exploration,

M. Rosenberg, T. Zhang, P. Perona, and M. Meister, “Mice in a labyrinth show rapid learning, sudden insight, and efficient exploration,” eLife 10, e66175 (2021)

2021
[24]

Phase transi- tions of brain evolution that produced human language and beyond,

R. V. Bretas, Y. Yamazaki, and A. Iriki, “Phase transi- tions of brain evolution that produced human language and beyond,” Neurosci. Res. 161, 1–7 (2020)

2020
[25]

Evolutionary transitions in learning and cognition,

S. Ginsburg and E. Jablonka, “Evolutionary transitions in learning and cognition,” Phil. Trans. R. Soc. B 376, 20190766 (2021)

2021
[26]

Neuronal energy consumption: biophysics, efficiency and evolution,

J. E. Niven, “Neuronal energy consumption: biophysics, efficiency and evolution,” Curr. Opin. Neurobiol. 41, 129–135 (2016)

2016
[27]

Updated en- ergy budgets for neural computation in the neocortex and cerebellum,

C. Howarth, P. Gleeson, and D. Attwell, “Updated en- ergy budgets for neural computation in the neocortex and cerebellum,” J. Cereb. Blood Flow Metabol. 32, 1222– 1232 (2012)

2012
[28]

An information- maximization approach to blind separation and blind de- convolution,

A. J. Bell and T. J. Sejnowski, “An information- maximization approach to blind separation and blind de- convolution,” Neural Comput. 7, 1129–1159 (1995)

1995
[29]

Phase transitions in pareto optimal complex networks,

L. F. Seoane and R. Sol´ e, “Phase transitions in pareto optimal complex networks,” Phys. Rev. E 92, 032807 (2015)

2015
[30]

Internal energy, fundamental thermodynamic relation, and gibbs’ ensemble theory as emergent laws of statistical counting,

H. Qian, “Internal energy, fundamental thermodynamic relation, and gibbs’ ensemble theory as emergent laws of statistical counting,” Entropy 26, 1091 (2024)

2024
[31]

Signature of pareto optimization in the es- cherichia coli proteome,

L. Ko¸ cillari, P. Fariselli, A. Trovato, F. Seno, and A. Maritan, “Signature of pareto optimization in the es- cherichia coli proteome,” Sci. Rep. 8, 9141 (2018)

2018
[32]

The scaling limit of high- dimensional online independent component analysis,

C. Wang and Y. M. Lu, “The scaling limit of high- dimensional online independent component analysis,” J. Stat. Mech. Theor. Exp. 2019, 124011 (2019)

2019
[33]

It contains addtional technical details of the theo- retical derivation, and additional numerical data to sup- port the main conclusions of this work

Online supplementary information accompanying this work. It contains addtional technical details of the theo- retical derivation, and additional numerical data to sup- port the main conclusions of this work
[34]

The energy-speed-accuracy trade-off in sensory adapta- tion,

G. Lan, P. Sartori, S. Neumann, V. Sourjik, and Y. Tu, “The energy-speed-accuracy trade-off in sensory adapta- tion,” Nature Phys. 8, 422–428 (2012)

2012
[35]

Tuning transduction from hidden observables to optimize information harvest- ing,

G. Nicoletti and D. M. Busiello, “Tuning transduction from hidden observables to optimize information harvest- ing,” Phys. Rev. Lett. 133, 158401 (2024)

2024
[36]

Thermodynamic cost of finite-time stochastic resetting,

K. S. Olsen, D. Gupta, F. Mori, and S. Krishna- murthy, “Thermodynamic cost of finite-time stochastic resetting,” Phys. Rev. Res. 6, 033343 (2024)

2024
[37]

Chaos in random neural networks,

H. Sompolinsky, A. Crisanti, and H. J. Sommers, “Chaos in random neural networks,” Phys. Rev. Lett. 61, 259– 262 (1988)

1988
[38]

An optimization-based equilib- 17 rium measure describing fixed points of non-equilibrium dynamics: application to the edge of chaos,

J. Qiu and H. Huang, “An optimization-based equilib- 17 rium measure describing fixed points of non-equilibrium dynamics: application to the edge of chaos,” Commun. Theor. Phys. 77, 035601 (2024)

2024
[39]

Frequency-dependent covariance reveals critical spatiotemporal patterns of synchronized activ- ity in the human brain,

R. Calvo, C. Martorell, G. B. Morales, S. Di Santo, and M. A. Mu˜ noz, “Frequency-dependent covariance reveals critical spatiotemporal patterns of synchronized activ- ity in the human brain,” Phys. Rev. Lett. 133, 208401 (2024)

2024
[40]

Signatures of criticality in efficient coding networks,

S. Safavi, M. Chalk, N. K. Logothetis, and A. Levina, “Signatures of criticality in efficient coding networks,” Proc. Natl. Acad. Sci. USA 121, e2302730121 (2024)

2024
[41]

Excitation- inhibition balance controls information encoding in neu- ral populations,

G. Barzon, D. M. Busiello, and G. Nicoletti, “Excitation- inhibition balance controls information encoding in neu- ral populations,” Phys. Rev. Lett. 134, 068403 (2025)

2025
[42]

Independent component anal- ysis: algorithms and applications,

A. Hyv¨ arinen and E. Oja, “Independent component anal- ysis: algorithms and applications,” Neural Networks 13, 411–430 (2000)

2000
[43]

From complex to simple: hierarchical free- energy landscape renormalized in deep neural networks,

H. Yoshino, “From complex to simple: hierarchical free- energy landscape renormalized in deep neural networks,” SciPost Phys. Core 2, 005 (2020)

2020
[44]

Phase transitions for the infor- mation bottleneck in representation learning,

T. Wu and I. Fischer, “Phase transitions for the infor- mation bottleneck in representation learning,” in Inter- national Conference on Learning Representations (2020)

2020
[45]

Deriving a genetic regulatory network from an opti- mization principle,

T. R. Sokolowski, T. Gregor, W. Bialek, and G. Tkaˇ cik, “Deriving a genetic regulatory network from an opti- mization principle,” Proc. Natl. Acad. Sci. USA 122, e2402925121 (2025)

2025
[46]

Energy-information trade-off makes the cortical critical power law the optimal coding,

T. Tatsukawa and J.-n. Teramae, “Energy-information trade-off makes the cortical critical power law the optimal coding,” eprint arXiv:2407.16215 [q-bio.NC] (2024)

2024 arXiv
[47]

Neural heterogeneity enhances reliable neu- ral information processing: Local sensitivity and glob- ally input-slaved transient dynamics,

S. Wu, H. Huang, S. Wang, G. Chen, C. Zhou, and D. Yang, “Neural heterogeneity enhances reliable neu- ral information processing: Local sensitivity and glob- ally input-slaved transient dynamics,” Science Adv. 11, eadr3903 (2025). 1 Discontinuous phase transitions of feature d...

2025
[48]

(S13) for the special case of a1 = 1/√1 − p0

as defined by Eq. (S13) for the special case of a1 = 1/√1 − p0. As another concrete example, we assume the non-Gaussian coefficient a1 is a continuous random variable sampled from the Laplace distribution, q(a1) = 1√ 2 exp − q 2a2 1 . (S18) 6 It is again easy to check that the...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [13]

Co-emergence of multi-scale cortical activities of irregular firing, oscil- lations and avalanches achieves cost-efficient information capacity,

D.-P. Yang, H.-J. Zhou, and C. Zhou, “Co-emergence of multi-scale cortical activities of irregular firing, oscil- lations and avalanches achieves cost-efficient information capacity,” PLoS Comput. Biol. 13, e1005384 (2017)

work page 2017

[2] [1]

Given an input signal ⃗s, the conditional distribution of the output signal ⃗x is then pout(⃗x|⃗s) = 1 (2πσ 2 0)N/2 exp h − ⃗x − (I + W )−1⃗s 2 2σ2 0 i . (B2) The mutual information between output ⃗x and input ⃗s is given by I ⃗x; ⃗s ≡ H pout(⃗x) − H ⃗x|⃗s , (B3) 13 where H ⃗x|⃗s is the conditional entropy of the output ⃗x given the input ⃗s: H ⃗x|⃗s ≡ − ...

work page

[3] [2]

When σ2 0 is small, we may assume H pout(⃗x) to be a smooth function of σ2

work page

[4] [3]

As a zeroth-order ap- proximation, we approximate the value of H pout(⃗x) by its limiting value at σ2 0 = 0, which is Eq. (S11). The W -dependent part of the mutual information I ⃗x; ⃗s is therefore approximated by I ⃗x; ⃗s ≈ −ln h det(I + W ) i = S . (B5) When the output noise ⃗ϵ of Eq. (B1) is not Gaussian, Eq. (B4) will no longer hold exactly, and the ...

work page

[5] [4]

Pre- dictive coding: a fresh view of inhibition in the retina,

M. V. Srinivasan, S. B. Laughlin, and A. Dubs, “Pre- dictive coding: a fresh view of inhibition in the retina,” Proc. R. Soc. Lond. B 216, 427–459 (1982)

work page 1982

[6] [5]

Predictive coding in the visual cortex: a functional interpretation of some extra- classical receptive-field effects,

R. P. N. Rao and D. H. Ballard, “Predictive coding in the visual cortex: a functional interpretation of some extra- classical receptive-field effects,” Nature Neurosci. 2, 79– 87 (1999)

work page 1999

[7] [6]

Predictive coding,

Y. Huang and R. P. N. Rao, “Predictive coding,” WIREs Cogn. Sci. 2, 580–593 (2011)

work page 2011

[8] [7]

Predictive coding is a consequence of energy efficiency in recurrent neural networks,

A. Ali, N. Ahmad, E. de Groot, M. A. J. van Gerven, and T. C. Kietzmann, “Predictive coding is a consequence of energy efficiency in recurrent neural networks,” Patterns 3, 100639 (2022)

work page 2022