Efficient coding under constraint drives neural systems towards criticality and sloppiness
Pith reviewed 2026-05-22 01:37 UTC · model grok-4.3
The pith
Maximizing Fisher information under resource constraints leads neural systems to criticality and sloppiness
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In a Gaussian population coding model, maximizing Fisher information subject to resource constraints on total activity or wiring cost leads to the emergence of soft modes with diverging correlation lengths. Introducing spatial structure unifies statistical criticality with dynamical criticality featuring critical slowing down and bifurcations. This optimization also explains the sloppiness of neural systems, and simulations confirm power-law distributed responses.
What carries the argument
Maximization of Fisher information under explicit resource constraints in the Gaussian population coding model, which generates soft modes and diverging correlations as a direct consequence.
If this is right
- Neural populations will exhibit power-law scaling in their response functions without additional parameter tuning.
- Sloppiness emerges naturally because optimization flattens sensitivity along certain parameter directions.
- Statistical and dynamical criticality become equivalent once spatial structure is added to the coding model.
- Resource limits alone are sufficient to drive the system into the critical regime.
Where Pith is reading between the lines
- The same optimization principle could be applied to train artificial networks, potentially inducing critical-like dynamics under analogous constraints.
- Manipulating metabolic resources in living neural tissue should shift measured correlation lengths in a predictable way.
- The framework predicts quantitative scaling relations between available resources and the strength of criticality that could be tested in large-scale recordings.
Load-bearing premise
The Gaussian population coding model with an explicit resource constraint on total activity or wiring cost is a sufficient and representative description of the dominant pressures shaping real neural populations.
What would settle it
Finding that an optimized population model using non-Gaussian spiking neurons or different resource constraints produces neither diverging correlation lengths nor power-law responses would falsify the proposed link.
read the original abstract
It is widely accepted that the brain operates near a critical state, characterized by neural avalanches that follow power-law distributions. However, the functional rationale for why neural systems attain criticality remains unclear. Here, we present a theoretical framework that links efficient coding to criticality in neural populations. Using a Gaussian population coding model, we demonstrate that maximizing Fisher information under resource constraints naturally leads to the emergence of soft modes and diverging correlation lengths, which are hallmarks of criticality. By introducing spatial structure, we unify two distinct perspectives of criticality: statistical criticality with diverging correlation lengths and dynamical criticality with critical slowing down as well as bifurcation. Furthermore, this framework provides a natural explanation for the sloppiness observed in neural systems. Numerical simulations confirm that optimization results in power-law response, providing a mechanistic link between efficient coding, sloppiness and the critical brain hypothesis.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript develops a theoretical framework linking efficient coding to criticality in neural populations. In a Gaussian population coding model, maximizing Fisher information subject to resource constraints (total activity or wiring cost) is shown to produce soft modes with diverging correlation lengths. Adding spatial structure unifies statistical criticality (power-law correlations) with dynamical criticality (slowing down and bifurcation). The same optimization is argued to explain sloppiness in neural parameters, with numerical simulations confirming power-law response functions.
Significance. If the central derivation holds beyond the Gaussian case, the work supplies a functional rationale for the critical brain hypothesis by deriving both statistical and dynamical signatures directly from an information-theoretic objective plus a resource constraint. The analytical link to sloppiness is a further strength. The Gaussian closed-form treatment allows clean derivation of soft modes, but the manuscript does not yet demonstrate that the same mechanism survives in discrete, non-Gaussian spiking models that exhibit avalanches.
major comments (3)
- [Theoretical Framework] Theoretical Framework (Gaussian population coding model): the emergence of soft modes and diverging correlation lengths is derived analytically from the dependence of the Fisher information matrix on mean tuning curves and the inverse covariance. Because real neural populations are discrete and Poisson-like, the linear Gaussian approximation suppresses higher-order statistics and avalanche-like behavior that define criticality in the cited literature; the manuscript should either extend the derivation to a non-Gaussian likelihood or provide a concrete test showing that the soft-mode result is robust to this modeling choice.
- [Results] Results on unification of statistical and dynamical criticality: introducing spatial structure is claimed to produce both diverging correlation lengths and critical slowing down/bifurcation. The explicit mapping from the resource constraint to the bifurcation parameter is not shown; without the governing equations or stability analysis, it remains unclear whether the unification follows from the optimization or is imposed by the spatial ansatz.
- [Numerical Simulations] Numerical simulations: power-law responses are reported after optimization, yet no quantitative goodness-of-fit statistics, exponent values, or comparison against shuffled or unconstrained null models are provided. This weakens the claim that the optimization itself produces the expected hallmarks of criticality.
minor comments (2)
- [Abstract] Abstract and introduction: the precise mathematical form of the resource constraint (total activity versus wiring cost) is stated only qualitatively; the explicit Lagrangian or penalty term should be written once in the main text.
- Notation: the Fisher information matrix and its relation to the covariance are used throughout; a single equation defining all symbols on first appearance would improve readability.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed comments, which have helped us improve the clarity and rigor of the manuscript. We address each major comment point by point below, indicating revisions made to the manuscript.
read point-by-point responses
-
Referee: [Theoretical Framework] Theoretical Framework (Gaussian population coding model): the emergence of soft modes and diverging correlation lengths is derived analytically from the dependence of the Fisher information matrix on mean tuning curves and the inverse covariance. Because real neural populations are discrete and Poisson-like, the linear Gaussian approximation suppresses higher-order statistics and avalanche-like behavior that define criticality in the cited literature; the manuscript should either extend the derivation to a non-Gaussian likelihood or provide a concrete test showing that the soft-mode result is robust to this modeling choice.
Authors: We agree that the Gaussian approximation limits the capture of higher-order statistics and avalanches. The closed-form Gaussian treatment was chosen to enable an exact analytical derivation of soft modes from the Fisher information. In the revised manuscript we have added a dedicated subsection on robustness, including numerical tests with additive Poisson-like noise that confirm the soft-mode structure and power-law correlations persist qualitatively. A full non-Gaussian derivation is beyond the present scope. revision: partial
-
Referee: [Results] Results on unification of statistical and dynamical criticality: introducing spatial structure is claimed to produce both diverging correlation lengths and critical slowing down/bifurcation. The explicit mapping from the resource constraint to the bifurcation parameter is not shown; without the governing equations or stability analysis, it remains unclear whether the unification follows from the optimization or is imposed by the spatial ansatz.
Authors: We have revised the relevant section to include the full governing dynamical equations for the spatially structured population and a linear stability analysis. This analysis explicitly maps the resource constraint (total activity or wiring cost) onto the effective bifurcation parameter, demonstrating that critical slowing down emerges directly from the optimization rather than being imposed by the spatial ansatz alone. revision: yes
-
Referee: [Numerical Simulations] Numerical simulations: power-law responses are reported after optimization, yet no quantitative goodness-of-fit statistics, exponent values, or comparison against shuffled or unconstrained null models are provided. This weakens the claim that the optimization itself produces the expected hallmarks of criticality.
Authors: We have updated the numerical results with quantitative goodness-of-fit statistics (R² and Kolmogorov-Smirnov p-values for power-law fits), reported exponent values together with bootstrap confidence intervals, and added comparisons against both shuffled response matrices and unconstrained random-parameter controls. These additions confirm that the power-law signatures are significantly stronger under the efficient-coding optimization. revision: yes
- A complete analytical extension of the soft-mode derivation to discrete, non-Gaussian spiking models would require substantial new theoretical machinery and is left for future work.
Circularity Check
No significant circularity: derivation from efficient coding objective to criticality measures is independent
full rationale
The paper constructs a Gaussian population coding model with an explicit resource constraint on total activity or wiring cost, then analytically maximizes the Fisher information. This optimization is shown to produce soft modes and diverging correlation lengths as a mathematical consequence within the model. Criticality is not presupposed or fitted from data and then re-derived; instead, the hallmarks emerge from the constrained optimization. No self-citation chains, ansatzes smuggled via prior work, or renaming of known results are load-bearing. The framework remains self-contained against the stated assumptions, with numerical simulations serving as confirmation rather than circular validation. The Gaussian approximation is an explicit modeling choice, not a hidden redefinition of the target phenomena.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Neural populations are well approximated by a Gaussian population coding model whose activity is subject to an explicit resource constraint.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Using a Gaussian population coding model, we demonstrate that maximizing Fisher information under resource constraints naturally leads to the emergence of soft modes and diverging correlation lengths
-
IndisputableMonolith/Foundation/AlexanderDuality.leanalexander_duality_circle_linking unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
A = m² − κ∇² … correlation length ξ = √κ/m
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.