Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime

Diyuan Wu; Marco Mondelli

REVIEW 2 major objections 4 minor 1 cited by

Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime

T0 review · 2 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that NC1, the vanishing of within-class feature variability, is a consequence of the loss landscape of a mean-field three-layer network trained with mean squared error, not a special property of the unconstrained…

desk verdict Genuine infinite-width theory for NC1 beyond UFM, but the finite-width gap and a liminf/noisy-gradient framing mean the abstract oversells the finite-network story. read the letter →

arxiv 2501.19104 v2 pith:ELFOI4R4 submitted 2025-01-31 cs.LG

Diyuan Wu , Marco Mondelli This is my paper · ORCID

classification cs.LG

keywords neuralcollapseNC1within-classvariabilitymean-fieldregimelosslandscapegradientflowmeansquarederrorthree-layernetwork

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural collapse's first property, NC1, says that after training, the feature vectors of samples from the same class become nearly identical. Earlier theory derived NC1 from an unconstrained features model that treats the features as free optimization variables and ignores the data. This paper claims NC1 is instead a landscape property of a concrete data-dependent network: a three-layer network with a mean-field feature layer followed by a linear layer, trained by gradient flow on the mean squared error. The paper proves that points that are nearly stationary and have small empirical loss approximately satisfy NC1, that gradient flow reaches such points, and that on well-separated data this happens while test error also vanishes. If correct, the result explains why NC1 is so prevalent in practical training.

What carries the argument

The load-bearing object is the free-energy stationarity identity of Theorem 4.2. At an $\epsilon_S$-stationary point, the first variation of the free energy with respect to the feature distribution nearly vanishes; substituting that stationarity condition into the definition of the feature matrix $H_\rho$ reduces the feature-learning problem to a linear equation whose solution is dominated by $\gamma^{-1}W(W^TW)^{-1}Y$. Since this dominant term already has zero within-class variance, the only obstacle to NC1 is the controlled error term. The proof that gradient flow actually reaches such points uses positive definiteness of the mean-field kernel $K_\rho(X,X)$ at initialization and a hitting-time argument showing that the dynamics stays near its starting point long enough for the empirical loss to become small.

What would settle it

Train a finite three-layer network as in the paper to a near-stationary point with very small MSE and gradient norm, and measure the penultimate-layer features. If the residual $\|H_\rho - \gamma^{-1}W(W^TW)^{-1}Y\|_F$ stays large while the loss and gradient norm are small, or if the NC1 metric does not follow the predicted scaling with the inverse temperature $\beta$, the landscape characterization fails. A cleaner version is to compute the bound in Corollary 4.4 at a numerically found near-stationary point and compare it with the measured NC1 value.

Watch

Extended reading notes

Core claim

The central result is a decomposition of the feature matrix at any approximately stationary point of the regularized free energy. Theorem 4.2 shows that at an $\epsilon_S$-stationary point with respect to the feature distribution $\rho$, the feature matrix obeys $H_\rho = \gamma^{-1}W(W^TW)^{-1}Y + E_2$, where the error $E_2$ is bounded in Frobenius norm by the stationarity tolerance $\epsilon_S$, the empirical loss $L_n(\rho,W)$, and the conditioning of $W$. Because $Y$ is the one-hot label matrix, the leading term $\gamma^{-1}W(W^TW)^{-1}Y$ assigns the same feature vector to every sample in a class, so its within-class variance is exactly zero. Corollary 4.4 turns the remaining error into an explicit upper bound on the NC1 metric, and Corollary 4.9 shows that gradient flow on the regularized MSE loss reaches the required small-loss, near-stationary regime for large enough inverse temperature and scale. The paper therefore establishes NC1 as a consequence of stationarity and small loss rather than as a special property of the unconstrained features model.

Load-bearing premise

The load-bearing premise is that the infinite-width mean-field feature map $h_\rho(x)=\mathbb{E}_\rho[a\sigma(u^\top x)]$ is a faithful stand-in for a finite-width network; the paper analyzes $h_\rho$ directly and only asserts, without a quantitative bound, that the finite-width version is close.

Editorial extensions

If this is right

Any optimizer that reaches a small-loss, near-stationary point will exhibit approximate NC1, so the collapse phenomenon is not tied to a particular training algorithm.
Gradient flow on the regularized MSE loss provably satisfies these conditions, so NC1 appears during the terminal phase of training rather than being imposed by the model or regularization.
For well-separated data, the same training run achieves both NC1 and test error that decays with the number of training samples, giving a concrete setting where collapse and generalization co-occur.
NC1 does not require the linear layers to be balanced, because the entropic regularization used here forces stationary points away from balancedness while the NC1 guarantee still holds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The finite-width transfer is asserted but not quantified; an explicit $O(1/\sqrt{N})$ or similar bound would turn the infinite-width guarantee into a practical finite-network statement, and the paper does not provide one.
The decomposition predicts a direct scaling law: the NC1 metric should be bounded by a term of order $\beta^{-1+2\epsilon_0}$ plus the residual loss and squared gradient norm, so measuring these quantities separately during training would test which one actually drives collapse.
The landscape identity suggests the same argument could extend to other losses with an affine gradient structure, but such an extension is not part of the paper.
Because the generalization result is a sufficient condition, the paper leaves open the possibility that NC1 occurs on non-separable data without good test error, which would match earlier warnings that collapse alone is not a generalization guarantee.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. The infinite-width core is genuinely good: Theorem 4.2 characterizes the learned features at approximate stationary points of the free energy, and the error bound depends on the gradient norm and the empirical loss. This gives a data-dependent route to NC1 and avoids the balancedness condition that Jacot et al. need. The dynamics analysis in Section 4.2 is also real work: for large β, regularized gradient flow reaches low empirical loss on a finite horizon, and combining that with the landscape result yields NC1 in the liminf sense. The appendix proofs are detailed and, as far as I can tell, the argument is coherent.

The soft spots are real but not fatal. The biggest one is the finite-width gap. All theorems concern the infinite-width distribution in Eq. (1). Section 3 says the difference with finite N 'can be readily bounded' using Mei et al., but no bound with rates appears anywhere. The time horizon t0 in Theorem 4.8 grows like β C5, and β is taken polynomially large in 1/δ0 in Corollary 4.9, so a valid finite-N transfer requires a quantitative statement over that horizon. Without it, the abstract's talk of a 'three-layer neural network' reaching NC1 is only established for the distributional law, not for the finite-width network in Eq. (2). The MNIST/CIFAR experiments use finite N but aren't tied to the theorem's regime, so they don't close the gap.

The second issue is framing. The dynamics is noisy gradient flow on the regularized free energy, not plain gradient flow on MSE, and Corollary 4.9 gives a liminf guarantee, not convergence. The abstract and introduction blur both points. These are mild, but they should be fixed.

Overall, if you are working on neural collapse theory or mean-field analysis of feature learning, this paper is worth your time. The infinite-width results are a solid contribution, and the missing finite-width transfer is a clear next step, not a sign of a broken argument. It deserves a serious referee. I would send it to review, with a request that the authors either provide the finite-N bound or rewrite the abstract to say plainly that the theory is infinite-width.

Referee Report

2 major / 4 minor

Summary. This paper studies the first neural-collapse property (NC1) in a three-layer architecture whose feature map is a mean-field two-layer network h_ρ(x)=E_ρ[aσ(u^T x)] followed by a linear map γW^T. The main results are: Theorem 4.2, which characterizes the feature matrix H_ρ at any ε_S-stationary point of the free energy with respect to ρ as γ^{-1}W(W^TW)^{-1}Y plus an error controlled by ε_S and the empirical loss; Corollary 4.4, which turns this into a bound on the NC1 metric; Theorem 4.8 and Corollary 4.9, which show that the noisy Wasserstein gradient flow of the regularized free energy reaches such points in the sense lim inf NC1(H_ρt) ≤ δ0; and Theorem 5.3, which gives a simultaneous NC1 and vanishing test-error guarantee for well-separated data under a two-stage procedure. The theorems are proved for the infinite-width feature map in Eq. (1), not for the finite-width network in Eq. (2).

Significance. If the results hold as stated, they are a genuine step beyond the unconstrained features model: NC1 is derived from the loss landscape of a data-dependent feature-learning model, with explicit quantitative error bounds and a generalization statement. The stationary-point decomposition in Theorem 4.2 is clean, the appendices are detailed, and the observation that entropic regularization makes stationary points imbalanced (Lemma 4.5) is a useful point of contrast with balancedness-based analyses. The main caveat is the unquantified finite-width transfer; once that is supplied, the paper would be a valuable contribution to the neural-collapse literature.

major comments (2)

[Section 3, Eqs. (1)-(2)] The paper's headline claim concerns a three-layer neural network, but every theorem is proved for the infinite-width feature map h_ρ(x)=E_ρ[aσ(u^T x)] in Eq. (1). The statement that its difference with the finite-width network h_N in Eq. (2) 'can be readily bounded using results from Mei et al. (2019, 2018)' is not supported by any quantitative bound in the manuscript. Because Corollary 4.9 runs the dynamics for t ≥ β C5 with β that grows polynomially in 1/δ0, a transfer would require an explicit statement of the form: for N ≥ N(β, δ0), sup_{t ≤ β C5} |L_{λ,n}(ρ_t^N, W_t^N) − L_{λ,n}(ρ_t, W_t)| ≤ …, and similarly for the NC1 metric. Without such a bound, the abstract's conclusion about 'a three-layer neural network' is established only for the distributional law ρ_t, not for the finite-width network (2). The finite-N experiments in Figure 1 do not close this gap because their parameters are not chosen as in Corollary 4.9.
[Abstract and Section 4.2, Eq. (20)] The abstract says that 'gradient flow on the mean squared error converges to NC1 solutions', but the dynamics actually studied is the noisy Wasserstein gradient flow on the regularized objective L_{λ,n}, with an explicit Brownian noise term sqrt(2β^{-1})dB_t and weight-decay terms. Moreover, Corollary 4.9 only proves lim inf_{t→∞} NC1(H_{ρ_t}) ≤ δ0, i.e., the inequality holds outside a finite Lebesgue measure set for large times; it does not imply convergence of NC1 to δ0. The abstract and introduction should be restated to reflect the regularized noisy dynamics and the liminf nature of the result.

minor comments (4)

[Section 3] The sentence about the difference between the infinite-width and finite-width networks being 'readily bounded' should be replaced by a precise statement or removed; as written it is an unsupported promise that is separate from the missing transfer theorem.
[Section 4.2, Figure 1] The numerical setup should be tied to the theoretical parameter regime: in setting (a) no noise is added (β^{-1}=0) and the last linear layer is fixed, while the theory requires β^{-1}>0 and joint training of W; in setting (b) the phrase 'λW = λρ = β = 10^{-4}' is ambiguous and does not obviously match the theoretical scaling λρ=λW=β^{-1}.
[Lemma 4.7] The notation σ(2k) is not defined; if it denotes the 2k-th derivative of σ, that should be stated explicitly. Also, the citation to 'v1 of the paper' should give the specific lemma number rather than relying on the reader to locate the version.
[Theorem C.1] The constants C3, C4, C5 are defined through a long list of auxiliary constants; it would help readers to state their polynomial dependence on n, d, p, and λ_*^{-1}, so that the scaling of the β lower bound in Corollary 4.9 with the problem dimension is transparent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: NC1 is derived from stationarity and small loss; the only self-citation is independent, and the finite-width gap is a completeness issue, not a circular one.

full rationale

This paper's central claim is derived, not assumed: Theorem 4.2 starts from the definition of an epsilon_S-stationary point, solves the linear equation (12) for H_rho, and obtains H_rho = gamma^{-1} W (W^T W)^{-1} Y + E_2 with an explicit bound on E_2 in terms of epsilon_S and the empirical loss L_n(rho, W). Corollary 4.4 then computes NC1(H_rho) directly from this decomposition, and the leading term has zero within-class variance because Y is one-hot; the target NC1 bound is not used as an input anywhere. No free parameters are fitted to NC1 data, and the numerical experiments in Figure 1 are illustrative rather than used to set constants in Theorems 4.8 or 4.9. The only self-citation with a load-bearing role is Lemma 4.7, which cites Nguyen and Mondelli (2020, Lemma 3.4) for lambda_min(K(X,X)) > 0; this is a parameter-free technical lemma on positive definiteness of a kernel whose stated assumptions (Assumption 2) do not involve NC1 or any conclusion of this paper, so it is independent support and does not raise the circularity score. I do flag one non-circular gap: Section 3 states that the difference between the finite-width network (2) and the infinite-width law (1) 'can be readily bounded using results from Mei et al. (2019, 2018)', but no explicit N-dependent bound or rate is proved, so the abstract's finite-width phrasing is not fully established; this is a completeness/correctness issue, not a circular reduction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard mean-field assumptions: smooth bounded activations, bounded data, Gaussian initialization, and positive-definite initial kernel. The regularization schedule λ_ρ = λ_W = β^{-1} and large γ are chosen by the theorems, not fitted to data. No new physical or mathematical entities are introduced. The most consequential structural assumption is the infinite-width idealization of the first layer, which is not quantitatively transferred to finite-width networks.

free parameters (3)

γ (last-layer scaling)
Model hyperparameter in Eq. (3); Theorem 4.8 requires γ > C_3 for a constant C_3 that depends only on n,d,p,C_1,λ_*. Not fitted to data, but the result only holds for sufficiently large amplification.
λ_ρ = λ_W (regularization coefficients) = 1/β
Set to β^{-1} in Theorem 4.8 so that regularization vanishes as the noise temperature β^{-1} goes to 0. This specific scaling is chosen for the proof; the authors note that Θ(β^{-1}) would suffice.
β (inverse temperature)
Controls entropic regularization strength in Eq. (5) and the noise level in the gradient flow (20). Theorems 4.8 and 4.9, and Theorem 5.3, state results for β sufficiently large; β is not estimated from data.

assumptions (4)

domain assumption Assumption 1: Gaussian initialization, bounded data, and activation with bounded derivatives up to third order.
Used throughout; sets the smoothness and boundedness required for the mean-field computations in Sections 4 and 5.
domain assumption Assumption 2: activation has non-zero even derivatives and the data have a non-degenerate, non-parallel coordinate.
Needed for Lemma 4.7 to ensure the initial kernel K_ρ0 is positive definite, which is the basis for the exponential loss decay in Lemma C.2.
domain assumption Assumption 3: sigmoid activation and (τ,M)-linearly separable data distribution.
Used in Section 5 to construct a low-loss interpolant (Lemma D.4) and to control the test error via Rademacher complexity.
standard math Talagrand inequality and Wasserstein-KL bounds for the Gaussian reference measure.
Used in Lemmas C.4 and C.5 to translate KL bounds into W_2 control of the feature distribution; standard in mean-field analyses (Chen et al., 2020).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime." pith.science (2026). https://pith.science/paper/ELFOI4R4

@misc{pith2026250119104,
  author       = {Pith},
  title        = {Pith review of: Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ELFOI4R4}},
  note         = {Machine review of arXiv:2501.19104}
}

read the original abstract

Neural Collapse is a phenomenon where the last-layer representations of a well-trained neural network converge to a highly structured geometry. In this paper, we focus on its first (and most basic) property, known as NC1: the within-class variability vanishes. While prior theoretical studies establish the occurrence of NC1 via the data-agnostic unconstrained features model, our work adopts a data-specific perspective, analyzing NC1 in a three-layer neural network, with the first two layers operating in the mean-field regime and followed by a linear layer. In particular, we establish a fundamental connection between NC1 and the loss landscape: we prove that points with small empirical loss and gradient norm (thus, close to being stationary) approximately satisfy NC1, and the closeness to NC1 is controlled by the residual loss and gradient norm. We then show that (i) gradient flow on the mean squared error converges to NC1 solutions with small empirical loss, and (ii) for well-separated data distributions, both NC1 and vanishing test loss are achieved simultaneously. This aligns with the empirical observation that NC1 emerges during training while models attain near-zero test error. Overall, our results demonstrate that NC1 arises from gradient training due to the properties of the loss landscape, and they show the co-occurrence of NC1 and small test error for certain data distributions.

Figures

Figures reproduced from arXiv: 2501.19104 by the authors.

**Figure 2.** plots the normalized balancedness metric NB(ρ, W) defined in (25) during training. The results clearly show that the network does not become balanced at convergence. (a) MNIST (b) CIFAR-100 [PITH_FULL_IMAGE:figures/full_fig_p035_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers
cs.LG 2025-05 conditional novelty 7.0 of 10

Neural collapse is globally optimal in deep regularized ResNets and transformers, with the approximation improving as depth grows.

Reference graph

Works this paper leans on

38 extracted references · 29 canonical work pages · cited by 1 Pith paper

[1]

Andriopoulos, G., Dong, Z., Guo, L., Zhao, Z., and Ross, K. W. (2024). The prevalence of neural collapse in neural multivariate regression. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024
[2]

I., and Yukimura, D

Ara \'u jo, D., Oliveira, R. I., and Yukimura, D. (2019). A mean-field limit for certain deep neural networks. arXiv preprint arXiv:1906.00193

arXiv 2019
[3]

Beaglehole, D., S \'u ken \' k, P., Mondelli, M., and Belkin, M. (2024). Average gradient outer product as a mechanism for deep neural collapse. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024
[4]

Chen, Z., Cao, Y., Gu, Q., and Zhang, T. (2020). A generalized neural tangent kernel analysis for two-layer neural networks. Advances in Neural Information Processing Systems , 33:13363--13373

work page 2020
[5]

Fang, C., He, H., Long, Q., and Su, W. J. (2021a). Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training. Proceedings of the National Academy of Sciences , 118(43):e2103091118

work page 2021
[6]

Fang, C., Lee, J., Yang, P., and Zhang, T. (2021b). Modeling from features: a mean-field framework for over-parameterized deep neural networks. In Conference on learning theory , pages 1887--1936. PMLR

work page 2021
[7]

Han, X., Papyan, V., and Donoho, D. L. (2022). Neural collapse under MSE loss: Proximity to and dynamics on the central path. In International Conference on Learning Representations

work page 2022
[8]

and Ling, S

Hong, W. and Ling, S. (2024a). Beyond unconstrained features: Neural collapse for shallow neural networks with general data. arXiv preprint arXiv:2409.01832

arXiv 2024

Show all 38 references

[9]

and Ling, S

Hong, W. and Ling, S. (2024b). Neural collapse for unconstrained feature model under cross-entropy loss with imbalanced data. Journal of Machine Learning Research , 25(192):1--48

2024
[10]

Hu, K., Ren, Z., S i s ka, D., and Szpruch, . (2021). Mean-field langevin dynamics and energy landscape of neural networks. In Annales de l'Institut Henri Poincare (B) Probabilites et statistiques , volume 57, pages 2043--2065. Institut Henri Poincar \'e

2021
[11]

Hui, L., Belkin, M., and Nakkiran, P. (2022). Limitations of neural collapse for understanding generalization in deep learning. arXiv preprint arXiv:2202.08384

2022 arXiv
[12]

Jacot, A., S \'u ken \' k, P., Wang, Z., and Mondelli, M. (2024). Wide neural networks trained with weight decay provably exhibit neural collapse. arXiv preprint arXiv:2410.04887

2024 arXiv
[13]

Javanmard, A., Mondelli, M., and Montanari, A. (2020). Analysis of a two-layer neural network via displacement convexity. The Annals of Statistics , 48(6):3619--3642

2020
[14]

and Suzuki, T

Kim, J. and Suzuki, T. (2024). Transformers learn nonlinear features in context: Nonconvex mean-field dynamics on the attention landscape. In Forty-first International Conference on Machine Learning

2024
[15]

Kothapalli, V. (2023). Neural collapse: A review on modelling principles and generalization. Transactions on Machine Learning Research

2023
[16]

and Tirer, T

Kothapalli, V. and Tirer, T. (2024). Kernel vs. kernel: Exploring how the data structure affects neural collapse. arXiv preprint arXiv:2406.02105

2024 arXiv
[17]

Lu, Y., Ma, C., Lu, Y., Lu, J., and Ying, L. (2020). A mean field analysis of deep resnet and beyond: Towards provably optimization via overparameterization from depth. In International Conference on Machine Learning , pages 6426--6436

2020
[18]

Ma, C., Wu, L., et al. (2022). The barron space and the flow-induced function spaces for neural network models. Constructive Approximation , 55(1):369--406

2022
[19]

Mei, S., Misiakiewicz, T., and Montanari, A. (2019). Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit. In Conference on learning theory , pages 2388--2464. PMLR

2019
[20]

Mei, S., Montanari, A., and Nguyen, P.-M. (2018). A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences , 115(33):E7665--E7671

2018
[21]

G., Parshall, H., and Pi, J

Mixon, D. G., Parshall, H., and Pi, J. (2022). Neural collapse with unconstrained features. Sampling Theory, Signal Processing, and Data Analysis , 20(2):11

2022
[22]

and Pham, H

Nguyen, P.-M. and Pham, H. T. (2023). A rigorous framework for the mean field limit of multilayer neural networks. Mathematical Statistics and Learning , 6(3):201--357

2023
[23]

Nguyen, Q. N. and Mondelli, M. (2020). Global convergence of deep networks with one wide layer followed by pyramidal topology. Advances in Neural Information Processing Systems , 33:11961--11972

2020
[24]

and Cao, X

Pan, L. and Cao, X. (2023). Towards understanding neural collapse: The effects of batch normalization and weight decay. arXiv preprint arXiv:2309.04644

2023 arXiv
[25]

Papyan, V., Han, X., and Donoho, D. L. (2020). Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences , 117(40):24652--24663

2020
[26]

Pham, H. T. and Nguyen, P.-M. (2021). Global convergence of three-layer neural networks in the mean field regime. In International Conference on Learning Representations

2021
[27]

Seleznova, M., Weitzner, D., Giryes, R., Kutyniok, G., and Chou, H.-H. (2024). Neural (tangent kernel) collapse. Advances in Neural Information Processing Systems , 36

2024
[28]

Shevchenko, A., Kungurtsev, V., and Mondelli, M. (2022). Mean-field analysis of piecewise linear solutions for wide relu networks. Journal of Machine Learning Research , 23(130):1--55

2022
[29]

and Mondelli, M

Shevchenko, A. and Mondelli, M. (2020). Landscape connectivity and dropout stability of sgd solutions for over-parameterized neural networks. In International Conference on Machine Learning , pages 8773--8784. PMLR

2020
[30]

H., and Mondelli, M

S \'u ken \' k, P., Lampert, C. H., and Mondelli, M. (2024). Neural collapse vs. low-rank bias: Is deep neural collapse really optimal? In The Thirty-eighth Annual Conference on Neural Information Processing Systems

2024
[31]

S \'u ken \' k, P., Mondelli, M., and Lampert, C. H. (2023). Deep neural collapse is provably optimal for the deep unconstrained features model. Advances in Neural Information Processing Systems , 36

2023
[32]

Suzuki, T., Wu, D., and Nitanda, A. (2024a). Mean-field langevin dynamics: Time-space discretization, stochastic gradient, and variance reduction. Advances in Neural Information Processing Systems , 36

2024
[33]

Suzuki, T., Wu, D., Oko, K., and Nitanda, A. (2024b). Feature learning via mean-field langevin dynamics: classifying sparse parities and beyond. Advances in Neural Information Processing Systems , 36

2024
[34]

and Suzuki, T

Takakura, S. and Suzuki, T. (2024). Mean-field analysis on two-layer neural networks from a kernel perspective. In Forty-first International Conference on Machine Learning

2024
[35]

R., Vakilian, V., and Behnia, T

Thrampoulidis, C., Kini, G. R., Vakilian, V., and Behnia, T. (2022). Imbalance trouble: Revisiting neural-collapse geometry. In Advances in Neural Information Processing Systems

2022
[36]

and Bruna, J

Tirer, T. and Bruna, J. (2022). Extended unconstrained features model for exploring deep neural collapse. In International Conference on Machine Learning , pages 21478--21505. PMLR

2022
[37]

Wainwright, M. J. (2019). High-Dimensional Statistics: A Non-Asymptotic Viewpoint . Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press

2019
[38]

Zhou, J., Li, X., Ding, T., You, C., Qu, Q., and Zhu, Z. (2022). On the optimization landscape of neural collapse under mse loss: Global optimality with unconstrained features. In International Conference on Machine Learning , pages 27179--27202. PMLR

2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Andriopoulos, G., Dong, Z., Guo, L., Zhao, Z., and Ross, K. W. (2024). The prevalence of neural collapse in neural multivariate regression. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024

[2] [2]

I., and Yukimura, D

Ara \'u jo, D., Oliveira, R. I., and Yukimura, D. (2019). A mean-field limit for certain deep neural networks. arXiv preprint arXiv:1906.00193

arXiv 2019

[3] [3]

Beaglehole, D., S \'u ken \' k, P., Mondelli, M., and Belkin, M. (2024). Average gradient outer product as a mechanism for deep neural collapse. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024

[4] [4]

Chen, Z., Cao, Y., Gu, Q., and Zhang, T. (2020). A generalized neural tangent kernel analysis for two-layer neural networks. Advances in Neural Information Processing Systems , 33:13363--13373

work page 2020

[5] [5]

Fang, C., He, H., Long, Q., and Su, W. J. (2021a). Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training. Proceedings of the National Academy of Sciences , 118(43):e2103091118

work page 2021

[6] [6]

Fang, C., Lee, J., Yang, P., and Zhang, T. (2021b). Modeling from features: a mean-field framework for over-parameterized deep neural networks. In Conference on learning theory , pages 1887--1936. PMLR

work page 2021

[7] [7]

Han, X., Papyan, V., and Donoho, D. L. (2022). Neural collapse under MSE loss: Proximity to and dynamics on the central path. In International Conference on Learning Representations

work page 2022

[8] [8]

and Ling, S

Hong, W. and Ling, S. (2024a). Beyond unconstrained features: Neural collapse for shallow neural networks with general data. arXiv preprint arXiv:2409.01832

arXiv 2024