{"id":"5463c177-222f-480b-a9ca-44272cf203ba","arxiv_id":"2411.17278","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"For the L-layer unconstrained feature model with bias and squared-error loss, the global minimizer's geometric structure is governed by the singular values of the matrix (I - n1^T/N)diag(√n).","lead":"This paper characterizes the geometry of optimal solutions for a simplified neural network with imbalanced classes and a bias term, showing which class directions collapse or stay orthogonal. It extends neural collapse theory to a setting closer to real class-imbalanced training, though the main theorem for deep linear layers rests on an unproved step.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.3's global-minimizer characterization rests on an unproved SVD-alignment inequality in Section B.3; the 'Otherwise... conflicts' argument does not control the products WL...W1 and \\bar HD, and repeated singular values of \\hat Y make the claimed uniqueness of \\hat U, \\hat V ill-posed.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the proof of Theorem 4.3 assumes that at a global minimizer the left singular vectors of W_L equal \\hat U and the right singular vectors of \\bar H D equal \\hat V, supported only by an unproved 'Otherwise... conflicts' inequality. My independent reading of Section B.3 confirms that the argument does not justify this inequality: the objective depends on the products W_L...W_1 and (W_L...W_1) \\bar H D, not on the individual singular vectors separately, so an SVD replacement cannot be assumed to decrease the loss. The regularization terms are invariant under orthogonal changes, which removes the only other mechanism that could yield a strict decrease. The non-uniqueness of SVD under repeated singular values makes the 'any global minimizer' formulation additionally fragile. The single-layer results (Theorems 4.1 and 4.2) and the singular-value interlacing estimates (Theorem 4.4) appear more solid and are given some independent support by the structure of the argument, so the concern is narrowly about the L-layer theorem. Since that theorem is the central novel claim of the paper, the appropriate disposition is the reader's REJECT; my critique reinforces it without changing the verdict.","tokens_in":26,"tokens_out":5257,"duration_ms":217961,"concrete_test":"Re-derive the inequality in Section B.3 for a minimal instance: K=2, n=(2,1), L=2, d0=d1=d2=2, with small symmetric regularization. For fixed W1 and a candidate global minimizer (W2', \\bar E') with SVD factors U' \\neq \\hat U or V' \\neq \\hat V, compute the objective difference f(\\hat U \\Sigma_{W2'} U_{W1}^T, W1, \\bar E'') - f(W2', W1, \\bar E'), where \\bar E'' = V_{W1} \\Sigma_{\\bar E'} \\hat V^T. If this difference is not strictly positive for some U', V', the 'conflicts' claim fails. Separately, with n=(2,2,1), check whether the alignment statement in Theorem 4.3 is invariant under the choice of SVD of \\hat Y; if different SVDs yield different row/column alignments, the theorem is not well-defined for repeated singular values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is in the proof of Theorem 4.3 (Section B.3). After writing SVDs for W1,...,WL and \\bar E = \\bar H D, the proof asserts U_{W_L} = \\hat U and V_{\\bar E} = \\hat V for any global minimizer, with the only justification being the sentence 'Otherwise ... \\hat f(W_L',...,\\bar E') > \\hat f(W_L'',...,\\bar E''), which conflicts with the definition of a global minimizer.' This is not an established inequality: W_L enters the objective through the product W_L W_{L-1} ... W_1 \\bar E, and \\bar E enters through the same product, so replacing only their outer singular vectors with \\hat U and \\hat V changes the product in a way that is not shown to decrease the squared-error term; the Frobenius regularization terms are invariant under this replacement, so no decrease follows. The claimed strict inequality therefore carries the entire 'any global minimizer' characterization. The issue is compounded when \\hat Y has repeated singular values: then \\hat U and \\hat V are not unique, and statements such as 'U_{W_L} = \\hat U' or 'the i-th row aligns with the i-th column' require a gauge-fixing argument that the paper does not provide. Since Theorem 4.3 is the paper's central contribution, this proof gap is the decisive weakness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies the global minimizers of an L-layer unconstrained feature model with squared loss, weight decay, and a bias term under imbalanced class sizes. The main claim (Theorem 4.3) is that every global minimizer has a geometry determined by the singular value decomposition of \\hat Y = (I_K - n 1_K^⊟/N)D: features collapse to class means, the left-transformed classifier product has zero rows for small singular values and orthogonal surviving rows, the right-transformed class-mean matrix has a matching column structure, and surviving rows and columns align. The paper also provides an explicit estimation of the singular values of \\hat Y (Theorem 4.4) and numerical experiments on imbalanced CIFAR-10 and EMNIST subsets. Theorems 4.1 and 4.2 cover the single-layer bias-free and biased cases.","tokens_in":24439,"tokens_out":12010,"duration_ms":110626,"significance":"If the results were fully established, they would give a fairly detailed characterization of neural collapse in multilayer linear unconstrained feature models with bias under imbalanced data, extending prior work by Dang et al. and Tirer and Bruna. The single-layer derivations via the nuclear-norm relaxation are useful, and Theorem 4.4's interlacing result and explicit formulas for m=2,3 are concrete and falsifiable. The paper does not provide code, but the experimental setup is described in enough detail to be reproducible in principle. However, the central L-layer theorem is not proved as written, and the SVD-uniqueness problem affects the meaning of the geometric claims, so the paper's main contribution is currently unsupported.","major_comments":[{"comment":"The assertion 'Then we can claim that U_WL = \\hat U, V_{\\bar E} = \\hat V' is justified only by the sentence 'Otherwise ... \\hat f(W'_L,…,\\bar E') > \\hat f(W''_L,W'_{L-1},…,W'_1,\\bar E''), which conflicts with the definition of a global minimizer.' Replacing the left singular vectors of W_L and the right singular vectors of \\bar E by \\hat U and \\hat V does not preserve the product W_L⋯W_1\\bar E, and the Frobenius regularization terms are invariant under the replacement, so the claimed strict inequality does not follow. Since this is the only step that connects every global minimizer to the singular vectors of \\hat Y, Theorem 4.3 is not proved as written.","section":"Section B.3, proof of Theorem 4.3"},{"comment":"When \\hat Y has repeated singular values, \\hat U and \\hat V are not unique; this occurs in the balanced case and more generally under Definition 4.1 whenever any ℓ_j > 1. The proof fixes one SVD and asserts U_WL = \\hat U and V_{\\bar E} = \\hat V for every global minimizer. A common orthogonal rotation within a degenerate singular subspace preserves the objective but changes these singular vectors, so the equality cannot hold for all global minimizers. The statement needs either an explicit gauge fixing or a reformulation in terms of subspaces or invariant quantities, and the 'i-th row aligns with the i-th column' claim requires a definite choice of SVD.","section":"Theorem 4.3 and Section B.3"},{"comment":"The statement says κ_1,…,κ_K are the eigenvalues of \\tilde Y D and writes \\tilde Y D = \\tilde U κ \\tilde V^⊟. The matrix \\tilde Y D = D - n√n^⊟/N is generally not symmetric, so its eigenvalues need not be real or nonnegative; the subsequent soft-thresholding formula (κ - N√(λ_Wλ_H))_+ is valid for singular values, not for eigenvalues. If κ is intended to denote singular values, the wording should be corrected throughout; if eigenvalues are intended, the formula is incorrect.","section":"Theorem 4.2"},{"comment":"After deriving D^{-1}Z^* = VΣ(Σ^2+Nλ_W I)^{-1}Σ V^⊟, the proof concludes V=I because D^{-1}Z^* is diagonal. The correct conclusion is only that V diagonalizes a diagonal matrix, i.e., V is a signed permutation matrix, with arbitrary orthogonal blocks on repeated singular values. Since D and V need not commute, the subsequent formulas W^*(W^*)^⊟ = D^2Σ^2(Σ^2+Nλ_W I)^{-2} and (\\bar H^*)^⊟\\bar H^* = Σ^2D^{-2} presuppose V=I and do not follow for arbitrary global minimizers.","section":"Section B.1, proof of Theorem 4.1"}],"minor_comments":[{"comment":"The proof says there are zeros of f in the intervals (N_i,N_{i+1}) for i=1,…,m-1; since N_1>…>N_m, the intended intervals should be (N_{i+1},N_i).","section":"Section B.4, proof of Theorem 4.4"},{"comment":"The symbol r is defined twice with different arguments, and d_L does not appear as a dimension of any matrix in the theorem (W_L ∈ ℝ^{K×d_{L-1}}); the block sizes in Υ_1 and Υ_2 should be stated consistently with the stated dimensions.","section":"Theorem 4.3"},{"comment":"In the definition of NC_H for the bias-free case, the denominator is written as ‖\\bar H^⊟‖_F but should be ‖\\bar H^⊟\\bar H‖_F to match the numerator.","section":"Appendix C"},{"comment":"There are several spacing and typographical errors, including 'F raud detection', 'T ext classification', 'Defect detection' in Section 1, and 'defind' in Appendix C; these should be corrected.","section":"Introduction and Appendix C"},{"comment":"The numerical section reports only plots of the NC metrics; it does not provide a quantitative comparison of the converged values with the singular-value thresholds predicted by Theorem 4.4, so the experimental support is weaker than the text suggests.","section":"Section 5"}],"recommendation":"reject","confidential_remarks":"The decisive issue is the proof of Theorem 4.3: the unproved SVD-alignment inequality and the non-uniqueness of \\hat U and \\hat V under repeated singular values undermine the central claim as stated. The same type of problem appears in the proof of Theorem 4.1. A substantial rewrite with a corrected statement and a complete proof would be needed before the paper could be considered further."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is not ready to publish, but it is not a waste of anyone's time. The genuinely new content is the bias term in the L-layer unconstrained feature model under imbalanced data: the reduction b* = n/N, the centered label matrix \\hat Y = (I_K - (1/N)n 1_K^T)D, and the claim that the geometry of any global minimizer is governed by the singular values of \\hat Y. The single-layer Theorems 4.1 and 4.2 go through the standard nuclear-norm relaxation cleanly, and Theorem 4.4's interlacing argument for grouped class sizes is plausible and useful. That is real progress over the bias-free L-layer result of DTO+23 and the cross-entropy UFM of HL24.\n\nThe soft spot is exactly where the reader and stress-test put it: Theorem 4.3, the central L-layer characterization. The proof assumes that at a global minimizer the outer singular vectors align with \\hat U and \\hat V. The only justification is an 'Otherwise... conflicts' sentence in Section B.3. That inequality is not established. W_L enters through the product W_L...W_1 \\bar E, and \\bar E enters through the same product; changing their outer singular vectors changes the product, and the squared-error term could go up or down. The Frobenius regularization terms are invariant, so no strict decrease follows. On top of that, when \\hat Y has repeated singular values, \\hat U and \\hat V are not unique, so statements like 'the i-th row aligns with the i-th column' need a gauge-fixing argument that is not present. Since Theorem 4.3 carries all the L-layer claims, this is a load-bearing gap, not a minor typo.\n\nThere are smaller issues. The numerical section gives datasets and optimizer but not the regularization parameters, so the experiments are not reproducible as reported. The theorem statements also have some notation drift (r defined twice, W_M vs W_L) that a referee would need cleaned up.\n\nWho is this for? Researchers working on neural collapse theory and the UFM. They will find the single-layer results and the singular-value interlacing useful, and the L-layer claim interesting enough to want the missing proof filled in.\n\nMy recommendation: send it to peer review, but make clear the central theorem needs a complete proof before acceptance. This is a fixable, worthwhile draft, not a paper to desk-reject and not a paper to accept as-is.","headline":"The single-layer results and the interlacing theorem are credible, but the central L-layer theorem rests on an unproved SVD-alignment step; the paper deserves a careful referee, not acceptance as-is.","tokens_in":24944,"tokens_out":2924,"would_cite":false,"duration_ms":26652,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Under imbalanced data, the entire neural-collapse geometry of a deep linear model is set by the singular values of a single class-size matrix.","keywords":["neural collapse","imbalanced data","L-extended unconstrained feature model","mean squared error","singular value decomposition","minority collapse","global minimizer","class-mean geometry"],"falsifier":"Solve (4.3) numerically for a small imbalanced case such as $K=3$ classes with sizes $[10,10,1]$, two layers, and small fixed regularization, using many random initializations; if any computed global minimizer shows a nonzero row of $\\hat U^\\top W_L\\cdots W_1$ for a class whose $\\hat Y$ singular value is below the paper's threshold, or shows surviving rows that are not mutually orthogonal, Theorem 4.3 would be contradicted. A cheaper check is to audit the unproved inequality in Section B.3 for the specific case of repeated class sizes, where the SVD of $\\hat Y$ is non-unique.","tokens_in":23870,"feed_emoji":"📉","tokens_out":10507,"duration_ms":87808,"temperature":0.7,"pith_summary":"Neural collapse describes how well-trained classifiers organize their final-layer features, but the classical picture assumes balanced classes. This paper asks what happens when class sizes differ, and claims that in the L-layer unconstrained feature model with bias and squared-error loss, the entire global-minimizer geometry is set by the singular values of a single matrix built from class sizes, $\\hat Y = (I_K - \\frac{1}{N}\\mathbf{n}1_K^\\top)D$. If the claim is right, the fate of each class is simple: when its singular value clears a regularization-dependent threshold the classifier keeps an orthogonal, aligned direction for that class, and otherwise the class direction collapses to zero. That matters because imbalanced data are the norm in applications, and minority classes are exactly where this collapse would occur. The paper also computes the singular values of $\\hat Y$ explicitly for grouped imbalance patterns, turning the geometric statement into a quantitative sample-size criterion.","feed_headline":"One matrix decides neural-collapse geometry under imbalanced data","feed_subtitle":"Class means still collapse; which classifier directions survive depends on the singular values of a class-size matrix.","key_machinery":"The load-bearing object is the matrix $\\hat Y = (I_K - \\frac{1}{N} \\mathbf{n} 1_K^\\top)D$, where $K$ is the number of classes, $\\mathbf{n}$ lists per-class sample sizes, $N$ is the total sample count, and $D = \\mathrm{diag}(\\sqrt{\\mathbf{n}})$. Its singular value decomposition $\\hat Y = \\hat U \\hat\\kappa \\hat V^\\top$ gives the coordinates in which the geometry of the global minimizer becomes simple. The argument works by first showing that at any critical point the bias is the class-frequency vector $\\mathbf{n}/N$ and the global feature mean is zero; a Jensen-type inequality then forces features within each class to equal their class mean, and the L-layer objective reduces to a nuclear-norm-regularized least-squares problem in the variable $Z = W_L\\cdots W_1 \\bar H D$. A singular-value-thresholding calculation identifies the optimal $Z$, and a scalar minimization (the lemma on minimizing $1/x^{L+1} + \\alpha x$) decides, singular value by singular value, which coordinates of $\\hat U^\\top W_L\\cdots W_1$ and $\\bar H D \\hat V$ survive as nonzero orthogonal directions.","core_discovery":"At the level of the model, the paper's central result is Theorem 4.3. For any global minimizer of the L-extended unconstrained feature model with bias and MSE loss, features within each class collapse to a single class mean, so the effective feature matrix is the class-mean matrix $\\bar H$ with entries repeated within each class. Writing $\\hat Y = (I_K - \\frac{1}{N}\\mathbf{n}1_K^\\top)D$ and its SVD as $\\hat Y = \\hat U \\hat\\kappa \\hat V^\\top$, the theorem says that $\\hat U^\\top W_L\\cdots W_1$ has rows that are either zero or mutually orthogonal, while $\\bar H D \\hat V$ has columns of the same kind, and for the surviving coordinates the $i$-th row of the classifier product aligns exactly with the $i$-th column of the scaled class-mean matrix. Which coordinates survive is governed by whether the corresponding singular value of $\\hat Y$ exceeds a threshold set by the layer count and the regularization weights. The bias is pinned down as the class-frequency vector and the global feature mean is zero, so the imbalance is carried entirely by $\\hat Y$ and not by the features.","pith_inferences":["The paper does not draw the practical corollary that its threshold is a minimal sample-size rule: the model can only represent a class when the associated singular value of $\\hat Y$ clears the regularization threshold, so classes below it are effectively dropped by the global optimum.","Because $\\hat Y$ is a rank-one perturbation of a diagonal matrix, the same SVD-driven geometry should plausibly appear under other losses whose centered label matrix has the same shape; testing cross-entropy or hinge versions of Theorem 4.3 would be a direct extension the paper does not attempt.","Repeated class sizes make the SVD of $\\hat Y$ non-unique, so the row-column alignment in Theorem 4.3 is only defined up to rotations inside the repeated eigenspace; a rotation-invariant statement would be needed for datasets with ties, and the paper does not address that case.","One could also use the paper's NC2 metrics as a diagnostic on real networks: measure how close the trained product of classifier layers is to the predicted zero/orthogonal pattern, and treat large deviations as evidence that the trained solution is far from the L-EUFM global optimum."],"forward_implications":["In this model, minority collapse is not a separate failure mode: a class is represented at the global optimum if and only if the singular value of $\\hat Y$ associated with it clears the regularization threshold, and otherwise its classifier row and feature column vanish.","For grouped imbalances, the singular values of $\\hat Y$ are explicit: within each group of classes sharing a sample size $N_j$, the repeated singular values are $\\sqrt{N_j}$, and the remaining singular values come from a small $m\\times m$ matrix $G$; in the two-group case the only extra singular value is $\\sqrt{K N_1 N_2/N}$.","With $L$ layers, the same zero-or-orthogonal geometry holds for the product of $L$ linear classifiers, and the survival threshold depends on the geometric mean of the weight-decay parameters across layers.","The bias absorbs the imbalance: the optimal bias is the vector of class frequencies $\\mathbf{n}/N$, and the global feature centroid is zero, so the centered label matrix $\\hat Y$ is the only place class sizes enter.","In the bias-free one-layer case, the survival condition reduces to $n_i > N^2 \\lambda_H \\lambda_W$, making the collapse threshold a direct inequality on per-class sample count."],"supporting_citations":[{"why":"Introduces the neural collapse phenomenon whose balanced-data geometry this paper generalizes.","marker":"[PHD20]"},{"why":"Establishes the unconstrained feature model that the paper extends to L layers with bias and imbalance.","marker":"[MPP20]"},{"why":"Supplies Lemma A.1, the nuclear-norm factorization identity used to reduce the MSE loss to a convex problem.","marker":"[ZDZ+21]"},{"why":"Provides the singular value thresholding result used to solve the reduced nuclear-norm problem.","marker":"[CCS10]"},{"why":"Extends neural collapse to deep linear networks under imbalanced data and supplies Lemma A.2 used in Theorem 4.3.","marker":"[DTO+23]"},{"why":"Identifies minority collapse in imbalanced training, the phenomenon this paper's zero-row result formalizes.","marker":"[FHLS21]"},{"why":"Gives an SVD-determined geometry for imbalanced neural collapse under hinge loss, the closest prior counterpart to this MSE analysis.","marker":"[TKVB22]"},{"why":"Develops the extended unconstrained features model for deep neural collapse that the L-layer part builds upon.","marker":"[TB22]"}],"fun_headline_variants":["A single matrix sets neural-collapse geometry for imbalanced data","Neural collapse under imbalance: one matrix decides which directions survive","Imbalanced data: class collapse persists, but one matrix picks directions","How a class-size matrix steers neural collapse in imbalanced settings","One matrix rules neural-collapse geometry in imbalanced data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that every global minimizer must rotate its singular vectors to match $\\hat Y$ is supported by an assertion in Section B.3 that a certain other configuration gives a strictly larger objective value, but the inequality behind that assertion is not demonstrated; if that inequality fails, the orthogonal and aligned structure is not established for every global minimizer.","fun_headline_variants_meta":{"raw":{"variants":["A single matrix sets neural-collapse geometry for imbalanced data","Neural collapse under imbalance: one matrix decides which directions survive","Imbalanced data: class collapse persists, but one matrix picks directions","How a class-size matrix steers neural collapse in imbalanced settings","One matrix rules neural-collapse geometry in imbalanced data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1496,"prompt_tokens":1087,"completion_tokens":409,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":322}},"tokens_in":703,"tokens_out":409,"duration_ms":4387,"temperature":1.0,"reasoning_tokens":322,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:19:41.131695+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Solve (4.3) numerically for a small imbalanced case such as $K=3$ classes with sizes $[10,10,1]$, two layers, and small fixed regularization, using many random initializations; if any computed global minimizer shows a nonzero row of $\\hat U^\\top W_L\\cdots W_1$ for a class whose $\\hat Y$ singular value is below the paper's threshold, or shows surviving rows that are not mutually orthogonal, Theorem 4.3 would be contradicted. A cheaper check is to audit the unproved inequality in Section B.3 for the specific case of repeated class sizes, where the SVD of $\\hat Y$ is non-unique.","supporting_citations":[],"review_version":1}