{"id":"4878b777-d5b9-42ab-864f-da91ad3fa12a","arxiv_id":"2606.00344","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Using the unrestricted feature model with MSE loss, the work shows that bias regularization drives one-hot class means from simplex ETF to orthogonal frame and that the classifier bias centers arbitrary label encodings.","lead":"The paper studies how different ways of encoding class labels affect the structure of neural collapse in the last hidden layer of classification networks. It finds that bias regularization can shift class-mean features from a simplex equiangular tight frame toward an orthogonal frame when labels are one-hot encoded.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Derivation of ETF-to-orthogonal transition relies on UFM with MSE loss and exact class balance; any deviation alters the global mean and may eliminate the transition","rationale":"The reader's weakest assumption correctly isolates the two idealizations (UFM + perfect balance) on which the transition depends. Because the paper supplies no robustness check outside these assumptions, the UNVERDICTED verdict with low confidence remains appropriate.","tokens_in":1640,"tokens_out":368,"duration_ms":15682,"concrete_test":"For K=3 classes and N=300 samples, solve the UFM stationarity equations analytically for λ ∈ {0, 0.1, 1, 10}; compute the Gram matrix of the three uncentered means and check whether off-diagonal entries move from -0.5 to 0. Then repeat with class counts (150,100,50) and verify whether the same λ sweep still produces an orthogonal Gram matrix at large λ.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim is obtained by jointly minimizing the UFM objective (features free, classifier linear) under MSE with one-hot targets plus bias regularization term λ||b||^{2}. At λ=0 the optimal uncentered class means \bar h_k form a simplex ETF; as λ\to∞ they become orthogonal because the bias centers the predictions to match the label origin. This step uses the closed-form stationarity condition that equates the weighted sum of features to the label means, which holds only when every class has exactly N/K samples. With imbalance the global mean shifts and the orthogonal fixed point ceases to be a critical point of the same objective.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that within the unrestricted feature model (UFM) trained under mean-squared-error loss, one-hot encoded labels on perfectly balanced data produce a transition in the uncentered class-mean features from a simplex equiangular tight frame to an orthogonal frame as the bias-regularization coefficient λ increases; for arbitrary label encodings the optimal bias centers the labels to compensate for the offset between the label global mean and the origin; the work further discusses implications for other neural-collapse properties.","tokens_in":1796,"tokens_out":483,"duration_ms":16185,"significance":"If the derivations are rigorous, the result supplies an explicit, closed-form account of how label encoding and bias regularization shape last-layer geometry inside the UFM, thereby clarifying why orthogonal-frame structures appear under strong regularization and extending prior NC analyses that typically fix λ = 0. The parameter-free limiting characterizations (λ → 0 and λ → ∞) constitute a genuine analytical contribution.","major_comments":[{"comment":"The central transition claim is obtained from the stationarity condition that equates the weighted sum of features to the label means; this identity holds only under exact class balance (N/K samples per class). The manuscript states the balanced-data assumption but does not display the explicit stationarity equation or show how the orthogonal fixed point ceases to be critical once the global mean shifts under imbalance.","section":"derivation of the transition (around the bias-regularization analysis)"},{"comment":"No numerical verification of the claimed ETF-to-orthogonal transition is provided inside the UFM, nor is the full step-by-step derivation of the stationarity conditions reproduced. Without either, it is impossible to confirm the absence of hidden assumptions or post-hoc choices in the optimization.","section":"main results and abstract"}],"minor_comments":[{"comment":"Notation for the bias-regularization coefficient λ and the uncentered means \bar h_k should be introduced with a single consolidated table or equation block rather than scattered across paragraphs.","section":null},{"comment":"The discussion of arbitrary encodings would benefit from an explicit statement of the centering condition (bias = –global label mean) as a separate proposition or corollary.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful review and constructive suggestions. We address each major comment below and commit to revisions that strengthen the manuscript's clarity and rigor.","responses":[{"response":"We agree that the stationarity condition is derived under the exact class-balance assumption (N/K samples per class), which is stated but not expanded upon. In the revision we will insert the explicit stationarity equation obtained from the UFM Lagrangian, and we will add a short analysis showing that the orthogonal fixed point is no longer critical when the global mean is shifted by imbalance; the transition result will be explicitly qualified to the balanced case.","revision_made":"yes","referee_comment":"The central transition claim is obtained from the stationarity condition that equates the weighted sum of features to the label means; this identity holds only under exact class balance (N/K samples per class). The manuscript states the balanced-data assumption but does not display the explicit stationarity equation or show how the orthogonal fixed point ceases to be critical once the global mean shifts under imbalance."},{"response":"We acknowledge the absence of both numerical verification inside the UFM and the complete derivation. The revised manuscript will include (i) a dedicated numerical experiment that sweeps λ inside the UFM and plots the transition of the uncentered class means from simplex ETF to orthogonal frame, and (ii) the full step-by-step derivation of the stationarity conditions placed either in the main text or as a self-contained appendix so that every algebraic step and assumption is transparent.","revision_made":"yes","referee_comment":"No numerical verification of the claimed ETF-to-orthogonal transition is provided inside the UFM, nor is the full step-by-step derivation of the stationarity conditions reproduced. Without either, it is impossible to confirm the absence of hidden assumptions or post-hoc choices in the optimization."}],"tokens_in":1296,"tokens_out":405,"duration_ms":13320,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that in the unrestricted feature model with MSE loss, increasing the bias regularization coefficient makes uncentered class means transition from a simplex equiangular tight frame to an orthogonal frame for one-hot labels on balanced data. The bias term also centers the predictions to match the label mean for arbitrary encodings.\n\nThe work does a solid job laying out how the bias interacts with label encoding inside this model. The explicit dependence of the geometry on the regularization strength and the centering observation for non-standard encodings look like genuine additions to the neural collapse literature.\n\nThe math stays clean because it works from stationarity conditions on the joint objective over features and classifier. That keeps the claims scoped and reproducible within the stated assumptions.\n\nThe soft spot is the exact class balance requirement. Any imbalance shifts the global mean, and the orthogonal fixed point is no longer a critical point of the same objective. The abstract already limits the claim to balanced data, but without exploring sensitivity to mild imbalance the transition stays fragile. The abstract also gives no numerical checks, so it is hard to see whether the closed-form conditions survive small perturbations or implementation details.\n\nThis is for people already working on neural collapse theory and the unrestricted feature model. A reader who cares about how label encoding and bias shape the geometry will get something concrete from it. The thinking is honest and the claims are properly scoped to the model, so the paper shows clear engagement with the literature.\n\nI would send it for peer review to verify the derivations and test whether the balance assumption can be relaxed or at least quantified.","headline":"The paper derives a bias-regularization shift from ETF to orthogonal class means in the UFM under MSE and one-hot labels, but only with exact balance.","tokens_in":2266,"tokens_out":393,"would_cite":false,"duration_ms":21809,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"For one-hot labels and balanced data, increasing bias regularization on the final classifier shifts uncentered class mean features from a simplex equiangular tight frame to an orthogonal frame.","keywords":["neural collapse","label encoding","one-hot encoding","unrestricted feature model","mean squared error","bias regularization","simplex equiangular tight frame","orthogonal frame"],"falsifier":"Train a network on balanced one-hot data while sweeping the bias-regularization coefficient and check whether the empirical class-mean vectors switch from simplex equiangular tight frame geometry to orthogonal geometry.","tokens_in":2549,"feed_emoji":"","tokens_out":658,"duration_ms":11855,"temperature":0.7,"pith_summary":"The paper examines how the choice of label encoding shapes the geometry of last-layer features under neural collapse. It relies on the unrestricted feature model together with mean-squared-error loss to isolate the effect of encoding. For one-hot labels and perfectly balanced classes, the mean feature vectors per class move from a simplex equiangular tight frame toward an orthogonal frame as the regularization strength on the classifier bias is raised. The bias term itself is shown to center the encoded labels, offsetting any offset between the label mean and the origin. The analysis extends to arbitrary encodings and notes that encoding choices influence additional collapse properties.","feed_headline":"Bias regularization turns class means from ETF to orthogonal frame","feed_subtitle":"For one-hot labels and balanced data the uncentered means switch geometry as the final classifier's bias penalty is increased.","key_machinery":"Unrestricted feature model (last-layer activations treated as free variables) under mean-squared-error loss, with bias regularization as the control parameter that drives the frame transition.","core_discovery":"Under the unrestricted feature model with mean-squared-error loss, one-hot encoded labels and balanced data produce uncentered class-mean features that transition from a simplex equiangular tight frame to an orthogonal frame when the bias-regularization coefficient of the final classifier is increased; for arbitrary encodings the bias acts to center the labels relative to the origin.","pith_inferences":["Practitioners could deliberately tune bias regularization to enforce a desired target geometry in the last-layer means.","The centering role of the bias suggests that label encodings with non-zero mean may require explicit mean-subtraction or adjusted regularization to reach the same collapse regime.","If real networks deviate from the unrestricted-feature idealization, the predicted transition may appear only in simplified linear or two-layer models."],"forward_implications":["The bias of the final classifier compensates for any discrepancy between the global mean of the encoded labels and the coordinate origin.","The particular geometry that emerges in neural collapse depends on the chosen label encoding.","Other neural-collapse properties besides the mean-feature frame are also modulated by the encoding scheme.","The orthogonal-frame limit recovered at high bias regularization matches the geometry of the one-hot labels themselves."],"fun_headline_variants":["Bias regularization shifts class means from ETF to orthogonal frame","Class means transition from simplex ETF to orthogonal with bias reg","Encoding and bias penalty determine neural collapse mean geometry","Bias centers labels while uncentered means form orthogonal frame"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The last-layer features behave exactly as free variables under mean-squared-error loss and the data classes are perfectly balanced.","fun_headline_variants_meta":{"raw":{"variants":["Bias regularization shifts class means from ETF to orthogonal frame","Class means transition from simplex ETF to orthogonal with bias reg","Encoding and bias penalty determine neural collapse mean geometry","Bias centers labels while uncentered means form orthogonal frame"]},"model":"grok-4.3","cost_usd":0.006066,"raw_usage":{"total_tokens":2821,"prompt_tokens":574,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":60662000,"prompt_tokens_details":{"text_tokens":574,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2186,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":574,"tokens_out":61,"duration_ms":13041,"temperature":1.0,"reasoning_tokens":2186,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T22:56:50.354203+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Train a network on balanced one-hot data while sweeping the bias-regularization coefficient and check whether the empirical class-mean vectors switch from simplex equiangular tight frame geometry to orthogonal geometry.","supporting_citations":[],"review_version":1}