{"id":"e375d5ba-7ac5-40a9-ab49-5f3033a707b1","arxiv_id":"2504.19351","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"The claimed exact asymptotic curve for double descent in a two-layer ReLU classifier does not follow from the paper's stated equations because a key step incorrectly drops a factor of r.","lead":"This paper tries to explain double descent, where test error dips, spikes, then drops again, in a simple two-layer ReLU classifier trained on a Gaussian mixture. The derivation behind the claimed curves has an algebraic error, so the main theoretical result is not supported by the paper's own equations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Deriving gamma* in (8.21) silently drops the factor r* from the paper's own equation (5.26); the Section 6 curves are not fixed-point solutions of Theorem 5.1, so the claimed double-descent peak at alpha = 1 is unsupported.","rationale":"I agree with the reader's REJECT, but my most load-bearing concern differs from the one listed in weakest_assumption (the CGMT identification in Theorem 8.1). The decisive problem is internal and algebraic: equation (5.26) of Theorem 5.1 contains r*, yet the reduction in Section 8.6 from (5.26) to (8.21) discards that factor. The paper's own computation of the limit evaluates (5.26) as α√r* · γ*√r*/(1+γ*) = 1 − 4λγ*, i.e. α γ* r*/(1+γ*) = 1 − 4λγ*, whose solution must contain αr*; (8.21) solves the corresponding equation with r* replaced by 1. At the plotted operating point (α = 1, λ = 10⁻⁵, ρ₁ = 0.5), the two sides of (5.26) differ by a factor of about 150, so the values in (8.27) are not fixed points of the paper's stated theory. The Section 6 figures therefore show curves the theory does not predict, and the headline claim of ratio-wise double descent peaking at α = 1 is unsupported. This is an internal inconsistency, not a disagreement with existing literature, and it can be settled by direct substitution. Credit where due: Theorem 4.1 is correctly derived; the Legendre-conjugate identities in (5.12) are standard and applied correctly; the derivative computations in Section 8.5 and the algebra from (8.22) through (8.26), conditional on (8.20), check out; and the CGMT strategy is a reasonable methodological choice. The flaw is localized to the γ* derivation but propagates into every plot in Section 6 and into the conclusions. The Theorem 8.1 proof is also under-supported: Section 8.4 bounds ω but asserts sup|...|→0 from boundedness; inequality (8.10) is justified by a sentence that concedes the right side may be infinite; boundedness of ||u||₂/d is assumed; and the CGMT conditions for the absolute-value (ReLU) term are not checked. That issue alone would justify at most CONDITIONAL with substantial revisions, but the algebraic inconsistency forces rejection: the stated equations and the reported fixed-point values cannot both be correct. The Section 2 empirical warm-up is suggestive but tests a different model (binary cross-entropy, 10 hidden units) and does not validate the theoretical curve. Verdict unchanged: REJECT.","tokens_in":22929,"tokens_out":14744,"duration_ms":120160,"concrete_test":"Substitution check: fix α = 1, λ = 10⁻⁵, ρ₁ = ρ₋₁ = 0.5, compute γ*, s*, b*, r* from (8.27) (γ* ≈ 158, r* ≈ 158), and evaluate both sides of (5.26), i.e. α√r* γ*√r*/(1+γ*) versus 1 − 4λγ*. The sides differ by two orders of magnitude, proving (8.27) is not a fixed point of Theorem 5.1. Then solve the corrected coupled system γ = [−(αr + 4λ − 1) + √((αr + 4λ − 1)² + 16λ)]/(8λ), s = 8αγρ₁ρ₋₁/(1 + γ + 4αγρ₁ρ₋₁), b = (2 − s)(2ρ₁ − 1), r = [αγ²((s − 2)² − b²) + (1 + γ)² s²]/[(1 + γ)² − αγ²] over α ∈ [0.05, 10], and plot R* from (4.1). If the corrected curve has no peak at α = 1, the claimed ratio-wise double descent is an artifact of the dropped factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 8.6 the limit in (5.26) is evaluated as α√r* · γ*√r*/(1+γ*), so equation (5.26) reduces to α γ* r*/(1+γ*) = 1 − 4λγ*. Solving this quadratic gives γ* = [−(αr* + 4λ − 1) ± √((αr* + 4λ − 1)² + 16λ)]/(8λ). The published (8.21) instead contains α where the correct expression must contain αr*; it solves (5.26) only after silently setting r* = 1, and nothing in the paper justifies r* ≡ 1. Direct substitution shows the inconsistency is large: for α = 1, λ = 10⁻⁵, ρ₁ = 0.5, the values in (8.27) give γ* ≈ 158 and r* ≈ 158, so the left side of (5.26) is ≈ 157 while the right side is ≈ 0.99. Hence (8.27) is not a solution of the paper's own Theorem 5.1. Because Figures 4–8 are computed from (8.27), the claimed test-error peak at α = 1, and with it the central double-descent conclusion, does not follow from the stated theory. The reader's re-solve of the stated equations for ρ₁ = 0.5 reportedly gives a monotone curve, consistent with the peak being an artifact of the dropped factor. Secondary but real: the proof of Theorem 8.1 (Sections 8.3–8.4) is a sketch, inequality (8.10) is asserted while conceding its right side may be infinite, and CGMT regularity conditions are not verified. The algebraic error is the decisive, standalone problem.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper analyzes the generalization error of a two-layer neural network with a single ReLU hidden unit for binary classification, with data generated from a Gaussian mixture teacher model. The authors derive an exact asymptotic expression for the test error as a function of three quantities r, s, and b (Theorem 4.1). They then use the Convex Gaussian Min-Max Theorem (CGMT) to characterize the asymptotic minimizers of the regularized empirical risk, obtaining a system of fixed-point equations in Theorem 5.1. Specializing to square loss, they solve this system in Section 8.6 and use the resulting formulas to plot the test error against alpha = n/d in Section 6. The reported curves exhibit a double-descent peak at alpha = 1, which the authors interpret as ratio-wise double descent. The central claim of the paper is that these curves are exact consequences of the derived theory.","tokens_in":23295,"tokens_out":5866,"duration_ms":55214,"significance":"The paper's central claim is the existence of ratio-wise double descent for this two-layer ReLU classification model, with the test-error peak occurring exactly at alpha = 1. If the derivation were correct, the paper would provide a rigorous CGMT-based asymptotic analysis of double descent in a simple neural network, which would be a useful contribution to the theoretical double-descent literature. The derivation of Theorem 4.1 is clean and correct, and the manuscript provides reproducible R code for the figures. However, as detailed in the major comments, the fixed-point solution in Section 8.6 contains an algebraic error that invalidates the numerical results; the paper's own equations do not support the claimed peak at alpha = 1. Since the plotted curves are the primary evidence for the paper's main conclusion, the significance of the manuscript is critically undermined.","major_comments":[{"comment":"The solution for gamma* in equation (8.21) does not follow from the paper's own equation (5.26). The limit computation in Section 8.6 correctly yields alpha * sqrt(r*) * [gamma* * sqrt(r*) / (1 + gamma*)] = 1 - 4 lambda gamma*, which simplifies to alpha * gamma* * r* / (1 + gamma*) = 1 - 4 lambda gamma*. Solving this quadratic for gamma* gives gamma* = [-(alpha r* + 4 lambda - 1) ± sqrt((alpha r* + 4 lambda - 1)^2 + 16 lambda)] / (8 lambda). The published equation (8.21) instead contains alpha in place of alpha r*, which is equivalent to silently setting r* = 1 with no justification. Direct substitution shows that the values in (8.27) do not solve (5.26): for alpha = 1, lambda = 10^-5, rho_1 = 0.5, the formulas in (8.27) give gamma* approximately 158 and r* approximately 158, so the left-hand side of (5.26) is approximately 157 while the right-hand side is approximately 0.99. Because Figures 4-8 are computed from (8.27), the claimed test-error peak at alpha = 1, and with it the central double-descent conclusion, does not follow from the stated theory. This algebraic error is load-bearing and invalidates the paper's main numerical claim.","section":"Section 8.6, Eqs. (5.26) and (8.21)"},{"comment":"The proof that the global training loss L*_lambda is asymptotically equal to the infimum of the lower bound omega_lambda^(d)(r,s) is only a sketch and leaves CGMT regularity conditions unverified. In particular, inequality (8.10) is merely asserted, and the text itself concedes that the right-hand side may be infinite when only omega_lambda^(d) is bounded. The boundedness of ||u||_2/d and the sup-norm convergence needed to apply CGMT are not rigorously established. Since Theorem 5.1 inherits this identification, the fixed-point system that underlies all subsequent calculations is not fully proved. This is a serious gap, although it is secondary to the algebraic error in Major Comment 1.","section":"Theorem 8.1 and Section 8.4"}],"minor_comments":[{"comment":"The text says \"CIFAR-10 and MINST\" but should say \"MNIST.\"","section":"Section 2, p.4"},{"comment":"The sentence following equation (3.1) contains malformed notation: \"xT_iβ√d+b,∈Rd\" should be written as \"x_i^T β / sqrt(d) + b in R.\"","section":"Section 3.2, Eq. (3.1)"},{"comment":"In the loop used to generate data for the Section 6 plots, the line \"x <- (eta*y)/sqrt(features) + epsilon\" overwrites the entire matrix x on each iteration instead of assigning to the row x[i, ]; the intended data matrix is therefore not stored. The theoretical curves in Section 6 do not actually use the simulated x, so this does not affect those plots, but the code is misleading as written.","section":"Section 9.1, R code"},{"comment":"The statement that \"when λ ≥ 1, the test error keeps unchanged until a specific α value is reached\" is vague; Figure 6 suggests the error stays near the Bayes-optimal value for small alpha, but the text should state this explicitly and relate it to the formulas in (8.27).","section":"Section 6.2"}],"recommendation":"reject","confidential_remarks":"The algebraic error in Section 8.6 is decisive: the numerical results that constitute the paper's main evidence for double descent are computed from formulas that do not solve the paper's own fixed-point equations. The corrected equations reportedly yield a monotone test-error curve for the parameters shown, which would mean the central claim is not merely unproved but false for the stated model. The CGMT proof is also too sketchy to support the theoretical framework. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take. The test-error formula in Theorem 4.1 is correct, and the use of CGMT to get a scalar fixed-point system is a legitimate strategy. But the square-loss specialization has a load-bearing algebra error: equation (5.26) reduces to αγ*r*/(1+γ*) = 1 − 4λγ*, and (8.21) solves it with r* silently set to 1. The R code uses that same wrong quadratic, so Figures 4–8 are not solutions of the paper's own Theorem 5.1. For α=1, λ=1e-5, ρ1=0.5, their (8.27) gives γ*≈158 and r*≈158; plugging back gives LHS≈157 and RHS≈0.99. The fixed point simply does not hold. Solving the stated equations correctly for ρ1=0.5 gives a monotone test-error curve, so the claimed double-descent peak at α=1 is an artifact. That is the main problem.\n\nWhat is genuinely useful: the derivation of the generalization error (4.1) is clean, and the derivative calculations in Section 8.5 are careful and documented. The fixed-point equations for s* and b* in (8.22)–(8.24) are also correct conditional on γ*. The paper is honest about the setup and the CGMT strategy, and it ships R code, though the code inherits the error.\n\nSecondary issues: Theorem 8.1 is asserted as an almost sure limit, but the proof is a sketch. Inequality (8.10) uses sup-norm bounding and concedes the right side may be infinite; the boundedness of ||u||/d is not fully verified. That is a real gap, but it is secondary to the algebra.\n\nBottom line: this is a paper with useful machinery and a central conclusion that is currently unsupported. It deserves a careful rewrite, not publication as-is. I would recommend the editor reject the current version with an invitation to resubmit after correcting the fixed-point equations and re-checking whether any non-monotone region survives for equal cluster sizes. If the corrected curve is monotone, the contribution becomes a narrow asymptotic error formula, still publishable in a specialized venue, but the double-descent story is gone.","headline":"The CGMT machinery is real, but one dropped factor of r* in the square-loss fixed point invalidates every double-descent curve in the paper.","tokens_in":23846,"tokens_out":4037,"would_cite":false,"duration_ms":40239,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62H30"],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-layer ReLU network trained by square loss has an asymptotic test-error curve that peaks exactly when the number of training samples equals the model dimension, then descends again.","keywords":["double descent","generalization error","two-layer neural network","ReLU activation","binary classification","Convex Gaussian Min-Max Theorem","square loss","high-dimensional asymptotics"],"falsifier":"Simulate the teacher–student model for a fixed small regularization, say $d=200$ and $n=\\alpha d$ with $\\alpha$ ranging from 0.2 to 5, train the two-layer ReLU student with square loss by running an optimizer from many restarts (or a global solver), and compare the empirical test-error curve to (8.28). The central claim fails if the empirical peak does not occur at $\\alpha\\approx1$ or if the curve deviates systematically from the closed form beyond finite-sample fluctuations. A sharper check is to verify the Section 8.4 claim that $\\lim_{n,d\\to\\infty}\\sup_{s^2\\le r}|\\tilde L_\\lambda(r,s)-\\omega_\\lambda^{(d)}(r,s)|=0$ using the boundedness of $\\|u\\|_2/d$; if that limit is nonzero for square loss, the lower-bound identification collapses.","tokens_in":22674,"feed_emoji":"📉","tokens_out":13816,"duration_ms":115916,"temperature":0.7,"pith_summary":"This paper derives a closed-form asymptotic formula for the test error of a two-layer ReLU network trained for binary classification, as a function of the ratio $\\alpha = n/d$ between training samples and model dimension. Working in a teacher–student setup where labels come from two Gaussian clusters, the authors show that under low $\\ell_2$ regularization the test-error curve has a U-shape in the over-parameterized regime, reaches its maximum exactly at $\\alpha = 1$, and then decreases monotonically as more data become available. The derivation avoids simulating training: it uses the Convex Gaussian Min-Max Theorem to replace the empirical risk minimizer by the minimizer of an auxiliary optimization problem, whose solution collapses to a scalar fixed-point system. If the identification is right, the paper turns double descent from an empirical curiosity into a computable curve whose peak location, sweet spot, and regularization sensitivity can be read off directly.","feed_headline":"Double descent peaks exactly at n=d in two-layer ReLU nets","feed_subtitle":"A closed-form asymptotic error formula shows the curve falls, peaks at α=1, then falls again; regularization erases the peak.","key_machinery":"The central object is the CGMT lower bound $\\omega_\\lambda^{(d)}(r,s) = \\lambda r/2 + (\\alpha/n)\\sum_{i=1}^n l(v_i)$, obtained by rewriting the ReLU risk through $\\sigma(z)=(z+|z|)/2$, applying the Legendre transformation to the margin loss, and passing to the auxiliary Gaussian optimization problem. Minimizing this bound under $s^2\\le r$ and sending $n,d\\to\\infty$ reduces the high-dimensional empirical risk minimization to the four scalar equations (5.26)–(5.29) for $r^*,s^*,b^*,\\gamma^*$. This scalar fixed-point system is what carries the argument: it turns an ERM over $\\mathbb{R}^d$ into a one-dimensional curve, and for square loss it closes explicitly in (8.27), giving $\\gamma^*$ as the positive root of the quadratic in (8.21).","core_discovery":"For the student model $f(x)=\\sigma(x^{\\mathsf T}\\beta/\\sqrt{d}+b)$ with ReLU activation, trained on data generated from the teacher model $x_i = \\eta y_i/\\sqrt{d} + \\epsilon_i$, the paper claims that as $n,d\\to\\infty$ with $\\alpha=n/d$ fixed, the limiting test error under square loss is $R^*(\\hat\\beta)=1-\\rho_1\\Phi((s^*+b^*)/\\sqrt{r^*})-\\rho_{-1}\\Phi((s^*-b^*)/\\sqrt{r^*})$, where $r^*,s^*,b^*$ solve the fixed-point system (8.27). The resulting curve exhibits ratio-wise double descent: error first falls, peaks at $\\alpha=1$, then falls again as the model enters the under-parameterized region. Theorems 4.1 and 5.1 are the two load-bearing results, with (8.28) giving the explicit square-loss specialization.","pith_inferences":["The closed-form curve in (8.28) can be differentiated with respect to $\\alpha$ and $\\lambda$ to locate the sweet-spot minimum as a function of regularization, a quantity the paper does not isolate.","Since the derivation fixes one dimension $d$ and sweeps $n$, the ratio-wise curve can be read as a smoothed theoretical version of sample-wise double descent for this teacher–student model, though the paper does not make that connection.","The same CGMT argument should yield analogous fixed-point systems for other convex activations, but the paper does not test that extension, so it remains an open possibility rather than a claim."],"forward_implications":["Under low regularization the test-error peak is located exactly at $\\alpha=1$, the interpolation threshold where $n=d$; the best model lies in the under-parameterized region $\\alpha>1$.","Raising the $\\ell_2$ regularization coefficient $\\lambda$ flattens the peak: for balanced classes with sufficiently large $\\lambda$ the test error decreases monotonically, matching the classical idea that more data are always better.","For uneven class probabilities (such as $\\rho_1=0.7$) strong regularization holds the test error flat until a large $\\alpha$, then lets it decline; the paper recommends a moderate range $0.1\\le\\lambda\\le1$ for best performance.","Because the fixed-point equations before specialization are derived for any margin-based convex loss, the same two-theorem framework would produce test-error curves for hinge or logistic loss by substituting the corresponding loss function."],"supporting_citations":[{"why":"Supplies the Convex Gaussian Min-Max Theorem used to convert the empirical risk minimization into the auxiliary optimization problem and to justify inequality (5.7).","marker":"[19]"},{"why":"Provides the CGMT machinery for regularized linear regression, including the convex-concave property invoked in Section 8.4 to bound the gap between auxiliary and lower-bound losses.","marker":"[20]"},{"why":"Gives the standard application procedure for CGMT that the paper follows to pass from the auxiliary problem to the scalar lower bound $\\omega_\\lambda^{(d)}(r,s)$.","marker":"[13]"},{"why":"Introduced the double-descent phenomenon in over-parameterized models, the empirical curve this paper seeks to explain.","marker":"[3]"},{"why":"Earlier analytic model of double descent in high-dimensional binary linear classification, the ratio-based setting this paper extends to a two-layer ReLU student.","marker":"[7]"},{"why":"Shows that optimal regularization can mitigate double descent; the paper's regularization experiments echo and extend that observation.","marker":"[16]"},{"why":"Theorem 18.5 supplies the Lagrange-multiplier optimality conditions used to solve the maximization over the Legendre variable $u$.","marker":"[17]"}],"fun_headline_variants":["Double descent peak pinned to α=1 in ReLU nets","Two-layer ReLU: test error curve re-peaks at n=d","Exact double descent: peak at n/d=1 for binary nets","ReLU network error: falls, peaks at n=d, then falls again","Double descent theory: peak at α=1 in two-layer nets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the auxiliary lower bound produced by the Convex Gaussian Min-Max Theorem converges to the true global training loss in the limit, so the minimizer of the easy problem can stand in for the empirical-risk minimizer. The proof of that identification is a sketch: it invokes boundedness of $\\|u\\|_2/d$ and sup-norm convergence without fully verifying all CGMT conditions.","fun_headline_variants_meta":{"raw":{"variants":["Double descent peak pinned to α=1 in ReLU nets","Two-layer ReLU: test error curve re-peaks at n=d","Exact double descent: peak at n/d=1 for binary nets","ReLU network error: falls, peaks at n=d, then falls again","Double descent theory: peak at α=1 in two-layer nets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000532,"raw_usage":{"total_tokens":2521,"prompt_tokens":864,"completion_tokens":1657,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":1563}},"tokens_in":480,"tokens_out":1657,"duration_ms":12136,"temperature":1.0,"reasoning_tokens":1563,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:56:47.645528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate the teacher–student model for a fixed small regularization, say $d=200$ and $n=\\alpha d$ with $\\alpha$ ranging from 0.2 to 5, train the two-layer ReLU student with square loss by running an optimizer from many restarts (or a global solver), and compare the empirical test-error curve to (8.28). The central claim fails if the empirical peak does not occur at $\\alpha\\approx1$ or if the curve deviates systematically from the closed form beyond finite-sample fluctuations. A sharper check is to verify the Section 8.4 claim that $\\lim_{n,d\\to\\infty}\\sup_{s^2\\le r}|\\tilde L_\\lambda(r,s)-\\omega_\\lambda^{(d)}(r,s)|=0$ using the boundedness of $\\|u\\|_2/d$; if that limit is nonzero for square loss, the lower-bound identification collapses.","supporting_citations":[{"cited_title":"conjugate","cited_arxiv_id":null,"evidence_quote":"Provides the CGMT machinery for regularized linear regression, including the convex-concave property invoked in Section 8.4 to bound the gap between auxiliary and lower-bound losses."},{"cited_title":"and Zdeborova, L., (2020), The role of regularization in classification of high-dimensional noisy gaussian mixture","cited_arxiv_id":null,"evidence_quote":"Gives the standard application procedure for CGMT that the paper follows to pass from the auxiliary problem to the scalar lower bound $\\omega_\\lambda^{(d)}(r,s)$."},{"cited_title":"and Mandal, S","cited_arxiv_id":null,"evidence_quote":"Introduced the double-descent phenomenon in over-parameterized models, the empirical curve this paper seeks to explain."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier analytic model of double descent in high-dimensional binary linear classification, the ratio-based setting this paper extends to a two-layer ReLU student."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Theorem 18.5 supplies the Lagrange-multiplier optimality conditions used to solve the maximization over the Legendre variable $u$."}],"review_version":1}