{"id":"17bffa95-73dd-4177-afb6-86e3a65cf360","arxiv_id":"2608.04882","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A variational potential gives both a lower and an upper bound on the quenched pressure of a perceptron, with matching supported numerically for logistic-type losses.","lead":"The authors prove matching lower and upper variational bounds for the free energy of a continuous-spin perceptron trained on Gaussian mixture data. The bounds coincide when two scalar optimizations commute, which the paper checks numerically for common losses.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Exact characterization hinges on the unproven sup-rho / inf-r exchange; the two-sided bounds themselves appear internally sound.","rationale":"The reader's CONDITIONAL verdict is appropriate and my analysis does not move it. I agree with the reader's identification of the unproven commutation of sup_ρ and inf_r as the key obstruction to the exact variational characterization and to the corollaries. I do not elevate Lemma 2 to a load-bearing concern: the inequality ⟨S12⟩_t ≥ ⟨S11⟩_t follows from log-concavity of the interpolating Boltzmann factor via Prekopa–Leindler, and the sign of the lower-bound velocities is secured by it. I also checked the sum rule's factor of 2: Proposition 3 displays β∫δ̇ρ̇, while the bounds use β∫δ̇ρ̇/2; a direct recombination of the Stein-lemma terms yields the half factor, so this appears to be a typographical inconsistency in Proposition 3 rather than a mathematical error affecting the bounds. The bounds themselves appear internally consistent: the upper-bound remainder is controlled by choosing (ρ̇, q̇, ṁ) as Gibbs averages, and the lower-bound remainder is controlled because the ODE choices force β(βṙ−δ̇)−E⟨S11⟩ = 0 and β²ṙ−E⟨S12⟩ = 0, leaving only covariances that vanish by concentration of Q11, Q12, M1 and bounded second moments of S11, S12, H1. The genuine substantive limitation is the missing proof of the outer minimax exchange; the numerical saddle-point evidence is suggestive but not dispositive, especially near the interpolation threshold where the potential may develop multiple stationary points. A direct numerical evaluation of the gap Δ, computed with global rather than local optimization, is the most concrete way to test whether the concern lands in the regimes the paper claims.","tokens_in":40510,"tokens_out":32710,"duration_ms":264738,"concrete_test":"For the logistic utility u_y(s) = −log(1 + e^{−ys}), no separable regularization (φ ≡ 0), Gaussian centroid θ ∼ N(0, I_N), and λ = 1, numerically compute the gap Δ = inf_{r≥0} sup_{ρ≥0} Φ⋆_u(ρ,r) − sup_{ρ≥0} inf_{r≥0} Φ⋆_u(ρ,r) at (α, β, κ) = (1, 1, 1), (3, 1, 1), (1, 100, 1), and (3, 100, 0.01). Use a global optimizer (e.g., differential_evolution with population size 50 and maxiter 2000) on a fine (ρ, r) grid, with inner optimizations over (q, m, δ, h) performed by L-BFGS-B at tolerance 1e−12. If any computed |Δ| exceeds 1e−6 on the scale of Φ⋆, the exactness claim fails in that region and the manuscript should explicitly limit its claims to the two-sided bounds. If Δ = 0 at all points, the result increases confidence in the commutation but does not prove it; a further analytic condition (e.g., monotonicity of ∂Φ⋆/∂ρ in r) would be needed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 1, as a pair of variational bounds, is supported by a coherent adaptive-interpolation argument: the upper bound uses Jensen on the convex Λφ with Gibbs-averaged (ρ, q, m) velocities, and the lower bound uses Jensen on the concave Ψ and Sion's theorem for the inner (q, m)/(δ, h) exchange. The potentially problematic step in the lower bound's remainder is neutralized because the ODE choices make β(β ṙ − δ̇) − ⟨S11⟩ and β²ṙ − ⟨S12⟩ have zero mean, so the remainder vanishes by concentration of Q11, Q12, M1 and bounded second moments of S11, S12, H1. The load-bearing weakness is elsewhere: the paper's headline claim that the model is 'solved' requires sup_{ρ≥0} inf_{r≥0} Φ⋆(ρ,r) = inf_{r≥0} sup_{ρ≥0} Φ⋆(ρ,r). This equality is not proven and does not follow from the convex–concave structure of Φ in (q, m) and (δ, h), since Sion only applies to the inner optimizations. The sole evidence for commutation is visual inspection of a saddle on a 50×50 grid for logistic and smooth-L1 losses. All downstream results—Corollaries 1 and 2, Proposition 2, and the training/generalization formulas—assume this matching for the original and perturbed utilities. If the two outer optimizations do not commute in some regime, the thermodynamic limit may fail to exist, the variational value may select the wrong order, and the derived observables would be unjustified. The paper is honest about this gap, but the conditional nature of the central exactness claim is precisely what should gate full acceptance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the finite-temperature quenched pressure of a continuous-spin perceptron trained on Gaussian-mixture data with labels, under a concave utility and a separable log-concave prior spin measure. The main result, Theorem 1, gives a lower bound liminf_N p_N >= sup_rho inf_r Phi*_u(rho,r) and an upper bound limsup_N p_N <= inf_r sup_rho Phi*_u(rho,r), where Phi*_u is a scalar variational potential obtained by optimizing four auxiliary parameters. The two bounds coincide whenever the outer sup/inf over (rho,r) commute; the paper then identifies the thermodynamic limit and, by differentiating the variational formula, derives the training loss (Corollary 1), generalization error (Corollary 2), and zero-temperature ground-state energy (Proposition 2). The proofs use adaptive interpolation with ODE-controlled paths, log-concavity (Prekopa-Leindler, Brascamp-Lieb), concentration estimates, and Sion's minimax theorem for the inner optimizations. Numerical saddle plots are provided for logistic and smooth-L1 losses.","tokens_in":44,"tokens_out":19614,"duration_ms":560148,"significance":"If the bounds are correct, the paper supplies a unified finite-temperature variational formulation for a non-Bayes-optimal perceptron with general log-concave priors, going beyond the usual Gaussian-prior/cavity setting. The proof is substantial and largely self-contained: the interpolation sum rule, the convexity analysis, the ODE existence criterion, and the concentration appendices are detailed. The authors are also honest: the exact-solution claim is explicitly conditioned on the unproven exchange of two scalar optimizations, and the observable formulas carry further local-matching assumptions. The main value is therefore the unconditional two-sided bounds plus a plausible variational route to exactness; unconditional exactness itself remains open.","major_comments":[{"comment":"The exact-solution claim, and consequently Corollaries 1 and 2, the matching version of Proposition 2, and Remark 3, rest on the exchange sup_{rho>=0} inf_{r>=0} Phi*_u(rho,r) = inf_{r>=0} sup_{rho>=0} Phi*_u(rho,r). This equality is not proven, and it does not follow from the Sion-minimax steps in Section 3.3, which only exchange the inner (q,m) and (delta,h) optimizations. The paper explicitly acknowledges the gap, but because all formulas for the training loss, generalization error, and ground-state energy assume the exchange, this is the load-bearing point of the paper rather than a cosmetic limitation. The authors should either prove the exchange in a nontrivial regime (for example lambda=0 random labels, Gaussian prior, small alpha, or high temperature) or reformulate the abstract and Section 2 so that the 'solution of the model' is presented as a conjecture conditional on an explicit, unverified assumption.","section":"Section 2, Theorem 1 and following paragraph"},{"comment":"The matching assumption in Theorem 1 is used at a single parameter point for u, but the training-loss formula (2.21) requires the two bounds to match for u_gamma = gamma u on an interval gamma in (1-epsilon,1+epsilon), and the generalization-error formula (2.23) requires matching for all gamma in [0,eta) and t in [-eta,eta] of the perturbed utility gamma(u_y + t W_y). These are strictly stronger hypotheses than the matching checked for the unperturbed u, and the numerical section provides no check of them. Since (2.21) and (2.23) are presented as the paper's learning-theoretic output, the authors should either verify these local matching conditions in at least the numerical settings considered in Section 4, or state explicitly that these predictions are conjectural.","section":"Corollaries 1 and 2, Section 2"},{"comment":"The evidence for the required exchange is visual inspection of a saddle surface on a 50x50 grid. A saddle in the (rho,r) plane does not by itself imply sup_rho inf_r F = inf_r sup_rho F for a function with multiple stationary points, and no quantitative value of the difference between the two orders is reported. The sentence 'Consequently, the lower and upper bounds in Theorem 1 coincide in these regimes' is therefore stronger than the data support. Please report the numerical gap, and if it is nonzero at grid resolution, rephrase the conclusion as evidence for a conjecture.","section":"Section 4, Figures 1 and 2"}],"minor_comments":[{"comment":"The apparent factor-of-2 discrepancy between the sum rule (3.15) and the upper-bound expression (3.49) is resolved if (3.15) is read as beta/2 integral [delta_dot rho_dot - beta r_dot(rho_dot - q_dot)] dt; this reading is forced by the lower-bound chain (3.54). Please typeset the fraction unambiguously so that it is clear the 1/2 multiplies the whole bracket.","section":"Equation (3.15) versus (3.49)"},{"comment":"There are several typos and spelling inconsistencies: 'ceneterd' should be 'centered', 'nor impossible' should be 'not impossible', 'Prekopa-Leindler' should be 'Prekopa-Leindler', 'Gronwall' should be 'Gronwall', and 'Cauchy-Schwartz' should be 'Cauchy-Schwarz'.","section":"Section 2 and appendices"},{"comment":"The set T_r is first introduced in (3.50) as 'a compact convex set to be chosen later', but the actual choice [0,alpha C] x [-sqrt(alpha r), sqrt(alpha r)] appears only after Eq. (3.57). Please define T_r explicitly before its first use.","section":"Section 3.3, Eq. (3.50) and Eq. (3.57)"},{"comment":"The claim that logistic regression with L2 regularization exhibits a learning transition at alpha=2 is stated without a citation at that point; please add a reference there.","section":"Section 4, first paragraph"},{"comment":"The color-scale ranges differ across panels, which makes visual comparison of the saddle structure difficult; please use a consistent scale or explain why the ranges differ.","section":"Figures 1 and 2 captions"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest about the conditional nature of the exactness claim, and the two-sided bounds are supported by a detailed proof. The main editorial risk is that the abstract and Section 2 may be read as an unconditional solution of the model, while the exactness, training loss, and generalization formulas all depend on an unproven exchange of two scalar optimizations and on additional local matching hypotheses that are not numerically tested. If the journal accepts conditional results backed by numerical evidence, the paper is a strong contribution; otherwise, the authors should be asked to prove the exchange in a special case or to reframe the central claims as conjectural. I saw no evidence of misconduct; the self-citations are relevant to the topic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a serious piece of work, and its central theorem is a real result: two-sided variational bounds for the quenched pressure of a continuous-spin perceptron on Gaussian-mixture data, for concave utilities and log-concave separable priors. The adaptive interpolation argument is adapted to a setting without Nishimori identities, and log-concavity supplies the missing correlation inequalities. The proof is detailed, self-contained in the appendices, and the bounds are honest.\n\nWhat is actually new is the finite-temperature treatment of this non-Bayes-optimal setting with general separable priors, and the observation that the only remaining gap between lower and upper bounds is the order of two scalar optimizations. The stationarity equations of the potential reproduce known cavity equations in the Gaussian-prior random-label limit, and the zero-temperature generalization formula matches existing results. That is a good sanity check and gives the paper credibility.\n\nThe soft spot is exactly the one the authors flag: the exact thermodynamic limit and all downstream corollaries require sup-rho inf-r of the reduced potential to equal inf-r sup-rho. That equality is not proved. Sion's theorem handles the inner (q,m)/(δ,h) exchanges, but not the outer pair. The numerical evidence is a 50x50 grid for two utilities; it is suggestive, not a proof. The paper would be stronger if the abstract and conclusions said 'conditional solution' more loudly, rather than just 'whenever the two optimizations commute' inside the text.\n\nThere is also an apparent factor of two between the sum rule (3.15), which has β∫ δ̇ρ̇, and the upper bound (3.49), which uses βδ̇ρ̇/2. Given that the variational potential and the lower bound consistently use βδρ/2, (3.15) looks like a typo. It should be corrected; as written it confuses.\n\nThis paper is for researchers in the statistical physics of learning and in rigorous interpolation methods. I would send it to a serious referee; the central bounds are worth publishing, and the unproven commutation is a clearly stated open problem, not a hidden flaw. The refereeing should push for the typo fix and for a cleaner separation of the proven bounds from the conditional exactness claims.","headline":"Genuine progress: rigorous variational bounds for a broad perceptron class, but exactness is conditional on an unproven sup-inf exchange and a sum-rule typo should be fixed.","tokens_in":41365,"tokens_out":5672,"would_cite":true,"duration_ms":48093,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["82B44","82C32","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"For perceptrons trained on Gaussian mixtures, the limiting free energy is trapped between two minimax variational bounds that differ only in the order of two scalar optimizations.","keywords":["continuous-spin perceptron","Gaussian mixture data","quenched pressure","variational bounds","adaptive interpolation","log-concavity","generalization error","fixed-point equations"],"falsifier":"Pick any admissible concave utility and log-concave prior, and compute the reduced potential $\\Phi^\\star_u(\\rho,r)$ on a fine grid; if the surface has no saddle point in $(\\rho,r)$, so that $\\sup_\\rho\\inf_r$ differs from $\\inf_r\\sup_\\rho$, then the two bounds cannot match and the exact variational formula in that regime would be false. A concrete candidate is logistic loss with smooth $L^1$ regularization at strong regularization strength, outside the parameter regimes plotted in the paper.","tokens_in":40299,"feed_emoji":"🧠","tokens_out":7206,"duration_ms":65148,"temperature":0.7,"pith_summary":"This paper tries to establish a variational characterization of the thermodynamic limit of a finite-temperature continuous-spin perceptron trained on a Gaussian mixture. The model covers a broad class of concave utility functions (losses) and log-concave separable priors on the weights. The authors prove lower and upper bounds on the quenched pressure; the bounds coincide whenever two scalar optimizations can be exchanged, in which case the pressure, ground-state energy, training loss, and generalization error all follow from one potential. A sympathetic reader should care because this gives a rigorous route to perceptron observables outside the Bayes-optimal setting, where the standard fixed-point identities are derived as stationarity conditions of a single variational function rather than assumed as a self-consistency system.","feed_headline":"Perceptron free energy pinned between two minimax bounds","feed_subtitle":"If two scalar optimizations commute, one variational potential gives the thermodynamic limit and the learning observables.","key_machinery":"The load-bearing object is the variational potential $\\Phi_u(\\rho,q,m,r,\\delta,h)$, built from two scalar channel functions: a pattern channel $\\Psi^\\beta_u(\\rho,q,m)$ and a spin channel $\\Lambda_\\phi(r,\\delta,h)$, glued by quadratic and centroid-aligning terms. The central identity is the sum rule of Proposition 3, which expresses the interpolating pressure as an endpoint potential plus an integrated remainder. The argument is carried by adaptive interpolation: an interpolating Hamiltonian connects the true model at $t=0$ to a gas of independent scalar channels at $t=1$, and differentiating along the path yields the sum rule whose remainder vanishes by concentration estimates. Log-concavity, via Pr\\'ekopa-Leindler and Brascamp-Lieb inequalities, supplies the convexity needed to apply Jensen and minimax arguments and, crucially, the sign inequality $\\langle S_{12}\\rangle_t\\ge\\langle S_{11}\\rangle_t$ that keeps the lower-bound interpolation path inside the physical region.","core_discovery":"On its own terms, the paper's central claim is Theorem 1: for any inverse temperature, sampling ratio, and signal strength satisfying hypotheses H1-H4, the liminf of the quenched pressure is at least $\\sup_{\\rho\\ge 0}\\inf_{r\\ge 0}\\Phi^\\star_u(\\rho,r)$, and the limsup is at most $\\inf_{r\\ge 0}\\sup_{\\rho\\ge 0}\\Phi^\\star_u(\\rho,r)$, where $\\Phi^\\star_u$ is the reduced variational potential obtained by optimizing the full potential over overlaps and conjugate fields. The two bounds differ only in the order of the maximum over the self-overlap and the minimum over the channel parameter; all remaining extrema are exchanged through convexity in $(\\delta,h)$ and concavity in $(q,m)$. Whenever these two outer optimizations commute, the thermodynamic limit exists and equals the common variational value, and the fixed-point equations of the cavity computation are recovered as stationarity conditions of the same potential. The paper also derives the zero-temperature ground-state energy and, under local matching conditions, explicit formulas for training loss and generalization error, including the standard $Q(\\lambda\\tilde{m}/\\sqrt{\\tilde{\\rho}})$ expression at zero temperature.","pith_inferences":["If a general commutativity criterion were found, the same potential would unify the free energy, ground-state energy, training loss, and generalization error for the whole H1-H4 class, making the fixed-point equations a derived rather than assumed object.","A concrete computational probe suggested by the variational structure is to evaluate the gap $\\sup_\\rho\\inf_r\\Phi^\\star_u(\\rho,r)-\\inf_r\\sup_\\rho\\Phi^\\star_u(\\rho,r)$ for admissible losses outside the plotted set, such as quadratic or smoothed-hinge utilities; a nonzero gap would mark regimes where the exact formula fails while the bounds remain valid.","The reliance on log-concavity indicates the method likely extends to other log-concave prior families, but not to non-log-concave priors without a new sign mechanism to replace the inequality $\\langle S_{12}\\rangle_t\\ge\\langle S_{11}\\rangle_t$.","The authors' multilayer discussion suggests low-rank multilayer perceptrons as a testbed where a finite set of overlaps might close the variational description, though this would require the convex structure to survive composition of layers."],"forward_implications":["If the two outer optimizations commute, the thermodynamic limit of the quenched pressure exists and equals one scalar variational value, and this does not require uniqueness of the full stationary-point system.","The same potential yields the cavity fixed-point equations as stationarity conditions, so the usual obstruction to an exact formula moves from uniqueness of a fixed-point system to exchangeability of two explicitly identified optimizations.","At zero temperature the bounds reorganize into a ground-state variational principle, and the generalization error takes the form $Q(\\lambda\\tilde{m}/\\sqrt{\\tilde{\\rho}})$, matching the known Gaussian-mixture classification result while allowing general log-concave priors.","For perturbed utilities, training loss and generalization error are obtained by differentiating the pressure, giving explicit scalar formulas in terms of the variational order parameters under local matching assumptions.","Numerical evaluation of the reduced potential for logistic loss with $L^2$ and smooth $L^1$ regularization shows a saddle structure in $(\\rho,r)$ across several $\\alpha,\\beta,\\kappa$, supporting commutativity in these regimes."],"supporting_citations":[{"why":"Supplies the adaptive interpolation scheme used to build the sum rule and the two-sided bounds.","marker":"[30]"},{"why":"Develops the adaptive interpolation method for proving replica formulas in settings beyond the present one.","marker":"[31]"},{"why":"Introduced the interpolation technique whose evolution this proof uses.","marker":"[28]"},{"why":"Provides the Guerra-type bounds that the adaptive construction refines.","marker":"[29]"},{"why":"Supplies the log-concavity and Brascamp-Lieb estimates underlying Lemma 2 and the moment controls.","marker":"[36]"},{"why":"Gives the cavity fixed-point equations that the variational potential recovers as stationarity conditions.","marker":"[27]"},{"why":"Establishes the role of regularization in Gaussian-mixture classification whose zero-temperature generalization formula is reproduced.","marker":"[16]"},{"why":"Proved the Gardner formula for a continuous-spin model that the present analysis extends to general log-concave priors.","marker":"[24]"},{"why":"Sion's minimax theorem justifies exchanging the inner extrema in the variational potential.","marker":"[45]"},{"why":"Provides the convexity argument that lets derivatives of the pressure be exchanged with the thermodynamic limit.","marker":"[49]"}],"fun_headline_variants":["Minimax swap pins perceptron free energy","Two bounds, one swap: perceptron solved when they agree","Perceptron bounds differ only by optimizer order","Commuting optimizations reveal exact perceptron pressure","Unified potential yields perceptron ground state and loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The lower-bound construction needs the inequality $\\langle S_{12}\\rangle_t\\ge\\langle S_{11}\\rangle_t$ to hold along the whole interpolation, a log-concavity consequence requiring the utility to be concave and the prior convex; and the exact formula additionally requires the supremum over $\\rho$ and the infimum over $r$ to be exchangeable, which the paper does not prove in general.","fun_headline_variants_meta":{"raw":{"variants":["Minimax swap pins perceptron free energy","Two bounds, one swap: perceptron solved when they agree","Perceptron bounds differ only by optimizer order","Commuting optimizations reveal exact perceptron pressure","Unified potential yields perceptron ground state and loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1434,"prompt_tokens":930,"completion_tokens":504,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":546,"tokens_out":504,"duration_ms":4869,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:37:23.698652+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick any admissible concave utility and log-concave prior, and compute the reduced potential $\\Phi^\\star_u(\\rho,r)$ on a fine grid; if the surface has no saddle point in $(\\rho,r)$, so that $\\sup_\\rho\\inf_r$ differs from $\\inf_r\\sup_\\rho$, then the two bounds cannot match and the exact variational formula in that regime would be false. A concrete candidate is logistic loss with smooth $L^1$ regularization at strong regularization strength, outside the parameter regimes plotted in the paper.","supporting_citations":[{"cited_title":"The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference","cited_arxiv_id":null,"evidence_quote":"Supplies the adaptive interpolation scheme used to build the sum rule and the two-sided bounds."},{"cited_title":"The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models","cited_arxiv_id":null,"evidence_quote":"Develops the adaptive interpolation method for proving replica formulas in settings beyond the present one."},{"cited_title":"The Thermodynamic Limit in Mean Field Spin Glass Models","cited_arxiv_id":null,"evidence_quote":"Introduced the interpolation technique whose evolution this proof uses."},{"cited_title":"Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model","cited_arxiv_id":null,"evidence_quote":"Provides the Guerra-type bounds that the adaptive construction refines."},{"cited_title":"Springer, 2010","cited_arxiv_id":null,"evidence_quote":"Gives the cavity fixed-point equations that the variational potential recovers as stationarity conditions."},{"cited_title":"The Role of Regularization in Classification of High-dimensional Noisy Gaussian Mixture","cited_arxiv_id":null,"evidence_quote":"Establishes the role of regularization in Gaussian-mixture classification whose zero-temperature generalization formula is reproduced."},{"cited_title":"On the Stability of the Quenched State in Mean Field Spin Glass Models","cited_arxiv_id":null,"evidence_quote":"Provides the convexity argument that lets derivatives of the pressure be exchanged with the thermodynamic limit."}],"review_version":2}