{"id":"e671a376-e4e4-4899-8cbd-f0460b056068","arxiv_id":"2412.17518","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Two-layer neural operators trained with early-stopped gradient descent achieve the same minimax convergence rates as kernel methods in the neural tangent kernel regime.","lead":"This paper analyzes two-layer neural operators in the neural tangent kernel regime and proves convergence rates for early-stopped gradient descent. If correct, it shows neural operators can match known optimal rates for learning solution operators of PDEs, while specifying how many hidden neurons and function evaluations are needed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed O(sqrt(n_U)) second-stage sample complexity in the well-specified case is not supported by the paper's own proof: for b=1, r=1/2, T=sqrt(n_U), Corollary 3.8 requires n_X = O(n_U), and with n_X=sqrt(n_U) the weight bound B_tau grows like sqrt(T), violating Theorem 3.5's conditions.","rationale":"The reader identified Assumption 3.3 (source condition) as the weakest assumption. That concern is legitimate but standard in RKHS-rate analyses, and the paper explicitly flags the assumption as an a priori smoothness condition. A more decisive problem is internal to the proof: the paper's advertised second-stage sample complexity for the well-specified case is inconsistent with the weight-stability theorem it relies on. The contradiction is concrete and checkable: Corollary 3.8 requires n_X >= n_U (for b=1, r=1/2, T = sqrt(n_U)) to get B_tau = O(log T), whereas the text claims n_X = O(sqrt(n_U)). With n_X = sqrt(n_U), the only available weight bound grows like sqrt(T), which makes the Taylor error and the n_X requirement in Theorem 3.5 too large to deliver the stated rate. This does not necessarily destroy the main rate theorem for all parameter regimes, because larger n_X may restore the proof, but it invalidates the paper's explicit sample-complexity claim as written and requires a correction or a substantially refined argument. The verdict therefore remains CONDITIONAL, but with the condition sharpened: the well-specified sample-complexity claims must be corrected and re-verified, or the theorem must be restated with the n_X dependence that the proof actually supports.","tokens_in":48345,"tokens_out":10492,"duration_ms":102067,"concrete_test":"Re-derive the well-specified case end-to-end with b=1, r=1/2, T = c n_U^{1/2}, testing n_X = sqrt(n_U). First, compute B_tau from Theorem 3.7's closed-form bound. Then check whether the assumptions of Theorem 3.5, especially n_X >= C_tilde B_tau^2 T^{2r} log^2 T and M >= C B_tau^6 log^2(T) T^{1+b(2r-1)}, are satisfied. Finally, verify whether the final excess-risk bound O(T^{-1/2}) follows from the three error terms in (3.9). If the n_X or M condition fails with n_X = sqrt(n_U), the paper's explicit claim 'n_X >= O(sqrt(n_U) log^2 n_U)' is false under its own proof.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's own sufficient conditions contradict the advertised sample complexity. After Corollary 3.6, the text states that in the well-specified case b=1, r=1/2, 'we only need M >= O(sqrt(n_U) log^2 n_U) and n_X >= O(sqrt(n_U) log^2 n_U)'. With T = n_U^{1/(2r+b)} = sqrt(n_U), Corollary 3.8(1) requires, for B_tau = O(log T), the lower bound n_X >= C T^{1+b} log^3(1/delta) = C n_U log^3(1/delta). If instead one takes n_X = sqrt(n_U) = T, then the formula in Theorem 3.7 gives B_delta(1/(alpha T)) = O(sqrt(T)) because N_{L_infty}(1/(alpha T)) = O(T) and the n_X-dependent term sqrt(alpha T / n_X) is O(1). Hence B_tau = O(sqrt(T) log T). Substituting this B_tau into Theorem 3.5's condition n_X >= C_tilde B_tau^2 T^{2r} log^2 T yields n_X >= C_tilde T * T log^4 T = n_U log^4 T, which is not satisfied by n_X = T. The Taylor error term in Proposition B.2 confirms the failure: it is bounded by C B_tau^2 / sqrt(M), which with B_tau = sqrt(T) and M = sqrt(T) is O(sqrt(T)), far larger than the target T^{-1/2}. Thus the proof does not establish the claimed rate with n_X = O(sqrt(n_U)); at best it supports n_X >= n_U polylog(n_U) for the advertised well-specified rate.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies two-layer neural operators trained by gradient descent in the neural tangent kernel (NTK) regime. It defines a vector-valued NTK and its RKHS, proves that any operator approximable by a neural operator with parameters near initialization is also approximable by an element of this RKHS (Theorem 3.2), and then analyzes early-stopped gradient descent. Under a Hölder source condition (Assumption 3.3) and an effective-dimension condition (Assumption 3.4), Theorem 3.5 gives a high-probability bound on the excess risk in terms of the stopping time T, the number of neurons M, the number of first-stage samples n_U, and the number of second-stage discretization points n_X. Choosing T = n_U^{1/(2r+b)} yields the rate n_U^{-r/(2r+b)} (Corollary 3.6), which is minimax optimal in the RKHS framework. Theorem 3.7 and Corollary 3.8 provide bounds on the weight deviation B_τ. A numerical illustration on a one-dimensional Poisson equation is included. The proofs are assembled from an error decomposition into Taylor, discretization, and kernel gradient descent errors, with many technical lemmas in the appendix.","tokens_in":48725,"tokens_out":12581,"duration_ms":102336,"significance":"If the results are correct, the paper would show that two-layer neural operators in the NTK regime achieve the same minimax-optimal rates as RKHS-based nonparametric regression, with explicit bounds on the required width and discretization size. The paper's strengths include a clean error decomposition, a nontrivial approximation theorem linking neural operators to the vvRKHS, and a derivation that is clearly not fitted to data. The main advertised sample-complexity claim for the well-specified case is, however, not supported by the paper's own sufficient conditions, which materially affects the paper's central message. The mathematical scaffolding is mostly solid, but the explicit 'O(√n_U) second-stage samples' claim needs correction, so the paper requires major revision.","major_comments":[{"comment":"","section":"Section 3.1 (after Corollary 3.6); Corollary 3.8(1); Theorem 3.5; Theorem 3.7"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the authors' own prior work (Nguyen and Mücke 2023) for the central kernel gradient descent bound (Proposition B.5) and for several technical lemmas; this is acceptable but makes the novel contribution largely the operator-specific approximation and weight bound analysis. The numerical section is a small illustration without error bars or multiple runs, which is acceptable for a theory paper but should be described as illustrative. The mismatch between the advertised second-stage sample complexity and the proved sufficient conditions is significant and should be fixed before acceptance. The paper's scope fits a statistics or machine learning journal, though the title may overstate the range of the results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: the paper proves real NTK-style generalization bounds for two-layer neural operators, but the advertised n_X = O(sqrt(n_U)) in the well-specified case is not supported by its own proof; the stated sufficient conditions force n_X = O(n_U) up to logs.\n\nWhat is new: they extend the shallow-network NTK analysis to operator-valued outputs, give an approximation theorem connecting neural operators to the vvRKHS, and derive explicit neuron and second-stage sample requirements under source and capacity conditions. The error decomposition is standard but carefully handled; the weight bound Theorem 3.7 is elaborate. The rates themselves match known RKHS minimax rates, so the upper-bound result is plausible under the stated assumptions.\n\nThe soft spot is not the source condition - that's standard, analogous to RKHS smoothness - but the mismatch I mentioned. After Corollary 3.6 they claim M and n_X = O(sqrt(n_U) log^2 n_U) suffice for the well-specified case b=1, r=1/2. But Corollary 3.8(1) requires n_X >= C T^{1+b} log^3(1/delta) = C n_U log^3(1/delta) for B_tau = O(log T). If you instead set n_X = sqrt(n_U), the weight bound B_tau grows like sqrt(T), which violates the condition n_X >= C_tilde B_tau^2 T^{2r} log^2 T in Theorem 3.5. The Taylor error term then blows up. So the proof establishes the rate only when n_X is of order n_U polylog, not sqrt(n_U). That is a large gap, because the sqrt(n_U) second-stage sample count is a headline contribution.\n\nThere's also no lower bound for the neural operator class itself, so 'minimax optimal' should be read as 'matches RKHS lower bounds,' not as a verified statement for this specific hypothesis class. The numerical section shows only a few curves without error bars or code; it is illustrative, not evidence.\n\nOverall: a competent extension with a genuine flaw in a headline claim. A referee should ask for a corrected statement and either a fix or a clear caveat. The paper deserves peer review, not desk rejection, because even the corrected result - n_X = n_U polylog with the stated rates - is a meaningful new result for neural operators.\n\nIf I were advising, I'd send it to review with the expectation of substantial revision, and I'd want the authors to check whether the sqrt(n_U) claim can be rescued by a sharper weight bound.\n\nRegards.","headline":"Neural operator NTK bounds are real, but the advertised sqrt(n_U) second-stage sample complexity is not supported by the paper's own conditions.","tokens_in":49265,"tokens_out":3145,"would_cite":false,"duration_ms":30205,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-layer neural operator trained by early-stopped gradient descent achieves the same excess-risk rate as minimax-optimal RKHS nonparametric regression, provided the true operator satisfies a source condition relative to the…","keywords":["neural operators","neural tangent kernel","operator learning","gradient descent","minimax optimal rates","reproducing kernel Hilbert space","source condition","PDE solution operators"],"falsifier":"Find a target operator $G^*$ that is approximable by neural operators but not in $\\mathrm{ran}(L_\\infty^r)$ for any $r>0$, train the two-layer neural operator with early-stopped gradient descent, and check whether the empirical excess risk decays like $n_U^{-r/(2r+b)}$; slower decay would locate the failure in the source condition.","tokens_in":48108,"feed_emoji":"🧠","tokens_out":7950,"duration_ms":71682,"temperature":0.7,"pith_summary":"Two-layer neural operators trained by early-stopped gradient descent can learn maps between function spaces, such as PDE solution operators, at the same convergence speed as the best nonparametric RKHS methods. The paper establishes an excess-risk bound of order $n_U^{-r/(2r+b)}$ when the true operator satisfies a H\\\"older-type source condition and the NTK integral operator has an effective dimension decaying like $\\lambda^{-b}$. It also gives explicit minimum numbers of hidden neurons and evaluation points needed to realize the rate, and proves that any operator approximable by a neural operator is approximable in the vector-valued RKHS of the neural tangent kernel, which justifies restricting attention to the NTK regime. The rates match known minimax lower bounds in the RKHS framework.","feed_headline":"Two-layer neural operators match RKHS minimax rates","feed_subtitle":"Early-stopped gradient descent matches RKHS minimax rates, with explicit neuron and sample counts.","key_machinery":"The central object is the vector-valued neural tangent kernel (vvNTK) of a two-layer neural operator, $K_M(u,u') = \\sum_{p=1}^P \\partial_p G_{\\theta_0}(u) \\otimes \\partial_p G_{\\theta_0}(u')$, together with its infinite-width limit $K_\\infty$ and the associated integral operator $L_\\infty G = \\int_\\mathcal{U} K_\\infty(u,\\cdot)G(u)\\,\\mu_u(du)$. The source condition $G^* = L_\\infty^r H^*$ measures the smoothness of the target operator, and the effective dimension $N_{L_\\infty}(\\lambda) = \\mathrm{tr}(L_\\infty(L_\\infty+\\lambda I)^{-1}) \\le c_b\\lambda^{-b}$ measures the capacity of the hypothesis space. Early-stopped gradient descent acts as spectral filtering on this operator, reproducing the classical bias-variance trade-off of RKHS regression, while the finite-width and finite-discretization errors are absorbed by taking $M$ and $n_X$ large enough.","core_discovery":"The central claim is Theorem 3.5 and Corollary 3.6: under Assumptions 2.1, 3.1, 3.3 and 3.4, early-stopped gradient descent with stopping time $T = n_U^{1/(2r+b)}$ achieves $\\lVert G_{\\theta_T} - G^*\\rVert_{L^2(\\mu_u)} \\le C\\log^3(2/\\delta)\\, n_U^{-r/(2r+b)}$ with high probability. The proof splits the excess risk into three errors: the Taylor remainder from linearizing the neural operator around its initialization, the finite-width and finite-discretization error, and the kernel gradient descent generalization error. The first is controlled by showing the weights remain in a ball around the initialization, the second by choosing the width $M$ and the number of second-stage samples $n_X$ large enough, and the third by spectral filtering estimates. The paper therefore claims that a shallow neural operator trained by gradient descent is a statistically optimal estimator in the vvNTK-RKHS class, and the 1D Poisson experiments illustrate the predicted plateau at $M \\sim n_X \\sim T \\sim \\sqrt{n_U}$ in the well-specified case.","pith_inferences":["The source condition $G^* \\in \\mathrm{ran}(L_\\infty^r)$ is not checked by any data-dependent procedure in the paper, so a practitioner would need to justify it from the structure of the target operator before quoting the rate.","Because the rates match RKHS minimax bounds, the paper suggests that neural operators inherit the statistical limits of kernel methods; their practical advantage is architectural flexibility rather than faster statistical convergence.","The same error decomposition should extend to deep or Fourier neural operators whenever their tangent kernels converge to a limiting kernel and their finite-width gradients are Lipschitz; verifying that is a direct next step.","The Poisson experiments test only one smooth, well-specified example; a nonlinear PDE with rough inputs would be a sharper check of whether the predicted $M$, $n_X$, and $T$ requirements are tight."],"forward_implications":["With enough neurons and second-stage samples, a two-layer neural operator trained by early-stopped gradient descent is a minimax-optimal estimator for learning PDE solution operators in the NTK regime.","The required width grows polynomially with the sample size: $M \\gtrsim T$ for $r<1/2$, $M \\gtrsim T^{1+b(2r-1)}$ for $1/2 \\le r < 1$, and $M \\gtrsim T^{2r}$ for $r>1$.","The number of second-stage samples only needs to scale roughly as $T^{2r}\\log^2 T$ in the relevant regime, so discretization is not the bottleneck when the smoothness parameter $r$ is small.","In the well-specified case $r=1/2$, $b=1$, the rate is $n_U^{-1/4}$ with only $O(\\sqrt{n_U})$ neurons and $O(\\sqrt{n_U})$ second-stage samples.","Stopping at $T = n_U^{1/(2r+b)}$ is the right schedule: before that the risk decreases at each step with high probability, and at that point the excess risk reaches the minimax-optimal rate."],"supporting_citations":[{"why":"Supplies the kernel gradient descent generalization bound and the refined spectral regularization estimates used for the third error term.","marker":"Nguyen and M\\\"ucke (2023)"},{"why":"Establishes the minimax-optimal rates and the effective-dimension framework in RKHS against which the paper's rates are matched.","marker":"Caponnetto and De Vito (2007)"},{"why":"Provides the universality and approximation results for neural operators that motivate the RKHS approximation theorem.","marker":"Kovachki et al. (2024c)"},{"why":"Gives the random-feature and operator concentration bounds used to control finite-width and finite-sample effects.","marker":"Rudi and Rosasco (2017)"},{"why":"Supplies spectral regularization and operator interpolation inequalities used in the error decomposition.","marker":"Blanchard and M\\\"ucke (2017)"},{"why":"Introduces the neural tangent kernel whose vector-valued analogue is the paper's central object.","marker":"Jacot et al. (2018)"},{"why":"Defines the easy and hard learning problem dichotomy that sets the paper's restriction $2r+b>1$.","marker":"Pillaud-Vivien et al. (2018)"}],"fun_headline_variants":["Neural operators hit minimax optimal convergence","Early-stopped GD makes neural operators minimax","Two-layer neural operators achieve minimax rates","Neural operators proven minimax optimal","Minimax convergence for neural operators via GD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole rate rests on Assumption 3.3: the true operator $G^*$ must lie in the image of a power of the NTK integral operator, $G^* = L_\\infty^r H^*$ with $\\lVert H^*\\rVert \\le R$, and this membership is not verified from data.","fun_headline_variants_meta":{"raw":{"variants":["Neural operators hit minimax optimal convergence","Early-stopped GD makes neural operators minimax","Two-layer neural operators achieve minimax rates","Neural operators proven minimax optimal","Minimax convergence for neural operators via GD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000607,"raw_usage":{"total_tokens":2812,"prompt_tokens":913,"completion_tokens":1899,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":1831}},"tokens_in":529,"tokens_out":1899,"duration_ms":13269,"temperature":1.0,"reasoning_tokens":1831,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:27:06.375825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a target operator $G^*$ that is approximable by neural operators but not in $\\mathrm{ran}(L_\\infty^r)$ for any $r>0$, train the two-layer neural operator with early-stopped gradient descent, and check whether the empirical excess risk decays like $n_U^{-r/(2r+b)}$; slower decay would locate the failure in the source condition.","supporting_citations":[{"cited_title":"Generalization properties of learning with random features","cited_arxiv_id":null,"evidence_quote":"Gives the random-feature and operator concentration bounds used to control finite-width and finite-sample effects."}],"review_version":1}