{"id":"1195a580-3dfa-4c79-90a2-aac022970bfe","arxiv_id":"2506.15199","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A learned linear solver converges only to the projection of the true solution operator onto the training-data function space, and richer polynomial training data can actually increase the finite-difference parameter bias.","lead":"This paper proves that machine-learning models trained on one family of force functions can look perfect on their training data yet fail catastrophically on inputs from another family, because linear models only learn the part of the physics that lies inside their training inputs. It tests eight model classes on a 25-dataset benchmark and proposes cross-family evaluation as a standard check for scientific machine learning.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.1's proof silently replaces the discrete grid loss with a continuous integral, so its exact bias formula and the 'richer data hurts' monotonicity are not established for finite Δx; the asymptotic O(Δx^q) floor likely survives, but the theorem is overstated as an equality.","rationale":"The linear-model projection ceiling in Theorem 3.2 is stable: for any finite training set with N ≥ p + 1 and full-rank coefficient covariance, the normal equations force W B = A B, so the converged weights equal A P + W_0(I − P); the iid assumption is stronger than needed, and the empirical cross-generalization maps in Figs. 4–8 are consistent with this and replicated across seeds and frameworks. The genuine soft spot is Theorem 3.1's proof. The paper says it is giving an a priori estimate, but Eq. (20) replaces the discrete sum from Eq. (7) with a continuous integral, and Eq. (8) asserts an equality with constants µ_m. This is not a harmless notational choice: at finite N_grid the minimizer is a ratio of discrete sums, not of integrals, so the closed-form bias and the strict monotonicity in p are not established for the actual loss. The reported overestimate in Fig. 2 is consistent with this gap. The Taylor-truncation floor O(Δx^q) remains a valid asymptotic statement, and the empirical FD results in Figs. 4a and 13 independently show the trend, so the central function-space ceiling argument is not overturned. The right verdict is therefore unchanged from the reader's CONDITIONAL: the exact quantitative claims of Theorem 3.1 need to be repaired or weakened, and the discrete minimizer should be checked and ideally shipped with code before the paper's rigorous framing is fully accepted.","tokens_in":18580,"tokens_out":11032,"duration_ms":119507,"concrete_test":"Recompute the exact minimizer of the discrete loss in Eq. (7) for the three-point stencil with N_grid = 22 and polynomial degrees p = 2 and p = 3, solving the scalar normal equation Σ_i [w/Δx^2 D_i − f_i] (1/Δx^2 D_i) = 0, where D_i = u_{i−1} − 2u_i + u_{i+1} and u, f are generated from the paper's Dirichlet solutions on the grid. Compare the resulting w to Eq. (35) and to the analogous p = 3 formula. If the discrete minimizer differs from the integral-based prediction by more than the overestimate already visible in Fig. 2, then Eq. (8) fails as an exact statement about the training loss and should be weakened to an asymptotic bound; if it matches to numerical precision, the integral substitution is benign for these grid sizes and the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is in the proof of Theorem 3.1, one of the paper's two theoretical pillars. The loss actually minimized in Eq. (7) is a sum over grid points x_i = iΔx; the proof in App. A.1 silently replaces this with the continuous integral L = E_c[∫_0^1 (w Σ b_m x^m − k Σ c_m x^m)^2 dx] in Eq. (20). This substitution changes the normal equations: the minimizer of the true discrete loss depends on discrete moments Σ_i x_i^k, not on the continuous inner-product constants 1/(m+j+1). Consequently, the closed forms in Eqs. (31)–(36) and the claim that each added polynomial degree strictly increases |w−k| are not proven for the stated training loss at finite Δx. The paper itself reports in Fig. 2 that the theoretical curve overestimates the measured error, which is consistent with this gap. What still stands is the asymptotic O(Δx^q) floor from Taylor truncation, and Theorem 3.2's projection formula is robust: it follows from the normal equations W B = A B for any full-rank finite sample, so the central 'function-space ceiling' narrative is not broken. However, one of the two 'rigorous' quantitative theorems is overstated as an exact equality rather than a bound or asymptotic statement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies generalization of machine-learning models for linear differential equations on 1D/2D Poisson and biharmonic problems. It proposes two theoretical results: (i) a finite-difference parameter fit has an irreducible bias of order Delta x^q that grows with the polynomial degree p of training data (Theorem 3.1), and (ii) a linear model trained by gradient descent converges to the projection of the true discrete solution operator onto the training-data subspace (Theorem 3.2). It then presents extensive cross-evaluation heatmaps for seven model families, showing that out-of-subspace test errors are large even when training MSE is near zero; introduces a Green's-function-based interpretability diagnostic; and recommends cross-set validation as a benchmark.","tokens_in":18715,"tokens_out":11959,"duration_ms":114588,"significance":"Theorem 3.2 is the strongest result: it is a parameter-free exact characterization of the learned weights, and it cleanly explains why near-zero training error does not imply operator recovery. The empirical comparison across finite-difference fits, linear models, MLPs, DeepONets, FNOs, PINNs, and PI-DeepONets is unusually broad, and the proposed cross-evaluation methodology is a practical benchmarking tool. The Green's-function extraction from trained weights is a useful diagnostic. The main intellectual contribution---that the training function space, not data quantity alone, sets the generalization ceiling for linear models---is significant for the SciML community, provided the issues with Theorem 3.1 are corrected.","major_comments":[{"comment":"The proof of Theorem 3.1 replaces the discrete grid loss of Eq. (7) with the continuous integral in Eq. (20) without justification. The minimizer of the discrete loss is governed by discrete moments such as sum_i x_i^k, not by the continuous values 1/(m+j+1); therefore the closed forms in Eqs. (28)-(31) and the claim that each added polynomial degree strictly increases |w-k| are not proven for the actual training loss at finite Delta x. The asymptotic O(Delta x^q) floor is plausible from Taylor truncation, but the theorem is stated as an exact equality and is overstated as written. Please restate Theorem 3.1 as an asymptotic result or provide a discrete proof.","section":"App. A.1, Theorem 3.1"},{"comment":"The expansion FD_q(u,Delta x) = sum_{m=0}^{q-1} c_m x^m + sum_{m=q}^p c_m(1+zeta_m Delta x^m)x^m does not describe the action of a standard centered stencil on monomials. For the three-point stencil applied to u=x^4, the truncation error is 2 Delta x^2, an additive constant, not a perturbation of the x^4 coefficient; in general the error contains lower-degree terms. Consequently Eq. (31) is not the exact minimizer of Eq. (7), which is consistent with the overestimation reported in Fig. 2. The theorem should be re-derived from the actual stencil expansion, or explicitly presented as a leading-order approximation.","section":"App. A.1, Eq. (18)"},{"comment":"Section 4.1 says the linear model is trained with W0=0 and no regularization, but App. B states that all models use AdamW with weight decay lambda=0.01. If the linear-model runs used weight decay, they minimize a regularized loss rather than the unregularized loss (10) in Theorem 3.2, so the near-machine-precision agreement in Fig. 2 (left) would not be expected from the theorem. Please clarify the configuration actually used; if weight decay was present, either report unregularized runs or quantify the effect of the regularization term.","section":"Sec. 4.1 vs. App. B"}],"minor_comments":[{"comment":"The summation index starts at m=-2; it should start at m=0 for the stated polynomial solutions.","section":"Eq. (16)"},{"comment":"The phrase 'Assuming the singular values of B are identical' is unnecessary and generally false for Vandermonde matrices; the convergence argument holds componentwise for any positive diagonal D^2 and the assumption should be removed.","section":"App. A.3"},{"comment":"The preamble says the relative error 'can be bounded from above,' but Theorem 3.1 is stated as an equality; align the wording after the theorem is revised.","section":"Sec. 3.2"},{"comment":"The phrase 'the analytical assumptions overestimate the error' undercuts the equality claim in Theorem 3.1; clarify whether the curves are bounds or exact predictions.","section":"Fig. 2 caption"},{"comment":"Consider adding a code/data availability statement for reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper is a strong empirical and theoretical contribution for a machine-learning-for-physics audience, but the proof of Theorem 3.1 and the regularization inconsistency in the linear-model experiments are real issues that should be addressed before publication. The central Theorem 3.2 appears sound. I do not see novelty disclosure concerns beyond standard prior work on Green's function learning and neural operators."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nIf you work on learned PDE solvers, this paper is worth a careful read. The core message is that for linear operator learning, cross-family generalization is capped by the projection of the true operator onto the training-data span (Thm 3.2), and that finite-difference parameter fitting has an irreducible O(Δx^q) bias that grows with the polynomial degree of the training data (Thm 3.1). That second result is genuinely counterintuitive and new, as far as I can tell. The 25-dataset × 8-model cross-evaluation study is broad, replicated across seeds and frameworks, and the theory curves are computed from the derived closed forms, not fitted to the data. The Green's function extraction trick is a nice diagnostic that could catch cases where a model looks good on the training set but hasn't learned the operator.\n\nThe central projection barrier holds up. Theorem 3.2 is just the normal-equations minimizer for a linear model, but that is the right thing to prove, and the presentation makes the 'function-space ceiling' concrete and data-quantity-independent. The authors are honest about what is prediction and what is measurement.\n\nThe main soft spot is Theorem 3.1. The loss being minimized in Eq. (7) is a sum over grid points, but the proof in App. A.1 replaces it with a continuous integral. The minimizer of the discrete loss depends on discrete moments Σ_i x_i^k, not the continuous 1/(m+j+1), so the exact closed form and the strict 'richer data hurts' monotonicity are not proven at finite Δx. The asymptotic O(Δx^q) floor from Taylor truncation almost certainly survives, and the empirical trend in Fig. 2 matches, but the theorem is stated as an equality and is stronger than the proof supports. The authors should either prove the discrete version, restrict the claim to an asymptotic/bound statement, or explicitly acknowledge the substitution.\n\nTwo minor complaints. The leap from linear theory to 'even the strongest physical priors may not mitigate the impact of the training data distribution' is too strong; the evidence is two physics-informed models, and while the analogy is plausible, it is not a theorem. And the paper ships no code or data, which hurts reproducibility for a benchmark proposal.\n\nWho is this for? Anyone evaluating neural operators or physics-informed methods, and anyone claiming OOD generalization in SciML. It deserves serious peer review; I would accept with the Theorem 3.1 caveat addressed, and I'd encourage the authors to release artifacts.","headline":"A useful paper on a real problem, with one theorem overstated; the projection barrier is solid, the finite-difference bias formula needs a caveat.","tokens_in":19411,"tokens_out":2141,"would_cite":true,"duration_ms":19805,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65N06","65N12"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that for linear differential equations, gradient-descent training drives a model only onto a projection of the true solution operator, so the subspace spanned by training data, not data quantity or grid fineness…","keywords":["operator learning","generalization bounds","Green's function","finite differences","function-space generalization","physics-informed neural networks","mechanistic interpretability","out-of-distribution generalization"],"falsifier":"Train a linear model on many samples of degree-$p$ polynomial forcings at a fixed fine grid, then test on a sine forcing of higher frequency: the theorem predicts the test MSE cannot fall below the squared projection residual $\\|A - A U U^T\\|_F^2$ no matter how many samples, so finding a test MSE below that bound would refute Theorem 3.2.","tokens_in":18173,"feed_emoji":"📐","tokens_out":7653,"duration_ms":76620,"temperature":0.7,"pith_summary":"This paper asks why machine-learning models of physics can show near-zero training error and still fail completely on new inputs. Its answer, proved for the simplest linear case and demonstrated across many architectures, is that the function space the training data fill, not just how much data there is or how fine the grid is, sets the ceiling on what the model can learn. A linear model trained by gradient descent converges to a projection of the true Green's-function operator onto the training subspace; the leftover part of the operator is invisible to training and cannot be recovered by more data or finer discretization. For a finite-difference parameter fit, the paper proves an irreducible bias of order $\\Delta x^q$ that actually grows with the polynomial degree of the training data. The same subspace barrier appears empirically in deep linear networks, MLPs, DeepONets, Fourier neural operators, and physics-informed variants, along with an interpretability method that extracts the learned Green's function from the weights.","feed_headline":"Models of physics learn only the subspace their data span","feed_subtitle":"A projection identity shows low training error can hide missing physics; cross-family tests expose it.","key_machinery":"The load-bearing object is the discretized Green's function matrix $A_{ij} = \\int_0^1 G(x_i,s)\\psi_j(s)\\,ds$, defined by expanding the forcing in the grid basis $\\psi_j$; it is the exact solution operator the model is trying to learn. The argument runs through the subspace-projection identity $W^* = A U U^T + W_0(I-UU^T)$, where $U$ comes from the SVD of the basis-evaluation matrix $B$, together with the Taylor-expansion coefficients of finite-difference stencils that produce the $\\Delta x^q$ bias terms. These two pieces turn training dynamics and parameter fitting into linear algebra, which is what makes the generalization ceiling computable a priori.","core_discovery":"The paper's central discovery is an exact characterization of what a linear model learns from solution-forcing pairs. When forcing functions are drawn from a $p$-dimensional subspace with basis matrix $B$, gradient descent on the mean-squared-error loss converges to $W^* = A U U^T + W_0(I - U U^T)$, where $A$ is the discretized Green's-function solution operator and $U$ is the left orthogonal basis of $B$. The learned operator is the true operator projected onto the training subspace, plus the part of the initial weights orthogonal to that subspace; the projection residual $A - A U U^T$ is invisible to the training loss and cannot be removed by adding more samples or refining the grid. In parallel, for a finite-difference parameter fit the paper proves that the relative parameter error carries an irreducible $O(\\Delta x^q)$ stencil-truncation term that grows as the polynomial degree of the training data increases, so richer training data can actively hurt. The same function-space barrier is then shown empirically across deep linear networks, MLPs, DeepONets, Fourier neural operators, and physics-informed variants, together with a mechanistic check that recovers Green's-function structure from the weights of models that do generalize.","pith_inferences":["A practical consequence the authors leave implicit is that data collection for scientific machine learning should target broad-spectrum or piecewise-rough forcings rather than smooth analytic families; their FEM-dataset results suggest such data can recover the full operator.","For real applications where forcing coefficients are correlated or spectrally decaying, the exact constants in Theorem 3.1 will change, but the qualitative claim that the training function space sets the generalization ceiling should persist, and this is testable by rerunning the heatmaps with real-world spectra.","The projection view suggests a diagnostic for large neural operators: compute the numerical rank of the training input distribution; if it is far below the spatial resolution, out-of-distribution failure is expected no matter how low the training loss.","By replacing the linear operator with the Jacobian of a nonlinear map, the projection identity would predict which tangent directions are learned, connecting the result to transfer learning for nonlinear dynamics."],"forward_implications":["Near-zero training error is not evidence that a model has learned the underlying physics; a converged model can be blind to every component of the operator that lies outside the training subspace.","Finer grids do not automatically help operator learning, because the training subspace is a smaller slice of a larger space of possible forcings, while finite-difference parameter error falls only at the stencil's truncation rate $\\Delta x^q$.","Richer training data can be harmful: for a finite-difference fit, each polynomial degree beyond the stencil order adds another positive $\\Delta x^m$ term to the parameter bias.","A cheap cross-subspace evaluation, training on one function family and testing on the others, reveals whether a model has subspace generalization, producing the block lower-triangular error structure seen in the experiments.","Extracting Green's functions by one-hot impulse probes gives a visual and algebraic diagnostic: generalized models show the correct kernel structure, and their learned inverse recovers a local stencil."],"supporting_citations":[{"why":"Defines the DeepONet operator architecture used as a baseline for subspace generalization.","marker":"Lu et al. 2021a"},{"why":"Provides the Fourier Neural Operator formulation and implementation used in the comparative experiments.","marker":"Kovachki et al. 2023"},{"why":"Supplies the physics-informed DeepONet loss that exhibits the same subspace limitation.","marker":"Wang et al. 2021"},{"why":"DeepXDE library used to train the PINN, DeepONet, and physics-informed DeepONet models.","marker":"Lu et al. 2021b"},{"why":"PySINDy package used for the finite-difference parameter-fitting experiments supporting Theorem 3.1.","marker":"Kaptanoglu et al. 2022"},{"why":"Prior Green's-function-based data-efficiency result that motivates treating the Green's function as the theoretical anchor.","marker":"Boullé et al. 2023"}],"fun_headline_variants":["Linear models learn only your data's subspace, not the physics","The unsampled physics is invisible to training loss","More data can't fix a missing function space","Green's function emerges in weights only when data covers it","Cross-family tests expose gaps that training loss hides"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The exact formulas assume the training forcing functions are random combinations of basis functions with zero mean, independent coefficients, and identical variance across all basis degrees; real data with correlated or decaying spectra would shift the constants, and the transfer of the mechanism to nonlinear networks is demonstrated empirically rather than proved.","fun_headline_variants_meta":{"raw":{"variants":["Linear models learn only your data's subspace, not the physics","The unsampled physics is invisible to training loss","More data can't fix a missing function space","Green's function emerges in weights only when data covers it","Cross-family tests expose gaps that training loss hides"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000372,"raw_usage":{"total_tokens":1982,"prompt_tokens":932,"completion_tokens":1050,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":973}},"tokens_in":548,"tokens_out":1050,"duration_ms":10908,"temperature":1.0,"reasoning_tokens":973,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:45:42.775231+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a linear model on many samples of degree-$p$ polynomial forcings at a fixed fine grid, then test on a sine forcing of higher frequency: the theorem predicts the test MSE cannot fall below the squared projection residual $\\|A - A U U^T\\|_F^2$ no matter how many samples, so finding a test MSE below that bound would refute Theorem 3.2.","supporting_citations":[],"review_version":1}