{"id":"ce1b0fcd-eb39-4f0b-9ac6-f098668067db","arxiv_id":"2508.19052","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A GNN trained on two-node graphs provably extrapolates the first-order finite-volume scheme for the heat equation, and with symbolic regression it rediscovers the update rule and suggests higher-order corrections.","lead":"The paper reports that graph neural networks trained on only two-cell graphs can recover the standard finite-volume update for the heat equation and generalize to large unstructured meshes. It then applies a physics-informed loss and symbolic regression so the networks propose higher-order time-stepping schemes, positioning GNNs as tools for inventing numerical methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.3/C.6's premise is vacuous: with L = MAE + η||θ||0 and ε < η, any model with at least one nonzero parameter has loss ≥ η > ε, and a zero-parameter model cannot fit Dtrain.","rationale":"The reader's weakest assumption pointed to Lemma C.4 and the parameter-counting lower bound. While that is a genuine flaw, the more load-bearing issue is that the theorem's premise is logically unsatisfiable as written. The loss in Eq. (21) includes η||θ||0, and the theorem requires L < ε with ε < η. Because ||θ||0 is integer-valued, this forces ||θ||0 = 0, making the GNN a constant function that cannot have small MAE on Dtrain. Thus Theorem 4.3/C.6 is vacuous. This directly undermines the paper's foundational claim that a GNN trained on two-node graphs is provably guaranteed to extrapolate the FV scheme. Even if Lemma C.4 were fixed, the vacuity remains. The empirical demonstrations and symbolic-regression results may still be interesting, but the central theoretical guarantee, as stated, does not hold. The reader's verdict of reject is therefore unchanged, though for a different reason than the one highlighted in the reader's weakest_assumption.","tokens_in":22751,"tokens_out":4208,"duration_ms":52022,"concrete_test":"Analytically verify the existence condition for Theorem 4.3. Compute L(θ) for the zero-weight network on Dtrain: for any nonzero training graph, e.g. G(1,1,0,0), the output is 0 while F is 1, so L = 1. For any θ with ||θ||0 ≥ 1, L ≥ η. Since ε < η < 1, no θ satisfies L < ε. If this check succeeds, the theorem's premise is empty; the central theoretical claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central OOD guarantee is stated as: if L(Mθ, Dtrain) = LMAE(Mθ, Dtrain) + η||θ||0 < ε, with ε < η, then ||Mθ(G)r − F(G)r||1 < 2ε(1 + 2/Δt) for every graph G (Theorem 4.3 and Theorem C.6). Since ||θ||0 is a count of non-zero parameters, it is a nonnegative integer. From L < ε < η we get η||θ||0 < η, hence ||θ||0 = 0. A zero-parameter network is constant (all weights and biases are zero), so its prediction on the training set is identically zero. But Dtrain contains, for example, G(1,1,0,0), whose FV target is T_i = 1; the MAE on this example is 1, so L = 1 > ε. Thus no Mθ satisfies the hypothesis: the theorem is vacuously true and provides no non-vacuous generalization guarantee. The same vacuousness infects Theorem C.5's claim that ηN_sparse is the global minimum of L. This is independent of the parameter-counting issue in Lemma C.4: even if that lemma were correctly proved, the loss definition still makes the premise unsatisfiable. To have a meaningful theorem, the authors would need to separate the MAE condition from the sparsity count (e.g., state 'if LMAE < ε and ||θ||0 = N_sparse, then ...'), but that is not what is written.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether graph neural networks (GNNs) trained on trivial two-cell graphs can learn and extrapolate finite-volume (FV) discretizations of the heat equation. It states a theorem (Thm 4.3 / C.6) that if a sparse-regularized GNN attains loss ε < η on a two-node dataset, then its prediction error on any unseen mesh is O(ε). It reports experiments with supervised and PINN losses, uses symbolic regression to claim rediscovery of the first-order FV update, and further claims discovery of a 2-hop second-order correction and an explicit midpoint scheme. The empirical design is clean and the symbolic-regression checks are a useful interpretability device, but the theoretical guarantee is vacuous as stated and the PINN loss is inconsistent with the explicit schemes claimed to be discovered.","tokens_in":23341,"tokens_out":9753,"duration_ms":91537,"significance":"If the results were valid, the paper would provide an important bridge between neural algorithmic alignment and numerical analysis, with a concrete route to interpretable scheme discovery. The two-cell training setup, the out-of-distribution and rollout comparisons, and the use of symbolic regression as an inspection tool are all methodologically appealing; I credit the authors for making the target scheme explicit and for checking the learned update symbolically. However, the central theorem is vacuous because of the integer L0 regularizer, the sparsity lower-bound proof has unresolved parameter-count inconsistencies, and the PINN loss in Eq. (17), as written, cannot be minimized by the explicit schemes the paper reports. These are load-bearing issues for the paper's main claims, not presentation concerns.","major_comments":[{"comment":"The premise of Theorem 4.3 is impossible for the stated loss. L(Mθ,Dtrain)=L_MAE+η||θ||0, and ||θ||0 is a nonnegative integer. If L<ε<η, then η||θ||0<η, hence ||θ||0=0. A zero-parameter network has zero weights and biases, so its prediction is constant zero, but Dtrain contains G(1,1,0,0) whose FV target is 1 (Sec. 4.1.3 and Remark 4.1). Thus no model satisfies the hypothesis, and the OOD bound (22) is vacuously true. The same integer-sparsity problem invalidates Theorem C.5's claim that ηN_sparse is the global minimum. To obtain a non-vacuous statement, the authors must decouple the MAE condition from the sparsity count, e.g., state 'if L_MAE<ε and ||θ||0=N_sparse, then ...', and then prove that version.","section":"Theorem 4.3 / Appendix C.6, Eq. (21)"},{"comment":"The parameter count underlying the sparsity argument is internally inconsistent. Lemma C.1 states that the sparse construction uses 2m(L+1)+4 non-zero parameters, but the proof's own construction (2m(L−1) for identity layers, 2m for the aggregation network, and 3+(m−1) for the update network) totals 2mL+m+2. Lemma C.4 then argues for a lower bound of 2mL+m+4 and later refers to 2mL+2. The proof of Lemma C.4 is a sign-preservation heuristic for zero-bias diagonal constructions, not a general lower bound over all ReLU MLP configurations, widths, and layer layouts. Since Theorem C.5 and C.6 both depend on Lemma C.4, the O(ε) transfer to arbitrary graphs is not established even after the vacuity issue is repaired.","section":"Lemmas C.1 and C.4, Theorem C.5"},{"comment":"The unsupervised/PINN claims are inconsistent with the loss as written. Eq. (17) computes the PDE residual using ∇²h_L, i.e., the Laplacian of the network's output (the next state). An explicit first-order FV update, or an explicit midpoint scheme, satisfies (h_L−h0)/Δt ≈ S+α∇²h0, not S+α∇²h_L; substituting the claimed schemes into Eq. (17) leaves a nonzero residual of order Δt or larger. Minimizing Eq. (17) should drive the network toward an implicit update, not toward the explicit schemes reported in Remark 4.4 and Eqs. (26)–(27). The authors need to specify how ∇² is discretized on graphs and on which temperature state it acts; otherwise the symbolic-regression results under PINN training do not support the claim that minimizing Eq. (17) rediscovers the explicit first-order or midpoint schemes.","section":"Eq. (17), Remark 4.4, Sec. 5.2"}],"minor_comments":[{"comment":"Typos and language issues: 'These findings follows', 'performancess', 'majoration constant'. The footnote about 'large modern transformers' is unsupported and not relevant to the paper's claims.","section":"Throughout"},{"comment":"The symbol ∇²h_L is not defined for graphs. Please define the discrete graph Laplacian or the finite-volume operator used, including boundary treatment.","section":"Eq. (17)"},{"comment":"The statement 'cannot have a non-zero error' is ambiguous; presumably it means 'cannot achieve zero MAE' or 'must have non-zero MAE'. Please rephrase.","section":"Lemma C.4"},{"comment":"The symbolic-regression step for the 2-hop aggregator and the midpoint network is described only through final formulas. Please report the Pareto fronts, coefficient values with uncertainties, and the number of training runs to show that the 1/2 coefficient and the midpoint structure are robust discoveries rather than post hoc selections.","section":"Sec. 5.2"},{"comment":"The numerical comparison in Section 4.4 appears to be a single-seed study. Given the small training set, error bars over multiple seeds would strengthen the empirical claim that sparsity, not chance, drives OOD generalization.","section":"Fig. 3"}],"recommendation":"reject","confidential_remarks":"The paper has an interesting empirical idea and the symbolic-regression checks are a useful addition to the literature, but the theoretical core is currently unsound: the main theorem is vacuous because of the integer L0 penalty, the sparsity lower bound has unresolved inconsistencies, and the PINN loss as written contradicts the explicit schemes claimed to be discovered. These are not local presentation issues; they affect the paper's central claims. If the authors can restate and prove a meaningful generalization guarantee and rerun the unsupervised experiments with a consistent residual, a revised manuscript might be viable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this before reading: the paper's headline theorem is vacuous. The loss is LMAE + η||θ||0, and with ε<η, any model satisfying L<ε must have ||θ||0=0, which cannot fit the training data. So Theorem 4.3/C.6 has no non-vacuous content. This isn't a minor gap; it's the central OOD guarantee.\n\nThat said, the empirical core is not nothing. Training a sparse GNN on two-node graphs and seeing it extrapolate to hundreds of cells (Fig. 3) is a neat demonstration, and symbolic regression recovering the first-order FV update from the trained MLPs is a satisfying sanity check. The paper also connects neural algorithmic alignment to continuous FV schemes, which is a reasonable research direction.\n\nThe soft spots are serious. The parameter-count Lemma C.1 gives 2m(L+1)+4, but the construction actually yields 2mL+m+2. Lemma C.4 is a sketch, not a proof. The PINN loss (Eq. 17) uses ∇²h_L (the new temperature), so the explicit forward-Euler scheme does not satisfy it; Remark 4.4 claiming the PINN recovers that scheme is therefore contradictory. And the 'discovered' 2-hop correction (Eq. 26) is dimensionally inconsistent—the second-hop term has no A/δ geometric factor—so it cannot be a valid discretization of the heat equation. There's also no convergence-rate table; Figure 7 is only a residual curve comparison over a few timesteps.\n\nI'd send this to reviewers because the topic is timely and the empirical observation is worth exploring, but they should be prepared to require substantial revisions: fix the loss definition or separate the MAE condition from the sparsity count, correct the parameter count, reconcile the PINN loss, and provide actual convergence tests for the higher-order claims. As it stands, I'd not cite it.\n\nBest.","headline":"Promising empirical idea undermined by a vacuous main theorem and inconsistent theory.","tokens_in":23675,"tokens_out":4741,"would_cite":false,"duration_ms":40670,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65M08","68T07","65M12"],"pacs":[],"model":"deepseek-v4-flash","headline":"A GNN trained on two-node graphs alone provably implements the standard finite-volume scheme for the heat equation: training error ε means O(ε) error on any unseen mesh.","keywords":["graph neural networks","finite volume method","heat equation","symbolic regression","physics-informed neural networks","out-of-distribution generalization","numerical scheme discovery","message passing"],"falsifier":"Find any sparse ReLU GNN fitting the two-node training set to loss below ε with fewer non-zero weights than the claimed N_sparse = 2m(L+1)+4, or with weights outside the zero-bias diagonal family, and show its one-step error on a small unstructured mesh exceeds 2ε(1 + 2/Δt); a pruning or evolutionary search over general weight structures at the training-loss optimum would either confirm or refute the parameter-counting premise.","tokens_in":22673,"feed_emoji":"🔥","tokens_out":11301,"duration_ms":92695,"temperature":0.7,"pith_summary":"This paper tries to establish that graph neural networks can do more than approximate physics: trained correctly, they can rediscover — and even extend — the numerical schemes that solvers use. The central result is a proof that a GNN trained only on graphs of two cells with one shared face, under a sparsity-regularized loss, implements the standard first-order finite-volume (FV) update for the heat equation on any unstructured mesh: training error ε on the minimal dataset implies error at most a constant times ε on every unseen mesh. Symbolic regression confirms the match is exact, recovering the textbook FV flux formula from the network's components, both when training is supervised and when it uses only the PDE residual (a physics-informed loss). Widening the network's receptive field then yields genuinely new schemes: a 2-hop variant learns a second-order correction term, and a two-layer variant rediscovers the classic explicit midpoint (second-order Runge–Kutta) scheme, both more accurate than the first-order baseline. If the claims hold, minimal data plus sparsity can turn a GNN into an automated discoverer of numerical methods with transferable guarantees.","feed_headline":"Two cells of training yield the exact heat-equation scheme","feed_subtitle":"A small training loss provably bounds error on any mesh, and symbolic regression recovers the textbook finite-volume update.","key_machinery":"The argument turns on aligning the GNN's update structure with the finite-volume flux exchange. The message-passing architecture splits into an edge MLP that computes a pairwise flux and a node MLP that sums aggregated fluxes into the temperature update — the same shape as the FV rule T ← T + (Δt/V)Σ flux. A sparsity penalty (L0 in theory, L1 in practice) on the weights, together with a parameter-counting lemma, pins any low-loss network to the minimal sparse configuration, which turns out to be exactly the FV scheme; this pinning is what converts small training error into a global O(ε) bound. The two-node training graph, with geometry chosen so the face-area-to-distance factor cancels the c","core_discovery":"Central claim (Theorem 4.3, appendix C.6): if an L-layer GNN with ReLU MLPs is trained on two-node graphs with a sparsity-regularized loss and achieves loss < ε (ε < η), then for every graph G and node r, ∥Mθ(G)_r − F(G)_r∥₁ < 2ε(1 + 2/Δt), with F the first-order finite-volume heat step. Small training error on the two-cell dataset therefore certifies O(ε) accuracy on any unseen mesh: sparsity forces the weights into the unique minimal configuration that fits the data, which is numerically the FV scheme. Symbolic regression confirms the match exactly — the learned edge function is α(A/δ)(T_N − T_P) and the node update is T_P + (Δt/V_P)Σfluxes + Δt S_P. With a pure PDE-residual (PINN) loss th","pith_inferences":["The transfer bound depends on the parameter-counting lemma, which is proven for a restricted zero-bias diagonal family of ReLU configurations; whether a different low-parameter structure can fit the two-cell dataset and escape the O(ε) bound is left open — an exhaustive search over sparse architectures at the claimed threshold would settle it.","The discovered 2-hop term, (1/2)α(T_Q − T_P) summed over second neighbors, reads like a discrete biharmonic (Laplacian-squared) correction; testing it on anisotropic diffusion or time-varying meshes would show whether it generalizes beyond the heat equation studied here.","The same minimal-dataset argument should carry over to other conservation laws whose FV updates are sums of pairwise fluxes (advection, nonlinear diffusion), since nothing in the proof uses linearity of the heat equation specifically beyond the flux form.","The O(ε) constant grows as 1/Δt, so the certified regime is exactly the explicit-marching regime; pushing the method to implicit or larger-timestep schemes would need a different stability argument than the Fourier condition used here."],"forward_implications":["Learned schemes inherit a transfer guarantee: a small loss on a two-node dataset certifies O(ε) accuracy on any mesh satisfying the Fourier stability condition, so minimal datasets can replace large simulation corpora for training PDE solvers.","The physics-informed variant means schemes can be derived from the PDE alone, with no ground-truth solver data — opening the same pipeline to equations whose discretizations are not yet known.","Wider receptive fields produce higher-order methods: the discovered 2-hop correction and the explicit midpoint scheme beat the first-order baseline, and the midpoint scheme matches Crank–Nicolson accuracy at lower memory cost.","Because symbolic regression returns closed forms, discovered schemes can be compiled into classical solver code or inspected by numerical analysts, closing the loop between learning and analysis.","The theory suggests sparsity is not a practical nicety but the load-bearing ingredient for out-of-distribution generalization: without the L1 penalty, training loss near zero on the minimal set leads to large OOD and rollout errors."],"supporting_citations":[{"why":"supplies the message-passing graph-network formalism that defines the GNN architecture.","marker":"[2]"},{"why":"supplies the physics-informed residual loss used for unsupervised scheme discovery.","marker":"[36]"},{"why":"supplies the symbolic-regression tool that turns the trained MLPs into closed-form expressions.","marker":"[6]"},{"why":"precedent for extracting symbolic physical laws from GNNs via sparsity and inductive bias; the method is built on this.","marker":"[7]"},{"why":"the neural-execution-of-algorithms paradigm the paper extends to continuous physics (algorithmic alignment).","marker":"[43]"},{"why":"recent evidence that GNNs extrapolate out of distribution on shortest paths; the claim the paper extends to PDE schemes.","marker":"[33]"},{"why":"the K-hop message-passing architecture behind the 2-hop discovery variant.","marker":"[13]"},{"why":"generates the unstructured meshes used to evaluate out-of-distribution generalization.","marker":"[16]"},{"why":"places the work within mesh-based learned simulation; the baseline paradigm for GNNs on unstructured meshes.","marker":"[35]"}],"fun_headline_variants":["GNNs trained on 2 nodes yield provable heat scheme","Two-node GNN training extrapolates to any mesh","Graph nets rediscover finite volume formulas exactly","No ground truth: GNN invents heat scheme via PDE loss","Symbolic regression confirms GNN's learned FV update"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The proof rests on the claim that the only way for a sparse ReLU network to fit the two-node training set is to take one specific parameter configuration — the finite-volume form — so a network with fewer or differently arranged parameters cannot match the data well.","fun_headline_variants_meta":{"raw":{"variants":["GNNs trained on 2 nodes yield provable heat scheme","Two-node GNN training extrapolates to any mesh","Graph nets rediscover finite volume formulas exactly","No ground truth: GNN invents heat scheme via PDE loss","Symbolic regression confirms GNN's learned FV update"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1359,"prompt_tokens":898,"completion_tokens":461,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":379}},"tokens_in":642,"tokens_out":461,"duration_ms":5291,"temperature":1.0,"reasoning_tokens":379,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:00:11.084250+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find any sparse ReLU GNN fitting the two-node training set to loss below ε with fewer non-zero weights than the claimed N_sparse = 2m(L+1)+4, or with weights outside the zero-bias diagonal family, and show its one-step error on a small unstructured mesh exceeds 2ε(1 + 2/Δt); a pruning or evolutionary search over general weight structures at the training-loss optimum would either confirm or refute the parameter-counting premise.","supporting_citations":[{"cited_title":"Karniadakis","cited_arxiv_id":null,"evidence_quote":"supplies the physics-informed residual loss used for unsupervised scheme discovery."},{"cited_title":"Discovering symbolic models from deep learning with inductive biases","cited_arxiv_id":null,"evidence_quote":"precedent for extracting symbolic physical laws from GNNs via sparsity and inductive bias; the method is built on this."},{"cited_title":"Neural execution of graph algorithms","cited_arxiv_id":null,"evidence_quote":"the neural-execution-of-algorithms paradigm the paper extends to continuous physics (algorithmic alignment)."},{"cited_title":"How powerful are k-hop message passing graph neural networks, 2023","cited_arxiv_id":null,"evidence_quote":"the K-hop message-passing architecture behind the 2-hop discovery variant."},{"cited_title":"Gmsh: A 3-d finite element mesh generator with built-in pre- and post-processing facilities","cited_arxiv_id":null,"evidence_quote":"generates the unstructured meshes used to evaluate out-of-distribution generalization."},{"cited_title":"Learning mesh-based simulation with graph networks","cited_arxiv_id":null,"evidence_quote":"places the work within mesh-based learned simulation; the baseline paradigm for GNNs on unstructured meshes."}],"review_version":1}