{"id":"62150229-680a-4904-9d49-3a655af7cda2","arxiv_id":"2504.15897","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A neural operator that performs attention on coefficients of functions in a Laplacian eigenfunction basis, improving accuracy on irregular-domain PDEs.","lead":"This paper introduces a new attention mechanism for neural networks that solve partial differential equations, treating whole solution fields as the items that attend to each other instead of individual mesh points. It uses the natural vibration modes of a shape as coordinates, which makes the method faster and more accurate on irregular shapes like airplane wings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Airfoil experiments compute the Laplacian basis once on the mean mesh and reuse it for every shape, so the claimed geometry-adaptive subspace is not actually defined per sample; the stated best configuration also conflicts with the ablation table.","rationale":"The reader's weakest assumption identifies exactly the issue I find most load-bearing: the Airfoil experiments reuse a Laplacian eigensubspace computed on the mean mesh for all geometry variations, so the subspace is not adapted to each sample domain. This matters because the paper's novelty for irregular domains rests on the claim that Laplacian eigenfunctions naturally adapt to the domain geometry and guarantee optimal approximation for smooth functions. That guarantee is only valid for functions on the domain whose Laplacian was used; for a different airfoil shape, the basis is not orthonormal and may not even be defined without an interpolation step that the paper does not describe. The empirical Airfoil result is therefore weaker evidence for the geometry-adaptive claim than the text suggests. I also note a concrete inconsistency between Table 10 (Airfoil best uses 64 basis functions) and Table 3 (64 basis gives 0.446, while 128/192 give 0.340), which further undermines the reproducibility of the headline number. The theoretical development in Sections 3–4 is otherwise coherent: the attention mechanism on functions is a valid generalization, the reduction to subspace coordinates is sound, and Appendix A's convergence argument works for bounded bilinear forms and linear operators when the basis is complete in the relevant space. The complexity analysis is mostly reasonable, though the projection step is really O(CMN) rather than O(CM) when N is not treated as a constant. Because the main unresolved issue is the basis-transfer assumption on Airfoil plus the configuration inconsistency, the existing CONDITIONAL verdict remains appropriate; no change in verdict is needed.","tokens_in":14947,"tokens_out":14290,"duration_ms":144273,"concrete_test":"Compute the Galerkin projection residual of the 200 Airfoil test solutions onto the mean-mesh Laplacian subspace (N=64, 128, 192) and compare it with the residual onto a per-sample Laplacian subspace for the same shapes, using the true L² inner product on each Ω_s. If the mean-mesh residual is substantially larger for off-mean shapes, the basis-transfer assumption is load-bearing for the Airfoil result; if the residuals are close, the transfer is benign. As a separate check, retrain the Airfoil model with the exact configuration in Table 10 and verify whether it reaches the Table 1 value of 0.34 or the Table 3 value of 0.446.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim that the Laplacian eigensubspace adapts to the domain and provides optimal approximation for smooth functions (Section 4.3, Appendix B) is not actually exercised on the Airfoil benchmark, the only irregular-domain test using it. Appendix C.1 states that the eigensubspace is computed once on the mean mesh and reused for all airfoil shapes. But the Laplace–Beltrami operator, and hence its eigenfunctions, depends on the sample domain Ω_s; a basis orthonormal in L²(Ω_mean) is not orthonormal in L²(Ω_s) and, absent an explicit mapping from each Ω_s to Ω_mean, is not even a basis there. Equation (12)'s projection formula is therefore not an L² projection on the sample domain. The convergence argument in Appendix A requires the basis to be complete in the space containing the target functions, which is not established for these per-sample spaces. The paper gives no analysis of this transfer error, and the Airfoil ablation (Table 3) is the only empirical support. There is also an internal inconsistency: Table 10 lists the Airfoil best configuration with 64 basis functions, while Table 3 reports 0.446 at 64 basis and 0.340 at 128/192 basis; the Table 1 value of 0.34 is not obviously reproducible from the stated 64-basis configuration. If the mean-mesh subspace is a poor approximation for off-mean shapes, the headline accuracy on irregular domains may not transfer to more varied geometries.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SUPRA, a neural operator that reformulates standard attention as a bilinear form plus a linear operator on L2(Ω), truncates the resulting infinite-dimensional attention to an N-dimensional subspace, and uses Laplacian eigenfunctions as the subspace on irregular domains. The authors claim that this yields a function-space attention mechanism with state-of-the-art accuracy (up to 33% lower relative L2 error) and near-linear computational cost on both regular and irregular PDE benchmarks, and Appendix A gives a convergence argument as N increases. Experiments cover Darcy, Navier-Stokes, Plasticity, Airfoil, and Pipe benchmarks, with ablations on basis count, basis type, normalization, model size, and cost.","tokens_in":15270,"tokens_out":6452,"duration_ms":57841,"significance":"If the claims hold, the paper offers a clean conceptual bridge between attention mechanisms and function-space operator learning: the bilinear-form/linear-operator formulation is principled, no fitted constants enter the convergence argument, and the Laplacian eigensubspace is a natural geometry-aware analogue of Fourier bases. The theoretical derivation in Section 4 and Appendix A is largely sound under stated boundedness assumptions, and the idea of parameterizing attention on subspace coordinates is potentially useful. However, the irregular-domain support is weakened by the mean-mesh Airfoil basis transfer, the complexity analysis omits an N factor in projection/reconstruction, and the experimental evidence is single-run without error bars. These issues are local enough to be fixable, but they are load-bearing for the paper's two headline claims: general-domain accuracy and near-linear cost.","major_comments":[{"comment":"The construction of the Laplacian eigensubspace for Airfoil is not geometry-adaptive as claimed. Appendix C.1 states that the eigensubspace is computed once on the mean mesh and reused for all airfoil shapes, but the Laplace–Beltrami operator is defined on each sample domain Ω_s; a basis that is orthonormal in L²(Ω_mean) is not orthonormal in L²(Ω_s) and, absent an explicit mapping from each Ω_s to Ω_mean, is not even a basis there. Consequently Eq. (12) is not an L² projection on the sample domain, and the convergence argument in Appendix A, which requires the basis to be complete in the space containing the target functions, does not apply to the Airfoil experiments. The paper needs either per-sample basis computation, an explicit transfer-error analysis, or a restriction of the irregular-domain claim.","section":"Section 4.3 and Appendix C.1, Eq. (12)"},{"comment":"The reported best Airfoil configuration is internally inconsistent. Table 10 lists #Basis = 64, but Table 3 reports Airfoil error 0.446 at 64 basis versus 0.340 at 128/192, and Table 5 uses 128 Laplacian eigenfunctions; the Table 1 value of 0.34 therefore cannot be reproduced from the stated optimal configuration. Please correct the configuration table or the ablation numbers.","section":"Table 10 vs Tables 3 and 5"},{"comment":"The complexity accounting omits the factor N in projection and reconstruction. In Eq. (12), computing N coefficients for one function requires sums over M sample points, i.e., O(NM) per function, and reconstruction from N coefficients to M points is also O(NM) per function. For C functions the total is O(C²N + CNM), not O(C²N + CM), and for the Laplacian basis on irregular meshes there is no fast transform to remove the N factor. This undermines the 'near-linear' efficiency claim and should be restated.","section":"Section 4.4, Complexity Analysis"},{"comment":"The experimental support for the headline accuracy improvement is thin. The baseline numbers are taken from prior papers rather than rerun under matched settings, and no error bars, standard deviations, or number of seeds are reported; for example, the Navier-Stokes gap between Transolver (7.83) and SUPRA (6.25) could be within run-to-run variation. Please provide repeated-run statistics or clearly label the comparison as previously reported values.","section":"Section 5.1, Table 1"}],"minor_comments":[{"comment":"The sentence 'we provide a Laplacian-based method to construct proper basis functions (?).' contains a literal '(?)' placeholder for a citation; please fill in the reference.","section":"Section 4.3"},{"comment":"The definition uses uppercase A(·,·) in the axioms but the rest of the paper uses lowercase a(·,·); align the notation.","section":"Definition 3.2"},{"comment":"Boundedness of a bilinear form should be stated as |a(u,v)| ≤ C||u|| ||v||, not a(u,v) ≤ C||u|| ||v||, since bilinear forms may be negative.","section":"Appendix A, Eq. (17)"},{"comment":"The statement that the infimum is achieved 'if and only if φ is an eigenfunction' is imprecise; scalar multiples of the eigenfunction also achieve it, and the naming of φ_k is ambiguous.","section":"Theorem B.1"},{"comment":"The claim that SUPRA 'will not degenerate when the number of basis is small' is not supported by the Airfoil row, which degrades from 0.340 to 0.446 when reducing from 128 to 64 basis functions; please qualify the claim.","section":"Section 5.2, Table 3"},{"comment":"The header of Table 5 reports units of ×10⁻³, while Tables 1 and 3 use ×10⁻²; this makes cross-comparison needlessly confusing. Unify the units or state the conversion explicitly.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The Airfoil mean-mesh issue is central to the paper's general-domain contribution and is stated explicitly in Appendix C.1, so it is not a review artifact. If the authors recompute the basis per sample, the computational cost may grow, which would also interact with the complexity claim. The internal inconsistency between Table 10 and Tables 3/5 should be resolved before the paper can be considered for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SUPRA is a genuinely useful reformulation. Rewriting attention weights as a bounded bilinear form and the value transform as a bounded linear operator on L2, then truncating in a basis, is simple and correct. Equations (10)-(11) show it reduces to standard attention on coefficient vectors, which the authors acknowledge, so the novelty is the framing plus the Laplacian-eigenfunction subspace for irregular geometries, not a new attention primitive. That framing is worth having. It gives a principled answer to why spectral and attention-based neural operators work and an explicit model-reduction route for function-space attention.\n\nThe paper does several things well. Appendix A gives a clean convergence argument under boundedness assumptions. The complexity analysis is honest: O(C^2N + CM), versus quadratic point attention. The basis-choice ablation is useful, and the efficiency numbers are plausible. I also credit the authors for noting normalization sensitivity and for not overselling the theory in the text.\n\nSoft spots, in order of importance:\n\n1. Airfoil basis mismatch. Appendix C.1 says the Laplacian eigensubspace is computed once on the mean mesh and reused for all airfoil shapes. But the convergence theorem and the 'adapts to geometry' claim require a basis appropriate to each sample domain. A basis orthonormal in L2(Omega_mean) is not automatically orthonormal in L2(Omega_sample) and, absent an explicit mapping from each sample mesh to the mean mesh, is not even well-defined there. This is the only irregular-domain test, so it carries real weight. The paper needs either a stated mapping and a transfer-error analysis or a numerical check that the basis still represents the relevant functions on off-mean shapes.\n\n2. Table inconsistency. The Airfoil row in Table 10 lists 64 basis functions as the best configuration, but Table 3 reports 0.446 at 64 basis functions and 0.340 at 128/192, and Table 1 reports 0.34. The reader cannot tell which configuration produced the headline number. That matters because Airfoil is the flagship irregular-domain result.\n\n3. Statistical support. Single runs, no error bars, baseline numbers taken from earlier papers, and no code release. The up-to-33% improvement is plausible but fragile; seed variation could shrink the Airfoil gap between SUPRA and LNO.\n\nThe citation pattern looks fair, and the distinction from Galerkin attention and LNO is accurate. This is not a case where the theory is hiding a fitted constant; the circularity burden is minimal.\n\nWho this is for: researchers building neural operators on non-uniform meshes and anyone working on attention in function spaces. The core formulation deserves a serious referee, and I would send it to review with the expectation of an experimental revision. The geometry-transfer question and the table conflict need to be resolved before publication.","headline":"Clean function-space attention framing with a correct approximation argument, but the Airfoil basis-reuse gap and a table inconsistency need fixing before the 'general domains' claim is credible.","tokens_in":15773,"tokens_out":3320,"would_cite":true,"duration_ms":32798,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attention between functions cuts PDE error by up to 33 percent.","keywords":["neural operator","attention mechanism","function space","Laplacian eigenfunctions","model reduction","irregular domains","PDE surrogate"],"falsifier":"Project the ground-truth output fields of an irregular-domain benchmark onto the best $N$-term Laplacian-eigenfunction subspace and measure the relative $L^2$ residual; if at $N=256$ the residual exceeds the reported test error, the claimed accuracy cannot be achieved by any attention mechanism confined to this subspace, and the paper's Airfoil result would then rest on cross-sample transfer of the eigensubspace rather than on the subspace approximation argument.","tokens_in":14733,"feed_emoji":"🧮","tokens_out":6137,"duration_ms":50206,"temperature":0.7,"pith_summary":"The paper proposes treating the functions that neural operators learn to map as the tokens of an attention mechanism, rather than treating mesh points as tokens. It observes that standard self-attention on vectors is built from a bilinear form and a linear operator, and it redefines both on $L^2(\\Omega)$ so attention acts directly between functions. Because that infinite-dimensional attention cannot be computed exactly, the paper approximates it inside a finite-dimensional subspace spanned by $N$ basis functions, chosen as Laplacian eigenfunctions on irregular domains so the basis respects the geometry. The resulting SUPRA operator is claimed to be as expressive as full attention while costing $O(C^2N + CM)$ rather than $O(CM^2)$, and experiments report relative $L^2$ errors up to 33 percent lower than four recent baselines on five PDE benchmarks.","feed_headline":"Function-space attention cuts PDE error by up to 33%","feed_subtitle":"SUPRA runs attention on function coordinates, beating four recent baselines on four of five PDE benchmarks.","key_machinery":"The load-bearing object is the subspace-parameterized attention (SUPRA) block: a projection of each input function onto $N$ basis coefficients $\\hat{u}_i$, standard attention applied to those coordinates using matrices $A$ and $B$ (decomposed into query/key/value matrices for multi-head use), and a reconstruction back to function values. The basis on general domains is the Laplacian eigensubspace, the span of the smallest eigenfunctions of $-\\Delta$ on $\\Omega$, which generalizes the Fourier basis and guarantees continuity and quasi-optimal approximation for smooth functions. The convergence argument rests on the boundedness (continuity) of the bilinear form and linear operator, so that coefficient truncation commutes with the attention computation in the limit.","core_discovery":"The central claim is that attention between functions, defined exactly as attention between vectors via a learned bilinear form $a(\\cdot,\\cdot)$ and a learned linear operator $b(\\cdot)$, can be carried out faithfully in a low-dimensional subspace: truncating the basis at $N$ terms turns the bilinear form into a matrix $A$ and the operator into a matrix $B$ acting on the coordinate vectors, so attention reduces to standard multi-head attention on those coordinates. Appendix A proves that as $N \\to \\infty$ the output converges to the infinite-dimensional attention. On irregular domains the paper uses the low-frequency eigenfunctions of the Laplace operator as the basis, which are continuous, orthonormal, and optimal approximants for smooth functions; this is what lets SUPRA avoid the discontinuity artifacts that FFT-based spectral convolution suffers when irregular domains are cut and flattened. On five standard benchmarks the method reports the lowest relative $L^2$ error on four of them and near-linear cost, including sharp-feature cases like transonic shock waves.","pith_inferences":["A testable consequence the paper does not develop: the same subspace parameterization could be plugged into other operator architectures beyond the FNO-style backbone, e.g. as a drop-in replacement for the spectral convolution layer, provided the basis projection cost stays manageable.","The Laplacian-eigenfunction choice suggests a natural error monitor: track the projection residual of the input and output functions; when it is large, SUPRA's accuracy ceiling is set by the subspace, not by attention.","The Airfoil practice of computing the eigensubspace once on the mean mesh and reusing it across shapes is a strong assumption; if it transfers, it implies geometries in the dataset are near-isospectral perturbations of the mean shape, which could be checked by measuring basis-consistency across samples."],"forward_implications":["SUPRA removes the need to cut irregular domains for spectral convolution, since the Laplacian eigenfunctions are defined directly on the physical domain.","The complexity $O(C^2N + CM)$ makes attention practical at mesh sizes where point-token attention at $O(CM^2)$ is prohibitive.","Because the same code path as standard multi-head attention can be reused, the method inherits optimized GPU attention implementations.","On the reported benchmarks, the method attains lower relative $L^2$ error than four recent baselines on four of five datasets, and the ablations show instance normalization and basis choice substantially affect accuracy."],"supporting_citations":[{"why":"Defines the standard attention mechanism that the paper generalizes to function spaces.","marker":"(Vaswani et al., 2017)"},{"why":"FNO, the spectral-convolution baseline whose FFT dependence on regular grids SUPRA replaces, and source of the Darcy and Navier-Stokes datasets.","marker":"(Li et al., 2020)"},{"why":"Geo-FNO, whose cut-and-flatten mapping introduces the domain-discontinuity problem and which supplies the irregular-domain benchmarks.","marker":"(Li et al., 2022b)"},{"why":"Fourier attention baseline that treats points as tokens and motivates the quadratic-complexity critique.","marker":"(Cao, 2021)"},{"why":"LNO/Galerkin attention baseline whose restricted input-span subspace SUPRA contrasts with its own basis-spanning attention.","marker":"(Wang & Wang, 2024)"},{"why":"Transolver, a general-geometry transformer baseline that SUPRA claims to beat in accuracy.","marker":"(Wu et al., 2024)"},{"why":"Architecture backbone whose spectral convolution blocks SUPRA replaces with SUPRA blocks.","marker":"(Kossaifi et al., 2023)"},{"why":"Supplies the theorem that Laplacian eigenfunctions form an orthonormal basis of $L^2(\\Omega)$.","marker":"(Gilbarg & Trudinger, 2013)"},{"why":"Gives the variational characterization of the lowest Laplacian eigenfunctions as the smoothest functions on the domain.","marker":"(Evans, 2010)"}],"fun_headline_variants":["SUPRA: 33% lower PDE error, no FFT grid needed","Function-space attention: 33% error cut on PDE benchmarks","Laplacian eigenfunctions adapt attention to irregular domains","Subspace attention beats four baselines on four of five PDE tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the solution fields, and the learned relations between them, are well approximated by just 64 to 256 smooth basis functions (Laplacian, Fourier, or Chebyshev), a quantitative sufficiency asserted from ablations rather than derived for each problem.","fun_headline_variants_meta":{"raw":{"variants":["SUPRA: 33% lower PDE error, no FFT grid needed","Function-space attention: 33% error cut on PDE benchmarks","Laplacian eigenfunctions adapt attention to irregular domains","Subspace attention beats four baselines on four of five PDE tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000932,"raw_usage":{"total_tokens":4006,"prompt_tokens":975,"completion_tokens":3031,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":2957}},"tokens_in":591,"tokens_out":3031,"duration_ms":20519,"temperature":1.0,"reasoning_tokens":2957,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:15:19.874318+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Project the ground-truth output fields of an irregular-domain benchmark onto the best $N$-term Laplacian-eigenfunction subspace and measure the relative $L^2$ residual; if at $N=256$ the residual exceeds the reported test error, the claimed accuracy cannot be achieved by any attention mechanism confined to this subspace, and the paper's Airfoil result would then rest on cross-sample transfer of the eigensubspace rather than on the subspace approximation argument.","supporting_citations":[{"cited_title":"Choose a transformer: Fourier or galerkin","cited_arxiv_id":null,"evidence_quote":"Fourier attention baseline that treats points as tokens and motivates the quadratic-complexity critique."},{"cited_title":"and Trudinger, N","cited_arxiv_id":null,"evidence_quote":"Supplies the theorem that Laplacian eigenfunctions form an orthonormal basis of $L^2(\\Omega)$."},{"cited_title":"Partial Differential Equations","cited_arxiv_id":null,"evidence_quote":"Gives the variational characterization of the lowest Laplacian eigenfunctions as the smoothest functions on the domain."}],"review_version":1}