Pith. sign in

REVIEW 3 major objections 7 minor 6 references

GFocal: A Global-Focal Neural Operator for Solving PDEs on Arbitrary Geometries

T0 review · 3 major / 7 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GFocal claims a Transformer neural operator that jointly learns global Nyström attention and slice-based focal attention, fused by a convolution-based gate, generalizes better to arbitrary PDE geometries than prior neural operators.

desk verdict Reasonable integration of existing attention blocks, but the '15.2% SOTA' headline needs a fairer comparison protocol and error bars before I'd trust the margins. read the letter →

arxiv 2508.04463 v1 pith:UBUREDVL submitted 2025-08-06 cs.LG

Fangzhi Fei , Jiaxin Hu , Qiaofeng Li , Zhenyu Liu This is my paper · ORCID
classification cs.LG
keywords neuraloperatorPDEsolvingTransformerNyströmattentionfocalarbitrarygeometriesmultiscalefeaturefusioncomputationalfluiddynamics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GFocal is a Transformer-based neural operator for parametric PDEs on arbitrary geometries that learns global and local physical features simultaneously and fuses them. Its global block uses Nyström attention to capture long-range correlations, while its focal block clusters points into physics-aware slices and applies attention among slice tokens to capture local detail. A convolution-based gating mechanism, aided by relative-coordinate position encoding, adaptively balances the two streams. The paper reports the lowest relative L2 error on five of six standard benchmarks it tests, with an average 15.2% gain over the second-best model, and strong design-oriented metrics on two industrial CFD tasks. The central claim is that coordinated multiscale feature fusion, not a single attention mechanism, drives the improvement.

What carries the argument

The load-bearing pair is the global-focal attention stream. The Global Block uses Nyström attention, approximating the softmax matrix as $\hat S = \mathrm{Softmax}(Q\tilde K^\top / \sqrt d) S_a^+ \mathrm{Softmax}(\tilde Q K^\top / \sqrt d)$ with landmark keys and queries, to model long-range correlations in linear time. The Focal Block assigns each point to $L$ learnable slices with soft weights $w_{i,j}$, aggregates them into physics-aware tokens $t_j = \sum_i w_{i,j} f_i / \sum_i w_{i,j}$, runs attention over tokens, and deslices back to points, which is meant to capture local physical detail and interdependencies among physical states. A convolution-based sigmoid gate $G = \$\sigma$(\mathrm{

What would settle it

Run GFocal and a matched-capacity variant whose global and focal attention are replaced by exact full self-attention on the Elasticity point-cloud benchmark. If the exact-attention variant matches or beats GFocal, the Nyström/slice split and gated fusion are not the cause of the reported gains. Alternatively, on a synthetic non-uniformly sampled domain with a known integral operator, check whether GFocal's error converges to the true solution as the number of points grows; if it plateaus, the uniform-measure Monte-Carlo approximation underlying the proof is biased.

Watch

Extended reading notes

Core claim

Existing Transformer neural operators for PDEs, in this paper's telling, emphasize either global aggregation or local tokenization but not the coordinated interaction of both; that omission shows up on problems where geometry conditions both large-scale structure and fine physical detail. GFocal's answer is a two-stream architecture: Nyström attention carries global correlations, slice-based physics attention encodes distinct physical states into tokens and models their interdependencies, gated convolutions fuse the streams, and position encoding supplies spatial context. On the paper's evidence this joint design yields the lowest errors on five of six standard benchmarks—point-cloud elastic

Load-bearing premise

The load-bearing premise is the appendix's assumption that the domain carries a uniform measure for the Monte-Carlo approximation of attention as an integral operator; the benchmarks include non-uniformly sampled point clouds and unstructured meshes, so if that bias is severe the proof no longer applies and the architecture rests on its empirical results alone.

Editorial extensions

If this is right

  • If GFocal's central claim is right, the practical recipe is explicit dual-scale attention: global Nyström attention for whole-domain correlation plus slice-token attention for local physical detail, rather than scaling up a single attention type.
  • The largest reported gains are on point-cloud and structured-mesh cases (28.3% on Elasticity, 23.9% on Pipe), so the fusion mechanism matters most where geometry strongly conditions the solution.
  • The design-metric results (lift/drag coefficients and Spearman ranks) improve even when field errors are close to Transolver's, which suggests the fusion helps ranking and design optimization, not just field reconstruction.
  • On temporal benchmarks (Plasticity and NS2d), the model reports gains, indicating that preserving local detail may help long-rollout stability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the gated global-focal module is architecture-agnostic; attaching it to other token-based neural operators would test whether the fusion mechanism transfers independently of the Nyström and slice estimators.
  • Inference: relaxing the uniform-measure assumption in the appendix proof suggests a natural extension—density-aware landmark sampling or measure-weighted token aggregation—whose payoff on non-uniform meshes is a testable prediction.
  • Inference: the paper's efficiency table records noticeably higher memory and per-epoch training time than Transolver; whether the accuracy and design-ranking gains justify that overhead on large 3D industrial tasks is an open question.
  • Inference: Darcy is the one benchmark where GFocal is second, which suggests the global-focal split matters less on smooth elliptic problems where global methods are already strong; mapping that boundary is a useful next study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces GFocal, a Transformer-based neural operator that combines Nyström-attention global blocks with slice-based focal-attention blocks, fused via convolution gating and an auxiliary position encoder. The intended contribution is a multiscale global-local architecture for PDE surrogates on arbitrary geometries. The authors report state-of-the-art relative L2 errors in five of six standard benchmarks (Elasticity, Plasticity, Airfoil, Pipe, Navier-Stokes), an average 15.2% relative gain over the second-best model, and strong performance on AirfRANS and Shape-Net Car. The appendix contains a proof attempt that attention is a Monte-Carlo approximation of an integral operator.

Significance. If the empirical results hold under a controlled comparison, GFocal would be a useful contribution: the modular combination of global Nyström attention, slice-based local attention, and gated fusion is well motivated, and the paper includes ablations, scaling experiments, and comparisons with more than fourteen baselines. The strongest assets are the breadth of benchmarks and the explicit attempt to combine global and local feature learning. However, the current manuscript does not yet substantiate the headline SOTA claim because the comparison protocol is asymmetric (per-dataset tuning for GFocal vs. defaults for baselines), results are single-run with no error bars, and the theoretical appendix contains a normalization error that invalidates the stated integral-operator equivalence.

major comments (3)
  1. [Implementation / Table 2 / Table 7] The central '5 out of 6 SOTA' claim rests on an unbalanced protocol. The Implementation section states that default hyperparameters were used for all baselines, while Table 7 shows that GFocal's architecture was selected by an extensive per-benchmark search (e.g., Global Depth varies 4-6, Focal Depth 3-6, Channels 128-256, Slices 32-64). Thus the reported margins on Elasticity (28.3%) and Pipe (23.9%) may reflect tuning effort rather than the architecture. Moreover, the 15.2% average is computed over five rows in which Plasticity is a tie (promotion 0%) and Darcy, where GFocal loses to Transolver (0.0057 vs 0.0052), is omitted. I request a matched tuning protocol, or at least several seeds with standard deviations for both GFocal and all baselines, and an average reported over all six benchmarks.
  2. [Appendix, Proof of Theorem, Eqs. (15)-(17)] The proof that attention is a Monte-Carlo approximation of an integral operator is not correct as written. In Eq. (15) the normalization integral is over ξ′ of exp((W_q a(ξ′))(W_k a(ξ))^T), i.e., it normalizes over query positions with a fixed key; in standard softmax attention the normalization is over keys for a fixed query. This inconsistency propagates to Eq. (17), whose denominator is ∑_j exp((W_q a(x_j))(W_k a(x_i))^T), not ∑_j exp((W_q a(x))(W_k a(x_j))^T). Consequently the claimed equivalence does not follow. In addition, the proof assumes Ω carries a uniform measure, which is violated by point clouds and unstructured meshes in the benchmarks, so the Monte-Carlo approximation would be biased even after the normalization is fixed. Please either correct the derivation or clearly mark the theorem as a heuristic analogy rather than a proof.
  3. [Table 2 / Experiment] No statistical uncertainty is reported. All tables give a single relative L2 value per model per benchmark, with no seeds, no standard deviations, and no confidence intervals. Several of the claimed gains are modest (Airfoil 0.0048 vs 0.0055; Navier-Stokes 0.0784 vs 0.0882; Darcy is a loss). Without repeated runs it is impossible to know whether these differences are significant. This is especially important because the paper does not release code or data, so the reader cannot check the reported numbers. Please add at least 3-5 seeds with mean±std and commit to releasing code/data.
minor comments (7)
  1. [Eqs. (6)-(7)] The text says Conv reduces the features to a single-channel gating map, but G is declared as R^{B×N×C}. Clarify how the one-channel map is broadcast across channels.
  2. [Eq. (9)] The phrase 'weighted combination' is written as an elementwise product {w_{i,j} f_i}; the subsequent weighted average in Eq. (10) should be stated explicitly as the intended aggregation.
  3. [Position Encoder, Eqs. (3)-(5)] The position encoder is described only for the 2D case, while Shape-Net Car is a 3D task. Please state the D=3 construction or clarify that the same formula is applied per coordinate.
  4. [Tables 3-4 / Abstract] The abstract says GFocal 'excels' in industry-scale simulations, but in Table 3 it is worse than Transolver on Volume (0.0041 vs 0.0037) and in Table 4 it is worse on Volume and Surf. Please qualify the claim to focus on the design-oriented metrics where GFocal is best.
  5. [Table 8 / Efficiency] It is unclear whether the reported memory is training or inference memory and what batch size was used for the memory measurement. The note says 'batch size during testing are set the same', but training time per epoch is measured; please clarify.
  6. [Appendix, Elasticity benchmark] The citation 'Pascon, 2019' appears in the constitutive model description but is missing from the reference list.
  7. [After Figure 10] There is a garbled encoding artifact (repeated '/uni00000013...' strings) in the source text after Figure 10; this should be cleaned.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GFocal's claims are held-out empirical comparisons, and its architectural components and integral-operator proof are explicitly grounded in cited prior work.

full rationale

The paper's central claim is an empirical performance comparison on standard benchmarks. GFocal's architecture combines Nyström attention (Xiong et al., 2021), slice-based tokens from Transolver (Wu et al., 2024), position encoding, and a learned gate; these are cited external components rather than self-cited results. The only derivation is the appendix proof that softmax attention can be written as a Monte-Carlo approximation of an integral operator with a kernel defined exactly to reproduce attention; this is a consistency/representation argument, not a prediction forced by fitted parameters. Hyperparameters are tuned per benchmark (Table 7) while baselines use default settings, which raises a comparison-fairness concern, but test errors are reported on held-out sets, so no fitted parameter is renamed as a prediction. There are no load-bearing self-citations, no uniqueness theorems imported from the authors, and no ansatz smuggled in via citation: both attention mechanisms are explicitly attributed. The uniform-measure assumption in the proof despite non-uniform benchmark meshes is a theoretical-validity concern, and the '5 out of 6 average' selectively excludes the Darcy loss and counts a tie, but these are reporting-robustness issues, not circularity. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical, so the main free parameters are architectural hyperparameters tuned per benchmark. The theoretical appendix introduces no new physical entities but relies on assumptions about uniform sampling and Monte-Carlo accuracy that are questionable for the tested geometries.

free parameters (5)
  • Number of slices (L) = 32 or 64 depending on benchmark (Table 7)
    Chosen by hyperparameter search; determines capacity of the focal block and directly affects performance.
  • Global depth (M) = 4, 5, or 6 (Table 7)
    Layer count of the Nystrom attention block, tuned per benchmark.
  • Focal depth (K) = 3, 4, or 5 (Table 7)
    Layer count of the physics attention block, tuned per benchmark.
  • Channel width (C) = 128 or 256 (Table 7)
    Feature width; Figure 8 shows width 256 overfits on Elasticity compared with width 128.
  • Nystrom landmark count = not reported
    Nystrom attention requires a number of landmark points; the paper does not state this value, hindering reproducibility.
assumptions (4)
  • domain assumption Monte-Carlo approximation of integrals with a uniform measure is valid for the domains considered.
    Stated in the appendix Proof of Theorem (Eq. 16), 'we assume that Ω carries a uniform measure for simplicity'. This is false for point clouds and unstructured meshes, which are non-uniformly sampled.
  • standard math The Nystrom method provides a sufficiently accurate approximation to full self-attention for PDE data.
    The global block relies on Nystrom attention without quantifying approximation error on the tested benchmarks.
  • domain assumption Training and test samples are drawn i.i.d. from the same distribution.
    Standard supervised learning assumption for the benchmark datasets.
  • standard math Softmax attention is a learnable integral operator.
    The appendix attempts to prove this, but the proof is flawed; the result is known in prior literature but is not correctly derived here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GFocal: A Global-Focal Neural Operator for Solving PDEs on Arbitrary Geometries." pith.science (2026). https://pith.science/paper/UBUREDVL

@misc{pith2026250804463,
  author       = {Pith},
  title        = {Pith review of: GFocal: A Global-Focal Neural Operator for Solving PDEs on Arbitrary Geometries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UBUREDVL}},
  note         = {Machine review of arXiv:2508.04463}
}
read the original abstract

Transformer-based neural operators have emerged as promising surrogate solvers for partial differential equations, by leveraging the effectiveness of Transformers for capturing long-range dependencies and global correlations, profoundly proven in language modeling. However, existing methodologies overlook the coordinated learning of interdependencies between local physical details and global features, which are essential for tackling multiscale problems, preserving physical consistency and numerical stability in long-term rollouts, and accurately capturing transitional dynamics. In this work, we propose GFocal, a Transformer-based neural operator method that enforces simultaneous global and local feature learning and fusion. Global correlations and local features are harnessed through Nystr\"{o}m attention-based \textbf{g}lobal blocks and slices-based \textbf{focal} blocks to generate physics-aware tokens, subsequently modulated and integrated via convolution-based gating blocks, enabling dynamic fusion of multiscale information. GFocal achieves accurate modeling and prediction of physical features given arbitrary geometries and initial conditions. Experiments show that GFocal achieves state-of-the-art performance with an average 15.2\% relative gain in five out of six benchmarks and also excels in industry-scale simulations such as aerodynamics simulation of automotives and airfoils.

Figures

Figures reproduced from arXiv: 2508.04463 by the authors.

Figure 1
Figure 1. Compared with ONO and Transolver prioritiz [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Overview of the GFocal architecture, consisted of 4 blocks: Global Block, Gated Mechanism, Position Encoder, and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview design of Global Block. We use Nystrom attention mechanism to reconstruct global interac- ¨ tions. Method This section outlines the architecture of the proposed GFo￾cal for solving PDEs on arbitrary geometries. We start with the problem setting, followed by the architecture overview. The processing modules are detailed in the following sub￾sections. Problem Setup In this work following (Li et al. 2023) and … view at source ↗
Figures from the paper (8 more)
Figure 3
Figure 3. Figure 3: Overall design of Focal Block. We use a series of slices to encode information into different tokens and modeling [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 4
Figure 4. Figure 4: Error map comparison between the state-of-the-art model Transolver and the proposed GFocal. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Scaling experiment results for different numbers [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization of a ground truth in AirfRANS. The left subfigure visualizes pressure field and the right visualizes [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Visualization of a ground truth in the ShapeNet Car dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Results of scaling experiments of GFocal on the Elasticity benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Ablation study visualization on the Airfoil and Elasticity benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Efficiency comparison on Elasticity (972 mesh points). Metrics are measured with the batch size of 2. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 3 canonical work pages

  1. [4]

    In International Conference on Machine Learning, 21658–21671

    NUNO: A general framework for learning parametric PDEs with non-uniform data. In International Conference on Machine Learning, 21658–21671. PMLR. Liu, X.; Xu, B.; and Zhang, L. 2022. HT-NET: Hierarchical Transformer based operator learning model for multiscale PDEs. Lu, L.; Jin, P.; Pang, G.; Zhang, Z.; and Karniadakis, G. E

    work page 2022

  2. [6]

    arXiv preprint arXiv:2204.11127

    U-NO: U-shaped Neural Operators. arXiv preprint arXiv:2204.11127. Raissi, M.; Perdikaris, P.; and Karniadakis, G. E. 2019. Physics-informed neural networks: A deep learning frame- work for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Compu- tational physics, 378: 686–707. Scarselli, F.; Gori, M.; Tsoi...

    arXiv 2019

  3. [2020]

    In International conference on machine learning, 5156–5165

    Transformers are RNNs: Fast autoregressive Trans- formers with linear attention. In International conference on machine learning, 5156–5165. PMLR. Li, Z.; Huang, D. Z.; Liu, B.; and Anandkumar, A. 2023. Fourier neural operator with learned deformations for PDEs on general geometries. Journal of Machine Learning Re- search, 24(388): 1–26. Li, Z.; Kovachki,...

    arXiv 2023

  4. [2021]

    Nature machine intelligence, 3(3): 218–229

    Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3): 218–229. Lu, L.; Meng, X.; Cai, S.; Mao, Z.; Goswami, S.; Zhang, Z.; and Karniadakis, G. E. 2022. A comprehensive and fair comparison of two neural operators (with practical exten- sions) based on fair data. Computer Meth...

    work page 2022

  5. [2022]

    Advances in Neural Information Process- ing Systems, 35: 23463–23478

    AirfRANS: High fidelity computational fluid dynam- ics dataset for approximating Reynolds-averaged Navier– Stokes solutions. Advances in Neural Information Process- ing Systems, 35: 23463–23478. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few...

    arXiv 2020

  6. [2023]

    Advances in Neu- ral Information Processing Systems, 36: 51830–51861

    Aligning optimization trajectories with diffusion models for constrained design generation. Advances in Neu- ral Information Processing Systems, 36: 51830–51861. Hao, Z.; Wang, Z.; Su, H.; Ying, C.; Dong, Y .; Liu, S.; Cheng, Z.; Song, J.; and Zhu, J. 2023. GNOT: A Gen- eral Neural Operator Transformer for Operator Learning. In International Conference on...

    work page 2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.