Pith. sign in

REVIEW 4 major objections 4 minor 5 cited by

Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries

T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Transolver++ claims the first neural PDE solver that trains on million-point meshes—1.2 million points on a single GPU and 2.5 million on four—by sharpening the learned physical-state assignment and parallelizing only the states rather…

desk verdict A credible incremental upgrade of Transolver with a genuinely useful parallel all-reduce scheme, but the headline industrial gain compares full-resolution training against 50k subsampled baselines, so the 'over 20%' claim is not yet established. read the letter →

arxiv 2502.02414 v2 pith:XD3S55Z6 submitted 2025-02-04 cs.LG

Huakun Luo , Haixu Wu , Hang Zhou , Lanxiang Xing , Yichen Di , Jianmin Wang , Mingsheng Long This is my paper · ORCID
classification cs.LG MSC 68T07
keywords neuralPDEsolversPhysics-Attentionphysicalstatesmillion-scalemeshesGumbel-Softmaxparallelismcomputationalfluiddynamicsoperatorlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transolver++ sets out to make neural PDE solvers usable at the mesh sizes industry actually runs, one to several million points, where previous deep surrogate models were limited to tens of thousands of points, with the strongest prior architecture reaching about 700,000. The paper's thesis is that the obstacle is not only GPU memory but also physics learning, because at large scales the learned point-to-state assignment tends to become uniform and attention collapses toward averaging. Its two fixes are eidetic physical states, obtained through a per-point adaptive temperature plus Gumbel-Softmax reparameterization so each point is sharply assigned to a mixture of physical states, and a parallelization scheme that communicates only the compact states across GPUs, at cost independent of the point count. On six standard PDE benchmarks the paper reports about 13% relative error reduction over the second-best baseline, and on million-point car and aircraft datasets it reports over 20% average relative improvement over strong baselines. If correct, this makes single-GPU training at 1.2 million points and four-GPU training at 2.5 million points a practical route for neural CFD surrogates.

What carries the argument

The central object is the slice-weight matrix that softly assigns every mesh point to one of $M$ eidetic physical states, with $M$ small relative to the point count, typically 32 or 64 in the experiments. Its defining formula is $w=\mathrm{Softmax}((\mathrm{Linear}(x)-\log(-\log \epsilon))/\tau)$ with a per-point temperature $\tau_i=\tau_0+\mathrm{Linear}(x_i)$ and Gumbel noise $\epsilon\sim U(0,1)$; states are weighted averages of point features, attention runs among states with $O(M^2)$ cost, and deslicing maps states back to points. Parallelism uses the same object: the weighted sums that define states are computed per GPU and combined with AllReduce, transferring only $M(C+1)$ numbers per GPU, so communication does not grow with the number of mesh points. This machinery carries the argument because the same state abstraction solves both failure modes: sharp adaptive assignments prevent attention degeneration, and compact states make million-point meshes computationally affordable.

What would settle it

Train Transolver++, Transolver, and GNOT on identical 50,000-point KNN-subsampled versions of DrivAerNet++ and AirCraft and, where feasible, train the baselines on the full meshes, then compare relative L2 and drag/lift coefficient errors; if the gap collapses at matched resolution, the claimed eidetic-state advantage is not the source of the industrial gains. Bootstrapping or enlarging the 10-case industrial test sets would also show whether the reported $R^2$ differences are statistically stable.

Watch

Extended reading notes

Core claim

The claim at the core of the paper is that Transolver's Physics-Attention can be scaled to million-point geometry without accuracy loss if the slice weights that cluster $N$ mesh points into $M$ compact physical states are made eidetic via Ada-Temp and Rep-Slice, preventing the uniform weights and average-pooling degeneration observed at scale, and if the state aggregation is split across GPUs and merged with AllReduce, so each GPU transfers only $M(C+1)$ values and communication volume is independent of $N$. With these changes, Transolver++ is described as the first neural solver whose single-GPU capacity reaches 1.2 million mesh points, scaling linearly to 2.5 million points on four A100 GPUs. On DrivAerNet++ (about 700k surface points and 2.5M full-field points) and the 300k-point AirCraft dataset, the paper reports relative-L2 improvements of roughly 11 to 30 percent over strong baselines on predicted fields, together with substantially reduced drag and lift coefficient errors and $R^2$ values near 0.997 or better on the 10-sample industrial test sets. The paper further argues that the learned states track fast- and slow-changing physical regions, which distinguishes the method from over-smooth baseline predictions.

Load-bearing premise

The headline industrial gains assume it is fair to compare Transolver++ trained on the full 2.5-million-point mesh with baselines trained on 50,000-point KNN-reconstructed subsamples, and to judge the result on only 10 test cases; if resolution and test-set size were matched, the reported >20% advantage could shrink.

Editorial extensions

If this is right

  • A neural PDE solver can be trained directly on high-fidelity industrial meshes rather than on subsampled approximations, so geometric details that drive drag and lift predictions enter the training signal instead of being discarded.
  • The memory ceiling on a single A100 moves to roughly 1.2 million points, and adding GPUs yields near-linear growth in addressable mesh size because inter-GPU traffic stays proportional to the number of states, not the point count.
  • The eidetic-state mechanism is what prevents deeper or wider PDE transformers from degenerating into average pooling, which is why the model keeps improving as layers and training data grow.
  • On aerodynamic coefficient prediction, the reported relative errors (for example, drag coefficient error nearly halved on DrivAerNet++) imply that learned surrogates can move from field visualization toward the engineering quantities design loops need.
  • The same state-parallel recipe should extend to even larger meshes, since the communication term is independent of point count and the remaining memory cost is from pointwise linear layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Implicit in the paper: the AllReduce-over-states recipe is general, so any architecture that first compresses many points into a bounded set of tokens can inherit communication independent of input size; the parallelism trick transfers beyond PDE solving.
  • Testable extension: Ada-Temp and Rep-Slice amount to a regularizer against slot collapse in clustering attention, so benchmarking them on generic point-cloud classification or object-centric scene decomposition would show whether the eidetic-state benefit is specific to physics fields.
  • The paper's scaling curves suggest a next experiment: train on the full 8,000-car DrivAerNet++ set rather than the 200-case subset, and at 10M or more points with additional GPUs, to see whether error keeps falling as both data and mesh resolution grow.
  • A matched-resolution ablation, using the same architecture at 50k points and at full resolution, would separate how much of the industrial gain comes from seeing more mesh points versus from the sharper state assignment, and would test the paper's claim that resolution itself, not just architecture, limits prior solvers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents Transolver++, a neural PDE solver designed for million-scale meshes. It extends Transolver's Physics-Attention with a local adaptive temperature (Ada-Temp) and Gumbel-Softmax reparameterization (Rep-Slice) to learn 'eidetic' physical states, and introduces a multi-GPU parallelization scheme that only all-reduces the physical states, making per-GPU communication volume independent of the input mesh size. Experiments on six standard PDE benchmarks report a 13% average relative improvement over the second-best baseline, and on two industrial datasets (DrivAerNet++ and AirCraft) the paper claims over 20% performance gains over baselines, with the largest run at 2.5 million points on four A100 GPUs.

Significance. If validated, the work would be a meaningful step toward practical neural surrogates for industrial CFD, as it demonstrates a route to training on million-point meshes with a communication-efficient parallel scheme. The standard-benchmark results and the ablations in Table 3 provide credible evidence that the proposed modifications improve over Transolver at conventional scales. The theoretical communication analysis (O(M(C+1)) per GPU, independent of N) is a clean and useful contribution. However, the headline industrial gains are not yet established as architecture-level improvements, because the comparison mixes resolution and architecture changes. The significance of the main claim is therefore conditional on a controlled experiment being provided.

major comments (4)
  1. [§5.2, Table 2, Appendix B.3.2] The comparison on industrial datasets is not controlled for training resolution. Baselines marked '*' are trained on random 50k-point KNN-reconstructed subsamples, while Transolver++ is trained on the full 2.5M-point DrivAerNet++ mesh (and the ~700k/300k meshes for the other two tasks). Since the paper's own Figure 6(c.2) shows that training on larger meshes improves relative L2, the reported gains (41.0% in CD, 24.1% and 30.4% in Surf, and the R2 improvements) may largely reflect a resolution effect rather than the eidetic-state or parallelization contributions. Please retrain all baselines at a common resolution, or train Transolver++ on the same 50k KNN subsamples, and report the comparisons under matched training and inference conditions.
  2. [Appendix B.1, Table 2, Eq. (9)] The industrial benchmark test sets contain only 10 samples each, so the reported R2 values (e.g., 0.997 versus 0.931) and relative L2 improvements are statistically fragile. With n=10, a single outlier can change the ranking. Please provide confidence intervals (e.g., bootstrap) or evaluate on a significantly larger held-out subset of DrivAerNet++ (the full dataset contains over 8,000 designs) to support the claimed improvements.
  3. [§4.1, Eq. (4), Algorithm 1] The Rep-Slice operation draws Gumbel noise during training, but the paper does not specify the inference-time procedure. If the stochastic sampling is retained at test time, the reported results are non-deterministic; if it is replaced by a deterministic softmax or argmax, that should be stated and justified. Please clarify, and if noise is used at inference, report the variance across repeated runs.
  4. [§4.2, Figure 3, §5.3] The paper claims 'linear complexity' and 'minimal overhead' when scaling across GPUs, but no actual scaling experiment is reported. The only multi-GPU run is the 4-GPU DrivAerNet++ Full experiment; there are no strong-scaling or weak-scaling curves varying the number of GPUs while holding per-GPU workload fixed. Please include such measurements (e.g., 1, 2, 4, 8 GPUs) to substantiate the linear-scaling claim, or soften the claim to a theoretical complexity analysis.
minor comments (4)
  1. [Table 3] Table 3 contains a typo: 'TRANSOVLER ++' should be 'TRANSOLVER ++'.
  2. [Figure 6(c.1)] The axes in Figure 6(c.1) are not clearly labeled; please specify which axis corresponds to running time and which to relative L2.
  3. [Appendix B.3.2] The number of subsamples used for test-time reconstruction and the KNN parameter k are not given; please specify them for reproducibility.
  4. [Eq. (5)] In Eq. (5), the ⊕ symbol is defined as AllReduce, but it is also used between scalar and vector terms; please clarify the operand shapes and the exact reduction semantics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are benchmarked externally and the main industrial-comparison concern is a validity issue, not a circular derivation.

full rationale

The paper's derivation chain is not circular. Transolver++ is an architectural extension of the authors' prior Transolver, but the central claims—13% average improvement on six standard benchmarks and the ability to handle million-scale meshes—are supported by external benchmark comparisons and memory/communication measurements, not by restating the model's own assumptions. The self-citations to Transolver and LSM are used as sources for baseline results and design context, not as load-bearing justification for the paper's conclusions. The ablations in Table 3 and the standard-benchmark results in Table 5 provide independent evidence for the proposed Ada-Temp, Rep-Slice, and speedup components. The main weakness is the industrial comparison in Table 2: baselines marked '*' are trained on 50k-point KNN-reconstructed subsamples while Transolver++ trains on full ~2.5M-point meshes, so the reported >20% gains may partly reflect training resolution rather than the proposed architecture. However, this is an experimental confound or fairness issue, not a case where a prediction is equivalent by construction to a fitted input or where a claimed result reduces to its own definition. No circular step satisfying the quoted-evidence standard was found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on standard hyperparameters (slice count, temperature seed), on the domain assumption that peaked softmax slice weights improve multi-scale physics representation, and on standard distributed-communication semantics. No new physical entities are introduced.

free parameters (3)
  • Number of physics states M (slices) = 32 or 64
    Chosen per dataset as a hyperparameter (Table 7). The attention cost and expressivity depend on M; the paper reports using 32 for DrivAerNet++ and 64 for most standard benchmarks.
  • Base temperature tau0 = unspecified
    Added to a per-point learned offset in Ada-Temp (Eq. 3). Its value is not reported, so the softmax sharpness is partially set by a hand-chosen constant.
  • Number of layers and channels = layers 4 or 8, channels 128/256
    Model capacity hyperparameters tuned per dataset (Table 7); they affect the reported relative errors, though such tuning is normal for neural PDE solvers.
assumptions (3)
  • domain assumption Softmax with learned temperature and Gumbel-Softmax produces more informative physical states than plain softmax
    The paper assumes peaked slice weights correspond to better physics representation (Section 4.1). This is validated only indirectly via downstream error and KL-divergence, not from first principles.
  • domain assumption Physical states aggregated by weighted averaging of mesh points preserve enough information for accurate PDE solving when meshes are million-scale
    The original Transolver assumption, carried over; the paper does not prove that a fixed M=32 states can represent the solution on very large complex geometries.
  • standard math AllReduce of partial sums is numerically equivalent to the global weighted average in Eq. (1)
    Used in Eq. (5) to justify linear scaling; this is a standard distributed-communication identity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries." pith.science (2026). https://pith.science/paper/XD3S55Z6

@misc{pith2026250202414,
  author       = {Pith},
  title        = {Pith review of: Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XD3S55Z6}},
  note         = {Machine review of arXiv:2502.02414}
}
abstract

Although deep models have been widely explored in solving partial differential equations (PDEs), previous works are primarily limited to data only with up to tens of thousands of mesh points, far from the million-point scale required by industrial simulations that involve complex geometries. In the spirit of advancing neural PDE solvers to real industrial applications, we present Transolver++, a highly parallel and efficient neural solver that can accurately solve PDEs on million-scale geometries. Building upon previous advancements in solving PDEs by learning physical states via Transolver, Transolver++ is further equipped with an extremely optimized parallelism framework and a local adaptive mechanism to efficiently capture eidetic physical states from massive mesh points, successfully tackling the thorny challenges in computation and physics learning when scaling up input mesh size. Transolver++ increases the single-GPU input capacity to million-scale points for the first time and is capable of continuously scaling input size in linear complexity by increasing GPUs. Experimentally, Transolver++ yields 13% relative promotion across six standard PDE benchmarks and achieves over 20% performance gain in million-scale high-fidelity industrial simulations, whose sizes are 100$\times$ larger than previous benchmarks, covering car and 3D aircraft designs.

Figures

Figures reproduced from arXiv: 2502.02414 by the authors.

Figure 1
Figure 1. (a) Comparison of model capability in handling large geometries. We plot the GPU memory change of each model when increasing input mesh points. The upper bound on a single A100 40GB GPU is depicted in the dotted line. (b) Comparison of experiment benchmarks. Transolver++ experiments on high-fidelity tasks with up to 2.5 million points, which is 100× larger than previous works. than real applications. Building on the… view at source ↗
Figure 2
Figure 2. (a) Overall design of Transolver++ block. Blocks highlighted in red represent modifications compared to the original Transolver. (b) Visualization of slice weights. Transolver++ learns more diverse and eidetic physical states. (c) Visualizations of physical quantity change ratio (difference between each point and its neighbors) and slice weights learned by models. The lighter color means faster change. can effective… view at source ↗
Figure 3
Figure 3. (a) Comparison with other parallel methods. Tailored to the unique physics learning design, our method only communicates physical states with an all-reduce operation. (b) Scalability of the communication overhead to the number of mesh points with 32 GPUs. Our parallel method stands out by only transferring 0.25MB of data, which does not scale with the size of input mesh points. Parallelism formulation Through carefu… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: (a) Visualization of standard benchmarks covering a wide range of physics scenarios, from solid physics to fluid dynamics. (b) Relative errors on standard benchmarks of the top-4 models selected based on overall performance. Full results can be found in [PITH_FULL_IMA…
Figure 5
Figure 5. Figure 5: Car and aircraft design to predict drag and lift coefficient under extremely complex geometries with million-scale meshes. which is of high quality and simulated by aerodynamicists. To fully evaluate the model performance under different scales, we split DrivAerNet++ i…
Figure 6
Figure 6. Figure 6: (a) Visualization of slice weight distributions. Transolver++ demonstrates a more diverse pattern than Transolver. (b) Error map of top-3 models on DrivAerNet++ and AirCraft. Transolver++ outperforms other models and captures some subtle variations. (c) Statistics anal…
Figure 7
Figure 7. Figure 7: Evaluation of the model scalability in terms of data size and parameter size. Our default setting is 150 cases and 4 layers. tion. While previous models (Li et al., 2021) claim to be resolution-invariant and aim to apply directly to large-scale geometries, our experime…
Figure 8
Figure 8. Figure 8: Visualizations of 32 physical or eidetic states learned in the final layer of models on DrivAetNet++ Surface. Transolver and Transolver++ are both plotted for a clear comparison. The lighter color means a higher weight in the corresponding physical state. AirCraft Tran…
Figure 9
Figure 9. Figure 9: Visualizations of 32 physical or eidetic states learned in the final layer of models on AirCraft. Transolver and Transolver++ are both plotted for a clear comparison. The lighter color means a higher weight in the corresponding physical state. C.2. Case Studies In addi…
Figure 10
Figure 10. Figure 10: Visualizations of 64 learned eidetic states of the final layer in Transolver++ on Airfoil. Transolver [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visualizations of 64 learned physical states of the final layer in Transolver on Airfoil [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Visualizations of 64 learned eidetic states of the final layer in Transolver++ on Elasticity [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Visualizations of 64 learned states of the final layer in Transolver on Elasticity. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Showcase comparison with Transolver and GNOT on AirCraft. A lighter color in the error map indicates a better performance. Ground Truth Transolver++ Prediction Transolver Prediction GNOT Prediction DrivAerNet++ Surface Error Map -1700 -600 500 -100 0 100 -1.80e+03 -67…
Figure 15
Figure 15. Figure 15: Showcase comparison with Transolver and GNOT on DrivAerNet++ Surface. A lighter error map means better performance. Ground Truth Transolver++ Prediction Transolver Prediction Elasticity Error Map Airfoil Error Map Ground Truth Transolver++ Prediction Transolver Predic…
Figure 16
Figure 16. Figure 16: Showcase comparison with Transolver on Elasticity and Airfoil. A lighter color in the error map indicates a better performance. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Mamba Neural Operators

    cs.LG 2026-07 reject novelty 6.0 of 10

    AMO builds adaptive Takenaka-Malmquist bases inside a Mamba state-space model for PDE operator learning, but the claimed equivalence to adaptive Fourier decomposition is not supported by the implemented recurrence.

  2. Evaluation of State-of-the-Art Deep Learning Architectures for Aerodynamical Predictions

    physics.flu-dyn 2026-07 conditional novelty 6.0 of 10

    Benchmarking four neural operators for airfoil and NASA CRM pressure prediction: Transolver best on 2D, BSMS-GNN best on 3D; UPT and GAOT lag.

  3. Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems

    cs.LG 2026-02 conditional novelty 6.0 of 10

    APT, a mesh-agnostic neural operator fusing graph-based local features with global attention, is claimed to be the first architecture trained directly on adaptive-mesh-refinement simulations and outperforms state-of-t...

  4. Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs

    cs.LG 2026-02 conditional novelty 6.0 of 10

    A geometry-conditioned Whitney-form neural network that solves a learned discrete conservation law improves out-of-distribution geometry generalization for steady-state PDEs compared with regression-based neural operators.

  5. Data-free neural PDE solvers based on Graph Neural Networks and weak forms

    cs.CE 2026-07 conditional novelty 5.0 of 10

    A graph-neural-network PDE solver trained on the weak-form force residual — no simulation data — reports residual convergence below 1% on unseen load cases and one modified geometry, with residual-based test-time refinement.

Reference graph

Works this paper leans on

19 extracted references · 9 canonical work pages · cited by 5 Pith papers

  1. [4]

    As shown in Table 5, Transolver++ demonstrates superior performance across six standard PDE benchmarks, achieving the lowest relative L2 error in all PDE-solving tasks with an averaged relative promotion of 13%. For models marked with an asterisk (*), we carefully reproduced the results by running the models more than three times and enabled fair and reli...

    work page 2024

  2. [5]

    Model performance on six standard PDE benchmarks is evaluated using the relative L2 error. The result marked with an asterisk (*) indicates a reproduced outcome, where the parameter counts and configurations of the baseline methods are carefully aligned to ensure a fair comparison. “/” means that the baseline cannot apply to this benchmark. MODEL RELATIVE...

    work page 2021

  3. [6]

    HT-net: Hierarchical transformer based operator learning model for multiscale PDEs

    Liu, X., Xu, B., and Zhang, L. HT-net: Hierarchical transformer based operator learning model for multiscale PDEs. arXiv preprint arXiv:2210.10890,

    arXiv

  4. [7]

    H., and Shi, B

    9 Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries Morris, E., Shen, H., Du, W., Sajjad, M. H., and Shi, B. Geometric instability of graph neural networks on large graphs. arXiv preprint arXiv:2308.10099,

    arXiv

  5. [10]

    Zhuang, Y ., Zheng, L., Li, Z., Xing, E., Ho, Q., Gonzalez, J., Stoica, I., Zhang, H., and Zhao, H

    URL https: //arxiv.org/abs/2410.05258. Zhuang, Y ., Zheng, L., Li, Z., Xing, E., Ho, Q., Gonzalez, J., Stoica, I., Zhang, H., and Zhao, H. On optimizing the communication of model parallelism. Proceedings of Machine Learning and Systems, 5,

    arXiv

  6. [13]

    Here are the details of these datasets. Elasticity This benchmark is generated by the simulations of the stress field in a hyper-elastic solid body under tension, which is governed by a stress-strain relationship using the Rivlin-Saunders material model (Li et al., 2022). Each case involves a unit cell of 972 points with a void in the middle , clamped at ...

    work page 2022

  7. [14]

    The whole field is discretized to unstructured meshes in the shape of 221 × 51 as the input and the output is the corresponding Mach number on these meshes

    Airfoil This benchmark is generated from simulations of transonic flow over an airfoil, governed by Euler’s equations (Li et al., 2022). The whole field is discretized to unstructured meshes in the shape of 221 × 51 as the input and the output is the corresponding Mach number on these meshes. The dataset includes 1000 training samples and 200 test samples...

    work page 2022

  8. [16]

    Training configurations are identical to previous methods (Wu et al., 2024; Hao et al., 2023; Deng et al., 2024; Elrefaie et al.,

    Implementation details of Transolver++ including training and model configuration. Training configurations are identical to previous methods (Wu et al., 2024; Hao et al., 2023; Deng et al., 2024; Elrefaie et al.,

    work page 2024

Show all 19 references
  1. [17]

    Lv and Ls refer to the loss on volume (surrounding area) and surface physics fields respectively

    and shared in all baselines. Lv and Ls refer to the loss on volume (surrounding area) and surface physics fields respectively. BENCHMARKS TRAINING CONFIGURATION (SHARED IN ALL BASELINES ) M ODEL CONFIGURATION LOSS EPOCHS INITIAL LR O PTIMIZER BATCH SIZE LAYERS L HEADS CHANNELS...

    2019

  2. [18]

    To ensure a rigorous comparison, we obtain the open-source implementations of these models and carefully verify their consistency with the original papers before training. B.3.1. S TANDARD BENCHMARKS We try to be as loyal to the original settings of baselines as possible. Howe...

    2024

  3. [19]

    and Transolver (Wu et al., 2024), as these works have conducted comprehensive and rigorous hyper-parameter tuning to ensure a fair and reliable comparison. B.3.2. I NDUSTRIAL APPLICATIONS For DrivAerNet++ and AirCraft datasets, most of the baseline models are unable to handle ...

    2024

  4. [64]

    The dataset also includes 1000 samples for training and 200 for testing

    The inputs are the velocity for the past 10 steps, while the outputs provide the future velocity for 10 timesteps. The dataset also includes 1000 samples for training and 200 for testing. Darcy This benchmark consists of simulations of the steady-state Darcy Flow in two dimens...

    2022

  5. [2013]

    Megatron-lm: Training multi- billion parameter language models using model paral- lelism

    Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., and Catanzaro, B. Megatron-lm: Training multi- billion parameter language models using model paral- lelism. arXiv preprint arXiv:1909.08053,

    1909 arXiv

  6. [2017]

    The devil in linear transformer.arXiv preprint arXiv:2210.10340,

    Qin, Z., Han, X., Sun, W., Li, D., Kong, L., Barnes, N., and Zhong, Y . The devil in linear transformer.arXiv preprint arXiv:2210.10340,

    arXiv

  7. [2019]

    A., Tanaka, M., Zhang, C., Zhang, M., Song, S

    Jacobs, S. A., Tanaka, M., Zhang, C., Zhang, M., Song, S. L., Rajbhandari, S., and He, Y . Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509,

    arXiv

  8. [2021]

    B., Choy, C., Li, B., Kossaifi, J., Otta, S

    Li, Z., Kovachki, N. B., Choy, C., Li, B., Kossaifi, J., Otta, S. P., Nabian, M. A., Stadler, M., Hundt, C., Azizzade- nesheli, K., and Anandkumar, A. Geometry-informed neural operator for large-scale 3d PDEs. In NeurIPS, 2023a. Li, Z., Kovachki, N. B., Choy, C., Li, B., Kossa...

    2003 arXiv

  9. [2022]

    Ring attention with blockwise transformers for near-infinite context

    Liu, H., Zaharia, M., and Abbeel, P. Ring attention with blockwise transformers for near-infinite context. arXiv preprint arXiv:2310.01889,

    arXiv

  10. [2023]

    Neural operator: Graph kernel network for partial differential equations

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020a. Li, Z., Kovachki, N. B., Azizzadenesheli, K., liu, B., Bhat-...

    2003 arXiv

  11. [2024]

    Drivaer- net++: A large-scale multimodal car dataset with com- putational fluid dynamics simulations and deep learning benchmarks

    Elrefaie, M., Morar, F., Dai, A., and Ahmed, F. Drivaer- net++: A large-scale multimodal car dataset with com- putational fluid dynamics simulations and deep learning benchmarks. arXiv preprint arXiv:2406.09624,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.