Pith. sign in

REVIEW 20 cited by

EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.12059 v3 pith:GCETK23F submitted 2023-06-21 cs.LG cs.AIphysics.comp-ph

Yi-Lun Liao , Brandon Wood , Abhishek Das , Tess Smidt This is my paper · ORCID
classification cs.LGcs.AIphysics.comp-ph
keywords betterdegreesequiformerv2equiformerequivarianthigheroc20convolutions
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Equivariant Transformers such as Equiformer have demonstrated the efficacy of applying Transformers to the domain of 3D atomistic systems. However, they are limited to small degrees of equivariant representations due to their computational complexity. In this paper, we investigate whether these architectures can scale well to higher degrees. Starting from Equiformer, we first replace $SO(3)$ convolutions with eSCN convolutions to efficiently incorporate higher-degree tensors. Then, to better leverage the power of higher degrees, we propose three architectural improvements -- attention re-normalization, separable $S^2$ activation and separable layer normalization. Putting this all together, we propose EquiformerV2, which outperforms previous state-of-the-art methods on large-scale OC20 dataset by up to $9\%$ on forces, $4\%$ on energies, offers better speed-accuracy trade-offs, and $2\times$ reduction in DFT calculations needed for computing adsorption energies. Additionally, EquiformerV2 trained on only OC22 dataset outperforms GemNet-OC trained on both OC20 and OC22 datasets, achieving much better data efficiency. Finally, we compare EquiformerV2 with Equiformer on QM9 and OC20 S2EF-2M datasets to better understand the performance gain brought by higher degrees.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 84 citations worldwide. Full citation record

  1. Pushing the limits of unconstrained machine-learned interatomic potentials

    physics.chem-ph 2026-01 conditional novelty 7.0 of 10

    Unconstrained non-equivariant and direct-force neural interatomic potentials scale to 730M parameters and match or beat equivariant state-of-the-art models on several atomistic benchmarks.

  2. MOFSimBench: Evaluating Universal Machine Learning Interatomic Potentials In Metal--Organic Framework Molecular Modeling

    cond-mat.mtrl-sci 2025-07 conditional novelty 7.0 of 10

    A new open benchmark shows that top universal machine learning interatomic potentials outperform classical force fields and a fine-tuned MOF-specific potential for most nanoporous materials modeling tasks.

  3. Dyna-Mat: End-to-end benchmarking of foundation machine learning interatomic potentials in finite-temperature ensembles

    cond-mat.mtrl-sci 2026-07 conditional novelty 6.5 of 10

    Dyna-Mat-v1.0 benchmarks 15 foundation MLIPs on finite-T MD observables, finding average force-error correlation with RDF/VDOS but systematic pressure failures and near-Pareto optimality of latest cross-trained models.

  4. E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding

    cs.CV 2026-07 conditional novelty 6.0 of 10

    3D Gaussian view-dependent colors are repacked as 3×3 matrices so geometry and color rotate together, giving exact rotation-equivariant recognition and world modeling in 3DGS.

  5. Faster Molecular Dynamics with Neural Network Potentials via Distilled Multiple Time-Stepping and Non-Conservative Forces

    physics.chem-ph 2026-02 conditional novelty 6.0 of 10

    Distilled non-conservative force models in a multi-time-step integrator accelerate neural-network-potential molecular dynamics by up to 5.6x on tested systems without degrading sampling accuracy.

  6. From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures

    cs.LG 2026-02 conditional novelty 6.0 of 10

    A bond-deformation benchmark plus a force-smoothness metric is proposed to detect PES artifacts and guide MLIP architecture design, with improvements shown on a new Transformer-style model.

  7. Global Plane Waves From Local Gaussians: Periodic Charge Densities in a Blink

    cond-mat.mtrl-sci 2026-01 conditional novelty 6.0 of 10

    ELECTRAFI predicts periodic electron densities by analytically Fourier-transforming a Gaussian mixture, reaching near-SOTA accuracy with up to 633× faster inference and ~20% end-to-end DFT speedups.

  8. E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory

    cs.LG 2026-01 conditional novelty 6.0 of 10

    E2Former-V2 combines SO(2) sparsification and a fused streaming Triton kernel to cut equivariant attention memory to linear in system size, claiming ~20x faster kernels and 100k-atom inference.

  9. Platonic Transformers: A Solid Choice For Equivariance

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Platonic Transformers achieve exact equivariance to translations plus discrete Platonic-solid rotations by lifting features into multiple reference frames and sharing one RoPE attention across them, with a linear-time...

  10. Insights into CO dimerization at electrified Cu interfaces from large-scale machine learning simulations

    cond-mat.mtrl-sci 2025-09 reject novelty 6.0 of 10

    OC25 is a large open dataset and baseline models for solid-liquid interfaces, but the claimed CO dimerization insights are absent from the manuscript body.

  11. Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow

    cs.LG 2025-07 conditional novelty 6.0 of 10

    SO(3)-Averaged Flow matching with reflow and distillation enables high-quality one-step molecular conformer generation, reporting new SOTA on GEOM-QM9 and strong one-step results on GEOM-Drugs.

  12. OrbitAll: A Unified Quantum Mechanical Representation Deep Learning Framework for All Molecular Systems

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A physics-informed graph neural network using spin-polarized orbital features from semi-empirical quantum mechanics predicts energies of charged, open-shell, and solvated molecules with claimed chemical accuracy and 1...

  13. Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A distributed equivariant GNN with a neighbor-minimizing graph partitioner scales electronic-structure (Hamiltonian) prediction to 512 GPUs and 190,000 atoms, with an 87% weak-scaling efficiency.

  14. A Scalable and Quantum-Accurate Foundation Model for Biomolecular Force Field via Linearly Tensorized Quadrangle Attention

    physics.chem-ph 2025-07 conditional novelty 6.0 of 10

    A new equivariant AI force field architecture, LiTEN, and its foundation model LiTEN-FF achieve state-of-the-art accuracy on multiple biomolecular benchmarks while scaling linearly with system size.

  15. Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning

    cs.CV 2025-06 reject novelty 6.0 of 10

    Across nine vision-language models, performance collapses when chemical composition is held out, but the reported magnitude and internal consistency of this collapse are not supported by the paper's own tables.

  16. Distillation of atomistic foundation models across architectures and chemical domains

    physics.comp-ph 2025-06 conditional novelty 6.0 of 10

    Distillation of atomistic foundation models via synthetic data yields 10x-100x faster student potentials with near-teacher accuracy.

  17. The Augmented Potential Method: Multiscale Modeling Toward a Spectral Defect Genome

    cond-mat.mtrl-sci 2025-02 conditional novelty 6.0 of 10

    The augmented potential method computes grain boundary segregation spectra for 1,036 binary alloys by pairing a foundation machine learning potential at the defect with a classical potential far away.

  18. Global Universal Scaling and Ultra-Small Parameterization in Machine Learning Interatomic Potentials with Super-Linearity

    cond-mat.mtrl-sci 2025-02 conditional novelty 6.0 of 10

    By rescaling atomic pair distances with element-pair-specific parameters, the authors make one shared radial function serve all elements, yielding an ultra-small machine learning interatomic potential with accuracy cl...

  19. Heterogeneous Ensemble Enables a Universal Uncertainty Metric for Atomistic Foundation Models

    cond-mat.mtrl-sci 2025-07 conditional novelty 5.0 of 10

    Weights from OMat24 force errors turn an eleven-model heterogeneous uMLIP ensemble into an uncertainty metric U that correlates with true force errors across material families and drives low-DFT distillation.

  20. Beyond Atomic Geometry Representations in Materials Science: A Human-in-the-Loop Multimodal Framework

    cs.LG 2025-05 reject novelty 5.0 of 10

    MCS-Set adds 2D projections and text labels to 20 synthetic crystal cluster families and exposes large, uneven errors across LLM baselines.

Pith tools