Accurate Self-Attention Wavefunctions at Large Scale

Filippo Gaggioli; Liang Fu; Sam Azadi

Self-attention neural wavefunctions for the 2D electron gas beat DMC energies up to 169 particles and recover the full collective-mode spectrum.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 09:45 UTC pith:OB3JFRKQ

load-bearing objection Abstract-only claim of self-attention NQS beating DMC on 2DEG at N=169 with full collective-mode spectrum; unverifiable without numbers or diagnostics. the 3 major comments →

arxiv 2607.08616 v2 pith:OB3JFRKQ submitted 2026-07-09 cond-mat.str-el cond-mat.mtrl-sci

Accurate Self-Attention Wavefunctions at Large Scale

Filippo Gaggioli , Sam Azadi , Liang Fu This is my paper

classification cond-mat.str-el cond-mat.mtrl-sci PACS 71.10.Ca02.70.Ss05.30.Fk

keywords self-attention wavefunctionstwo-dimensional electron gasvariational Monte Carloneural quantum statescollective modesplasmonrotonthermodynamic limit

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that self-attention neural networks can serve as accurate variational wavefunctions for the two-dimensional homogeneous electron gas even at large particle numbers. Applied to systems of up to 169 electrons, these wavefunctions produce ground-state energies that lie systematically below those of state-of-the-art diffusion Monte Carlo. Because the optimized wavefunction is available directly, the authors extract the full collective-mode dispersion of the liquid phase, spanning the small-momentum plasmon branch down to a roton-like minimum near twice the Fermi wavevector. Near-perfect agreement between observables computed at 91 and 169 particles indicates that the results have essentially reached the thermodynamic limit. The work therefore demonstrates that the expressivity of self-attention architectures can be retained at scales previously inaccessible to neural quantum states.

Core claim

Self-attention variational wavefunctions applied to the two-dimensional homogeneous electron gas for system sizes up to N=169 yield energies systematically lower than the best diffusion Monte Carlo results, give direct access to the complete collective-mode dispersion from the plasmon branch to a roton-like minimum near q=2k_F, and produce observables that agree almost perfectly between N=91 and N=169, signalling convergence to the thermodynamic limit.

What carries the argument

Self-attention neural-network variational wavefunctions: a flexible, permutation-equivariant ansatz whose attention layers capture long-range electron correlations and whose parameters are optimized by variational Monte Carlo energy minimization.

Load-bearing premise

The reported energy lowering and the extracted collective-mode spectrum are free of uncontrolled variational bias, finite-size artifacts, and incomplete optimization at the largest system sizes.

What would settle it

An independent fixed-node or released-node diffusion Monte Carlo calculation at N=169 that returns a lower energy, or a direct comparison of the computed structure factor and density-response peak positions against high-resolution experimental electron-gas data in the same density regime.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Abstract-only claim of self-attention NQS beating DMC on 2DEG at N=169 with full collective-mode spectrum; unverifiable without numbers or diagnostics.

read the letter

The one thing you need to know is that this abstract asserts self-attention variational wavefunctions for the continuum 2DEG stay accurate out to N=169, deliver energies systematically below SOTA DMC, recover the full collective-mode dispersion (plasmon down to a roton-like minimum near 2k_F), and show near-perfect agreement between N=91 and N=169. That package, if real, is a clear methodological step for neural quantum states on continuum fermions.

What is actually new is the scale plus the spectrum extraction. Self-attention and transformer-style ansätze already exist in the NQS literature; pushing them to N~100–169 on the homogeneous electron gas and reading out the entire collective branch is a solid, concrete advance rather than a first-of-its-kind idea. The variational framing against external DMC and the independent observable (dispersion) keep circularity low. Credit where due: if the full paper ships the energies, the structure factor or dynamic structure factor data, and the N-convergence plots, that is useful evidence for the community.

The soft spot is exactly what the stress-test flags and is load-bearing because we only have the abstract. No numbers, no error bars, no variance, no optimization trajectories, no multiple runs, no fixed-node comparisons, no finite-size scaling. High-capacity neural ansätze have non-convex landscapes; incomplete optimization can produce energies that look better than DMC while still missing the true ground state, and the extracted modes inherit the same bias. The “near-perfect” N=91/169 agreement could just mean both sizes share similar residual artifacts. That concern is real and uncontrolled on present evidence; it is not fatal if the paper later supplies the diagnostics, but right now the central numerical claims sit unchecked.

This is for people who already work on neural quantum states, continuum QMC, or 2DEG physics and want a scalable fermionic ansatz that also gives dynamics. It is not for a general condensed-matter audience yet. It deserves a serious referee: the claim shape is important enough and the method is formally grounded enough that an editor should send it out rather than desk-reject. I would not cite it until the full paper and numbers appear, and I would only bring the abstract to reading group as a “watch this space” item. If the diagnostics hold up, it becomes a yes; if they do not, the energy-lowering claim collapses. Send it to review.

Referee Report

3 major / 3 minor

Summary. The manuscript applies self-attention neural-network variational wavefunctions to the two-dimensional homogeneous electron gas for system sizes up to N=169. It reports variational energies systematically lower than state-of-the-art diffusion Monte Carlo (DMC), recovers the full collective-mode dispersion of the liquid phase from the small-q plasmon branch to a roton-like minimum near q=2k_F, and finds near-perfect agreement of observables between N=91 and N=169, which is interpreted as convergence to the thermodynamic limit.

Significance. If the numerical claims hold under controlled diagnostics, the work would be a substantial advance for neural quantum states: it would show that high-capacity self-attention ansatze remain optimizable and accurate at large particle number, can undercut established DMC benchmarks for the 2D HEG, and give direct wavefunction access to collective excitations that are otherwise hard to extract. The reported N=91/169 consistency would further support practical thermodynamic-limit studies with this class of ansatz. The abstract-level claims are therefore of clear interest to the strongly correlated and quantum Monte Carlo communities.

major comments (3)

[Abstract] Abstract claim of energies systematically lower than SOTA DMC: for a high-capacity non-convex ansatz at N=169 this is load-bearing only if optimization completeness is demonstrated. The abstract supplies no energy variances, statistical error bars, optimization trajectories, multiple independent runs, or fixed-node comparisons. Without those controls it is not possible to distinguish genuine variational improvement from incomplete descent or residual bias that can still produce energies below a given DMC reference.
[Abstract] Abstract claim that near-perfect agreement of observables at N=91 and N=169 indicates thermodynamic-limit convergence: two-size agreement alone does not establish the TL for the 2D Coulomb gas, where shell effects, long-range interactions, and finite-size corrections are known to be substantial. An explicit finite-size scaling analysis (or equivalent controls such as twist averaging and extrapolation) is required for this conclusion to be load-bearing.
[Abstract] Abstract claim of recovering the full collective-mode dispersion (plasmon branch to roton-like minimum near q=2k_F): the extraction protocol is unspecified (e.g., dynamic structure factor from imaginary-time correlations versus direct excitation operators). The spectrum inherits any residual variational bias of the ground-state wavefunction and must be validated against the known small-q plasmon asymptotics and prior literature before the recovery claim can be accepted as controlled.

minor comments (3)

[Abstract] The abstract should state the density parameter (r_s) range studied; 2D HEG physics and the location of any roton-like feature depend strongly on r_s.
[Abstract] The phrase “state-of-the-art DMC” should be tied to specific references and matching system parameters (N, r_s, boundary conditions) once the full text is available.
[Abstract] A brief definition of the self-attention architecture and the observables used for the N=91 vs N=169 comparison would help non-specialist readers assess the scope of the claims.

Circularity Check

0 steps flagged

No circularity in available abstract; claims are numerical variational results vs external DMC, not definitional or fitted-by-construction.

full rationale

Only the abstract is available. It reports a computational application of self-attention variational wavefunctions to the 2D homogeneous electron gas (N up to 169), with energies lower than external state-of-the-art DMC, extraction of collective-mode dispersion from the obtained wavefunction, and empirical near-agreement of observables at N=91 and N=169. None of these steps reduce by construction to their inputs: there are no equations defining a quantity in terms of the reported prediction, no fitted parameter renamed as a prediction of a closely related observable, no uniqueness theorem or ansatz imported via self-citation, and no renaming of a known empirical pattern. The energy comparison is an external benchmark; the dispersion and finite-size agreement are independent observables extracted after optimization. Per the rules, absence of quotable self-definitional or fitted-input reductions yields score 0 with empty steps. (Optimization completeness and missing diagnostics are correctness/risk issues, not circularity.)

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

Abstract-only review. No free parameters, axioms, or invented entities can be extracted with precision. The work rests on standard variational Monte Carlo assumptions (energy upper bound, antisymmetry of the ansatz, thermodynamic-limit extrapolation from finite N) and on the domain assumption that self-attention networks are sufficiently expressive and optimizable for continuum fermions at these sizes. No new particles or forces are introduced.

axioms (3)

standard math Variational principle: the expectation value of the Hamiltonian in a normalized antisymmetric trial wavefunction is an upper bound to the true ground-state energy.
Implicit foundation of any VMC energy comparison to DMC; not stated but required for the 'energies systematically lower' claim.
domain assumption Self-attention neural networks can represent the ground-state wavefunction of the continuum 2DEG with controllable bias at N up to ~169.
Core modeling premise of the work; expressivity and trainability at this scale are assumed rather than proved.
domain assumption Near-perfect agreement of observables at N=91 and N=169 implies convergence to the thermodynamic limit for the reported quantities.
Finite-size extrapolation assumption stated in the abstract; two system sizes alone do not rigorously establish the limit.

pith-pipeline@v1.1.0-grok45 · 6037 in / 2618 out tokens · 21716 ms · 2026-07-15T09:45:18.528365+00:00 · methodology

0 comments

read the original abstract

Self-attention neural networks provide powerful variational wavefunctions that surpass the expressivity of traditional variational ansatze. This expressivity, however, comes with increased computational complexity, raising a pressing question about scalability -- can such wavefunctions retain their accuracy at large system sizes? We apply self-attention wavefunctions to the two-dimensional homogeneous electron gas for up to N=169 particles, obtaining energies systematically lower than state-of-the-art DMC. Direct access to the ground state wavefunction further lets us recover the full collective-mode dispersion of the liquid phase, from the small-q plasmon branch to a roton-like minimum near q=2k_F. Observables at N=91 and N=169 are in near-perfect agreement, indicating convergence to the thermodynamic limit.