Pith. sign in

REVIEW 3 major objections 4 minor 13 references

Permutation-Invariant Spectral Learning via Dyson Diffusion

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read Graph spectra can be learned by diffusing eigenvalues with Dyson Brownian motion, bypassing permutation-equivariant architectures and outperforming GNN-based diffusion models.

desk verdict DyDM reframes permutation invariance as a property of the diffusion dynamics rather than the network architecture; the derivations are clean, but the backward SDE is only sketched and the experiments lack error bars. read the letter →

arxiv 2510.08535 v2 pith:PYRD53J5 submitted 2025-10-09 stat.ML cs.LGmath.PR

classification stat.MLcs.LGmath.PR MSC 60B2060H10
keywords DysonBrownianmotiongraphdiffusionmodelsspectrallearningpermutationinvarianceOrnstein-UhlenbeckprocessrandommatrixtheoryWeisfeiler-Lemanequivalencescore-basedgenerativemodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Graph diffusion models normally diffuse the entire adjacency matrix, forcing them to either learn up to n! representations per graph or rely on permutation-equivariant networks. This paper claims to bypass that trade-off analytically: for an Ornstein-Uhlenbeck process on the adjacency matrix, the ordered eigenvalues evolve according to Dyson Brownian Motion, independent of the eigenvectors. That decoupling means the spectrum—a permutation-invariant signature—can be learned by a generic score network, and the eigenvector dynamics remain recoverable through a separate SDE. The authors introduce the Dyson Diffusion Model (DyDM) and show it learns graph spectra more accurately than GNN- and graph-transformer-based diffusion models, without ad hoc spectral feature augmentation. The broader point is that the inductive bias for permutation invariance can live in the dynamics rather than in the learning architecture.

What carries the argument

The key object is Dyson Brownian Motion, the SDE for the ordered eigenvalues of a symmetric-matrix Ornstein-Uhlenbeck process; its pairwise 1/(λ_k−λ_l) repulsion prevents eigenvalue crossings and is exactly the term that makes the spectrum permutation invariant. The machinery also includes the time-reversed SDE (Eq. 5) used for sampling, the Eigenvector-SDE (Theorem 3.2) that keeps non-spectral information accessible, and a loss function derived from a small-time Gaussian approximation of the transition density. An adaptive step-size controller conditions on the probability-1 non-crossing event so forward simulation stays inside the Weyl chamber.

What would settle it

Take a dataset whose spectral distribution is known exactly (e.g., i.i.d. GOE spectra) and train DyDM with a score network; compare the sampler's output to the exact distribution via a two-sample test on spectra. If the backward SDE fails to reproduce the target even in the limit of fine discretization and a perfectly learned score, then the time-reversal justification is false.

Watch

Extended reading notes

Core claim

The central claim is that diffusing the adjacency matrix of an undirected graph with an entry-wise Ornstein-Uhlenbeck process induces a spectral process that is exactly Dyson Brownian Motion: each ordered eigenvalue obeys dλ_k = (α Σ_{l≠k} 1/(λ_k−λ_l) − βλ_k)dt + √(2α) dW_k. Because this SDE involves only the eigenvalues, the spectral dynamics are independent of the eigenvectors and therefore permutation invariant. The paper proves an Eigenvector-SDE that, conditioned on the eigenvalue path, describes the remaining degrees of freedom, so no information is lost. Building on this, the authors construct a diffusion model that learns the score of the spectral SDE by simulating forward paths with

Load-bearing premise

The entire generative procedure rests on the assumption that the time-reversed Dyson SDE with singular drift (Eq. 5) is a valid time reversal that targets the data distribution — a step the paper supports only by a sketched regularization argument, not a full proof.

Editorial extensions

If this is right

  • Learning graph spectra no longer requires GNNs or graph transformers; any score network, including a simple MLP, can be used because the spectrum is permutation invariant by construction.
  • The WL-expressivity blind spot of GNN-based diffusion models is avoided for spectral quantities: DyDM can distinguish WL-equivalent graphs whose spectra differ.
  • Because the Eigenvector-SDE retains all remaining degrees of freedom, full graph generation is possible in principle without losing the Θ(n²) non-spectral information.
  • Ad hoc feature augmentation (e.g., cycle counts and a few Laplacian eigenvalues) is unnecessary for accurate spectral learning; DyDM matches or beats DiGress with the 'trick'.
  • The method extends to any symmetric-matrix data (covariance matrices, graph Laplacians), not just adjacency matrices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the spectral dynamics are truly decoupled, a natural next step is a two-stage generator: sample the spectrum with DyDM, then sample eigenvectors from the conditioned Eigenvector-SDE; this would yield a permutation-invariant generative model for full graphs.
  • The paper's principle — pushing permutation invariance into the stochastic dynamics rather than the architecture — could apply to other symmetry groups, e.g., rotational invariance in point clouds or unitary invariance in quantum systems.
  • A testable extension: apply DyDM to the graph Laplacian spectrum and check whether it recovers the same WL-distinguishability advantages on heterophilic graphs, where the adjacency spectrum is less informative.
  • The exponential time schedule motivated by Dyson's mixing conjecture suggests the method may scale to large n; an empirical scaling study on graphs with n ≫ 10 would sharpen that claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces DyDM, a score-based generative model for graph spectra. It starts from an entry-wise OU SDE on symmetric adjacency matrices (Eq. 1), recalls that ordered eigenvalues follow Dyson-BM (Eq. Dyson-BM, Theorem 3.1), and derives a conditional eigenvector SDE (Theorem 3.2). Training uses an approximate denoising loss (Eq. 18) based on Euler-Maruyama Gaussian transition densities; sampling uses the time-reversed Dyson-BM (Eq. 5) with an adaptive step size and a shooting mechanism. Experiments compare spectral statistics against EDP-GNN, GDSS, ConGress, and DiGress on a WL-bimodal dataset, Community Small, and 15k brain ego-graphs. The main claims are that the spectral score can be learned with an arbitrary architecture, avoiding GNN/transformer WL-blindness, and that DyDM matches or outperforms existing graph diffusion models.

Significance. The analytical decomposition is the paper's key contribution. If Eq. (5) is rigorously justified and the loss approximation is controlled, the paper offers a principled way to make graph diffusion permutation-invariant at the dynamics level rather than through equivariant architectures. The detailed proofs of Theorems 3.1 and 3.2 are strengths, and the promise of released code and samples supports reproducibility. The current version, however, leaves the singular time-reversal and the discretization bias as open gaps, and the empirical support is not yet at the level of the advertised comparison.

major comments (3)
  1. [Section D, Eq. (5)] The reverse SDE is the only bridge from the forward DBM to actual graph-spectrum generation, but the proof of Anderson's time-reversal theorem for this singular drift is not supplied. The regularization φ_R is Lipschitz for each R, yet the text asserts without proof that the score contribution 'does not cause complications as R→0' because paths stay inside the Weyl chamber. The score s(λ,t) is learned from data and can in principle diverge near the boundary; no uniform L^p or Novikov-type bound is established. The commutation of the R→0 limit with time reversal is exactly the point that needs proof. A concrete fix: prove weak convergence of the regularized reverse SDEs to Eq. (5) and verify Anderson's integrability conditions for the limiting drift, or construct the reverse transition directly. Without this, a perfectly learned score may be used in the wrong generator and the sampler doe
  2. [Section A, Eq. (18)] The training objective replaces the intractable transition density p_{t+h|t} by the Gaussian density of an Euler-Maruyama step. The approximation error is never quantified; the schedule in Table 2 uses step sizes as large as 2·dt = 0.1 near the end of the forward process, where Dyson-BM is rapidly mixing and the conditional density is strongly non-Gaussian. Minimizing Eq. (18) therefore estimates the score of a discretized Dyson-BM, not necessarily the score of the continuous-time process required by Eq. (5). The manuscript should quantify this bias (e.g., via a weak error bound for the Euler scheme in the score) or use an unbiased estimator; otherwise the learned score network is trained on the wrong target.
  3. [Table 1 and Appendix N.6] The main comparison consists of point estimates with no standard errors or repeated seeds. In the Brain row the marginal Wasserstein values are 0.0275 (DyDM) versus 0.0285 (DiGress with trick), while Appendix N.6 itself warns that these extra digits are not statistically significant. That difference is smaller than plausible Monte Carlo error, so the claim of being 'on par or improves' is not established; the same applies to the WL-Bimodal marginal values 0.0076 versus 0.0073. The paper should report mean ± std over multiple retraining seeds and sampling runs, or confidence intervals, and assess significance of the µ and W_marg differences. Given that baselines were re-trained with extensive hyperparameter search (Section N), reporting the selection procedure alone does not quantify variability.
minor comments (4)
  1. [Algorithm 3, line 8] The backward drift is computed using s_θ(λ, t_{j+1}) for a backward step starting at t. If t lies in [t_j, t_{j+1}], the score should be evaluated at t or interpolated; as written the time label is ambiguous and potentially incorrect.
  2. [Section 2 and Appendix K] The text refers to 'Theorem K.1' in the main part, but the appendix states 'Corollary K.1'. The numbering/labeling should be made consistent.
  3. [Eq. (5)] The Brownian motion dW_k on the right-hand side is not explicitly declared to be a backward-in-time Brownian motion; state this to avoid sign/interpretation ambiguity.
  4. [Section I.2] The ε-perturbation for degenerate eigenvalues is a preprocessing step that may bias the learned distribution. The paper should report how often the perturbation is triggered and the sensitivity of the results to the chosen ε.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the core DBM and eigenvector derivations are independent of the learned score and test data.

full rationale

The central derivation chain is self-contained against external mathematical results rather than against the paper's own fitted outputs. Theorem 3.1 (Dyson-BM) is proved in Appendix B from the matrix OU process (1) via Ito calculus; Theorem 3.2 (Eigenvector-SDE) is proved in Appendix C from the same process. Neither uses the learned score or the experimental data. The backward sampler Eq. (5) is Anderson's time reversal applied to DBM, and Section D explicitly acknowledges the non-Lipschitz difficulty and sketches a regularization argument. Even if that argument is incomplete---the R->0 limit is asserted rather than fully proved---an unproved technical step is a rigor/correctness concern, not circularity: no equation in Section D is derived from the quantity being predicted. The loss (6) is an approximation of score matching for the forward DBM, with the intractable transition approximated by the Euler-Maruyama Gaussian; this is a standard tractability approximation, not a fitted parameter renamed as a prediction. Experiments compare against held-out test spectra and external baseline models, not against quantities constructed from the training fit. There are no load-bearing self-citations: the cited results by Dyson, Anderson, Anderson et al., Katori-Tanemura, Yang, and Allez et al. are external, and the paper supplies its own proofs for the main novel SDE claims. Hence no circular step can be exhibited.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The model builds on known RMT results; the paper's additions are algorithmic. No new entities are postulated. The main assumptions are the applicability of time reversal to non-Lipschitz SDEs and the Gaussian approximation of the transition density.

free parameters (4)
  • diffusion coefficients α, β = not reported in main text
    Control the OU/DBM dynamics and invariant prior. The paper does not state the exact values used in experiments; they are hyperparameters.
  • final diffusion time T and exponential schedule = T=12 for Brain dataset
    Chosen to cover mixing; schedule piecewise constant step sizes listed in Table 2.
  • spectrum rescaling bounds λ_min, λ_max = λ_min=-5, λ_max=5 for benchmarks
    Preprocessing affine transform applied to all spectra; data-dependent.
  • epsilon perturbation for degenerate eigenvalues = depends on closest eigenvalue distance in dataset
    Used to ensure simple spectrum; postprocessed by re-merging eigenvalues within epsilon.
assumptions (3)
  • ad hoc to paper Anderson's time reversal theorem applies to the DBM backward SDE (Eq. 5) with singular drift and learned score
    Section D acknowledges non-Lipschitz drift and gives only a sketch via regularization; the R→0 limit is asserted.
  • domain assumption Euler-Maruyama Gaussian transition density approximates the true DBM transition kernel for the score-matching loss (Eq. 18)
    Used to make the loss tractable; no error bounds or correction terms provided.
  • domain assumption Graph adjacency matrices in the datasets have simple spectra (or can be perturbed to)
    Theorems 3.1 and 3.2 require simple spectrum; the paper argues generic graphs satisfy this and uses epsilon perturbation for multiplicities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Permutation-Invariant Spectral Learning via Dyson Diffusion." pith.science (2026). https://pith.science/paper/PYRD53J5

@misc{pith2026251008535,
  author       = {Pith},
  title        = {Pith review of: Permutation-Invariant Spectral Learning via Dyson Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYRD53J5}},
  note         = {Machine review of arXiv:2510.08535}
}
abstract

Diffusion models are central to generative modeling and have been adapted to graphs by diffusing adjacency matrix representations. The challenge of having up to $n!$ such representations for graphs with $n$ nodes is only partially mitigated by using permutation-equivariant learning architectures. Despite their computational efficiency, existing graph diffusion models struggle to distinguish certain graph families and their spectra, unless graph data are augmented with ad hoc features. This shortcoming stems from enforcing the inductive bias within the learning architecture. In this work, we leverage random matrix theory to analytically extract the spectral properties of the diffusion process, allowing us to push most of the inductive bias from the architecture into the dynamics. Building on this, we introduce the Dyson Diffusion Model, which employs Dyson's Brownian motion to capture the spectral dynamics of an Ornstein-Uhlenbeck process on the adjacency matrix. Furthermore, conditioned on the spectral dynamics, we formulate a Lie group diffusion, appropriately modeling the remaining degrees of freedom. Strikingly, the resulting learning problem becomes permutation invariant at the Lie algebra level. We demonstrate that the Dyson Diffusion Model learns graph spectra accurately and outperforms existing graph diffusion models.

Figures

Figures reproduced from arXiv: 2510.08535 by the authors.

Figure 1
Figure 1. Dyson Diffusion model and its application to graph spectra: A graph on [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Struggle of GNN-based and graph-transformer-based models with two WL-equivalent [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Dyson Diffusion Model (training): The Dyson-BM is evolved forward in time with an adaptive step size ensuring that the paths remain in the Weyl Chamber. The step-size controller conditions on the probability 1 event of non-crossing as detailed in Section F.1 and Algorithm 2. Chamber for any sensible step size. Since the score is not defined outside the Weyl Chamber, this is problematic. On the other hand, the step s… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Plot of the invariant density of Dyson-BM for d = 2 and α = β = 1 C INFERRING THE EIGENVECTOR DYNAMICS Theorem 3.2 (restated). Denote by (v1(t), . . . , vn(t)) the orthonormal eigenvectors associated to the eigenvalues of Theorem 3.1. Assuming that the initial matrix M…
Figure 5
Figure 5. Figure 5: Forward step size controller which exploits that non-crossing of paths happens with prob [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: Sampling from the Dyson Diffusion Model. The eq. ( [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]
Figure 7
Figure 7. Figure 7: Learning dynamics of EDP-GNN on the WL-bimodal dataset (ground truth = [PITH_FULL_IMAGE:figures/full_fig_p035_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 3 linked inside Pith

  1. [1]

    The eigenvectors of Gaussian matrices with an external source.arXiv preprint arXiv:1412.7108,

    Romain Allez, Jo¨el Bun, and Jean-Philippe Bouchaud. The eigenvectors of Gaussian matrices with an external source.arXiv preprint arXiv:1412.7108,

  2. [2]

    Table 2: Example of exponential time grid, here for the brain data set which contains in total15′000 graphs.dtis0.05, and the final time isT= 12.0. from to stepsize 0 1/8 1/64·dt 1/8 1/4 1/32·dt 1/4 1/2 1/16·dt 1/2 1 1/8·dt 1 2 1/4·dt 2 3 1/2·dt 3 7 1·dt 7T2·dt I.2 PREPROCESSING AND DEALING WITH DIFFERENT DIMENSIONS As is the case for an Ornstein-Uhlenbec...

  3. [5]

    X i (vm)i(vk)i #

    We wrote it in the above form withp t|0 denoting the conditional density, since the three integrals can be replaced by Et∼U[0,T] h · · ·Ex∼p0 h Ey∼pt|0(·|x) h ∥· · ·∥2 2 iii , which we can sample from if we assume (1) sam- ple access top 0, (2) known density at anytgiven a dirac-deltap 0, (3) the term in the norm is tractable. (1) is assumed by the proble...

  4. [6]

    γ(η)-SDE

    Indeed, we have (omitting “(0)” in the notation) (vk(0) + dvk(0))T (vl(0) + dvl(0)) = 0 +v T k dvl +v T l dvk + dvT k dvl =√α X i̸=l 1 λl −λ i vT k vidwil + √α X j̸=k 1 λk −λ j vT l vjdwjk +α X j̸=k X i̸=l vT j vidwildwjk (λk −λ j)(λl −λ i) +O(dt 3/2) = √α λl −λ k (dwkl −dw lk)| {z } =0 +α X j̸=k X i̸=l δijδikδjldt (λk −λ j)(λl −λ i) +O(dt 3/2) =0 +O(dt 3...

  5. [9]

    We drew among all permutations5 ′000permutations uniformly at random and shuffled the graphs

    adjacency matrices. We drew among all permutations5 ′000permutations uniformly at random and shuffled the graphs. The first80%of this dataset are used for training, the remaining20%are used for testing. Community-smallThis standard benchmark dataset (Niu et al., 2020; Jo et al., 2022; You et al.,

  6. [10]

    In the hidden MLP, we employ as nonlinearities scaled SiLU functions, as argued by Karras et al. (2024). To make full use of the GPU memory, in one epoch, we sampleNpaths paths in parallel. If the dataset is too small to fill the GPU memory, we sample multiple, independent, paths of the same data points in parallel. From these samples, we update the score...

  7. [11]

    We comment in Section N.1 on the small dataset set (100graphs) compared to the big dimension (up to 20 vertices) and the thereby induced effect of undersampling

    consists of100graphs of size up to20vertices. We comment in Section N.1 on the small dataset set (100graphs) compared to the big dimension (up to 20 vertices) and the thereby induced effect of undersampling. BrainWe report this dataset in our repository. In detail, we construct from the brain graph Amunts et al. (2013); Rossi & Ahmed (2015) so-called ego-...

  8. [12]

    neigh- borhoods of vertices, and consider the induced subgraph. From those, we generate15 ′000graphs of sizen= 5ton= 10vertices with eigenvector multiplicity up to3, with the closest eigenval- ues – which are not multiplicities – having distance0.036. We take70%as train graphs,15%as validation, and the remaining15%as test graphs. N COMPARING TOBENCHMARKMO...

Show all 13 references
  1. [13]

    community small

    containsonly100 graphs, and each has a size of up ton= 20vertices. Thus, a comparison from the learned distribution based on (few) training samples to (very few) test samples 32 Preprint suffers from undersampling. This becomes very stark if one considers the following issue: ...

  2. [2018]

    A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI.arXiv preprint arXiv:2303.13336,

    Chenshuang Zhang, Chaoning Zhang, Sheng Zheng, Mengchun Zhang, Maryam Qamar, Sung-Ho Bae, and In So Kweon. A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI.arXiv preprint arXiv:2303.13336,

  3. [2020]

    community small

    using80%of the data as train data and the remaining20%as test data. Conversely, the standard benchmark set “community small” (Niu et al., 2020; Jo et al., 2022; You et al.,

  4. [2022]

    Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?arXiv preprint arXiv:2502.02488,

    Xiyuan Wang, Yewei Liu, Lexi Pang, Siwei Chen, and Muhan Zhang. Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?arXiv preprint arXiv:2502.02488,

  5. [2024]

    Monique M. Tirion. Large Amplitude Elastic Motions in Proteins from a Single-Parameter, Atomic Analysis.Physical Review Letters, 77(9):1905–1908,

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.