pith. sign in

arxiv: 2606.02600 · v1 · pith:L522SR6Anew · submitted 2026-05-23 · ❄️ cond-mat.dis-nn · cs.LG

High-Dimensional Latents Should Be Diagnosed Through Phase Structure

Alejandro Ascarate , Leo Lebrat , Rodrigo Santa Cruz , Clinton Fookes , Olivier Salvado This is my paper

Pith reviewed 2026-06-30 12:20 UTC · model grok-4.3

classification ❄️ cond-mat.dis-nn cs.LG
keywords autoencodersvariational autoencodersspin-glass theorylatent spacesanomaly detectionimage generationphase structurehyperspherical prior
0
0 comments X

The pith

Latent spaces in autoencoders form spin-glass phases whose edge-of-stability regime improves generation and anomaly detection.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper maps the latent space of autoencoders and variational autoencoders onto a spin-glass model. The reconstruction term plus a hyperspherical prior creates an effective Hamiltonian on the latent sphere, with latent coordinates acting as continuous spins. Standard spin-glass tools such as overlap distributions, susceptibility, and block-spin coarse-graining then identify ordered, disordered, and edge-of-stability phases. Positioning the system at the edge of the topological trivialization regime produces measurable gains: lower self-FID on CIFAR-10 and CelebA64 with preserved reconstruction, and stronger performance on both unsupervised and conditional out-of-distribution detection across image benchmarks and real-world datasets.

Core claim

By formalizing a latent-space spin-glass dictionary, the reconstruction term together with the hyperspherical prior induces a Hamiltonian on the latent sphere; latent coordinates play the role of continuous spins and the prior supplies an external field. This dictionary imports operational diagnostics—overlap distributions, susceptibility, and block-spin coarse-graining—to detect ordered, disordered, and edge-of-stability phases. Deliberately driving the latent system to the edge-of-stability of the topological trivialization regime improves the reconstruction-generation trade-off on CIFAR-10 and CelebA64 and raises accuracy in anomaly detection on CIFAR-10/100, Imagenette, Mars Rover, and G

What carries the argument

The latent-space spin-glass dictionary that converts the reconstruction term and hyperspherical prior into a Hamiltonian on the sphere, allowing overlap distributions, susceptibility, and block-spin coarse-graining to identify phases that affect downstream tasks.

If this is right

  • Hyperspherical compression at the edge of stability lowers self-FID on CIFAR-10 and CelebA64 while preserving or improving reconstruction quality.
  • The same semi-ordered latent geometry raises both fully unsupervised and conditional out-of-distribution detection performance on CIFAR-10/100, Imagenette, Mars Rover, and Galaxy Zoo benchmarks.
  • Spin-glass observables can be used alongside standard reconstruction and generation metrics to evaluate whether a latent representation sits in a regime that supports or undermines task performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same phase diagnostics could be tested on latent spaces of other generative models that use different priors or loss terms.
  • Latent representations that remain stuck in fully disordered or fully trivialized phases may explain persistent failure modes in high-dimensional generation and detection tasks.
  • Coarse-graining the latent sphere at multiple block sizes could reveal whether the identified phases persist or change across different length scales of the representation.

Load-bearing premise

The reconstruction term together with the hyperspherical prior induces a Hamiltonian on the latent sphere to which spin-glass diagnostics can be directly and meaningfully applied to identify phases that causally affect downstream performance.

What would settle it

Training an autoencoder or VAE, measuring its overlap distribution and susceptibility to locate the edge-of-stability regime, then observing that this regime produces no reduction in self-FID on CIFAR-10 or no gain in out-of-distribution detection accuracy.

Figures

Figures reproduced from arXiv: 2606.02600 by Alejandro Ascarate, Clinton Fookes, Leo Lebrat, Olivier Salvado, Rodrigo Santa Cruz.

Figure 1
Figure 1. Figure 1: Detecting phases via Block–spin–like coarse–graining of latent coordinates µ (neighboring–dimension averaging down to R 3 from an initial 128−dimensional latent). Left: a standard 2D Ising model Block–spin coarse–graining (dashed yellow line illustrating a 3 × 3 → 1 reduction, new spin = sign(Σ9 spins), i.e., b = 3; these are not the real scale factors b and length scale L of the lattice in the displayed i… view at source ↗
Figure 2
Figure 2. Figure 2: Replica angle. From right to left, the direction of the applied magnetic field in each experiment is shifted from the equator (‘zero’–compression mode), to (1, 1, . . . , 1) (‘half’), and then towards the north pole (‘full’) to bias the transition during training (first two, right to left, are the experiments of Fig.1). 4.2 Renormalisation-group and block-spin transformations Renormalisation-group (RG) tec… view at source ↗
Figure 3
Figure 3. Figure 3: Each horizontal slice at some vertical index value shows the color coded histogram (red, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: At the edge of topological trivialization. The k−NN method on latent space for anomaly detection is natural in this case, since it directly measures the Euclidean distances between replicas (cf.4.1). In this way, the histograms for the score display quite directly the law of the overlap P(R). At the edge of stability (B), both topologically trivial and continuous RSB phases are observed at the same time, t… view at source ↗
Figure 5
Figure 5. Figure 5: Decay test (σ refers to the variance of the noise for the mollification) [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Evolution of the energy landscape in terms of the applied external magnetic field of intensity [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: A full RSB region should be full of peaks around different, but very close, values of distance [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Results of a typical fully compressed VAE training at epoch [PITH_FULL_IMAGE:figures/full_fig_p029_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Results of the same fully compressed VAE training at final epoch [PITH_FULL_IMAGE:figures/full_fig_p030_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: AD metrics improvement at the edge-of-stability in GZ (Middle panel/row: green, normal [PITH_FULL_IMAGE:figures/full_fig_p031_10.png] view at source ↗
read the original abstract

We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory. The paper has two components. First, we formalize a latent-space spin-glass dictionary: for a fixed decoder, the reconstruction term together with a hyperspherical coordinates prior induces a Hamiltonian on the latent sphere, where latent coordinates play the role of continuous spins and the prior acts as an external magnetic field. This allows us to import operational spin-glass diagnostics -- overlap distributions, susceptibility, and block-spin coarse-graining -- to detect ordered, disordered, and edge-of-stability phases in trained latent representations. Second, we show that deliberately driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences. In generation, hyperspherical compression improves the reconstruction-generation trade-off on CIFAR-10 and CelebA64, yielding lower self-FID while preserving or improving reconstruction. In anomaly detection, the same semi-ordered latent geometry improves both fully unsupervised and conditional OOD detection, including real-world Mars Rover and Galaxy Zoo datasets, as well as CIFAR-10/100 and Imagenette-based OOD benchmarks. We therefore advocate a phase-aware evaluation paradigm for AEs/VAEs, in which spin-glass observables complement standard ML metrics and expose the latent regimes that underlie downstream success or failure in many cases.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a spin-glass theory framework for diagnosing latent spaces in autoencoders and VAEs. It defines a Hamiltonian on the latent sphere induced by the reconstruction term (for fixed decoder) plus a hyperspherical prior, treats latent coordinates as continuous spins, and imports diagnostics such as overlap distributions, susceptibility, and block-spin coarse-graining to identify ordered, disordered, and edge-of-stability phases. The central claim is that deliberately operating at the edge-of-stability of the topological trivialization regime yields improved generation (lower self-FID on CIFAR-10 and CelebA64) and anomaly detection (across CIFAR, real-world Mars Rover and Galaxy Zoo datasets).

Significance. If the Hamiltonian construction validly maps to the empirical latent distribution and the identified phases are shown to causally drive the reported performance gains, the work would introduce a novel physics-inspired diagnostic paradigm for latent representations that complements standard ML metrics. The approach of importing operational spin-glass observables and demonstrating downstream consequences on multiple benchmarks would be a substantive contribution to understanding high-dimensional latent geometry.

major comments (2)
  1. [Abstract / latent-space spin-glass dictionary] The Hamiltonian H(z) is defined via the reconstruction term for a fixed decoder together with the hyperspherical prior (abstract and latent-space spin-glass dictionary section). Spin-glass diagnostics are then applied directly to the encoder outputs q(z|x). No derivation or argument is supplied showing that the measure induced by joint encoder-decoder optimization approximates the Boltzmann measure of this H; if the two measures differ due to encoder capacity or optimization dynamics, the diagnosed phases are not demonstrably those of the stated Hamiltonian, undermining the causal attribution of performance improvements to phase structure.
  2. [Experimental results on generation and anomaly detection] The central experimental claim—that driving the system to the edge-of-stability of the topological trivialization regime produces the reported gains in self-FID and OOD detection—rests on the phase identification step. Without evidence that the overlap distributions and susceptibility computed on q(z|x) correspond to equilibrium statistics of H, the correlation between these observables and downstream metrics could be an artifact of the training objective rather than a spin-glass phenomenon.
minor comments (2)
  1. [latent-space spin-glass dictionary] Notation for the hyperspherical prior and the mapping of latent coordinates to continuous spins should be introduced with explicit equations early in the dictionary section to avoid ambiguity when importing overlap and susceptibility formulas.
  2. The manuscript would benefit from a short table summarizing the spin-glass observables (overlap distribution, susceptibility, block-spin coarse-graining) and their precise definitions in the latent setting.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We address the two major points below, clarifying the role of the effective Hamiltonian and the empirical nature of our diagnostics and results.

read point-by-point responses

  1. Referee: [Abstract / latent-space spin-glass dictionary] The Hamiltonian H(z) is defined via the reconstruction term for a fixed decoder together with the hyperspherical prior (abstract and latent-space spin-glass dictionary section). Spin-glass diagnostics are then applied directly to the encoder outputs q(z|x). No derivation or argument is supplied showing that the measure induced by joint encoder-decoder optimization approximates the Boltzmann measure of this H; if the two measures differ due to encoder capacity or optimization dynamics, the diagnosed phases are not demonstrably those of the stated Hamiltonian, undermining the causal attribution of performance improvements to phase structure.

    Authors: We appreciate the referee's emphasis on this theoretical link. The Hamiltonian is constructed as an effective model for a fixed decoder, with the reconstruction term supplying the energy and the hyperspherical prior the external field; latent coordinates are treated as continuous spins. The encoder-decoder training does not guarantee that the resulting q(z|x) is exactly the Boltzmann measure of this H, owing to finite model capacity and the dynamics of optimization. The spin-glass observables (overlap distributions, susceptibility, block-spin coarse-graining) are therefore applied operationally to the empirical distribution of latents actually produced by the trained encoder. These empirical statistics are what govern downstream generation and detection performance. The framework supplies a physically motivated language for characterizing latent geometry rather than a claim of exact equilibrium equivalence. We will add a short clarifying paragraph in the latent-space spin-glass dictionary section to make this effective-model status explicit. revision: partial

  2. Referee: [Experimental results on generation and anomaly detection] The central experimental claim—that driving the system to the edge-of-stability of the topological trivialization regime produces the reported gains in self-FID and OOD detection—rests on the phase identification step. Without evidence that the overlap distributions and susceptibility computed on q(z|x) correspond to equilibrium statistics of H, the correlation between these observables and downstream metrics could be an artifact of the training objective rather than a spin-glass phenomenon.

    Authors: The reported improvements in self-FID (CIFAR-10, CelebA64) and OOD detection (CIFAR, Mars Rover, Galaxy Zoo, Imagenette) are obtained by deliberately tuning hyperparameters so that the empirical overlap distribution and susceptibility on samples from q(z|x) reach the edge-of-stability regime. While we do not demonstrate that these statistics are precisely those of the equilibrium Boltzmann measure of H, the observables are computed directly on the latent distribution that the model actually uses at inference time. Across multiple datasets the same tuning procedure consistently yields better reconstruction-generation trade-offs and improved anomaly scores, indicating that the diagnosed phase structure serves as a practical control knob. We view the spin-glass diagnostics as complementary descriptors of latent geometry rather than a strict causal proof; the empirical correlation with performance is the primary evidence offered. No revision is required on this point, as the manuscript already presents the results as correlations between observable phase diagnostics and task metrics. revision: no

Circularity Check

0 steps flagged

No circularity: Hamiltonian is an explicit modeling choice; downstream claims are empirical.

full rationale

The paper defines a Hamiltonian on the latent sphere from the reconstruction term plus hyperspherical prior for a fixed decoder, then applies standard spin-glass observables (overlap distributions, susceptibility, block-spin coarse-graining) to the encoder outputs of trained models. It reports empirical improvements in self-FID and OOD detection on CIFAR-10, CelebA64, Mars Rover, Galaxy Zoo and other benchmarks when the latent geometry is driven toward a particular regime. These steps do not reduce any claimed result to its inputs by construction; the phase diagnostics are a proposed analysis tool whose correlation with performance is tested externally rather than derived tautologically from the training objective. No self-citation chain or fitted-input-as-prediction pattern appears in the load-bearing claims.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Based solely on the abstract, the central claim rests on the applicability of spin-glass theory to the induced latent Hamiltonian and on the assumption that detected phases correlate with downstream metrics. No explicit free parameters or invented physical entities with independent evidence are stated.

axioms (1)
  • domain assumption Spin-glass theory and its operational diagnostics apply to the Hamiltonian induced on the latent sphere by reconstruction loss plus hyperspherical prior
    The paper imports overlap distributions, susceptibility, and block-spin coarse-graining directly from spin-glass literature under the assumption that the mapping is valid.
invented entities (1)
  • latent-space spin-glass dictionary no independent evidence
    purpose: To equate latent coordinates with continuous spins and the prior with an external field
    Conceptual mapping introduced to enable the phase diagnostics; no external falsifiable evidence provided in the abstract.

pith-pipeline@v0.9.1-grok · 5783 in / 1317 out tokens · 46278 ms · 2026-06-30T12:20:46.042920+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 4 canonical work pages · 2 internal anchors

  1. [1]

    Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications , series =

    Marc M. Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications , series =. 1987 , publisher =

    1987

  2. [2]

    Cugliandolo and Jorge Kurchan , title =

    Leticia F. Cugliandolo and Jorge Kurchan , title =. Physical Review Letters , volume =. 1993 , doi =

    1993

  3. [3]

    Journal de Physique I , volume =

    Silvio Franz and Giorgio Parisi , title =. Journal de Physique I , volume =

  4. [4]

    Fyodorov and C

    Yan V. Fyodorov and C. Critical Behaviour of the Number of Minima of a Random Landscape at the Glass Transition Point and the Tracy--Widom Distribution , journal =. 2012 , doi =

    2012

  5. [5]

    Fyodorov , title =

    Yan V. Fyodorov , title =. Journal of Statistical Mechanics: Theory and Experiment , volume =. 2016 , doi =

    2016

  6. [6]

    2011 , publisher =

    Michel Talagrand , title =. 2011 , publisher =

    2011

  7. [7]

    2013 , publisher =

    Dmitry Panchenko , title =. 2013 , publisher =

    2013

  8. [8]

    Random Matrices and Complexity of Spin Glasses , journal =

    Antonio Auffinger and G. Random Matrices and Complexity of Spin Glasses , journal =. 2013 , doi =

    2013

  9. [9]

    Complexity of Random Smooth Functions on the High-Dimensional Sphere , journal =

    Antonio Auffinger and G. Complexity of Random Smooth Functions on the High-Dimensional Sphere , journal =. 2013 , doi =

    2013

  10. [10]

    Inventiones mathematicae , volume =

    Eliran Subag , title =. Inventiones mathematicae , volume =. 2017 , doi =

    2017

  11. [11]

    Hopfield , title =

    John J. Hopfield , title =. Proceedings of the National Academy of Sciences , volume =. 1982 , doi =

    1982

  12. [12]

    Ackley and Geoffrey E

    David H. Ackley and Geoffrey E. Hinton and Terrence J. Sejnowski , title =. Cognitive Science , volume =. 1985 , doi =

    1985

  13. [13]

    Amit and Hanoch Gutfreund and Haim Sompolinsky , title =

    Daniel J. Amit and Hanoch Gutfreund and Haim Sompolinsky , title =. Physical Review Letters , volume =. 1985 , doi =

    1985

  14. [14]

    The Loss Surfaces of Multilayer Networks , booktitle =

    Anna Choromanska and Mikael Henaff and Michael Mathieu and G. The Loss Surfaces of Multilayer Networks , booktitle =. 2015 , pages =

    2015

  15. [15]

    Comparing Dynamics: Deep Neural Networks versus Glassy Systems , booktitle =

    Marco Baity-Jesi and Levent Sagun and Mario Geiger and Stefano Spigler and G. Comparing Dynamics: Deep Neural Networks versus Glassy Systems , booktitle =. 2018 , pages =

    2018

  16. [16]

    Comparing Dynamics: Deep Neural Networks versus Glassy Systems , journal =

    Marco Baity-Jesi and Levent Sagun and Mario Geiger and Stefano Spigler and G. Comparing Dynamics: Deep Neural Networks versus Glassy Systems , journal =. 2019 , doi =

    2019

  17. [17]

    Dauphin and Razvan Pascanu and Caglar Gulcehre and Kyunghyun Cho and Surya Ganguli and Yoshua Bengio , title =

    Yann N. Dauphin and Razvan Pascanu and Caglar Gulcehre and Kyunghyun Cho and Surya Ganguli and Yoshua Bengio , title =. NeurIPS , year =

  18. [18]

    A Dynamic Programming Approach to the Parisi Functional

    J. Jagannath and I. Tobasco , title =. arXiv preprint arXiv:1502.04398 , year =. 1502.04398 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    J. M. Yeomans , title =. 1992 , isbn =

    1992

  20. [20]

    Foundations and Trends in Machine Learning , volume =

    Andrea Montanari and Subhabrata Sen , title =. Foundations and Trends in Machine Learning , volume =. 2024 , publisher =

    2024

  21. [21]

    Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions , journal =

    Valentina Ros and G. Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions , journal =. 2019 , doi =

    2019

  22. [22]

    Wilson , title =

    Kenneth G. Wilson , title =. Scientific American , year =

  23. [23]

    Why Are You Wrong? Coun- terfactual Explanations for Language Grounding with 3D Objects

    Alejandro Asc. Improving the Generation of. 2025 International Joint Conference on Neural Networks (IJCNN) , year =. doi:10.1109/IJCNN64981.2025.11227651 , url =

    work page doi:10.1109/ijcnn64981.2025.11227651 2025

  24. [24]

    VAE with Hyperspherical Coordinates: Improving Anomaly Detection from Hypervolume-Compressed Latent Space

    Alejandro Asc. VAE with Hyperspherical Coordinates: Improving Anomaly Detection from Hypervolume-Compressed Latent Space , year =. doi:10.48550/arXiv.2601.18823 , url =. 2601.18823 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2601.18823

  25. [25]

    2009 , institution =

    Alex Krizhevsky , title =. 2009 , institution =

    2009

  26. [26]

    Advances in Neural Information Processing Systems 30 (NeurIPS 2017) , pages =

    Heusel, Martin and Ramsauer, Hubert and Unterthiner, Thomas and Nessler, Bernhard and Hochreiter, Sepp , title =. Advances in Neural Information Processing Systems 30 (NeurIPS 2017) , pages =. 2017 , publisher =

    2017

  27. [27]

    Fu, Hao and Li, Chunyuan and Liu, Xiaodong and Gao, Jianfeng and Celikyilmaz, Asli and Carin, Lawrence , title =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , editor =. 2019 , month = jun, address =. doi:10.18653/v1/N19-102...

    work page doi:10.18653/v1/n19-1021 2019