EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

Ioannis Kakogeorgiou; Nikos Komodakis; Spyros Gidaris; Theodoros Kouzelis

REVIEW 3 major objections 6 minor 6 cited by

EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that fine-tuning an image autoencoder with an equivariance regularizer reshapes its latent space so downstream generative models train several times faster, reaching lower FID in far fewer steps.

desk verdict EQ-VAE is a genuinely useful empirical result - fine-tuning a pretrained autoencoder with an implicit equivariance objective speeds up latent generative training - but the 7x speedup claim outruns the reported numbers and the mechanism story is looser than the title suggests. read the letter →

arxiv 2502.09509 v3 pith:IEHKLWFQ submitted 2025-02-13 cs.LG

Theodoros Kouzelis , Ioannis Kakogeorgiou , Spyros Gidaris , Nikos Komodakis This is my paper · ORCID

classification cs.LG

keywords latentgenerativemodelsautoencoderequivariancespatialtransformationsdiffusiontransformersmaskedmodelingspaceregularizationintrinsicdimensionimagesynthesis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing autoencoders used in latent image generation encode scaled or rotated versions of an image into latent codes that do not transform in the same way, so the generative model has to learn unnecessary nonlinear structure. This paper proposes EQ-VAE, a regularization that fine-tunes a pretrained autoencoder so that decoding a transformed latent reproduces the transformed image, implicitly pushing the encoder toward equivariance. After only five epochs of fine-tuning, equivariance error and the intrinsic dimension of the latent manifold drop while reconstruction quality is preserved. Downstream generative models trained on these latents converge substantially faster, with the paper reporting DiT-XL/2 reaching a generative FID of 8.8 at 1.5 million iterations versus 9.6 at 7 million for the baseline, and corresponding speedups for SiT, REPA, and MaskGIT.

What carries the argument

The central object is the EQ-VAE objective, Equation (5), an implicit equivariance regularization that replaces the autoencoder's standard loss with $\mathcal{L}_{\mathrm{rec}}(\tau \circ x, D(\tau \circ E(x))) + \lambda_{\mathrm{gan}}\mathcal{L}_{\mathrm{gan}}(D(\tau \circ E(x))) + \lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}$, where $\tau$ is sampled from compositions of anisotropic scaling and 90-degree rotations. The mechanism is indirect: the decoder can only faithfully invert transformed latents if those latents already carry the transformation structure, so gradient descent through reconstruction reshapes the encoder into an approximately equivariant map without a direct latent-norm penalty. To preserve the autoencoder's original reconstruction behavior, the identity transform is sampled with probability $p_\alpha = 0.5$, and the method is adapted to discrete autoencoders by applying $\tau$ to encoder features before quantization.

What would settle it

Fine-tune the same autoencoder with the EQ-VAE objective but stop gradients from the equivariance term at the encoder, so only the decoder adapts; if generative FID improves as much as with full fine-tuning, the encoder's equivariance is not the causal driver. A companion check is to add a control fine-tuning that rescales latent statistics without any equivariance loss and compare generative FID curves.

Watch

Extended reading notes

Core claim

The paper's central claim is that the latent space of standard autoencoders is a major, overlooked bottleneck in latent generative modeling because it is not equivariant under scaling and rotation, and that this bottleneck can be removed by a simple change to the training objective. The authors introduce an implicit equivariance regularizer: instead of minimizing reconstruction loss on ordinary pairs, they transform the latent code with $\tau$, decode it, and minimize reconstruction, adversarial, and regularizer losses against the transformed image $\tau \circ x$. Directly penalizing $\| \tau \circ E(x) - E(\tau \circ x)\|^2$ collapses the latent space, so the implicit route is the one that works. The paper shows that fine-tuning four continuous autoencoders and VQ-GAN with this objective reduces equivariance error without hurting reconstruction, lowers the intrinsic dimension of the latent manifold, and accelerates training of DiT, SiT, REPA, and MaskGIT, with the strongest result being a 7x speedup for DiT-XL/2.

Load-bearing premise

The load-bearing premise is that the implicit reconstruction loss genuinely makes the encoder equivariant, rather than only making the decoder tolerant of transformed latents, and that the resulting equivariance, not a side effect of extra fine-tuning, is what accelerates downstream generative training.

Editorial extensions

If this is right

Diffusion and masked generative models built on an EQ-VAE-finetuned tokenizer inherit faster convergence with no changes to the generative architecture.
Autoencoder quality can be assessed and reported in terms of equivariance error and intrinsic dimension, not only reconstruction FID, giving model developers a cheap proxy for downstream training difficulty.
Because both continuous and discrete autoencoders benefit, the regularizer transfers across diffusion, flow-based, and masked token modeling families.
The method needs no architectural changes, so it can be dropped into existing latent generative pipelines as a first-stage fine-tuning step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Testable implication: equivariance error of a tokenizer could serve as a cheap predictor of downstream generative training difficulty, enabling autoencoder selection without training a full generator.
The paper's correlation between intrinsic dimension and generative FID suggests a control experiment: deliberately inflating latent intrinsic dimension, without changing equivariance, should slow generative training if the mechanism is geometric.
The transformation set is limited to sub-unit scaling and multiples of 90 degrees; extending to translations, flips, or continuous rotations may yield further gains or reveal a trade-off with reconstruction fidelity.
Models that already apply heavy spatial augmentation during generative training might see less benefit, since the generator would already be learning the transformation structure.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here is the bottom line: fine-tune a pretrained latent autoencoder for a few epochs with an objective that asks the reconstruction of a transformed latent (D(tau o E(x))) to match the transformed image (tau o x), and downstream latent generative models - DiT, SiT, REPA, MaskGIT - train substantially faster and reach better FID. The effect is consistent across continuous and discrete autoencoders, reconstruction quality is preserved or slightly improved, and Table 6 rules out the boring explanation that extra fine-tuning alone does it. That is worth knowing.

The novelty is real, if modest. Equivariance regularization for VAEs is not new (Sinha & Dieng 2021; Ryu 2024), and the paper says so. What's new is applying an implicit equivariance objective to off-the-shelf autoencoders and showing the downstream generative payoff. That is a useful contribution, and the paper's honest handling of prior work helps.

Soft spots, in proportion. First, the advertised 7x speedup is not what the tables show. Table 2 reports EQ-VAE DiT-XL/2 at 1.5M iterations with GFID 8.8 versus baseline 9.6 at 7M - that's roughly 4.7x, and the baseline's 8.8 point is not reported. The 7x needs a definition or a correction. Second, the headline results are single runs with no variance. In a regime where a 0.3 FID difference is treated as meaningful, one seed is thin; the authors should report a few seeds for at least the main comparisons. Third, the mechanism story is ahead of the evidence. Table 7 shows that an explicit equivariance loss reduces equivariance error far more than EQ-VAE yet destroys generation (GFID 141). So reducing equivariance error is not what drives the improvement; the particular reconstruction-based regularizer matters, and the causal channel is not isolated. The authors acknowledge the explicit loss's collapse in the appendix, but they do not provide an intervention that would pin the benefit on equivariance per se. This is a caveat, not a deal-breaker - the method works - but the narrative should be softened. Fourth, the transformation hyperparameters were tuned on the same ImageNet benchmark used for evaluation; a cross-dataset check would help.

Bottom line: this is a solid, practical paper for anyone doing latent generative modeling. It deserves a serious referee. I would send it out, ask for a corrected speedup claim and seed variance, and not block on the mechanism question.

Referee Report

3 major / 6 minor

Summary. The paper proposes EQ-VAE, a regularization for autoencoders used in latent generative models. The authors observe that existing autoencoders such as SD-VAE produce latent codes that are not equivariant under scaling and rotation. They define an implicit loss (Eq. 5) that penalizes the reconstruction error between a transformed latent code and the correspondingly transformed input, and fine-tune pretrained autoencoders for five epochs. They report that this reduces equivariance error and intrinsic dimension of the latent space, and that downstream generative models (DiT, SiT, REPA, MaskGIT) trained on the regularized latents converge faster and achieve lower FID, with a claimed ×7 speedup for DiT-XL/2. They also extend the approach to discrete VQ-GAN.

Significance. If the central claims hold, EQ-VAE is a simple, architecture-agnostic plug-in that improves the trainability of latent generative models without reconstruction quality loss. The paper has notable strengths: it evaluates across four autoencoders (continuous and discrete) and four generative model families; it includes a control for extra training (Table 6) and an explicit-versus-implicit comparison (Table 7); it quantifies equivariance error and intrinsic dimension; and it provides code. However, the causal interpretation is not fully supported, and the headline speedup is not rigorously defined. These issues are fixable, and the empirical contribution is likely to be of interest to the community.

major comments (3)

[§4.3, Table 2, Figure 1, Abstract] The headline '×7 speedup' is not substantiated by the reported numbers. The comparison cited is EQ-VAE DiT-XL/2 at 1.5M iterations (GFID 8.8) versus the baseline at 7M iterations (GFID 9.6). These are different FID values at different iteration counts; a speedup factor requires the iteration count at which each model reaches a common GFID threshold. As reported, the iteration-count ratio is 7M/1.5M ≈ 4.7 and the FID values are not matched. Please report the iteration at which the baseline reaches GFID 8.8 (or EQ-VAE reaches 9.6) and compute the speedup at a matched FID. The same issue applies to the curves in Figure 1 and to the abstract's '×7 speedup' claim.
[§4.4, Table 7] The causal claim that equivariance-error reduction drives the improvement is undermined by the explicit-loss experiment. Equation (4) reduces equivariance error far more (rotation 0.32 vs. 0.49; scaling 0.11 vs. 0.15) yet yields GFID 141.3, far worse than the baseline 43.5. The paper attributes this to mode collapse, but no quantitative evidence (e.g., latent variance or identity-reconstruction quality) is provided. More importantly, this result shows that lower equivariance error is not sufficient for the benefit, so the improvement from Eq. (5) may be due to the particular reconstruction-based regularizer rather than equivariance per se. Since the paper's central claim is that equivariance simplifies the latent space and thereby accelerates generative training, an interventional control that varies equivariance error while holding the reconstruction regularizer fixed is needed, or the claims should be weakened to state that the proposed regularization improves generation.
[§3.3, Eq. (5)] A direct baseline is missing: fine-tuning the autoencoder with the standard objective (Eq. 1) on randomly scaled/rotated input images (i.e., input data augmentation) would isolate the effect of applying transformations to the latent code rather than to the input. The paper distinguishes itself from input augmentation but does not provide this comparison. If standard input augmentation yields similar GFID improvements, the equivariance mechanism is not the driver. Please add this control. Additionally, specify in Eq. (5) whether the adversarial loss Lgan is computed with a discriminator that is updated on transformed images and whether the GAN loss is conditioned on the transformation; as written, the behavior of the GAN term under transformed latents is underspecified.

minor comments (6)

[§3.2 title] The section title contains typos: 'Equivarance' should be 'Equivariance' and 'Tansformations' should be 'Transformations'.
[§4.1, Table 13] The text says 'we finetune all autoencoders on OpenImages', but Table 13 lists VQ-GAN as originally trained on ImageNet; please clarify whether VQ-GAN is also fine-tuned on OpenImages and whether this affects the comparison with the published MaskGIT baseline.
[Table 4] The caption and text refer to 'SD-VAE-FT-MAE', which appears to be a typo for 'SD-VAE-FT-EMA'.
[Figure 1 and throughout] There are several minor formatting issues, such as 'ourEQ-VAE' missing a space in Figure 1 and in the introduction; a final proofread would catch these.
[§4.4, Table 5] The claim that 'combining transformations lowers ID' is based on differences (41.0 vs. 39.4 vs. 38.9) that are small relative to the known noise sensitivity of the TwoNN estimator acknowledged in Appendix B; please report repeated estimates or confidence intervals for the ID values.
[§4.1 and Appendix E] The paper does not state the number of fine-tuning steps for each autoencoder beyond '5 epochs'; since batch sizes and dataset sizes may differ, please provide step counts for SD-VAE, SDXL-VAE, SD3-VAE, SD-VAE-16, and VQ-GAN.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EQ-VAE objective is defined without reference to downstream FID, the reported gains are empirical measurements, and the only self-citation is a non-load-bearing related-work reference.

full rationale

The paper's central derivation chain is not circular. The EQ-VAE objective (Eq. 5) is defined purely in autoencoder terms: Lrec(τx, D(τE(x))) plus adversarial and regularizer terms, with no term that references GFID, intrinsic dimension, or generative-model convergence. The reported equivariance-error reduction is an empirical outcome, not an identity: Eq. 5 optimizes agreement between D(τE(x)) and τx, while the measured equivariance error is ||τE(x) − E(τx)||² / ||E(τx)||²; these quantities coincide only under additional injectivity/equivariance assumptions that the paper does not make. Table 7 actually demonstrates the distinction: the explicit loss (Eq. 4) achieves far lower equivariance error (0.32/0.11 vs. 0.49/0.15) yet yields GFID 141.3 versus 34.1, so reducing equivariance error is neither equivalent to nor sufficient for the generative gain. The 'additional training' control in Table 6 and the epoch ablation in Figure 5 separate the effect from mere fine-tuning, and the speedups in Tables 2–4 are measured against external baselines. The single author-overlapping citation (Marcos et al. 2017, co-authored by Komodakis) appears only in the related-work survey of equivariant networks and is not load-bearing. The choice of transformations and pα=0.5 was tuned on the DiT-B GFID benchmark family that later reports improvements; this is a potential selection/overfitting concern but not circularity, since the final numbers are not implied by the objective by construction. The paper's own Table 7 does expose a causal-interpretation gap — lower equivariance error alone does not explain the gains — but a missing causal identification is a correctness risk, not a circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method introduces no new entities. Its load-bearing ingredients are the hyperparameters governing the equivariance loss and a set of domain assumptions about how equivariance, intrinsic dimension, and generative learnability relate; these are grounded in correlations and ablations rather than formal guarantees.

free parameters (4)

p_alpha = 0.5
Probability of applying the EQ-VAE objective vs original VAE loss in Eq. (7); set by default and ablated in Sec A.2.
finetuning epochs = 5
Number of epochs of autoencoder finetuning on OpenImages; Figure 5 shows performance varies with epochs, so this is a tunable hyperparameter.
transformation ranges = s in (0.25, 1), theta in {90, 180, 270} deg
Sampling ranges for scaling and rotation in Eq. (6); chosen to avoid corner artifacts and tuned via ablation (Table 5).
interpolation method = bicubic
Interpolation used for downsampling and rotating latents; affects how the discrete latent grid is transformed.

assumptions (5)

domain assumption Pretrained latent autoencoders are not equivariant under scaling and rotation.
Sec 3.2 and Figures 2-3 demonstrate the lack of equivariance empirically; this motivates the regularization.
domain assumption Minimizing Eq. (5) approximately enforces E(tau o x) = tau o E(x) without mode collapse.
Sec 3.3 and Sec A.1: explicit loss (Eq. 4) collapses, implicit loss does not. This is an observed behavior, not a theorem.
domain assumption Lower intrinsic dimension of the latent manifold makes the generative modeling task easier.
Sec 4.4, Table 5; correlation between ID and GFID is shown, but no causal proof is provided.
domain assumption Downstream generative model hyperparameters tuned for SD-VAE-FT-EMA latents remain suitable for EQ-VAE latents.
Sec 4.1: models follow default settings; if the latent distribution changes scale, retuning might be needed.
domain assumption Fine-tuning autoencoders on OpenImages transfers to ImageNet latent generation.
Sec 4.1 Implementation Details; a domain shift exists between datasets.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling." pith.science (2026). https://pith.science/paper/IEHKLWFQ

@misc{pith2026250209509,
  author       = {Pith},
  title        = {Pith review of: EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IEHKLWFQ}},
  note         = {Machine review of arXiv:2502.09509}
}

read the original abstract

Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: https://eq-vae.github.io/.

Figures

Figures reproduced from arXiv: 2502.09509 by the authors.

**Figure 1.** Latent Space Structure (Left) Top three principal components of SD-VAE and SDXL-VAE, with and without EQ-VAE, demonstrating visually that our regularization produces smoother latent representations without compromising reconstruction (See [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Latent Space Equivariance. Reconstructed images using SD-VAE (Rombach et al., 2022) and our EQ-VAE when applying scaling transformation τ , with factor s = 0.5, to the input images D(E(τ ◦ x)) versus directly to the latent representations D(τ ◦ E(x)). Our approach preserves reconstruction quality under latent transformations, whereas SD-VAE exhibits significant degradation. See Figure 6 for additional examples. i… view at source ↗

**Figure 4.** EQ-VAE accelerates generative modeling. We compare results from two DiT-XL/2 models at 50K, 100K, and 400K iterations, one trained with SD-VAE-FT-EMA (top) and with EQ-VAE (bottom). The same noise and number of sampling steps are used for both models, without classifier-free guidance. Our approach delivers faster improvements in image quality, demonstrating accelerated convergence. both DiT-B and DiT-XL models. Spec… view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: Rapid Improvement via EQ-VAE Fine-tuning. Even a single epoch of EQ-VAE fine-tuning significantly improves generative modeling performance, reducing GFID from 43.5 to 36.7. Generative modeling with DiT-B/2. train a DiT-B/2 on each latent distribution. In [PITH_FULL_IM…

**Figure 6.** Figure 6: Latent Space Equivariance. Reconstructed images using SD-VAE (Rombach et al., 2022) and EQ-VAE when applying transformations τ to the input images (D(E(τ ◦ x))) versus directly to the latent representations (D(τ ◦ E(x))). We present results for scaling and rotation tra…

**Figure 7.** Figure 7: Additional comparisons of latent representations across different VAE models. EQ-VAE (+Ours) produces smoother latent representations for both SD-VAE and SDXL-VAE. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

**Figure 8.** Figure 8: Uncurated samples 256 × 256 DiT-XL/2 /w EQ-VAE. Classifier-free guidance scale = 4.0. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents
cs.CV 2026-08 conditional novelty 7.0 of 10

SPAE compresses pretrained vision latents into a 64-channel bottleneck with channel-wise masking, improving image generation while keeping reconstruction and semantic understanding strong.
AnchorMark: Robust Diffusion Watermarking via Latent-Space Rotation Synchrony
cs.CR 2026-07 conditional novelty 6.0 of 10

Image rotation is shown to rotate the recovered latent by the same angle, and a central phase anchor exploits this to estimate and undo rotation before decoding the watermark.
ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling
cs.LG 2026-07 conditional novelty 6.0 of 10

A single-step IMLE generator with per-stage supervision and a robust loss reports FID 2.56 on ImageNet-256 by filtering ~5% of samples at test time.
Missing Fine Details in Images: Last Seen in High Frequencies
cs.CV 2025-09 conditional novelty 6.0 of 10

A wavelet-based VAE that trains low- and high-frequency branches separately improves image reconstruction and diffusion generation.
Home-made Diffusion Model from Scratch to Hatch
cs.CV 2025-09 conditional novelty 5.0 of 10

A consumer-grade diffusion model trains on four RTX5090 GPUs for $535-620 and produces 1024x1024 anime-style images, claiming competitive quality and emergent camera control.
Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders
cs.SD 2025-07 conditional novelty 5.0 of 10

A post-hoc Re-Bottleneck network trained only in latent space can impose ordering, semantic alignment, or equivariance on pre-trained audio autoencoder latents with little extra compute.

Reference graph

Works this paper leans on

22 extracted references · 6 canonical work pages · cited by 6 Pith papers

[6]

W., Cohen, T

Hoogeboom, E., Peters, J. W., Cohen, T. S., and Welling, M. Hexaconv. arXiv preprint arXiv:1803.02108,

arXiv
[8]

Exploring the rep- resentation manifolds of stable diffusion through the lens of intrinsic dimension

Kvinge, H., Brown, D., and Godfrey, C. Exploring the rep- resentation manifolds of stable diffusion through the lens of intrinsic dimension. arXiv preprint arXiv:2302.09301,

arXiv
[9]

Autoregres- sive image generation without vector quantization

Li, T., Tian, Y ., Li, H., Deng, M., and He, K. Autoregres- sive image generation without vector quantization. arXiv preprint arXiv:2406.11838,

arXiv
[14]

Visual autoregressive modeling: Scalable image generation via next-scale prediction

Tian, K., Jiang, Y ., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905,

arXiv
[16]

Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y

Yu, J., Li, X., Koh, J. Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y . Vector-quantized image modeling with improved VQGAN. In ICLR, 2022a. Yu, J., Xu, Y ., Koh, J. Y ., Luong, T., Baid, G., Wang, Z., Vasudevan, V ., Ku, A., Yang, Y ., Ayan, B. K., et al. Scal- ing autoregressive models for content-rich text-to-image genera...

arXiv
[17]

Fast training of diffusion models with masked transformers

Zheng, H., Nie, W., Vahdat, A., and Anandkumar, A. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305,

arXiv
[18]

Scaling the codebook size of VQ-GAN to 100,000 with a utilization rate of 99%

Zhu, L., Wei, F., Lu, Y ., and Chen, D. Scaling the codebook size of VQ-GAN to 100,000 with a utilization rate of 99%. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. Zhu, R., Pan, Y ., Li, Y ., Yao, T., Sun, Z., Mei, T., and Chen, C. W. Sd-dit: Unleashing the power of self-supervised discrimination in diffusion tra...

arXiv
[19]

We perform two experiments, with lower (pα = 0.7) and higher (pα = 0.3) regularization strength

AUTOENCODER pα GFID ↓ RFID ↓ EQ-VAE 0.3 35.4 0.78 EQ-VAE 0.7 34.4 0.88 EQ-VAE 0.5 34.1 0.82 Table 8: Ablation on regularization strength. We perform two experiments, with lower (pα = 0.7) and higher (pα = 0.3) regularization strength. We observe that our method is relatively robust to choices of pα. We highlight the setting used throughout all our experim...

work page 2023

Show all 22 references

[20]

Further Pope et al

have utilized ID to measure the complexity of latent representations in deep learning modeling. Further Pope et al. (2021) has demonstrated a strong correlation between a dataset’s relative difficulty and its ID. We compute the ID of the latent representations using the TwoNN ...

2021
[21]

The distance is calculated based on the assumption that both feature distributions follow multivariate Gaussian distributions

quantifies the feature distance between the distributions of two image datasets by leveraging the Inception-v3 network (Szegedy et al., 2016). The distance is calculated based on the assumption that both feature distributions follow multivariate Gaussian distributions. • sFID ...

2016
[22]

• LDM (Rombach et al.,

utilizes a bidirectional transformer decoder to synthesize images by iteratively predicting masked visual tokens produced by a VQ-GAN (Esser et al., 2021). • LDM (Rombach et al.,

2021
[1969]

Besnier, V

doi: 10.1109/TIT.1969.1054365. Besnier, V . and Chen, M. A pytorch reproduction of masked generative image transformer. arXiv preprint arXiv:2310.14400,

1969
[2004]

Weiler, M

doi: 10.1109/TIP.2003.819861. Weiler, M. and Cesa, G. General e (2)-equivariant steerable cnns. Advances in neural information processing systems, 32,

2003
[2016]

K., Motwani, S., and Wen, Y

Dai, X., Hou, J., Ma, C.-Y ., Tsai, S., Wang, J., Wang, R., Zhang, P., Vandenhende, S., Wang, X., Dubey, A., Yu, M., Kadian, A., Radenovic, F., Mahajan, D., Li, K., Zhao, Y ., Petrovic, V ., Singh, M. K., Motwani, S., and Wen, Y . Emu: Enhancing image generation models us- ing...

arXiv
[2017]

Nash, C., Menick, J., Dieleman, S., and Battaglia, P. W. Generating images with sparse representations. arXiv preprint arXiv:2103.03841,

arXiv
[2018]

N-body networks: a covariant hierarchical neu- ral network architecture for learning atomic potentials

Kondor, R. N-body networks: a covariant hierarchical neu- ral network architecture for learning atomic potentials. arXiv preprint arXiv:1803.01588,

arXiv
[2019]

Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds

Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., and Riley, P. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219,

arXiv
[2021]

A., Garnelo, M., Lee, M

Dilokthanakul, N., Mediano, P. A., Garnelo, M., Lee, M. C., Salimbeni, H., Arulkumaran, K., and Shanahan, M. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648,

arXiv
[2022]

Very deep convolutional networks for large- scale image recognition

Simonyan, K. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556,

arXiv
[2023]

Does equivariance matter at scale? arXiv preprint arXiv:2410.23179,

Brehmer, J., Behrends, S., de Haan, P., and Cohen, T. Does equivariance matter at scale? arXiv preprint arXiv:2410.23179,

arXiv
[2024]

Adversarial autoencoders

Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. Adversarial autoencoders. arXiv preprint arXiv:1511.05644,

arXiv
[2025]

Bridging information-theoretic and geometric compression in lan- guage models

Cheng, E., Kervadec, C., and Baroni, M. Bridging information-theoretic and geometric compression in lan- guage models. arXiv preprint arXiv:2310.13620,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [6]

W., Cohen, T

Hoogeboom, E., Peters, J. W., Cohen, T. S., and Welling, M. Hexaconv. arXiv preprint arXiv:1803.02108,

arXiv

[2] [8]

Exploring the rep- resentation manifolds of stable diffusion through the lens of intrinsic dimension

Kvinge, H., Brown, D., and Godfrey, C. Exploring the rep- resentation manifolds of stable diffusion through the lens of intrinsic dimension. arXiv preprint arXiv:2302.09301,

arXiv

[3] [9]

Autoregres- sive image generation without vector quantization

Li, T., Tian, Y ., Li, H., Deng, M., and He, K. Autoregres- sive image generation without vector quantization. arXiv preprint arXiv:2406.11838,

arXiv

[4] [14]

Visual autoregressive modeling: Scalable image generation via next-scale prediction

Tian, K., Jiang, Y ., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905,

arXiv

[5] [16]

Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y

Yu, J., Li, X., Koh, J. Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y . Vector-quantized image modeling with improved VQGAN. In ICLR, 2022a. Yu, J., Xu, Y ., Koh, J. Y ., Luong, T., Baid, G., Wang, Z., Vasudevan, V ., Ku, A., Yang, Y ., Ayan, B. K., et al. Scal- ing autoregressive models for content-rich text-to-image genera...

arXiv

[6] [17]

Fast training of diffusion models with masked transformers

Zheng, H., Nie, W., Vahdat, A., and Anandkumar, A. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305,

arXiv

[7] [18]

Scaling the codebook size of VQ-GAN to 100,000 with a utilization rate of 99%

Zhu, L., Wei, F., Lu, Y ., and Chen, D. Scaling the codebook size of VQ-GAN to 100,000 with a utilization rate of 99%. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. Zhu, R., Pan, Y ., Li, Y ., Yao, T., Sun, Z., Mei, T., and Chen, C. W. Sd-dit: Unleashing the power of self-supervised discrimination in diffusion tra...

arXiv

[8] [19]

We perform two experiments, with lower (pα = 0.7) and higher (pα = 0.3) regularization strength

AUTOENCODER pα GFID ↓ RFID ↓ EQ-VAE 0.3 35.4 0.78 EQ-VAE 0.7 34.4 0.88 EQ-VAE 0.5 34.1 0.82 Table 8: Ablation on regularization strength. We perform two experiments, with lower (pα = 0.7) and higher (pα = 0.3) regularization strength. We observe that our method is relatively robust to choices of pα. We highlight the setting used throughout all our experim...

work page 2023