Pith. sign in

REVIEW 5 cited by

Improving the Diffusability of Autoencoders

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.14831 v3 pith:WWOBF6FI submitted 2025-02-20 cs.CV cs.AIcs.LG

Ivan Skorokhodov , Sharath Girish , Benran Hu , Willi Menapace , Yanyu Li , Rameen Abdal , Sergey Tulyakov , Aliaksandr Siarohin This is my paper
classification cs.CVcs.AIcs.LG
keywords diffusiongenerationlatentautoencodersqualityautoencodercodecomponents
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Latent diffusion models have emerged as the leading approach for generating high-quality images and videos, utilizing compressed latent representations to reduce the computational burden of the diffusion process. While recent advancements have primarily focused on scaling diffusion backbones and improving autoencoder reconstruction quality, the interaction between these components has received comparatively less attention. In this work, we perform a spectral analysis of modern autoencoders and identify inordinate high-frequency components in their latent spaces, which are especially pronounced in the autoencoders with a large bottleneck channel size. We hypothesize that this high-frequency component interferes with the coarse-to-fine nature of the diffusion synthesis process and hinders the generation quality. To mitigate the issue, we propose scale equivariance: a simple regularization strategy that aligns latent and RGB spaces across frequencies by enforcing scale equivariance in the decoder. It requires minimal code changes and only up to 20K autoencoder fine-tuning steps, yet significantly improves generation quality, reducing FID by 19% for image generation on ImageNet-1K $256^2$ and FVD by at least 44% for video generation on Kinetics-700 $17 \times 256^2$. The source code is available at https://github.com/snap-research/diffusability.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents

    cs.CV 2026-08 conditional novelty 7.0 of 10

    SPAE compresses pretrained vision latents into a 64-channel bottleneck with channel-wise masking, improving image generation while keeping reconstruction and semantic understanding strong.

  2. Reputation Effects: Robustness and Fragility

    econ.TH 2026-05 unverdicted novelty 7.0 of 10

    Vanishingly small misspecification about signal structure eliminates reputation effects, bounding the long-lived strategic player's payoff by the complete-information level.

  3. SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks

    eess.AS 2026-08 conditional novelty 6.0 of 10

    SwanTale unifies instruction-driven and zero-shot speech and audio generation in one 48 kHz model, with a large captioning pipeline, and reports leading scores on several expressiveness and instruction-following benchmarks.

  4. Missing Fine Details in Images: Last Seen in High Frequencies

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A wavelet-based VAE that trains low- and high-frequency branches separately improves image reconstruction and diffusion generation.

  5. Home-made Diffusion Model from Scratch to Hatch

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A consumer-grade diffusion model trains on four RTX5090 GPUs for $535-620 and produces 1024x1024 anime-style images, claiming competitive quality and emergent camera control.

Pith tools