Pith. sign in

REVIEW 3 major objections 6 minor 57 references

MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorize

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-scale diffusion scheme that generates a low-resolution base then high-frequency residual details matches or beats full-resolution transformers while sampling over four times faster.

desk verdict A genuinely new two-scale residual-latent DiT that beats DiT-XL on FID with 4x sampling speedup; the decomposition is exact, but the paper overstates step comparability and needs a few sanity checks. read the letter →

arxiv 2501.13349 v2 pith:7K3IQLLT submitted 2025-01-23 cs.CV

Haohang Xu , Longyu Chen , Yichen Zhang , Shuangrui Ding , Zhipeng Zhang This is my paper · ORCID
classification cs.CV
keywords diffusionmodelslatentmulti-scalefactorizationresiduallearningefficientsamplingImageNetgenerationrectifiedflowTransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a diffusion model can generate high-resolution images by splitting the VAE latent into a low-resolution base and high-frequency residual components, generating the base first and then the residual with far fewer denoising steps. The author argues that residual information is easier to model than full-resolution content, so the two-stage pipeline costs less than direct full-resolution generation while improving fidelity. On class-conditional ImageNet, the method reports FID 2.08 at 256x256 and 2.47 at 512x512, beating the DiT baseline, and samples 64 images at 512x512 in 76 seconds versus 326 seconds for DiT-XL. A sympathetic reader would care because it suggests a simple architectural decomposition, not extra distillation or a new solver, can make diffusion transformers both cheaper and better.

What carries the argument

The load-bearing mechanism is multi-scale residual latent factorization over a pretrained VAE. Given a full latent $\hat{f}$, the base $f_0$ is obtained by encoding a downsampled image (latent-space downsampling is rejected as lossy), and each subsequent residual is $f^{h_i}_{\text{resi}} = \text{Down}(\hat{f} - \text{Up}(f^{h_{i-1}}_{\text{resi}}))$ (Algorithm 1). During sampling, priors are built by accumulating upsampled residuals and downsampling (Algorithm 2), and the model autoregressively predicts each residual conditioned on the class label and prior (Algorithm 3). The network is a single DiT trained with a rectified-flow velocity objective, with per-scale segment embeddings; residual targets are trained with teacher forcing. These operations turn the expensive full-resolution denoising task into a low-resolution base task plus short residual refinements that are easier to model.

What would settle it

Encode an ImageNet image with the pretrained VAE, run Algorithms 1 and 2 to decompose and re-accumulate the latent, decode it, and compare to the original image with FID, LPIPS, and mean squared error in latent space; if the reconstruction error is large or the accumulated latent differs substantially from $\hat{f}$, then the residual targets are not a faithful decomposition and the quality gains must come from elsewhere, not from the claimed factorization.

Watch

Extended reading notes

Core claim

The central discovery is that factorizing the denoising target into a base and residuals, rather than denoising the full-resolution latent directly, shifts most of the sampling cost to a low-resolution stage while a short, guidance-free residual stage supplies texture. Concretely, MSF trains one DiT backbone to predict the low-resolution base from a class label, then to predict residual latents at the target resolution conditioned on the accumulated prior; during sampling, residuals are upsampled and added to reconstruct the full latent, which the VAE decodes. With 100 steps at scale 0 and 20 at scale 1, it reports FID 2.08 (256x256) and 2.47 (512x512) on ImageNet, versus 2.27 and 3.04 for DiT-XL, and a more than 4x wall-clock speedup. The paper also shows the residual stage needs no classifier-free guidance and works with as few as 4 steps, and that combining scale-0 step distillation with MSF reaches 16 images per second at 512x512.

Load-bearing premise

The approach assumes that subtracting an upsampled base from a full VAE latent, downsampling the difference, and later accumulating residuals by upsampling and addition yields a latent that the decoder can turn into a high-quality image, meaning the latent space is closed under these linear resampling operations.

Editorial extensions

If this is right

  • Sampling cost at the residual stage can be cut to 4 steps with FID still comparable to DiT, meaning high-resolution texturing is the cheap part of generation.
  • Classifier-free guidance is unnecessary for the residual stage, halving the forward passes at that scale without hurting FID.
  • The method is orthogonal to step distillation: applying PeRFlow to scale 0 yields 16 images per second at 512x512 with competitive FID, so MSF can stack with existing accelerators.
  • Because the base stage dominates cost, further gains come from making the low-resolution stage faster, not the high-resolution stage.
  • Three-scale decomposition degrades quality (FID-5K 8.29 vs 5.44 for two-scale), attributed to error accumulation, so two scales is the recommended configuration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If residual latents are genuinely easier to model, the same base or residual factorization should transfer to text-to-image and video latents, where the prior from a cheap low-resolution stage could condition later detail stages.
  • The method's quality ceiling likely depends on how faithfully the linear accumulation in Algorithms 1-3 reconstructs the original VAE latent; a direct reconstruction-error test on ImageNet would separate factorization error from model error.
  • The 4x speedup is measured for a fixed step count on an A800; a fairer comparison would match total compute or training iterations, since MSF also trains the base stage to convergence first.
  • Because scale 1 needs no CFG, the residual stage could potentially run at lower precision or with token pruning (as in MSF-L*), suggesting further hardware-friendly optimizations not explored in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MSF, a multi-scale latent factorization framework for diffusion models. Given a pretrained VAE latent, MSF decomposes it into a low-resolution base and a sequence of high-resolution residuals via down/up-sampling operations (Algorithms 1-3). A single DiT backbone generates the base at low resolution and the residuals at higher resolutions with fewer sampling steps, and the outputs are accumulated to form the final latent. On class-conditional ImageNet, MSF-L reports FID 2.08 at 256x256 and 2.47 at 512x512, outperforming DiT-XL (2.27 and 3.04) while being roughly 4x faster in wall-clock sampling time, and the paper also shows a combination with PeRFlow step distillation.

Significance. If the reported results hold, MSF provides a practical, orthogonal speed-up for latent diffusion transformers, with particularly notable gains at 512x512 (FID 2.47 vs. DiT-XL 3.04 at a 4x speed-up). The paper contains direct comparisons against the DiT family with matched architecture sizes, ablations on CFG and on the number of scales, and an integration with PeRFlow that demonstrates compatibility with existing acceleration techniques. The main caveats are that the fidelity of the proposed latent decomposition is not verified, and the abstract's claim of 'same number of sampling steps' is inaccurate when checked against Table 2. If these issues are properly addressed with additional experiments and corrected exposition, the contribution would be a useful building block for efficient high-resolution generation.

major comments (3)
  1. [Sec. 3.2, Algorithms 1 and 3] The paper never verifies that the multi-scale residual decomposition actually reconstructs the original VAE latent. Under teacher forcing, the quantity f_hat - Up(f^{h0}_resi) - Up(f^{h1}_resi) (the leftover after the last scale) is discarded, and no experiment reports its magnitude or its effect on decoded images. If this discarded residual is large, the residual stage is not modeling high-frequency detail but is simply omitting part of the signal, and the reported FID improvement could come from the low-resolution base plus decoder robustness rather than from the proposed factorization. Please report a reconstruction error (e.g., latent MSE or decoded pixel-level metrics such as LPIPS/PSNR) when ground-truth residuals are used in Algorithm 3, and ablate the effect of discarding the final residual.
  2. [Abstract and Sec. 4.2, Table 2] The abstract claims a '4x speed-up with the same number of sampling steps,' but Table 2 compares MSF-L with 100+20 steps (120 total) against DiT-XL with 100 steps. The step counts are not the same. Please correct the wording (e.g., 'with comparable sampling steps' or 'at a 4x wall-clock speed-up') or provide an additional comparison in which DiT-XL also uses 120 steps, so the speed-up claim is precisely quantified.
  3. [Table 1, ImageNet 256x256] MSF-L achieves FID 2.08 vs. DiT-XL's 2.27 but its Inception Score is 245.5, substantially lower than DiT-XL's 278.2. The paper's claim of 'superior generation quality' rests entirely on FID and does not discuss this trade-off. Please address this discrepancy, for example by reporting additional metrics (sFID, FID with different feature layers) or by explaining why FID is the appropriate primary metric for this comparison.
minor comments (6)
  1. [Section 5 heading] The heading contains a typo: 'Effects of the Nmuber of Scales' should read 'Effects of the Number of Scales.'
  2. [Equation (4)] The conditioning term in the loss is written as f^{h_i-1}_resi; this should be f^{h_{i-1}}_resi for clarity, and it would be helpful to clarify how this term relates to the prior \f\(f^{h_i}\) defined in Algorithm 2 and used in Algorithm 3.
  3. [Section 4.1] The text states that the number of training iterations for scale 1 is approximately 1/4 of that for scale 0, but Table 1 reports 200K iterations for scale 1 and 1025K for scale 0, which is closer to 1/5; please correct or rephrase.
  4. [Figure 2] Figures 1 and 2 are information-dense; the Up/Down operations that mirror Algorithms 1-3 are not explicitly labeled in the figure, which makes it hard to connect the diagram to the algorithm boxes. Please annotate them.
  5. [General] The paper does not state whether code or trained models will be released; adding a reproducibility statement would strengthen the submission.
  6. [References] Reference [30] (Rectified Flow) is missing a publication year and venue; please standardize the reference format across the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MSF's FID and speed claims are measured against external baselines, and its multi-scale residual decomposition is an operational construction rather than a result derived from its own outputs.

full rationale

I walked the derivation chain from the proposed multi-scale residual extraction (Alg. 1), through the priors and next-scale prediction objective (Alg. 2, Eq. 4), to the autoregressive sampling and accumulation (Alg. 3). The residual latents are defined by an explicit subtract-and-downsample procedure on the pretrained VAE latent, and the sampling procedure is the corresponding accumulate-and-decode operation. This is an operational factorization, not a hidden identity: the generated residuals are sampled from a learned velocity field, and the final image is decoded from their accumulated sum. The reported FID numbers are measured with the standard external ADM 50K evaluation script against external baselines (DiT-XL, DiT-L, ADM, etc.), and the speed comparison is a wall-clock measurement on an A800 GPU (Table 2). The CFG scale and step-count choices are ordinary hyperparameter tuning on FID-5K, not a fitted parameter relabeled as a prediction. I found no load-bearing self-citations: the cited components (Stable Diffusion VAE, DiT, rectified flow, PeRFlow) are independent external works. The paper does assert, rather than prove, that residual information is easier to model, and it does not report a reconstruction-error check for the full residual chain; those are correctness and evidence gaps, not circularity. No equation in the paper is defined in terms of the claim it is used to support, and no reported result is forced by construction from the method's own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on a small set of tuned hyperparameters and two domain assumptions: latent-space linearity of the VAE and the residual-easy hypothesis. No new physical or architectural entities are invented; the multi-scale residual latents are a representational device, not an independent entity.

free parameters (3)
  • Classifier-free guidance scale at scale 0 = 1.3
    Tuned on FID-5K in Table 3a; affects reported FID and the speed claim because CFG is disabled at scale 1.
  • Sampling steps for scale 0 and scale 1 = 100 + 20
    Chosen in Sec. 4.2 and Table 2; the efficiency claim depends on this budget and the base stage still uses 100 steps.
  • Number of scales and base resolution = 2 scales, base at 192x192
    Selected from Table 4 experiments; a three-scale version degrades FID, so the central result is tied to the two-scale configuration.
assumptions (3)
  • domain assumption The pretrained VAE latent space supports interpolation, subtraction, and accumulation without information loss sufficient for high-quality decoding.
    Algorithms 1-3 and Sec. 3.2 rely on this; the paper acknowledges direct latent-space downsampling loses information for the base, yet applies similar operations to residuals and never reports a reconstruction error check.
  • domain assumption High-frequency residual information is intrinsically easier to model and requires fewer sampling steps than full-resolution generation.
    This is the efficiency premise stated in Sec. 1 and used to justify the reduced sampling budget at scale 1; it is validated only indirectly through FID tables, not by an information-theoretic or controlled complexity argument.
  • domain assumption The pretrained Stable Diffusion VAE provides a sufficiently good latent space for class-conditional generation.
    Standard LDM assumption, cited via [43]; the paper does not analyze failure cases of the VAE decoder on accumulated residual latents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorize." pith.science (2026). https://pith.science/paper/7K3IQLLT

@misc{pith2026250113349,
  author       = {Pith},
  title        = {Pith review of: MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorize},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7K3IQLLT}},
  note         = {Machine review of arXiv:2501.13349}
}
abstract

While diffusion-based generative models have made significant strides in visual content creation, conventional approaches face computational challenges, especially for high-resolution images, as they denoise the entire image from noisy inputs. This contrasts with signal processing techniques, such as Fourier and wavelet analyses, which often employ hierarchical decompositions. Inspired by such principles, particularly the idea of signal separation, we introduce a diffusion framework leveraging multi-scale latent factorization. Our framework uniquely decomposes the denoising target, typically latent features from a pretrained Variational Autoencoder, into a low-frequency base signal capturing core structural information and a high-frequency residual signal that contributes finer, high-frequency details like textures. This decomposition into base and residual components directly informs our two-stage image generation process, which first produces the low-resolution base, followed by the generation of the high-resolution residual. Our proposed architecture facilitates reduced sampling steps during the residual learning stage, owing to the inherent ease of modeling residual information, which confers advantages over conventional full-resolution generation techniques. This specific approach of decomposing the signal into a base and a residual, conceptually akin to how wavelet analysis can separate different frequency bands, yields a more streamlined and intuitive design distinct from generic hierarchical models. Our method, \name\ (Multi-Scale Factorization), demonstrates its effectiveness by achieving FID scores of 2.08 ($256\times256$) and 2.47 ($512\times512$) on class-conditional ImageNet benchmarks, outperforming the DiT baseline (2.27 and 3.04 respectively) while also delivering a $4\times$ speed-up with the same number of sampling steps.

Figures

Figures reproduced from arXiv: 2501.13349 by the authors.

Figure 1
Figure 1. An illustration of MSF, where diffusion models generate hierarchical representations [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of the proposed MSF. Left: The figure on the left illustrates the process of extracting multi-scale residual latents f hi resi using the pretrained VAE. Middle: The middle depicts the construction process of multi-scale priors fˆhi and how the diffusion model takes fˆhi as input to generate the ground-truth residual latents f hi resi. Right: During sampling, we first generate the residual latents f hi re… view at source ↗
Figure 3
Figure 3. Some generated 256 × 256 and 512 × 512 samples by MSF trained on ImageNet. the sampling steps of all scales. The results indicate that the generation quality degrades compared to the two-scale setting. We hypothesize that this is due to error accumulation as the number of scales increases. Therefore, we recommend using a two-scale configuration for MSF. 6 Conclusion [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 18 canonical work pages

  1. [1]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22669–22679, 2023

    2023

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  3. [3]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023

    2023

  4. [4]

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11305–11315, 2022

    work page 2022

  5. [5]

    Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.04692, 2024

    arXiv 2024

  6. [6]

    Pixart- {\delta}: Fast and controllable image generation with latent consistency models

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart- {\delta}: Fast and controllable image generation with latent consistency models. arXiv preprint arXiv:2401.05252, 2024

    arXiv 2024

  7. [7]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

    arXiv

  8. [8]

    Weiss, Mohammad Norouzi, Najim Dehak, and William Chan

    Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, Najim Dehak, and William Chan. Wavegrad 2: Iterative refinement for text-to-speech synthesis. ArXiv, abs/2106.09660, 2021

    arXiv 2021

Show all 57 references
  1. [9]

    Infogan: Interpretable representation learning by information maximizing generative adversarial nets

    Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016

    2016

  2. [10]

    Emu: Enhancing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, et al. Emu: Enhancing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023

    2023 arXiv

  3. [11]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  4. [12]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  5. [13]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12873–12883, 2021

    2021

  6. [14]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23164–23173, 2023

    2023

  7. [15]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 11

    2014

  8. [16]

    TeaForN: Teacher-forcing with n-grams

    Sebastian Goodman, Nan Ding, and Radu Soricut. TeaForN: Teacher-forcing with n-grams. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8704–8717, Online, Novemb...

    2020

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  10. [18]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022

    arXiv

  11. [19]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  12. [20]

    Cascaded diffusion models for high fidelity image generation

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1–33, 2022

    2022

  13. [21]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  14. [22]

    Scaling up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023

    2023

  15. [23]

    Guided-tts: A diffusion model for text-to- speech via classifier guidance

    Heeseung Kim, Sungwon Kim, and Sungroh Yoon. Guided-tts: A diffusion model for text-to- speech via classifier guidance. In International Conference on Machine Learning, 2021

    2021

  16. [24]

    Distilling ode solvers of diffusion models into smaller steps

    Sanghwan Kim, Hao Tang, and Fisher Yu. Distilling ode solvers of diffusion models into smaller steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9410–9419, 2024

    2024

  17. [25]

    Understanding diffusion objectives as the elbo with simple data augmentation

    Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation. Advances in Neural Information Processing Systems, 36, 2024

    2024

  18. [26]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  19. [27]

    Diffwave: A versatile diffusion model for audio synthesis

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations

  20. [28]

    Sdxl-lightning: Progressive adversarial diffusion distillation, 2024

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation, 2024

    2024

  21. [29]

    Playground v3: Improving text-to-image alignment with deep-fusion large language models

    Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Improving text-to-image alignment with deep-fusion large language models. arXiv preprint arXiv:2409.10695, 2024

    2024 arXiv

  22. [30]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations

  23. [31]

    Pyramid diffusion for fine 3d large scene generation

    Yuheng Liu, Xinke Li, Xueting Li, Lu Qi, Chongshou Li, and Ming-Hsuan Yang. Pyramid diffusion for fine 3d large scene generation. arXiv preprint arXiv:2311.12085, 2023

    2023 arXiv

  24. [32]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  25. [33]

    Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022

    2022 arXiv

  26. [34]

    Fit: Flexible vision transformer for diffusion model

    Zeyu Lu, Zidong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, and Lei Bai. Fit: Flexible vision transformer for diffusion model. arXiv preprint arXiv:2402.12376, 2024. 12

    2024 arXiv

  27. [35]

    Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740, 2024

    2024 arXiv

  28. [36]

    Conditional generative adversarial nets

    Mehdi Mirza. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014

    2014 arXiv

  29. [37]

    Swiftbrush: One-step text-to-image diffusion model with variational score distillation

    Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One-step text-to-image diffusion model with variational score distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7807–7816, June 2024

    2024

  30. [38]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021

    2021

  31. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  32. [40]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  33. [41]

    Vadim Popov, Ivan V ovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail A. Kudinov. Grad-tts: A diffusion probabilistic model for text-to-speech. In International Conference on Machine Learning, 2021

    2021

  34. [42]

    Generating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aäron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. In Neural Information Processing Systems, 2019

    2019

  35. [43]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  36. [44]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  37. [45]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022

    2022

  38. [46]

    Stylegan-xl: Scaling stylegan to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022

    2022

  39. [47]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Learning Representations

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  41. [50]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  42. [51]

    Givt: Generative infinite-vocabulary transformers

    Michael Tschannen, Cian Eastwood, and Fabian Mentzer. Givt: Generative infinite-vocabulary transformers. In European Conference on Computer Vision, pages 292–309. Springer, 2025

    2025

  43. [52]

    Fitv2: Scalable and improved flexible vision transformer for diffusion model

    ZiDong Wang, Zeyu Lu, Di Huang, Cai Zhou, Wanli Ouyang, et al. Fitv2: Scalable and improved flexible vision transformer for diffusion model. arXiv preprint arXiv:2410.13925, 2024. 13

    2024 arXiv

  44. [53]

    PeRFlow: Piecewise rectified flow as universal plug-and-play accelerator

    Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, qiang liu, and Jiashi Feng. PeRFlow: Piecewise rectified flow as universal plug-and-play accelerator. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  45. [54]

    Vector-quantized image modeling with improved vqgan

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. ArXiv, abs/2110.04627, 2021

    2021 arXiv

  46. [55]

    Diffusion models need visual priors for image generation

    Xiaoyu Yue, Zidong Wang, Zeyu Lu, Shuyang Sun, Meng Wei, Wanli Ouyang, Lei Bai, and Luping Zhou. Diffusion models need visual priors for image generation. arXiv preprint arXiv:2410.08531, 2024

    2024 arXiv

  47. [56]

    Show-1: Marrying pixel and latent diffusion models for text-to-video generation

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to-video generation. International Journal of Computer Vision, pages 1–15, 2024

    2024

  48. [57]

    Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics

    Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. Advances in Neural Information Processing Systems, 36:55502–55542, 2023

    2023

  49. [58]

    Fast ode-based sampling for diffusion models in around 5 steps

    Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7777–7786, 2024. 14

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.