Pith. sign in

REVIEW 4 major objections 6 minor 31 references

Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diffusion models can be converted into one-step generators by GAN fine-tuning alone, without any distillation loss, and diffusion training itself may be viewed as generative pre-training.

desk verdict Worth reading for the freezing recipe; the 'diffusion as pre-training' claim needs a from-scratch control it doesn't have. read the letter →

arxiv 2506.09376 v1 pith:EJ7VEY7Q submitted 2025-06-11 cs.LG

Bowen Zheng , Tianming Yang This is my paper
classification cs.LG
keywords diffusionmodelsone-stepgenerationdistillationGANobjectivegenerativepre-trainingparameterfreezingfrequency-domainanalysisimage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speeding up diffusion models usually means distillation: training a one-step student model to copy the outputs of a multi-step teacher. This paper argues that distillation is fighting a structural mismatch, because the teacher and student settle into different local minima, so direct imitation is the wrong target. It shows that a standalone GAN objective, with no distillation loss, converts a pre-trained diffusion model into a competitive one-step generator, and it reads this as evidence that diffusion training is generative pre-training that a light GAN fine-tune unlocks. If the claim holds, existing diffusion checkpoints can be turned into fast one-step generators with a small data budget and most parameters left frozen.

What carries the argument

The load-bearing mechanism is the local-minima mismatch between multi-step teacher and one-step student, measured by the FID between their output sets; the paper's remedy is the standalone GAN objective, which replaces instance-level imitation with distribution-level discrimination against real images. The main working system is D2O/D2O-F: a pre-trained diffusion U-Net acts as the generator, a Projected-GAN discriminator with $r_1$ regularization provides the learning signal, and in D2O-F the convolutional layers, 85.8% of parameters, are frozen while normalization layers, QKV projections, and skip connections are tuned. The frequency analysis uses the log-frequency difference $\operatorname{DFFT}(x_1,x_2)=\log(|\mathrm{FFT}(x_1)|+1)-\log(|\mathrm{FFT}(x_2)|+1)$ to show that during inference the model enhances low-frequency components first and higher frequencies later, which the authors offer as a preliminary explanation of the one-step capability gained in pre-training.

What would settle it

Train D2O with the same generator architecture, Projected-GAN discriminator, augmentation, and 5M-image budget from random initialization on CIFAR-10; if that from-scratch generator reaches an FID close to 1.54-1.66, the claim that diffusion pre-training supplies the one-step capability would be falsified, while a collapse or much worse FID would support it.

Watch

Extended reading notes

Core claim

The paper's central claim is that teacher and student in diffusion distillation converge to different local minima, because the teacher runs the network several times and has more effective parameters, while the one-step student passes through it once; forcing the student to match the teacher instance-by-instance therefore drags it toward a solution that is not its own optimum. A GAN objective sidesteps this: instead of copying the teacher's output, the student is scored against the real data distribution, so it can find a good one-step solution in its own parameter space. The authors show that this standalone GAN objective alone, applied to a pre-trained EDM-style diffusion U-Net with a Projected-GAN discriminator, reaches FID 1.66 on CIFAR-10 and FID 1.42 on ImageNet 64x64; freezing 85.8% of the generator's parameters, the convolutional layers, and tuning only normalization, QKV projections, and skip connections improves this to FID 1.54 and 1.16 with roughly 5 million training images. They interpret this as evidence that diffusion pre-training endows the network with one-step generative capability, and their frequency analysis shows inference progressively enhancing low then high frequencies, a pattern they take as a preliminary mechanism.

Load-bearing premise

The paper's central interpretation rests on the assumption that the strong one-step results come from the pre-trained diffusion weights, but no control trains the same GAN pipeline from a randomly initialized generator, so the gains could in principle come from the GAN objective and hyperparameters alone.

Editorial extensions

If this is right

  • One-step generation from a diffusion checkpoint becomes a light fine-tuning problem: roughly 5M images and a small subset of tunable layers replace a full distillation run.
  • The resulting one-step generators match or beat the multi-step teacher on FID across CIFAR-10, AFHQv2 64x64, FFHQ 64x64, and ImageNet 64x64, while sampling in a single network evaluation.
  • Distillation losses can be dropped: adding a consistency-distillation loss slows early convergence, doubles training cost, and does not beat the frozen GAN-only model.
  • Freezing most convolutional layers stabilizes GAN training and reduces mode collapse, so the usual instability of GAN fine-tuning is largely avoided.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The generative-pre-training interpretation would be tested directly by training D2O from a randomly initialized generator with the same discriminator and 5M-image budget; the paper does not run this control.
  • If the frequency-specialization story holds, one-step fine-tuning should work best when the frozen base already has well-developed low-to-high frequency dynamics, which predicts that the recipe transfers to DiT-style and latent diffusion models once the frequency analysis is repeated there.
  • The freezing result suggests a practical multi-task setup: one pre-trained diffusion base plus per-task fine-tuned normalization and skip layers could serve many one-step generators from a single checkpoint, although the paper does not explore this.
  • The standalone-GAN recipe may also apply to masked or autoregressive generative pre-training, since the paper sketches how its destruction-and-recovery view generalizes; that generalization is speculative.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that diffusion training can be reinterpreted as generative pre-training, and that a standalone GAN objective, without any instance-level distillation loss, is sufficient to convert a pre-trained diffusion model into an efficient one-step generator. The authors introduce D2O, which fine-tunes an EDM score model with a Projected-GAN discriminator, and D2O-F, which freezes 85.8% of the generator's parameters during fine-tuning. They report one-step FID values of 1.66 and 1.54 on CIFAR-10, 1.42 and 1.16 on ImageNet 64x64, and strong results on AFHQv2 and FFHQ 64x64 using 5-10M training images, which is substantially less than typical distillation budgets. A frequency-domain analysis of time-step-dependent low-to-high frequency enhancement is offered as a preliminary mechanistic explanation.

Significance. If the central attribution holds, the paper offers a practical recipe for reusing diffusion checkpoints as one-step generators and a conceptual reframing of diffusion training as generative pre-training. The empirical recipe is potentially valuable: evaluations span four datasets, CLIP-FID results are provided to address FID leakage concerns, the freezing ablation is informative, and the frequency analysis is a useful descriptive contribution. However, the central attribution rests on a missing from-scratch control, several cross-model comparisons are not fully controlled, and the local-minima mechanism is supported only by correlational evidence. The significance of the conceptual claim is therefore conditional on additional experiments.

major comments (4)
  1. [Sec. 4.1-4.2, Table 6] The load-bearing attribution claim, that D2O's data efficiency comes from diffusion generative pre-training rather than from the GAN fine-tuning recipe itself, is not tested against the correct control. The text argues that 0.2 million images are 'unlikely' to be sufficient to learn a complex distribution from scratch, but this is a heuristic appeal to conventional GAN data requirements. The D2O pipeline is not a standard GAN: it uses a Projected-GAN discriminator built on pretrained VGG16-BN, EfficientNet-lite0, and DeiT feature networks, r1 regularization, tuned learning rates, EMA, and no augmentation, and Table 6 itself lists StyleGAN2-XL reaching FID 1.51 on ImageNet 64x64 from scratch with a projected-GAN-style objective. To support the generative-pretraining interpretation, the authors should train a randomly initialized generator with the identical D2O recipe, discriminator, augmentation, EMA, batch size, and 5M-image budget. If that baseline reaches comparable FID, the observed performance cannot be assigned to the pretrained diffusion weights; if it does not, the gap is the evidence needed for the claim.
  2. [Sec. 2.3, Figure 2] The evidence in Section 2.3 establishes only a correlation, not the causal claim in the abstract that 'mismatched step sizes and parameter numbers between the teacher and the student model lead them to converge to different local minima, rendering direct imitation suboptimal.' The experiment shows that teacher-student FID increases with teacher steps while both have similar FID to the dataset, which is consistent with the stated hypothesis but also with other explanations, such as multi-step teachers being simply less similar to any single-step function. A direct test would train the same instance-level distillation loss with teachers of different step counts and measure the final one-step student's FID against the dataset; unless such a test is included, the local-minima mechanism should be presented as a hypothesis rather than a demonstrated limitation. This matters because it is the stated reason for discarding the distillation loss.
  3. [Tables 4-6, Appendix D] All FID values are reported as point estimates without confidence intervals, multiple seeds, or stated evaluation-sample sizes. Several headline comparisons are numerically close, such as D2O-F at 1.16 versus DMD2 at 1.26 on ImageNet 64x64, D2O-F at 1.54 versus StyleGAN2-XL at 1.51 on CIFAR-10, and D2O at 1.66 versus CTM at 1.98, so without uncertainty estimates the reader cannot judge whether the reported advantages are meaningful. Please report bootstrap confidence intervals or repeated-seed standard deviations, and specify how many generated images and which reference statistics were used.
  4. [Sec. 5.2, Table 10] The claim that 'D2O-F outperforms D2O by a significant margin in most datasets' is confounded on AFHQv2 and FFHQ, where Table 10 shows that D2O-F is trained with 10M training images while D2O uses 5M. On CIFAR-10 and ImageNet the budgets match, but on AFHQv2 and FFHQ the comparison should either train D2O with the same 10M budget or explicitly restrict the cross-model comparison to the matched-budget datasets. The current presentation lets the reader infer that freezing alone improves performance, when part of the improvement may be due to twice the training data.
minor comments (6)
  1. [Sec. 4.2] The sentence 'Based on these results, the D2O model is trained with most of the original parameters locked' should read 'D2O-F model', since the freeze ablation defines D2O-F.
  2. [Sec. 4.2] The paper alternates between 'training steps' and 'training images' when describing the 0.2M and 5M budgets; please make the unit consistent throughout, and clarify in Table 10 whether 'Training Images' means distinct images seen or gradient steps.
  3. [Sec. 5.1, Table 10] The text says the pre-trained diffusion models are from EDM, but Table 10 lists the ImageNet 64x64 generator architecture as ADM; please clarify which architecture and EDM checkpoint were used for each dataset, and whether the NCSN++ description in Section 3.2 applies only to CIFAR-10.
  4. [Appendix D] The CLIP-FID table reports D2O-F FID values that differ from the main tables on the same datasets (e.g., CIFAR-10 1.56 versus 1.54, ImageNet 1.13 versus 1.16, FFHQ 0.83 versus 0.85); please state whether these are separate runs, different checkpoints, or evaluation discrepancies.
  5. [Figure 2] Panel B caption should define what 'sigma' refers to for the intermediate time step and how it was varied, since this is central to interpreting the teacher-student FID trend.
  6. [Abstract] The phrase 'without relying a distillation loss' should be 'without relying on a distillation loss'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is validated against external FID/IS benchmarks, and the generative-pretraining interpretation is an attribution hypothesis rather than a definitional reduction.

full rationale

The paper's central empirical claim—that a standalone GAN objective can convert a pretrained diffusion model into a one-step generator—is evaluated with external FID, Inception Score, precision, and recall against held-out data distributions. No fitted parameter is renamed as a prediction: the D2O/D2O-F losses, discriminators, regularization, and freezing choices are hyperparameter selections, and the headline numbers (CIFAR-10 FID 1.54, ImageNet 64 FID 1.16) are direct measurements, not outputs forced by construction. The local-minima argument against distillation is supported by teacher/student FID comparisons, which are empirical observations rather than equations that assume the conclusion. The stronger claim that diffusion training constitutes 'generative pre-training' is inferred from data efficiency (0.2M or 5M images) and from the freezing ablation, but this is an attribution that would benefit from a from-scratch control; it is a missing-baseline / underdetermination issue, not a circular reduction of the kind where the conclusion is identical to the input by definition. The paper contains no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The frequency analysis is explicitly preliminary and descriptive. Under the stated criteria for circularity, the derivation chain is self-contained and empirically grounded.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central empirical method depends mainly on standard GAN training assumptions and on the quality of pretrained EDM checkpoints. The generative-pretraining conclusion additionally depends on an untested counterfactual (from-scratch baseline), captured as a red flag. No new physical entities are introduced.

free parameters (5)
  • gamma_r1 (R1 regularization weight) = 1e-4 on CIFAR/AFHQ/FFHQ, 4e-4 on ImageNet
    Chosen via ablation in Table 2; FID varies from 1.66 to 1.98 depending on the value, so the central results depend on this tuning.
  • Projected GAN discriminator configuration = VGG16-BN plus EfficientNet-lite0 for CIFAR; DeiT plus EfficientNet-lite0 for other datasets
    Selected via ablation in Table 1; discriminator choice changes results from collapse to FID 1.66 and is a key component of the method.
  • EMA half-life = 0.5 Mimg on CIFAR/AFHQ/FFHQ, 50 Mimg on ImageNet
    Set differently per dataset in Table 10; affects sample quality and is part of the training recipe.
  • Freezing mask = 85.8% convolutional parameters frozen; QKV and skip connections tuned
    Chosen via ablation in Table 3; FID ranges from 1.54 to 2.51 depending on which layer groups are frozen, so the freezing choice directly drives the main result.
  • Time-step schedule constants = N=18, rho=7, sigma_min=0.002, sigma_max=80
    Borrowed from Consistency Distillation and used for all models; these constants define the diffusion sampler and are inputs from prior literature rather than fitted here.
assumptions (5)
  • domain assumption Non-saturating GAN training with R1 regularization converges and pushes the generator toward the real data distribution.
    The whole method relies on GAN stability; Table 1 shows that some discriminator choices collapse, so convergence is assumed rather than guaranteed.
  • domain assumption The pretrained EDM checkpoints contain reusable generative structure worth preserving.
    Used as initialization and as the frozen base; if the checkpoints were weak, the D2O and D2O-F results would not demonstrate pretraining value.
  • ad hoc to paper Mismatched teacher and student step counts induce different local minima for the generator.
    Section 2.3 supports this only with FID correlations between teacher and student outputs, not with a direct test of optimization landscapes. It is a speculation used to justify abandoning distillation losses.
  • domain assumption FID and CLIP-FID are valid measures of generation quality and of divergence between generators.
    Used for all comparisons; known to be imperfect metrics, though the CLIP-FID appendix partially addresses representation leakage concerns.
  • domain assumption Log-frequency differences between network inputs and outputs reflect the mechanism of denoising.
    Section 4.5 and Appendix E present this as a descriptive analysis, and the authors call it preliminary; it is used to explain, not to derive, the one-step capability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation." pith.science (2026). https://pith.science/paper/EJ7VEY7Q

@misc{pith2026250609376,
  author       = {Pith},
  title        = {Pith review of: Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJ7VEY7Q}},
  note         = {Machine review of arXiv:2506.09376}
}
read the original abstract

Diffusion distillation is a widely used technique to reduce the sampling cost of diffusion models, yet it often requires extensive training, and the student performance tends to be degraded. Recent studies show that incorporating a GAN objective may alleviate these issues, yet the underlying mechanism remains unclear. In this work, we first identify a key limitation of distillation: mismatched step sizes and parameter numbers between the teacher and the student model lead them to converge to different local minima, rendering direct imitation suboptimal. We further demonstrate that a standalone GAN objective, without relying a distillation loss, overcomes this limitation and is sufficient to convert diffusion models into efficient one-step generators. Based on this finding, we propose that diffusion training may be viewed as a form of generative pre-training, equipping models with capabilities that can be unlocked through lightweight GAN fine-tuning. Supporting this view, we create a one-step generation model by fine-tuning a pre-trained model with 85% of parameters frozen, achieving strong performance with only 0.2M images and near-SOTA results with 5M images. We further present a frequency-domain analysis that may explain the one-step generative capability gained in diffusion training. Overall, our work provides a new perspective for diffusion training, highlighting its role as a powerful generative pre-training process, which can be the basis for building efficient one-step generation models.

Figures

Figures reproduced from arXiv: 2506.09376 by the authors.

Figure 1
Figure 1. A. Comparison between our methods (D2O and D2O-F) and other methods on ImageNet 64x64. Our models show competitive results with a much smaller training set than the competing models. B. D2O-F model. We initialize the generator with a pre-trained diffusion model and freeze most convolutional layers during the fine-tuning. A simple GAN objective is adopted. C. A detailed illustration of the freezing method used in D2O… view at source ↗
Figure 2
Figure 2. A. illustrates the FIDs for teacher models with varying numbers of steps. While the FID values computed with the target dataset are similar across teacher models, those with more steps show higher FID values when com￾pared with the one-step student model. This suggests a greater divergence from the student model as the number of teacher steps increases. Additionally, when we fix the teacher model’s steps to two and … view at source ↗
Figure 3
Figure 3. A. Performance of the D2O and CD models with (D2O-F and CD-F) and without (D2O and CD) freezing the convolutional layers. B. Effects of adding an extra CD loss to the D2O and D2O-F models. diffusion U-Net. D2O-F produces satisfying images with as few as 0.2 million training steps (FID=4.12). The per￾formance further reaches near the SOTA level with only 5 million steps (FID=1.54). In comparison, training a gen￾erati… view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: A. Spatial and frequency domain visualization. First row: Inputs at different time steps. Second row: Outputs at different time steps. Third and Fourth row: the difference between the current outputs and the previous outputs in the spatial domain (enhanced for clearer …
Figure 5
Figure 5. Figure 5: Sample comparison between EDM (Top), D2O (Middle) and D2O-F (Bottom) on AFHQv2 (left) and FFHQ (right). trained diffusion models are from EDM. We report FID for all datasets, Inception Score (IS) (Salimans et al., 2016) for CIFAR-10, precision and recall metric (Kynka¨…
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: CIFAR-10, EDM, NFE=18, FID=1.96 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: CIFAR-10, D2O, NFE=1, FID=1.66 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: CIFAR-10, D2O-F, NFE=1, FID=1.54 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: CIFAR-10 (conditional), EDM (VE), NFE=18, FID=1.82 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: CIFAR-10 (conditional), D2O, NFE=1, FID=1.58 [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: CIFAR-10 (conditional), D2O-F, NFE=1, FID=1.44 17 [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: AFHQv2 64x64, VE, NFE=79, FID=2.17 [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: AFHQv2 64x64, D2O, NFE=1, FID=1.23 [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: AFHQv2 64x64, D2O-F, NFE=1, FID=1.31 18 [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: FFHQ 64x64, EDM (VE), NFE=79, FID=2.60 [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: FFHQ 64x64, D2O, NFE=1, FID=1.08 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: FFHQ 64x64, D2O-F, NFE=1, FID=0.85 19 [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]
Figure 19
Figure 19. Figure 19: ImageNet 64x64 (conditional), EDM (VE), NFE=79, FID=2.36 [PITH_FULL_IMAGE:figures/full_fig_p020_19.png]
Figure 20
Figure 20. Figure 20: ImageNet 64x64 (conditional), D2O, NFE=1, FID=1.42 [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]
Figure 21
Figure 21. Figure 21: ImageNet 64x64 (conditional), D2O-F, NFE=1, FID=1.16 20 [PITH_FULL_IMAGE:figures/full_fig_p020_21.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 9 canonical work pages

  1. [5]

    Ho, J., Jain, A., and Abbeel, P

    URL https://arxiv.org/abs/ 2207.12598. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models,

    arXiv

  2. [6]

    Plug-and-Play Diffusion Distillation

    URL https://arxiv.org/abs/ 2406.01954. Huang, L., Fang, R., Zhang, A., Song, G., Liu, S., Liu, Y ., and Li, H. Fouriscale: A frequency perspective on training-free high-resolution image synthesis,

    work page Pith review arXiv

  3. [7]

    Ioffe, S

    URL https://arxiv.org/abs/2403.12963. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    arXiv

  4. [8]

    Karras, T., Laine, S., and Aila, T

    URL https://arxiv.org/abs/2405.05967. Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks,

    arXiv

  5. [9]

    Diffusion Model Compression for Image-to-Image Translation

    Kim, D., Lai, C.-H., Liao, W.-H., Murata, N., Takida, Y ., Uesaka, T., He, Y ., Mitsufuji, Y ., and Ermon, S. Consis- tency trajectory models: Learning probability flow ode trajectory of diffusion, 2024a. Kim, G., Kim, B., Park, E., and Cho, S. Diffusion model compression for image-to-image translation, 2024b. URL https://arxiv.org/abs/2401.17547. Krizhev...

    work page Pith review arXiv

  6. [10]

    Lee, S., Jung, S.-W., and Seo, H

    URL https://arxiv.org/ abs/2203.06026. Lee, S., Jung, S.-W., and Seo, H. Spectrum translation for refinement of image generation (stig) based on contrastive learning and spectral filter profile,

    arXiv

  7. [11]

    Spectrum Translation for Refinement of Image Generation (STIG) Based on Contrastive Learning and Spectral Filter Profile

    URL https: //arxiv.org/abs/2403.05093. Li, J., Cao, J., Zou, Z., Su, X., Yuan, X., Zhang, Y ., Guo, Y ., and Yang, X. Distillation-free one-step diffusion for real-world image super-resolution, 2024a. URL https: //arxiv.org/abs/2410.04224. Li, L., Li, H., Zheng, X., Wu, J., Xiao, X., Wang, R., Zheng, M., Pan, X., Chao, F., and Ji, R. Autodiffusion: Traini...

    work page Pith review arXiv

  8. [13]

    Ma, X., Fang, G., and Wang, X

    URL https://arxiv.org/abs/2311.05556. Ma, X., Fang, G., and Wang, X. Deepcache: Acceler- ating diffusion models for free,

    arXiv

Show all 31 references
  1. [14]

    Mescheder, L., Geiger, A., and Nowozin, S

    URL https: //arxiv.org/abs/2312.00858. Mescheder, L., Geiger, A., and Nowozin, S. Which training methods for gans do actually converge?,

    arXiv

  2. [15]

    Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Den- ton, E., Ghasemipour, S

    URL https://arxiv.org/abs/2206.13397. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Den- ton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., Salimans, T., Ho, J., Fleet, D. J., and Norouzi, M. Photorealistic text-to-image diffusion models with de...

    arXiv

  3. [16]

    10 Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., and Rombach, R

    URL https: //arxiv.org/abs/2311.17042. 10 Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., and Rombach, R. Fast high-resolution image synthesis with latent adversarial diffusion dist...

    arXiv

  4. [17]

    Simonyan, K

    URL https://arxiv.org/abs/2403.12015. Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition,

    arXiv

  5. [18]

    and Dhariwal, P

    Song, Y . and Dhariwal, P. Improved techniques for training consistency models.ArXiv, abs/2310.14189,

    arXiv

  6. [19]

    Takida, Y ., Imaizumi, M., Shibuya, T., Lai, C.-H., Ue- saka, T., Murata, N., and Mitsufuji, Y

    URL https://arxiv.org/abs/ 2410.23274. Takida, Y ., Imaizumi, M., Shibuya, T., Lai, C.-H., Ue- saka, T., Murata, N., and Mitsufuji, Y . San: Inducing metrizability of gan with discriminative normalized lin- ear layer,

    arXiv

  7. [20]

    URL https://arxiv.org/abs/ 2301.12811. Tan, M. and Le, Q. V . Efficientnet: Rethinking model scaling for convolutional neural networks,

    arXiv

  8. [21]

    Xu, Y ., Zhao, Y ., Xiao, Z., and Hou, T

    URLhttps://arxiv.org/abs/2308.10510. Xu, Y ., Zhao, Y ., Xiao, Z., and Hou, T. Ufogen: You forward once large scale text-to-image generation via diffusion gans,

    arXiv

  9. [22]

    Xue, S., Liu, Z., Chen, F., Zhang, S., Hu, T., Xie, E., and Li, Z

    URL https://arxiv.org/ abs/2311.09257. Xue, S., Liu, Z., Chen, F., Zhang, S., Hu, T., Xie, E., and Li, Z. Accelerating diffusion sampling with optimized time steps,

    arXiv

  10. [23]

    Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W

    URL https://arxiv.org/abs/ 2402.17376. Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W. T., and Park, T. One-step diffusion with distribution matching distillation,

    arXiv

  11. [24]

    Zhang, R., Isola, P., Efros, A

    URL https://arxiv.org/abs/2405.14867. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric,

    arXiv

  12. [25]

    Zheng, H., Nie, W., Vahdat, A., Azizzadenesheli, K., and Anandkumar, A

    URL https://arxiv.org/abs/ 2410.03456. Zheng, H., Nie, W., Vahdat, A., Azizzadenesheli, K., and Anandkumar, A. Fast sampling of diffusion models via operator learning,

    arXiv

  13. [26]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation, 2024a

    Zhou, M., Zheng, H., Wang, Z., Yin, M., and Huang, H. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation, 2024a. Zhou, Z., Chen, D., Wang, C., and Chen, C. Fast ode-based sampling for diffusion models in around 5...

    arXiv

  14. [27]

    11 Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation A

    URL https://arxiv.org/abs/ 2410.05317. 11 Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation A. Related Work Image GenerationGAN (Goodfellow et al., 2014; Brock et al., 2019; Karras et al., 2020b) models have dominated the image generation field f...

    2014 arXiv

  15. [29]

    Accelerating Diffusion InferenceMany recent works tried to accelerate the inference process of diffusion models, often focusing on the redundancy inherent in these models

    and are widely used in different fields. Accelerating Diffusion InferenceMany recent works tried to accelerate the inference process of diffusion models, often focusing on the redundancy inherent in these models. This typically involves both architectural and temporal redundan...

    2023

  16. [30]

    Distillation models with GAN were introduced recently

    have been proposed to distill diffusion models to one-step generators. Distillation models with GAN were introduced recently. GAN loss has been used as an auxiliary loss of distillation loss (Sauer et al., 2023; Xu et al., 2023; Kang et al., 2024; Sauer et al., 2024; Li et al....

    2023

  17. [80]

    CLIP-FID Results Potential data leakage in FID when using a discriminator pre-trained on ImageNet has been a concern (Kynk ¨a¨anniemi et al., 2023)

    D. CLIP-FID Results Potential data leakage in FID when using a discriminator pre-trained on ImageNet has been a concern (Kynk ¨a¨anniemi et al., 2023). We provide CLIP-FID in Table D. Our method consistently shows superior or competitive performance with significantly less tra...

    2023

  18. [2009]

    Dhariwal, P

    doi: 10.1109/CVPR.2009.5206848. Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis,

    2009

  19. [2021]

    Diffusion models have achieved great success in image generation (Dhariwal & Nichol, 2021; Nichol et al., 2022; Ramesh et al., 2022; Saharia et al.,

    or score-based generative models try to learn an accuracy estimation of scores (the gradient of the log probability density) to sample from a perturbed distribution with a Gaussian kernel to the image distribution. Diffusion models have achieved great success in image generati...

    2021

  20. [2022]

    Accelerating vision diffusion transformers with skip branches, 2024a

    Chen, G., Zhao, X., Zhou, Y ., Chen, T., and Cheng, Y . Accelerating vision diffusion transformers with skip branches, 2024a. URL https://arxiv.org/abs/ 2411.17616. Chen, P., Shen, M., Ye, P., Cao, J., Tu, C., Bouganis, C.-S., Zhao, Y ., and Chen, T.δ-dit: A training-free acce...

  21. [2023]

    Esser, P., Rombach, R., and Ommer, B

    URL https://sander.ai/2023/07/20/ perspectives.html. Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis,

    2023

  22. [2024]

    Goodfellow, I

    URL https: //arxiv.org/abs/2406.14548. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y . Generative adversarial networks,

    arXiv

  23. [2025]

    Lu, C., Zhou, Y ., Bao, F., Chen, J., Li, C., and Zhu, J

    URLhttps: //arxiv.org/abs/2410.11081. Lu, C., Zhou, Y ., Bao, F., Chen, J., Li, C., and Zhu, J. Dpm- solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps, 2022a. Lu, C., Zhou, Y ., Bao, F., Chen, J., Li, C., and Zhu, J. Dpm-solver: A fast ode...

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.