The reviewed record of science sign in
Pith

arxiv: 2607.06335 · v1 · pith:KAWI6CS5 · submitted 2026-07-07 · cs.CV

Bridging Diffusion Pruning and Step Distillation with Teacher-Aligned Repair

Jincheng Ying , Li Wenlin , Minghui Xu , Yinhao Xiao This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 09:19 UTCglm-5.2pith:KAWI6CS5record.jsonopen to challenge →

classification cs.CV
keywords diffusion modelsstructured pruningstep distillationone-step generationteacher alignmentmodel compressionImageNet-512EDM2
0
0 comments X

The pith

Short repair stage lets pruned diffusion models skip retraining

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper addresses a practical bottleneck in compressing diffusion image generators: the two main efficiency strategies, pruning (shrinking the network) and step distillation (reducing sampling iterations), do not compose directly. When you prune a diffusion teacher and immediately try to distill it into a one-step generator, the result is unusable—the pruned model's denoising behavior has drifted too far from the teacher for the distillation loss to provide useful gradients. The authors introduce a short intermediate stage called teacher-alignment repair, which matches the pruned model's outputs to the teacher on noisy real-image latents at the highest noise level relevant to one-step generation. This repair is not full retraining; it only nudges the pruned model into a region where step distillation can take over. On ImageNet-512, a 20%-pruned EDM2-XS model repaired and then distilled to one step achieves FID 3.12, improving over the original 63-step teacher (FID 3.53) while using 21% fewer parameters and 63x fewer network evaluations. A 30%-pruned variant reaches FID 4.26 with one step.

Core claim

The central finding is that the failure of direct pruning-to-distillation transfer is caused by local teacher-student mismatch on the noisy latent states where one-step generation begins, and that this mismatch can be reduced enough by a short, targeted alignment procedure at a single high-noise level. The repair stage does not need to recover a full denoiser; it only needs to align the pruned model with the teacher on the specific noise distribution that initializes one-step generation. Once that local alignment is achieved, downstream distillation methods (the paper demonstrates with SiDA on ImageNet-512 and Diff-Instruct on CIFAR-10) can successfully train a compact one-step generator.

What carries the argument

The teacher-alignment repair loss: given a pruned generator G_S and teacher T, the repair objective matches their denoising outputs on noisy real-image latents at a fixed high-noise level sigma_r, minimizing the squared error between G_S and T on those states. This is paired with a teacher-aware block sensitivity score that ranks U-Net blocks by how much their removal increases the local teacher-student denoising mismatch, guiding which structures to prune.

Load-bearing premise

The paper assumes that reducing teacher-student mismatch on noisy real latents at a single high-noise level is sufficient to place the pruned model in a region where one-step distillation converges. This is a local smoothness assumption: if the distillation loss landscape has barriers or saddle points that the repair does not address, the bridge could fail for other architectures or pruning ratios.

What would settle it

Try the repair bridge at a substantially higher pruning ratio (e.g., 50%+) or on a different architecture family. If the repaired checkpoint still fails under distillation at that ratio, the single-noise-level local alignment is insufficient and the assumption of local stability is violated.

Figures

Figures reproduced from arXiv: 2607.06335 by Jincheng Ying, Li Wenlin, Minghui Xu, Yinhao Xiao.

Figure 1
Figure 1. Figure 1: ImageNet-512 samples from our 20% pruned one-step generator. This model reaches an FID of 3.12 with [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: CIFAR-10 samples from our 34% pruned one-step generator distilled with Diff-Instruct. [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
read the original abstract

Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. Existing compression pipelines usually attack these costs separately. Pruning reduces the network, but most pruning methods still rely on a long post-pruning retraining stage to recover a many-step sampler. Step distillation reduces the number of denoising steps, but it usually assumes a student that can already follow the teacher well enough to receive useful distillation gradients. This paper asks whether post-pruning retraining can be replaced by step distillation. We find that the direct replacement fails: after pruning an EDM2-XS teacher, starting SiDA from the pruned checkpoint produces unusable samples. We introduce a short teacher-alignment repair stage as a bridge between pruning and step distillation. The bridge matches the pruned generator to the teacher on noisy real-image latents, then hands the repaired checkpoint to one-step distillation. On ImageNet-512, the original EDM2-XS baseline uses 124.713M parameters and 63 network evaluations, reaching an FID of 3.53. With a suitable distillation objective, our 20% pruned one-step generator uses 98.826M parameters and one network evaluation, reaching an FID of 3.12. With 30% pruning, the model uses 88.029M parameters and one network evaluation, with an FID of 4.26.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 4 minor

Summary. The paper addresses the problem of combining structured pruning with one-step distillation for diffusion models. The central observation is that directly applying step distillation (SiDA) to a pruned EDM2-XS checkpoint fails catastrophically (FID 112.56), likely because the pruned model's denoising field is too misaligned with the teacher's. To bridge this gap, the authors introduce a short 'teacher-alignment repair' stage that matches the pruned model to the teacher on noisy real-image latents at a high noise level before initiating distillation. On ImageNet-512, the proposed 20% pruned, one-step generator achieves an FID of 3.12 (98.8M parameters), and the 30% pruned variant achieves an FID of 4.26 (88M parameters), compared to the 63-step EDM2-XS teacher baseline of 3.53. The method is also tested with Diff-Instruct on CIFAR-10, demonstrating that the repair bridge is somewhat generalizable across distillation objectives.

Significance. The paper tackles a practical and relevant problem: unifying network compression (pruning) and sampling acceleration (distillation) for diffusion models. The identification of the initialization mismatch problem (direct distillation from a pruned checkpoint fails) is a valuable empirical finding. The proposed solution—a lightweight, teacher-alignment repair stage—is conceptually simple and well-motivated. The inclusion of ablations (Table 2) isolating the repair bridge, pruning form, and block-selection score is a strength, as is the cross-method validation on CIFAR-10 using Diff-Instruct (Table 4). The approach yields a falsifiable and reproducible pipeline with clear parameter and NFE reductions.

major comments (2)
  1. Table 3 & Abstract: The headline claim states the 20% pruned model 'improves FID from 3.53 to 3.12'. This comparison is made against the 63-NFE EDM2-XS teacher. However, Table 3 also lists the unpruned one-step SiDA baseline (FID 2.228 ± 0.037). The more relevant comparison for evaluating the cost of pruning is pruned one-step (3.12) vs. unpruned one-step (2.228), which shows a 0.9 FID degradation. The paper does not discuss this comparison. Furthermore, the 0.41 FID improvement over the 63-NFE teacher is reported without error bars or variance estimates for the proposed method's FID (3.12). Given that the SiDA baseline reports ±0.037, the absence of variance estimates for the core results makes the 'improvement' claim statistically unsupported. The framing should be revised to accurately reflect the tradeoff (pruning cost relative to the unpruned one-step baseline) and include error bar
  2. Table 2: The ablation study reports an FID of 3.16 for the 'Full pipeline', while Table 3 reports an FID of 3.12 for the 'Ours (20% pruned)' model. The text in §4.4 states that Table 2 changes 'one component at a time'. Please clarify if the 3.16 in Table 2 is a different run (e.g., a different random seed or slightly different hyperparameters) or if there is a specific component difference between the 'Full pipeline' ablation and the final benchmark model. If it is just seed variance, this further underscores the need for error bars on the main benchmark results.
minor comments (4)
  1. §3.6, Eq. (17): The assumption that the distillation loss gradient norm is bounded by C1 + C2*sqrt(epsilon) is stated but the constants C1 and C2 are never defined or discussed. While it is explicitly framed as an assumption, a brief sentence explaining what these constants represent (or acknowledging they are abstract) would improve clarity.
  2. Table 5: The branch names (e.g., 'enc.8x8_block0') are helpful for reproducibility. However, the text in §4.2 mentions that 'The more compressed model in Table 3 removes low-importance residual-attention branches from low-resolution U-Net stages'. Table 5 only lists the disabled branches for the 30% pruned (88.029M) model. Please add the corresponding disabled branches for the 20% pruned (98.826M) model to ensure full reproducibility.
  3. Table 3: The 'Inference Time' and 'Speedup' columns contain dashes for all baseline methods except EDM2-XS and the proposed models. While obtaining these metrics for all baselines may be infeasible, a footnote explicitly stating why they are missing (e.g., 'not measured under our setup' is currently in the caption but could be more prominent) would help readers interpret the table.
  4. Abstract and §1: The phrasing 'improves FID from 3.53 to 3.12' could be misinterpreted as an improvement over a one-step baseline. It should be clarified earlier that 3.53 is the 63-step teacher baseline.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback. Both major comments are well-taken and identify legitimate gaps in our reporting. We will revise the manuscript to address them.

read point-by-point responses

  1. Referee: Table 3 & Abstract: The headline claim states the 20% pruned model 'improves FID from 3.53 to 3.12'. This comparison is made against the 63-NFE EDM2-XS teacher. However, Table 3 also lists the unpruned one-step SiDA baseline (FID 2.228 ± 0.037). The more relevant comparison for evaluating the cost of pruning is pruned one-step (3.12) vs. unpruned one-step (2.228), which shows a 0.9 FID degradation. The paper does not discuss this comparison. Furthermore, the 0.41 FID improvement over the 63-NFE teacher is reported without error bars or variance estimates for the proposed method's FID (3.12). Given that the SiDA baseline reports ±0.037, the absence of variance estimates for the core results makes the 'improvement' claim statistically unsupported. The framing should be revised to accurately reflect the tradeoff (pruning cost relative to the unpruned one-step baseline) and include error bar

    Authors: The referee is correct on both points. First, we should have explicitly discussed the comparison between our pruned one-step model (FID 3.12) and the unpruned one-step SiDA baseline (FID 2.228 ± 0.037). This comparison is indeed the most informative one for evaluating the cost of pruning within the one-step regime, and the 0.9 FID degradation is a real tradeoff that our paper should state plainly rather than leaving implicit. Second, the absence of variance estimates for our core FID results is a legitimate gap, especially given that the SiDA baseline reports ±0.037. We will address both issues in the revision. Specifically: (1) We will add discussion of the pruned-vs-unpruned one-step comparison in both the abstract and Section 4.5, framing the 0.9 FID gap as the cost of 20% parameter reduction within the one-step setting. The comparison to the 63-NFE teacher will remain as context for the combined benefit of pruning plus step distillation, but it will no longer be the sole framing. (2) We will run multiple evaluation seeds and report FID with standard deviation for our main results, following the same protocol used for the SiDA baseline. The 'improvement' claim will be restated to include error bars and to avoid implying statistical significance where the variance does not support it. revision: yes

  2. Referee: Table 2: The ablation study reports an FID of 3.16 for the 'Full pipeline', while Table 3 reports an FID of 3.12 for the 'Ours (20% pruned)' model. The text in §4.4 states that Table 2 changes 'one component at a time'. Please clarify if the 3.16 in Table 2 is a different run (e.g., a different random seed or slightly different hyperparameters) or if there is a specific component difference between the 'Full pipeline' ablation and the final benchmark model. If it is just seed variance, this further underscores the need for error bars on the main benchmark results.

    Authors: The referee's observation is accurate. The 3.16 in Table 2 and the 3.12 in Table 3 are from the same pipeline configuration with no intended hyperparameter difference; the small discrepancy is due to run-to-run variance from different random seeds used in the ablation and benchmark runs. We agree this should be clarified and that it reinforces the need for error bars. In the revision we will: (1) add a note in Section 4.4 explicitly stating that the 'Full pipeline' entry in Table 2 and the 'Ours (20% pruned)' entry in Table 3 use the same configuration and that the 0.04 FID difference reflects seed variance; and (2) report error bars on the main benchmark results as described in our response to the first comment, which will make the magnitude of this variance explicit. revision: yes

Circularity Check

0 steps flagged

No circularity: the repair objective, pruning sensitivity score, and distillation loss are independently defined and evaluated against external FID benchmarks.

full rationale

The paper's derivation chain is self-contained and non-circular. The repair objective (Eq. 13) minimizes teacher-student mismatch on noisy real latents — an independent objective that does not involve the downstream distillation loss or target FID. The pruning sensitivity score (Eq. 9) measures the increase in alignment error when a block is removed, which is a legitimate importance signal defined before and independently of the distillation stage. The downstream distillation (Eq. 16) uses SiDA (Eq. 18), an externally cited method [11], with the repaired checkpoint as initialization. The FID results (3.12, 4.26) are measured against the standard ImageNet-512 benchmark and compared against external baselines (EDM2-XS, SiDA, and others in Table 3). The ablation in Table 2 (112.56 without repair vs. 3.16 with) provides independent empirical evidence that the repair stage is necessary. The assumption in Eq. 17 (gradient bound) is explicitly stated as an assumption, not a theorem, and is not used to define or force the results. No step reduces to its inputs by construction, no prediction is a renamed fit, and no self-citation chain is load-bearing. The concerns raised by the skeptic (missing error bars, baseline selection) are statistical and framing issues, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

The paper introduces no new entities (particles, forces, dimensions, etc.). The free parameters are hyperparameters of the pruning and repair pipeline, which is expected for an empirical ML method. The two ad-hoc axioms (Eq. 17 local stability, single-noise-level repair sufficiency) are the main theoretical assumptions; both are empirically motivated but not formally proven.

free parameters (5)
  • sigma_star (calibration noise level) = 2.5
    Chosen as the noise level for block sensitivity estimation; stated in §3.3 and Appendix A.1. Not fitted to the target FID but chosen based on the one-step initialization noise level.
  • sigma_r (repair noise level) = 2.5
    Set to max(sigma_init, sigma_1, ..., sigma_K) per Eq. 15. In practice equals sigma_init = 2.5. The choice is motivated by the one-step generation starting region but the specific value is not derived from first principles.
  • Number of repair updates M_repair = not stated for ImageNet-512; 12k steps for CIFAR-10
    Stated in Algorithm 1 and Appendix A.2 (CIFAR-10 only). The ImageNet-512 repair update count is not reported.
  • Pruning ratio (20% and 30%) = 0.20 and 0.30
    Chosen target parameter reduction ratios. Not derived but selected to demonstrate the method at two compression levels.
  • Protected channel fraction = 0.9
    Stated in Table 7 for CIFAR-10. Prefix channels protected to maintain EDM2 layout and attention head structure. Not used for ImageNet-512 block pruning.
axioms (3)
  • ad hoc to paper The distillation loss D(G;T,A) is locally stable near a teacher-aligned initialization, formalized as the gradient bound in Eq. 17.
    §3.6, Eq. 17. This is the core assumption enabling the repair bridge. It is stated as a 'mild way to state the assumption used in practice' but is not proven or verified empirically.
  • ad hoc to paper Aligning the pruned model to the teacher at a single high-noise level (sigma_r = sigma_init) is sufficient to reduce mismatch across the noisy states encountered during one-step distillation.
    §3.5, Eq. 15. The paper argues that one-step generation starts from high-noise latents, so aligning there is most important. But the distillation loss (Eq. 18) also evaluates at other noise levels sigma, and the repair only covers sigma_r.
  • domain assumption The teacher EDM2-XS is a fixed, correct reference model.
    Standard assumption in distillation literature. The teacher's FID (3.53) is used as the baseline.

pith-pipeline@v1.1.0-glm · 15778 in / 3120 out tokens · 421348 ms · 2026-07-08T09:19:52.753639+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · 10 internal anchors

  1. [1]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems, 2020. 9 Bridging Pruning and Step Distillation

    work page 2020

  2. [2]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. InAdvances in Neural Information Processing Systems, 2021

    work page 2021

  3. [3]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    work page 2022

  4. [4]

    Analyzing and Improving the Training Dynamics of Diffusion Models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models.arXiv preprint arXiv:2312.02696, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  5. [5]

    Structural pruning for diffusion models

    Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. InAdvances in Neural Information Processing Systems, 2023

    work page 2023

  6. [6]

    LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insights

    Thibault Castells, Hyoung-Kyu Song, Bo-Kyeong Kim, and Shinkook Choi. LD-Pruner: Efficient pruning of latent diffusion models using task-agnostic insights.arXiv preprint arXiv:2404.11936, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

    Samarth N. Ramesh and Zhixue Zhao. Efficient pruning of text-to-image models: Insights from pruning stable diffusion.arXiv preprint arXiv:2411.15113, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. InInternational Conference on Learning Representations, 2022

    work page 2022

  9. [9]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. InInternational Conference on Machine Learning, 2023

    work page 2023

  10. [10]

    Freeman, and Taesung Park

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    work page 2024

  11. [11]

    Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step

    Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step.arXiv preprint arXiv:2410.14919, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both weights and connections for efficient neural networks.arXiv preprint arXiv:1506.02626, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  13. [13]

    Learnable sparsity for vision generative models.arXiv preprint arXiv:2412.02852, 2024

    Yang Zhang, Er Jin, Wenzhong Liang, Yanfei Dong, Ashkan Khakzar, Philip Torr, Johannes Stegmaier, and Kenji Kawaguchi. Learnable sparsity for vision generative models.arXiv preprint arXiv:2412.02852, 2024

    work page arXiv 2024

  14. [14]

    BK-SDM: A lightweight, fast, and cheap version of stable diffusion

    Bo-Kyeong Kim, Hyoung-Kyu Song, Thibault Castells, and Shinkook Choi. BK-SDM: A lightweight, fast, and cheap version of stable diffusion. InEuropean Conference on Computer Vision, 2024

    work page 2024

  15. [15]

    Snapfusion: Text-to-image diffusion model on mobile devices within two seconds

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snapfusion: Text-to-image diffusion model on mobile devices within two seconds. InAdvances in Neural Information Processing Systems, 2023

    work page 2023

  16. [16]

    U-DiTs: Downsample tokens in u-shaped diffusion transformers

    Yuchuan Tian, Zhijun Tu, Hanting Chen, Jie Hu, Chao Xu, and Yunhe Wang. U-DiTs: Downsample tokens in u-shaped diffusion transformers. InAdvances in Neural Information Processing Systems, 2024

    work page 2024

  17. [17]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InInternational Conference on Computer Vision, 2023

    work page 2023

  18. [18]

    Fast training of diffusion models with masked transformers.Transactions on Machine Learning Research, 2024

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers.Transactions on Machine Learning Research, 2024

    work page 2024

  19. [19]

    Learning stackable and skippable LEGO bricks for efficient, reconfigurable, and variable-resolution diffusion modeling.International Conference on Learning Representations, 2024

    Huangjie Zheng, Zhendong Wang, Jianbo Yuan, Guanghan Ning, Pengcheng He, Quanzeng You, Hongxia Yang, and Mingyuan Zhou. Learning stackable and skippable LEGO bricks for efficient, reconfigurable, and variable-resolution diffusion modeling.International Conference on Learning Representations, 2024

    work page 2024

  20. [20]

    Large scale GAN training for high fidelity natural image synthesis

    Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. InInternational Conference on Learning Representations, 2019

    work page 2019

  21. [21]

    Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalization

    Qihao Liu, Zhanpeng Zeng, Ju He, Qihang Yu, Xiaohui Shen, and Liang-Chieh Chen. Alleviating distortion in image generation via multi-resolution diffusion models and time-dependent layer normalization.arXiv preprint arXiv:2406.09416, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  22. [22]

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. MaskGIT: Masked generative image transformer. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    work page 2022

  23. [23]

    Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G

    Lijun Yu, José Lezama, Nitesh B. Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion – tokenizer is key to visual generation. In International Conference on Learning Represent...

    work page 2024

  24. [24]

    Scalable adaptive computation for iterative generation

    Allan Jabri, David Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. InInternational Conference on Machine Learning, 2023

    work page 2023

  25. [25]

    Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation

    Diederik P. Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmenta- tion.arXiv preprint arXiv:2303.00848, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. InAdvances in Neural Information Processing Systems, 2024

    work page 2024

  27. [27]

    StyleGAN-XL: Scaling StyleGAN to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. StyleGAN-XL: Scaling StyleGAN to large diverse datasets. ACM Transactions on Graphics, 2022

    work page 2022

  28. [28]

    Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models

    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models.arXiv preprint arXiv:2305.18455, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  29. [29]

    Score-based generative modeling in latent space

    Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. InAdvances in Neural Information Processing Systems, 2021

    work page 2021

  30. [30]

    Poisson flow generative models

    Yilun Xu, Ziming Liu, Max Tegmark, and Tommi Jaakkola. Poisson flow generative models. InAdvances in Neural Information Processing Systems, 2022

    work page 2022

  31. [31]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. InAdvances in Neural Information Processing Systems, 2022

    work page 2022

  32. [32]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. InInternational Conference on Learning Representations, 2021

    work page 2021

  33. [33]

    DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. InAdvances in Neural Information Processing Systems, 2022

    work page 2022

  34. [34]

    Fast sampling of diffusion models with exponential integrator

    Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. InInternational Conference on Learning Representations, 2023

    work page 2023

  35. [35]

    UniPC: A unified predictor-corrector framework for fast sampling of diffusion models

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. UniPC: A unified predictor-corrector framework for fast sampling of diffusion models. InAdvances in Neural Information Processing Systems, 2023

    work page 2023

  36. [36]

    Tackling the generative learning trilemma with denoising diffusion GANs

    Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion GANs. InInternational Conference on Learning Representations, 2022

    work page 2022

  37. [37]

    Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed

    Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. InarXiv preprint arXiv:2101.02388, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  38. [38]

    Truncated diffusion probabilistic models and diffusion-based adversarial auto-encoders

    Huangjie Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Truncated diffusion probabilistic models and diffusion-based adversarial auto-encoders. InInternational Conference on Learning Representations, 2022

    work page 2022

  39. [39]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022. 11 Bridging Pruning and Step Distillation Appendix A Reproducibility Details The main text keeps the experimental narrative at the protocol level. This appendix records the run-level settings...

    work page internal anchor Pith review Pith/arXiv arXiv 2022