Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diffusion-based discriminator replaces reverse-KL loss to distill diffusion models

desk verdict Strong empirical distillation recipe with an unsupported TVD claim; the theory needs to be fixed or removed. read the letter →

arxiv 2507.18569 v1 pith:6JPH3JL4 submitted 2025-07-24 cs.CV

Yanzuo Lu , Yuxi Ren , Xin Xia , Shanchuan Lin , Xing Wang , Xuefeng Xiao , Andy J. Ma , Xiaohua Xie
show 1 more author
Jian-Huang Lai
This is my paper
classification cs.CV
keywords diffusiondistillationscoreadversarialtrainingdistributionmatchingtotal-variationdistanceone-stepgenerationtext-to-imagesynthesistext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Distribution Matching Distillation (DMD) compresses pre-trained diffusion models into fast generators, but its reverse-KL objective is zero-forcing: in one-step or few-step settings it drives low-probability modes to zero and causes mode collapse, gradient vanishing, or gradient explosion. This paper proposes Adversarial Distribution Matching (ADM), which replaces the fixed DMD divergence with a discriminator built from the frozen teacher diffusion model plus trainable heads, applied to the score predictions of real and fake estimators along the probability-flow ODE. Optimizing the Hinge loss then corresponds to minimizing total-variation distance rather than reverse KL, giving a symmetric, bounded objective. Combined with an adversarial pre-training stage on teacher ODE pairs (DMDX), the method reports one-step SDXL performance above DMD2 with less GPU time, and multi-step ADM distillation sets new benchmarks on SD3-Medium, SD3.5-Large, and CogVideoX.

What carries the argument

The load-bearing object is the diffusion-backbone discriminator: a frozen pretrained diffusion model (the teacher) with trainable heads attached to intermediate UNet or DiT blocks, which scores the PF-ODE predictions of real and fake score estimators one timestep $\Delta t$ ahead. In ADM this discriminator supplies the implicit discrepancy measure that replaces the reverse-KL divergence; in the adversarial pre-training stage, a second hybrid discriminator operates on a latent-space sample and a pixel-space image from a pretrained vision encoder to align the student with teacher ODE pair endpoints, using a cubic generator timestep schedule that biases training toward high noise levels. The Hinge loss is the engine that converts discriminator training into total-variation-distance minimization.

What would settle it

Measure the trained ADM discriminator's accuracy on held-out pairs of real and fake score predictions across noise levels; if that accuracy is far from chance or saturates too early, Equations (7)-(8) are not minimizing total-variation distance and the theoretical advantage over reverse KL is unsupported. Equivalently, replace the frozen teacher discriminator with a fully fine-tuned discriminator of comparable size: if results do not improve, the frozen backbone was already sufficient, and if they improve substantially, the paper's discriminator-capacity premise fails.

Watch

Extended reading notes

Core claim

ADM's central claim is that score distillation need not be tied to a predefined divergence such as reverse KL. The paper constructs a discriminator $D_\tau(x_t,t)$ from the frozen teacher diffusion backbone with multiple learnable heads attached to intermediate blocks. For a generator output $\hat{x}_0$, it diffuses $\hat{x}_0$ to noise level $t$, then solves the PF-ODE of both the fake score estimator $f_\psi$ and the real teacher $F_\phi$ backward by a small interval $\Delta t$; the discriminator sees these two score predictions $x^{\text{fake}}_{t-\Delta t}$ and $x^{\text{real}}_{t-\Delta t}$ and is trained with the Hinge loss. The paper argues that this adversarial alignment of latent predictions is the DMD loss itself, played by an implicit, data-driven discrepancy measure, and that at convergence the Hinge objective minimizes total-variation distance, which is symmetric and bounded, avoiding the mode-seeking pathologies of reverse KL. For one-step distillation, DMDX adds an adversarial pre-training stage that matches the student to teacher-sampled ODE pairs through hybrid latent- and pixel-space discriminators, providing the support overlap that one-step score distillation needs, and then applies ADM as fine-tuning.

Load-bearing premise

The load-bearing premise is that a frozen teacher diffusion backbone with a few trainable heads is a good enough discriminator that optimizing the Hinge loss actually minimizes total-variation distance between the student and teacher distributions; the paper does not test that optimality.

Editorial extensions

If this is right

  • In the paper's comparison on SDXL-Base, one-step DMDX beats DMD2 on all four reported metrics (CLIP, PickScore, HPSv2, MPS) while using 2240 GPU hours instead of 3840.
  • As a standalone multi-step method, ADM sets higher reported scores than TSCD, PCM, Flash, and LADD on 4-step SD3-Medium and SD3.5-Large, and its 8-step CogVideoX generators match or exceed the 100-step base models on VBench.
  • Because the adversarial loss plays the role of the DMD loss, ADM needs no ODE-based or GAN-based regularizer to counterbalance reverse-KL mode collapse.
  • The diversity evaluation reports higher pairwise LPIPS for DMDX than for ADD, LCM, Lightning, DMD2, or the teacher, indicating reduced mode collapse.
  • TTUR scaling from 1x to 8x yields only marginal metric gains while nearly tripling training time, supporting the paper's claim that good initialization, not the update rule, is what stabilizes one-step training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if support-set overlap is the real bottleneck for one-step distillation, then other score-distillation losses (Fisher-divergence variants, for instance) should gain as much as DMD does from the same adversarial pre-training; the paper only demonstrates this for DMD and ADM.
  • Inference: the paper's own limitation (teacher score prediction needs CFG, blocking guidance-distilled models such as FLUX.1-dev) suggests a direct test: apply ADM to one of those models and see whether the bounded, symmetric total-variation objective tolerates inaccurate teacher scores better than reverse KL does.
  • Inference: a fully trainable discriminator, or discriminators with more capacity per noise level, would test whether the frozen-backbone design is near-optimal or is the limiting factor; the paper reports only the frozen-backbone configuration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Adversarial Distribution Matching (ADM), a score-distillation method that replaces the DMD objective's reverse-KL loss with an adversarial loss on predictions of frozen teacher and learned fake score estimators. It also introduces Adversarial Distillation Pre-training (ADP), using latent- and pixel-space discriminators on ODE pairs, and combines the two into a pipeline called DMDX for one-step SDXL distillation. The authors report quantitative results on SDXL, SD3-Medium, SD3.5-Large, CogVideoX-2b, and CogVideoX-5b, including human-preference metrics, VBench scores, training-time comparisons with DMD2, and ablations on the proposed components.

Significance. If the central theoretical claim held, the paper would be significant: it would show that an adversarial, data-driven discrepancy can replace a predefined reverse-KL divergence in score distillation, thereby avoiding mode collapse while retaining the efficiency of one-step or few-step generation. The empirical scope is also broad—the method is demonstrated on multiple model families (UNet and DiT, image and video) and the paper includes training-time comparisons and ablations that are useful for practitioners. The main weakness is that the theoretical bridge from Hinge loss to total-variation distance is load-bearing and, as written, does not apply to the actual quantities being compared; the empirical margins over strong baselines are also small and are reported without uncertainty quantification. The paper is nonetheless well positioned as a strong empirical contribution, provided the theory is either repaired or substantially reframed.

major comments (4)
  1. [Sec. 4.3.3, Algorithm 1] The theoretical bridge is load-bearing and does not transfer to the trained objective. In Algorithm 1 (lines 18-21), both x_fake_{t-Δt} and x_real_{t-Δt} are obtained by solving PF-ODEs from the same noisy student output xt = q(xt | Gθ(z, t')); the discriminator in Eqs. (7)-(8) therefore compares two conditional score-prediction distributions given the common xt, not independent samples from the student and teacher marginal distributions pfake and preal. The Hinge-loss/TVD equivalence cited from [68] applies to a standard GAN comparing draws from pfake and preal; it does not imply that optimizing Eqs. (7)-(8) minimizes TVD(pfake, preal) even with an arbitrarily rich discriminator. Consequently the symmetry and boundedness advantages claimed for the student distribution, and the illustration in Fig. 4(c), are not established. The authors should either prove the stated equivalence for the conditional score-prediction game or substantially soften the theoretical claim.
  2. [Sec. 5.2, Fig. 3] The evidence that ADM implicitly optimizes the DMD objective is confounded. Fig. 3 plots Eq. (6) using a fake score estimator fψ that is co-trained in the loop (Algorithm 1, line 11); a decrease in the plotted loss can reflect fψ adapting to the current generator rather than the generator's distribution approaching the teacher. Furthermore, the authors themselves note that the DMD loss "didn't converge well at 8K iterations" and show only two training-length points; this does not establish that the discriminator is playing the role of the DMD loss. Please provide a controlled comparison, e.g., with fψ held fixed, or frame Fig. 3 as descriptive rather than causal evidence.
  3. [Tables 1, 2, 3] Several headline improvements are small relative to likely run-to-run variance, and no confidence intervals, multiple seeds, or significance tests are reported. In Table 1, DMDX vs. DMD2 differences are 0.04 CLIP, 0.18 PickScore, 0.25 HPSv2, and 0.50 MPS; in Table 3, the 8-step ADM CogVideoX-5b final score (82.067) is within about 0.8 of the teacher (81.226) when no CFG is used, and the CFG variant is lower than the no-CFG variant. Without uncertainty quantification the claim of "superior performance" and "new benchmark" is not yet supported. Please report per-seed variance or statistical tests, or identify which conclusions are robust.
  4. [Sec. 4.3.3, Eq. (12)] Even setting aside the conditional-distribution issue, the argument assumes that the Hinge discriminator approximates the optimal discriminator; the practical discriminator is a frozen teacher backbone with a small number of trainable heads (Sec. B.1), and no evidence is given that this family is rich enough to represent the density ratio between text-conditioned high-dimensional distributions. Without this verification, the claim that ADM minimizes TVD and thereby overcomes reverse-KL mode collapse remains an assumption. A falsifiable test would be to measure a mode-coverage metric (e.g., recall or diversity) under controlled support mismatch and compare against reverse-KL distillation.
minor comments (5)
  1. [Sec. 2, 'Score Distillation' paragraph] The sentence 'the student model distribution at a specific noise level is with the same probability as it does in the teacher model distribution,.' contains a stray comma-period and an incomplete clause; rephrase.
  2. [Sec. 5.2] The phrase 'didn't converge well' contains a typo ('did't'), and the subsequent sentence 'the DMD loss is also approximately optimized by the learnable discriminator' uses causal language that goes beyond the observational plot; recommend hedging.
  3. [Eqs. (2)-(3)] The expectation subscripts such as 'Ex0,t,ϵ∼N (0,I)' lack spacing and are visually cluttered; use a consistent notation, e.g., \mathbb{E}_{x_0,t,\epsilon}.
  4. [General] The paper does not state whether code or model weights will be released; given the reproducibility value of the training-time comparisons in Table 7, please add an availability statement.
  5. [Sec. 4.2.3] The sentence 'our motivation to perform adversarial distillation on synthetic data is inspired by LADD, but differs in a lot that we 1)...' is ungrammatical; revise for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ADM objective is not fitted to or defined by the quantities it is claimed to predict.

full rationale

The derivation chain in Secs. 4.1-4.3 does not reduce a prediction to its inputs. The generator loss Eq. (7) and discriminator loss Eq. (8) are a genuine Hinge-GAN objective on score-prediction endpoints; the paper never fits a parameter to a held-out metric and then reports that metric as a prediction. The theoretical claim in Sec. 4.3.3 that Hinge loss minimizes TVD cites an external theorem [68], and whether that theorem transfers to the conditional score-prediction game is a correctness question, not a definitional identity. Self-citations to SDXL-Lightning [23], APT [24], and Hyper-SD [55] are used for design inspiration and implementation details, not as load-bearing uniqueness or existence claims; none forces the ADM conclusion. The Fig. 3 DMD-loss diagnostic may be confounded by co-training the fake score estimator, but it is a supporting observation, not the objective being optimized, and no result is defined in terms of it. The central benchmarks (COCO, PickScore, HPSv2, MPS, VBench) are external to the training losses, and the CFG limitation in Sec. 6 concerns applicability, not circularity. Therefore the paper's core contribution is self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces several hand-tuned hyperparameters (loss weights, CFG ranges, learning rates, timestep schedule) and relies on standard GAN and distillation assumptions. No new physical entities are postulated. The central empirical claim is not derived from first principles.

free parameters (5)
  • lambda1, lambda2 (ADP balancing weights) = 0.85, 0.15
    Set based on empirical observation that lambda1=0.85, lambda2=0.15 yields visually coherent results; ablated in A3/A4.
  • Delta t (ADM ODE step) = T/64
    Fixed timestep interval for solving PF-ODE of score estimators before discrimination; chosen without extensive tuning.
  • Generator learning rates = 1e-6 (ADP) and 1e-7 (ADM) for SDXL; 1e-6 SD3-Medium LoRA; 1e-8 SD3.5-Large; 1e-7 CogVideoX
    Adjusted per model; the authors state they only adjust generator LR across experiments.
  • CFG ranges for real model = [6,8] SDXL/SD3; [3,4] SD3.5; [5,7] CogVideoX
    Chosen based on recommended CFG values from baseline inference; not derived.
  • Cubic timestep schedule exponent = 3
    Maps uniform timesteps via [1-(t/T)^3]*T to bias toward high noise; motivated by mode exploration, not derived.
assumptions (4)
  • standard math Hinge GAN loss minimizes Total-Variation Distance at the optimal discriminator.
    Cited from [68]; used in Sec 4.3.3 to argue ADM is theoretically superior to reverse-KL DMD loss. Requires a sufficiently rich discriminator, which is an unvalidated assumption in their constrained setup.
  • domain assumption Teacher model with CFG provides accurate score predictions for distillation.
    The paper states CFG is required for the real model, matching DMD practice; this limits applicability to guidance-distilled models (Sec 6).
  • domain assumption ODE pairs collected from the teacher provide a valid synthetic training distribution for pre-training.
    The pre-training stage uses synthetic teacher-generated data (Sec 4.2); assumes teacher distribution is a good proxy for real data.
  • domain assumption The fake score estimator f_psi, initialized from the teacher and trained with the standard denoising objective, faithfully tracks the generator distribution.
    Required for the ADM discriminator inputs to be meaningful; this is a standard assumption in DMD-style methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis." pith.science (2026). https://pith.science/paper/6JPH3JL4

@misc{pith2026250718569,
  author       = {Pith},
  title        = {Pith review of: Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6JPH3JL4}},
  note         = {Machine review of arXiv:2507.18569}
}
read the original abstract

Distribution Matching Distillation (DMD) is a promising score distillation technique that compresses pre-trained teacher diffusion models into efficient one-step or multi-step student generators. Nevertheless, its reliance on the reverse Kullback-Leibler (KL) divergence minimization potentially induces mode collapse (or mode-seeking) in certain applications. To circumvent this inherent drawback, we propose Adversarial Distribution Matching (ADM), a novel framework that leverages diffusion-based discriminators to align the latent predictions between real and fake score estimators for score distillation in an adversarial manner. In the context of extremely challenging one-step distillation, we further improve the pre-trained generator by adversarial distillation with hybrid discriminators in both latent and pixel spaces. Different from the mean squared error used in DMD2 pre-training, our method incorporates the distributional loss on ODE pairs collected from the teacher model, and thus providing a better initialization for score distillation fine-tuning in the next stage. By combining the adversarial distillation pre-training with ADM fine-tuning into a unified pipeline termed DMDX, our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time. Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cross-Resolution Distribution Matching for Diffusion Distillation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Cross-resolution distribution matching with logSNR timestep alignment and predicted-noise re-injection enables high-fidelity few-step multi-resolution cascaded diffusion distillation.

  2. Distribution Matching Distillation Meets Reinforcement Learning

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Combining DMD distillation with RL during training produces few-step text-to-image models that outperform their multi-step teacher on several benchmarks.

Reference graph

Works this paper leans on

94 extracted references · 56 canonical work pages · cited by 2 Pith papers

  1. [68]

    Calibrated ad- versarial algorithms for generative modelling

    Zhiqiang Tan, Yunfu Song, and Zhijian Ou. Calibrated ad- versarial algorithms for generative modelling. Stat, 8:e224,

    work page

  2. [1]

    Flux.1-dev

    Black Forest Labs. Flux.1-dev. https : / / huggingface.co/black- forest- labs/FLUX .1- dev, 2024. 8

    2024

  3. [2]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Naresh Babu Bynagari. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InNeurIPS,

  4. [3]

    Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation

    Clement Chadebec, Onur Tasar, Eyal Benaroche, and Ben- jamin Aubin. Flash diffusion: Accelerating any condi- tional diffusion model for few steps image generation. arXiv preprint arXiv:2406.02347, 2024. 1, 7, 2

    work page Pith review arXiv 2024

  5. [4]

    Training deep nets with sublinear memory cost

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174, 2016. 2

    arXiv 2016

  6. [5]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. 1

    2021

  7. [6]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yan- nik Marek, and Robin Rombach. Scaling rectified flow trans- formers for high-resolution image synthesis. In ICML, 2024. 2, 7, 1

    2024

  8. [7]

    Consistency models made easy

    Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. In ICLR,

Show all 94 references
  1. [8]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS,

  2. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 1

    2016

  3. [10]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. 1

    2016 arXiv

  4. [11]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS Workshops, 2021. 7, 8

    2021

  5. [12]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 1, 3

    2020

  6. [13]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022. 7, 8

    2022

  7. [14]

    Vbench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Com- prehensive benchmark suite for video generative models....

    2024

  8. [15]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Min- jia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023. 2

    2023 arXiv

  9. [16]

    Jon Ryu, and Gregory Wornell

    Tejas Jayashankar, J. Jon Ryu, and Gregory Wornell. Score- of-mixture training: Training one-step generative models made simple via score estimation of mixture distributions. arXiv preprint arXiv:2502.09609, 2025. 1, 2, 3

    2025 arXiv

  10. [17]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022. 1, 3

    2022

  11. [18]

    Consistency trajectory mod- els: Learning probability flow ode trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Mu- rata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory mod- els: Learning probability flow ode trajectory of diffusion. In ICLR, 2024. 1, 3

    2024

  12. [19]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. In ICCV, pages 3992– 4003, 2023. 5, 8, 1

    2023

  13. [20]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In NeuriPS, 2023. 7

    2023

  14. [21]

    Imagine flash: Accelerating emu diffusion models with backward distillation

    Jonas Kohler, Albert Pumarola, Edgar Sch ¨onfeld, Artsiom Sanakoyeu, Roshan Sumbaly, Peter Vajda, and Ali Tha- bet. Imagine flash: Accelerating emu diffusion models with backward distillation. arXiv preprint arXiv:2405.05224 ,

    arXiv

  15. [22]

    Geometric gan

    Jae Hyun Lim and Jong Chul Ye. Geometric gan. arXiv preprint arXiv:1705.02894, 2017. 4, 5, 6

    2017 arXiv

  16. [23]

    Sdxl- lightning: Progressive adversarial diffusion distillation

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl- lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402.13929, 2024. 1, 2, 3, 5, 7

    2024 arXiv

  17. [24]

    Diffusion adversarial post-training for one-step video generation

    Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, Lu Jiang, and ByteDance Seed. Diffusion adversarial post-training for one-step video generation. arXiv preprint arXiv:2501.08316, 2025. 1, 3

    2025

  18. [25]

    Lawrence Zitnick, and Piotr Doll ´ar

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Doll ´ar. Microsoft coco: Common objects in context. In ECCV, 2014. 7

    2014

  19. [26]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2023. 3

    2023 arXiv

  20. [27]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2022. 1, 3, 5, 8

    2022

  21. [28]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation

    Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and Qiang Liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In ICLR, 2024. 1, 3

    2024

  22. [29]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 2

    2019

  23. [30]

    Simplifying, stabilizing and scaling continuous-time consistency models

    Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.11081, 2024. 1, 3

    2024 arXiv

  24. [31]

    Ma, Xi- aohua Xie, and Jianhuang Lai

    Yanzuo Lu, Manlin Zhang, Yiqi Lin, Andy J. Ma, Xi- aohua Xie, and Jianhuang Lai. Improving pre-trained masked autoencoder via locality enhancement for person re- identification. In PRCV, pages 509–521, 2022. 5

    2022

  25. [32]

    Mlnet: Mutual learning network with neighbor- hood invariance for universal domain adaptation

    Yanzuo Lu, Meng Shen, Andy J Ma, Xiaohua Xie, and Jian- Huang Lai. Mlnet: Mutual learning network with neighbor- hood invariance for universal domain adaptation. In AAAI, pages 3900–3908, 2024. 5

    2024

  26. [33]

    Coarse-to-fine latent diffusion for pose- guided person image synthesis

    Yanzuo Lu, Manlin Zhang, Andy J Ma, Xiaohua Xie, and Jianhuang Lai. Coarse-to-fine latent diffusion for pose- guided person image synthesis. In CVPR, pages 6420–6429,

  27. [34]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 1, 2, 7

    2023 arXiv

  28. [35]

    Lcm-lora: A universal stable-diffusion acceleration module

    Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolin´ario Passos, Longbo Huang, Jian Li, and Hang Zhao. Lcm-lora: A universal stable-diffusion acceleration module. arXiv preprint arXiv:2311.05556, 2023. 1, 2

    2023 arXiv

  29. [36]

    Diff-instruct++: Training one-step text-to- image generator model to align with human preferences

    Weijian Luo. Diff-instruct++: Training one-step text-to- image generator model to align with human preferences. In TMLR, 2024. 3

    2024

  30. [37]

    Diff-instruct: A universal approach for transferring knowledge from pre-trained diffu- sion models

    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffu- sion models. In NeurIPS, pages 76525–76546, 2023. 1

    2023

  31. [38]

    Zico Kolter, and Guo-jun Qi

    Weijian Luo, Zemin Huang, Zhengyang Geng, J. Zico Kolter, and Guo-jun Qi. One-step diffusion distillation through score implicit matching. In NeurIPS, 2024. 1, 2

    2024

  32. [39]

    David and goliath: Small one-step model beats large diffusion with score post-training

    Weijian Luo, Colin Zhang, Debing Zhang, and Zhengyang Geng. David and goliath: Small one-step model beats large diffusion with score post-training. In ICML, 2025. 3

    2025

  33. [40]

    You only sample once: Taming one-step text-to- image synthesis by self-cooperative diffusion gans

    Yihong Luo, Xiaolong Chen, Xinghua Qu, Tianyang Hu, and Jing Tang. You only sample once: Taming one-step text-to- image synthesis by self-cooperative diffusion gans. In ICLR,

  34. [41]

    Ms-detr: Towards effective video moment retrieval and highlight detection by joint motion-semantic learning

    Hongxu Ma, Guanshuo Wang, Fufu Yu, Qiong Jia, and Shouhong Ding. Ms-detr: Towards effective video moment retrieval and highlight detection by joint motion-semantic learning. In ACMMM, 2025. 5

    2025

  35. [42]

    Fine-grained zero-shot ob- ject detection

    Hongxu Ma, Chenbo Zhang, Lu Zhang, Jiaogen Zhou, Ji- hong Guan, and Shuigeng Zhou. Fine-grained zero-shot ob- ject detection. In ACMMM, 2025. 5

    2025

  36. [43]

    Osv: One step is enough for high-quality image to video generation

    Xiaofeng Mao, Zhengkai Jiang, Fu-Yun Wang, Wenbing Zhu, Jiangning Zhang, Hao Chen, Mingmin Chi, and Yabiao Wang. Osv: One step is enough for high-quality image to video generation. arXiv preprint arXiv:2409.11367, 2024. 1

    2024 arXiv

  37. [44]

    Data synthesis with diverse styles for face recognition via 3dmm-guided diffusion

    Yuxi Mi, Zhizhou Zhong, Yuge Huang, Qiuyang Yuan, Xuan Zhao, Jianqing Xu, Shouhong Ding, Shaoming Wang, Rizen Guo, and Shuigeng Zhou. Data synthesis with diverse styles for face recognition via 3dmm-guided diffusion. In CVPR, pages 21203–21214, 2025. 5

    2025

  38. [45]

    Divergence measures and message passing

    Thomas Minka. Divergence measures and message passing. Microsoft Research, Technical Report, 2005. 2

    2005

  39. [46]

    Movie gen: A cast of media foun- dation models

    Movie Gen Team. Movie gen: A cast of media foun- dation models. https : / / ai.meta.com / static - resource/movie-gen-research-paper , 2024. 2

    2024

  40. [47]

    Openvid-1m: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to- video generation. arXiv preprint arXiv:2407.02371 , 2024. 7

    2024 arXiv

  41. [48]

    Dinov2: Learning robust visual features with- out supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mah- moud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michae...

    2024

  42. [49]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 4, 1

    2023

  43. [50]

    Open-sora-plan- v1.1.0

    PKU-Yuan Lab and Tuzhan AI. Open-sora-plan- v1.1.0. https : / / huggingface.co / datasets / LanguageBind/Open-Sora-Plan-v1 .1.0, 2024. 7

    2024

  44. [51]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In ICLR,

  45. [52]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 7, 6

    2021

  46. [53]

    Zero: Memory optimizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC, 2020. 2

    2020

  47. [54]

    Prajit Ramachandran, Barret Zoph, and Quoc V . Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017. 1

    2017 arXiv

  48. [55]

    Hyper-sd: Trajectory segmented consistency model for efficient image synthesis

    Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. In NeurIPS, 2024. 1, 3, 7

    2024

  49. [56]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 1, 7, 5

    2022

  50. [57]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 4, 1

    2015

  51. [58]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022. 1, 2, 3

    2022

  52. [59]

    Multistep distillation of diffusion models via moment matching

    Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. In NeurIPS, 2024. 1, 2

    2024

  53. [60]

    Fast high- resolution image synthesis with latent adversarial diffusion distillation

    Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high- resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015, 2024. 1, 2, 3, 5, 7

    2024 arXiv

  54. [61]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In ECCV, 2024. 1, 7

    2024

  55. [62]

    Meng Shen, Yanzuo Lu, Yanxu Hu, and Andy J. Ma. Collab- orative learning of diverse experts for source-free universal domain adaptation. In ACM MM, pages 2054–2065, 2023. 5

    2023

  56. [63]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 1, 3

    2021

  57. [64]

    Improved techniques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In ICLR, 2024. 2

    2024

  58. [65]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 1, 3

    2021

  59. [66]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023. 1, 2

    2023

  60. [67]

    Journeydb: A benchmark for generative image under- standing

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Limin Wang, and Hongsheng Li. Journeydb: A benchmark for generative image under- standing. arXiv preprint arXiv:2307.00716, 2023. 7

    2023 arXiv

  61. [69]

    Phased consistency model

    Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, Hong- sheng Li, and Xiaogang Wang. Phased consistency model. arXiv preprint arXiv:2405.18407, 2024. 1, 3, 7, 2

    2024 arXiv

  62. [70]

    Animatelcm: Computation-efficient personalized style video generation without personalized video data

    Fu-Yun Wang, Zhaoyang Huang, Weikang Bian, Xiaoyu Shi, Keqiang Sun, Guanglu Song, Yu Liu, and Hongsheng Li. Animatelcm: Computation-efficient personalized style video generation without personalized video data. In SIGGRAPH ASIA Technical Communications, 2024. 1

    2024

  63. [71]

    Rectified diffusion: Straightness is not your need in rectified flow.arXiv preprint arXiv:2410.07303,

    Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow.arXiv preprint arXiv:2410.07303,

    arXiv

  64. [72]

    Bfloat16: The se- cret to high performance on cloud tpus

    Shibo Wang and Pankaj Kanwar. Bfloat16: The se- cret to high performance on cloud tpus. https : / / cloud .google.com / blog / products / ai - machine - learning / bfloat16 - the - secret - to-high-performance-on-cloud-tpus , 2019. 2

    2019

  65. [73]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tie...

    arXiv

  66. [74]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. In NeurIPS, 2023. 2

    2023

  67. [75]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  68. [76]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In ECCV,

  69. [77]

    Building fast and compact convolu- tional neural networks for offline handwritten chinese char- acter recognition

    Xuefeng Xiao, Lianwen Jin, Yafeng Yang, Weixin Yang, Jun Sun, and Tianhai Chang. Building fast and compact convolu- tional neural networks for offline handwritten chinese char- acter recognition. Pattern Recognition, 72:72–81, 2017. 5

    2017

  70. [78]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In NeurIPS, 2023. 3, 7

    2023

  71. [79]

    Ufogen: You forward once large scale text-to-image genera- tion via diffusion gans

    Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou. Ufogen: You forward once large scale text-to-image genera- tion via diffusion gans. In CVPR, 2024. 3

    2024

  72. [80]

    Hunyuanportrait: Implicit condition con- trol for enhanced portrait animation

    Zunnan Xu, Zhentao Yu, Zixiang Zhou, Jun Zhou, Xiaoyu Jin, Fa-Ting Hong, Xiaozhong Ji, Junwei Zhu, Chengfei Cai, Shiyu Tang, et al. Hunyuanportrait: Implicit condition con- trol for enhanced portrait animation. InCVPR, pages 15909– 15919, 2025. 5

    2025

  73. [81]

    Perflow: Piecewise rectified flow as universal plug-and-play accelerator

    Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. In NeurIPS, 2024. 1, 3

    2024

  74. [82]

    Vript: A video is worth thousands of words

    Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: A video is worth thousands of words. arXiv preprint arXiv:2406.06040, 2024. 7

    2024 arXiv

  75. [83]

    Cogvideox: Text-to-video dif- fusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xi- aohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang, Yean Cheng, Ting Liu, Bin Xu, Yux- iao Dong, and Jie Tang. Cogvideox: Text-to-video dif- fusion m...

    2024 arXiv

  76. [84]

    Magic 1-for-1: Generating one minute video clips within one minute

    Hongwei Yi, Shitong Shao, Tian Ye, Jiantong Zhao, Qingyu Yin, Michael Lingelbach, Li Yuan, Yonghong Tian, Enze Xie, and Daquan Zhou. Magic 1-for-1: Generating one minute video clips within one minute. arXiv preprint arXiv:2502.07701, 2025. 1

    2025 arXiv

  77. [85]

    Tianwei Yin, Micha ¨el Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T. Freeman. Im- proved distribution matching distillation for fast image syn- thesis. In NeurIPS, 2024. 1, 2, 3, 5, 7, 8

    2024

  78. [86]

    Freeman, and Taesung Park

    Tianwei Yin, Micha ¨el Gharbi, Richard Zhang, Eli Shecht- man, Fr´edo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, pages 6613–6623, 2024. 1, 2, 3

    2024

  79. [87]

    From slow bidirectional to fast causal video generators

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Free- man, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast causal video generators. arXiv preprint arXiv:2412.07772, 2024. 1

    2024

  80. [88]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content...

    2022

  81. [89]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pages 586– 595, 2018. 8

    2018

  82. [90]

    Learning multi- dimensional human preference for text-to-image generation

    Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingt- ing Gao, Di Zhang, and Zhongyuan Wang. Learning multi- dimensional human preference for text-to-image generation. In CVPR, pages 8018–8027, 2024. 7

    2024

  83. [91]

    Sf-v: Single forward video generation model

    Zhixing Zhang, Yanyu Li, Yushu Wu, Yanwu Xu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris Metaxas, Sergey Tulyakov, and Jian Ren. Sf-v: Single forward video generation model. In NeurIPS,

  84. [92]

    Tra- jectory consistency distillation: Improved latent consistency distillation by semi-linear consistency function with trajec- tory mapping

    Jianbin Zheng, Minghui Hu, Zhongyi Fan, Chaoyue Wang, Changxing Ding, Dacheng Tao, and Tat-Jen Cham. Tra- jectory consistency distillation: Improved latent consistency distillation by semi-linear consistency function with trajec- tory mapping. arXiv preprint arXiv:2402.19159 ,...

    2024 arXiv

  85. [93]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation

    Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In ICML, 2024. 1, 2

    2024

  86. [94]

    =0.3A4: 𝜆!=1.0,𝜆

    Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step. In ICLR, 2025. 3 Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis Su...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.