Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Improved Training Technique for Latent Consistency Models

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Latent-space consistency training fails because latent codes contain rare huge outliers; the paper's Cauchy-loss recipe brings one-step FID from 37 to 7.

desk verdict A useful, mostly empirical fix for latent-space consistency training with a plausible but under-tested outlier diagnosis; worth engaging, but treat the causal story as a hypothesis. read the letter →

arxiv 2502.01441 v2 pith:UOP2IAYV submitted 2025-02-03 cs.CV cs.LG

Quan Dao , Khanh Doan , Di Liu , Trung Le , Dimitris Metaxas This is my paper · ORCID
classification cs.CVcs.LG
keywords consistencymodelslatentspaceCauchylossimpulsiveoutlierstrainingoptimaltransportcouplingNon-scalingLayerNormfew-stepgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Consistency models generate images in one or a few steps by learning a direct mapping from noise to data, but the improved training recipe that works in pixel space fails badly when moved to the latent space used by most large-scale generators. This paper identifies the cause: latent codes from a pretrained autoencoder contain rare, extremely large values—impulsive outliers—that blow up the temporal-difference errors the consistency objective optimizes. To fix this, the paper replaces the standard Pseudo-Huber loss with a Cauchy loss whose logarithmic growth damps extreme residuals, and adds four supporting changes: a diffusion loss at early timesteps, optimal-transport coupling of noise to data, an adaptive schedule for the loss's scale parameter, and a LayerNorm variant without a learnable scaling term. On three 256x256 datasets the combined recipe improves one-step FID from 37.15 to 7.27 on CelebA-HQ, with similar jumps on LSUN Church and FFHQ, nearly closing the gap to multi-step latent diffusion models. If the diagnosis generalizes, the same recipe could make single-step generation practical for text-to-image and video models.

What carries the argument

The load-bearing object is the Cauchy loss $d_{\mathrm{Cauchy}}(\mathbf{x},\mathbf{y})=\log\!\left(1+\frac{\|\mathbf{x}-\mathbf{y}\|_2^2}{2c^2}\right)$, whose logarithmic tail is what keeps extreme TD residuals from dominating the gradient; the scale parameter $c$ controls where the loss transitions from quadratic to log behavior. The paper couples this with an adaptive scheduler $c=\exp(-1.18\log(\mathrm{NFE}(k)-1)-0.72)$ that shrinks $c$ as the number of discretization steps grows, tracking the observed reduction in TD variance. Two secondary mechanisms carry the rest: a diffusion loss $\|f_\theta(\mathbf{x}_{t_i},t_i)-\mathbf{x}_0\|_2^2$ applied only for $i\le N/4$ to avoid the consistency objective's error accumulation at small noise levels, and Non-scaling LayerNorm, which removes the shared learnable scaling term from LayerNorm so a single outlier cannot amplify across all feature channels.

What would settle it

Train iLCT on CelebA-HQ latent codes whose extreme outliers are removed (e.g., clipping all entries to the 99.9th percentile of the training distribution) with no other changes; if one-step FID stays above 30, outliers are not the main cause. Separately, run the full proposed recipe on a different VAE (for instance a KL-4 autoencoder) and check whether the one-step FID gain persists; if it collapses, the method is tied to the specific latent statistics of one encoder.

Watch

Extended reading notes

Core claim

The paper's central claim is that latent consistency training fails not because of the consistency objective itself but because latent data is statistically unlike pixel data: normalized latent codes have a heavy-tailed distribution with a small fraction of entries reaching values of −9 to 7 while the interquartile range spans only about −1.4 to 1.4. These outliers propagate into the temporal-difference (TD) errors used to train the consistency model, which range from −3.2 to 5 in latent space versus −1.5 to 1.7 in pixel space, and the Pseudo-Huber loss used by iCT grows only linearly in the residual, so it cannot contain them. The paper shows that switching to the Cauchy loss $\log(1+\|\mathbf{x}-\mathbf{y}\|_2^2/(2c^2))$, which grows logarithmically, reduces FID from 37.15 to 13.02 on CelebA-HQ by itself. Adding a diffusion loss on the first quarter of timesteps, minibatch optimal-transport coupling, an adaptive scaling-$c$ schedule $c=\exp(-1.18\log(\mathrm{NFE}(k)-1)-0.72)$, and Non-scaling LayerNorm (LayerNorm with the learnable scaling term fixed to 1) brings the one-step FID to 7.27 and recall from 0.12 to 0.50. The authors interpret this as evidence that consistency training is highly sensitive to training-data statistics, and that latent-space outliers are the specific obstacle the proposed techniques overcome.

Load-bearing premise

The paper's entire fix is motivated by the claim that impulsive outliers in one particular pretrained VAE's latent codes, measured on three image datasets, are what break latent consistency training; if those outliers are peculiar to that VAE, or if the real driver is something else like EMA target drift, the recipe may not transfer to the text-to-image and video latent spaces the paper aims for.

Editorial extensions

If this is right

  • Consistency training can be run directly in latent space without distillation from a pretrained diffusion model, removing an entire training pipeline.
  • One-step and two-step samplers for latent generators become competitive with multi-step latent diffusion, narrowing the FID gap from over 30 points to a few points at 256x256 resolution.
  • Each proposed component contributes independently: ablation shows monotone FID improvement from 37.15 to 13.02 (Cauchy), then 9.11 (+diffusion loss), 8.89 (+OT), 7.76 (+scaled c), and 7.27 (+NsLN).
  • The scaling parameter of robust losses should not be fixed during consistency training; it must track the TD variance as the discretization curriculum progresses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The outlier diagnosis was tested on one pretrained VAE, so the recipe's portability to other autoencoders or to text-to-image/video latents is plausible but unverified; retuning the diffusion threshold $r$ and the $c$ schedule may be needed per latent space.
  • The diffusion-loss-at-small-timestep term suggests that a principled interpolation between consistency and diffusion objectives could replace the heuristic threshold $r=0.25$.
  • Because the adaptive $c$ scheduler is hand-fitted to the three datasets, a scale-invariant robust loss that estimates its own scale might achieve similar gains without the extra hyperparameter.
  • A complementary fix would be to regularize the VAE itself to produce lighter-tailed latents, which would benefit any downstream model sensitive to outliers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript studies why improved consistency training (iCT) performs poorly when moved from pixel space to the latent space of a pretrained VAE, and proposes a set of training modifications. It first presents a statistical analysis (Section 4.1) arguing that latent representations contain impulsive outliers that produce large temporal-difference errors, and that Pseudo-Huber losses are insufficient. The proposed method replaces Pseudo-Huber with Cauchy loss, adds an L2 diffusion loss at early timesteps, uses minibatch optimal-transport coupling of noise and data, introduces an adaptive scaling-c scheduler, and replaces scaling LayerNorm with a Non-scaling LayerNorm. On CelebA-HQ, FFHQ, and LSUN Church at 256x256, the method improves one-step FID from 37.15 to 7.27 (CelebA-HQ), 52.45 to 8.87 (LSUN Church), and 48.82 to 8.72 (FFHQ) compared with an iLCT baseline, with corresponding Recall improvements. Ablations on CelebA-HQ attribute gains to each component.

Significance. If the reported results are reproducible, this is a practically important step: it shows consistency training can be made to work in latent space without distillation, with one- or two-step sampling, and the gains over the iLCT baseline are large. The paper has several strengths: it releases code, uses the same architecture and training budget for baselines and ablations, includes a controlled LDM-8-dagger baseline trained under the same VE scheduler, and provides component ablations. The main weakness is that the causal story (impulsive outliers are the dominant cause of iLCT failure) is supported only by observational evidence, and all quantitative claims rest on single-run point estimates. If the causal diagnosis does not transfer to other VAEs or to text-to-image/video latents, the recipe may still work, but the generalization claim in the abstract is not established.

major comments (3)
  1. [Section 4.1, Abstract, and Eq. (9)] The causal claim that impulsive outliers in latent space 'significantly degrade' iCT is not tested by any intervention. Section 4.1 compares pixel and latent training and shows box plots of TD values for one VAE, but no experiment removes, clips, or injects outliers to show that they cause the iLCT failure. This is load-bearing because the Cauchy loss, the adaptive c scheduler, and NsLN are all motivated by this mechanism, and the abstract generalizes the diagnosis to text-to-image and video latents. The tension is sharpened by Eq. (9): the diffusion loss is plain L2 applied to the same outlier-laden x0 and improves FID by about 4 points (Table 2a); if impulsive outliers were the dominant failure mode, an L2 term on x0 would be expected to aggravate them. Please add an outlier intervention experiment (e.g., clipping latent values, or training with a VAE whose latents have no impulsive outliers) or soften the causal claim.
  2. [Tables 1 and 2] All FID and Recall numbers are single-run point estimates, with no standard errors or number of seeds reported. This matters because several ablations are small relative to typical run-to-run variation in generative-model training: the diffusion-threshold comparison r=0.25 vs r=0.6 differs by 0.06 in FID (7.27 vs 7.33), OT adds only 0.22 (9.11 to 8.89), and NsLN beats GN by 0.49 (7.76 vs 7.27). The word 'significantly' is used throughout Section 5.2 without statistical support. Please report mean and standard deviation over at least three seeds for the main baseline, the full method, and at least the smaller ablation contrasts, or restrict the language to point estimates.
  3. [Section 4.5, Eq. (11)] The adaptive c scheduler is presented as a fitted exponential, but the paper does not show the TD-variance measurements that motivate it or the fitting procedure. Eq. (11) contains two fitted constants (-1.18 and -0.72) that are presented as universal across datasets, yet no evidence is given that this schedule transfers to other datasets, VAEs, or training budgets. Since the c schedule is one of the five proposed components and the ablation attributes more than 1 FID point to it, please provide the underlying variance data and a clear statement of how the constants were obtained and their sensitivity.
minor comments (5)
  1. [Section 4.3] The word 'heuristicly' should be 'heuristically'.
  2. [Appendix A] The heading 'CelebaA-HQ' should be 'CelebA-HQ'.
  3. [Table 2c] The row labeled 'Huber' should be labeled 'Pseudo-Huber' to match Eq. (7); if it is a different loss, define it.
  4. [Section 4.5] Eq. (10) is taken from Song & Dhariwal (2023); please cite the source in the text or caption rather than only referring to 'eq. (10)'.
  5. [Figures 4 and 5] The captions do not state whether the samples are uncurated; Appendix A says its samples are uncurated, so state the same for the main-text figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical training recipe validated by ablations and external benchmarks; the adaptive c schedule is openly fitted, not presented as a prediction.

full rationale

I find no circular step in this paper. The central contribution is an empirical training recipe for latent consistency models, and the main results are FID/Recall measurements on CelebA-HQ, FFHQ, and LSUN Church, compared against a self-implemented iLCT baseline and against external latent diffusion baselines (LDM, LFM, LSGM, DDMI, DIMSUM). The outlier analysis in Section 4.1 is observational and the causal claim is framed as a hypothesis, but none of the proposed components—Cauchy loss, early-timestep diffusion loss, OT coupling, adaptive scaling-c, or Non-scaling LayerNorm—is defined in terms of the target FID; each is independently evaluated in ablations in Table 2. Equation (11) is a fitted scheduler, and the paper openly states 'we fit the c scheduler to get the scheduler equation'; it is a hyperparameter schedule derived from observed TD variance, not a prediction of the benchmark, so it does not force the reported performance by construction. Self-citations (Dao et al. 2023, 2024a, 2024b; Phung et al. 2023, 2024; Van Le et al. 2023) appear only as baseline comparisons or related work and are not load-bearing for the main claim. The OT motivation relies on Zhang et al. 2023a, which is an external citation rather than a self-citation, and it is used as justification for variance reduction, not as a uniqueness theorem. No equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction. Therefore the derivation chain is self-contained in the sense relevant to circularity analysis.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on established consistency-training theory, on an empirical diagnosis of latent outliers made with one VAE, and on a fitted scheduler. No new particles, forces, or independent entities are postulated; NsLN is an architectural variant, not an invented entity.

free parameters (2)
  • Adaptive c scheduler coefficients = -1.18 and -0.72
    Equation (11), c = exp(-1.18 * log(NFE-1) - 0.72), is explicitly fit to the observed reduction of TD variance as the discretization steps grow, and is then used on the same datasets for which performance is reported.
  • Diffusion-loss threshold r = 0.25
    Chosen heuristically in Section 4.3; the ablation in Table 2b compares only 1.0, 0.6, and 0.25, so the selected value is a tuning choice rather than a derived quantity.
assumptions (5)
  • standard math The consistency model and iCT framework, including the PF-ODE, discretization, and EMA target updates, are valid as specified in Song et al. 2023 and Song and Dhariwal 2023.
    The paper builds directly on the preliminaries in Section 3 without re-deriving the consistency training objective.
  • domain assumption Impulsive outliers measured in sd-vae-ft-ema KL-8 latents are representative of latent spaces used for text-to-image and video generation.
    Section 4.1 measures outlier statistics on three datasets with a single pretrained VAE, but the abstract generalizes the conclusion to large-scale latent-space tasks.
  • domain assumption Temporal-difference loss in consistency models shares instability properties with DQN TD learning, motivating the same style of robust-loss fixes.
    The DQN analogy in Section 4.1 is used to justify focusing on the loss function rather than on other potential sources of instability.
  • domain assumption Minibatch optimal transport variance reduction transfers from flow matching to consistency training because consistency and diffusion models produce close outputs for the same noise input.
    Section 4.4 relies on the cited result of Zhang et al. 2023a rather than measuring this correspondence directly within the paper's setup.
  • ad hoc to paper The exponential form of the adaptive c schedule in Eq. (11) is an adequate model of how TD variance shrinks as the discretization curriculum progresses.
    The schedule is fit to observed TD variance and convergence curves in Section 4.5; it is not derived from first principles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improved Training Technique for Latent Consistency Models." pith.science (2026). https://pith.science/paper/UOP2IAYV

@misc{pith2026250201441,
  author       = {Pith},
  title        = {Pith review of: Improved Training Technique for Latent Consistency Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UOP2IAYV}},
  note         = {Machine review of arXiv:2502.01441}
}
abstract

Consistency models are a new family of generative models capable of producing high-quality samples in either a single step or multiple steps. Recently, consistency models have demonstrated impressive performance, achieving results on par with diffusion models in the pixel space. However, the success of scaling consistency training to large-scale datasets, particularly for text-to-image and video generation tasks, is determined by performance in the latent space. In this work, we analyze the statistical differences between pixel and latent spaces, discovering that latent data often contains highly impulsive outliers, which significantly degrade the performance of iCT in the latent space. To address this, we replace Pseudo-Huber losses with Cauchy losses, effectively mitigating the impact of outliers. Additionally, we introduce a diffusion loss at early timesteps and employ optimal transport (OT) coupling to further enhance performance. Lastly, we introduce the adaptive scaling-$c$ scheduler to manage the robust training process and adopt Non-scaling LayerNorm in the architecture to better capture the statistics of the features and reduce outlier impact. With these strategies, we successfully train latent consistency models capable of high-quality sampling with one or two steps, significantly narrowing the performance gap between latent consistency and diffusion models. The implementation is released here: https://github.com/quandao10/sLCT/

Figures

Figures reproduced from arXiv: 2502.01441 by the authors.

Figure 1
Figure 1. Box and Whisker Plot: Impulsive noise comparison between pixel and latent spaces. The right column shows the statistics of TD values at 21 discretization steps. Other discretization steps exhibit same behavior, where impulsive outliers are consistently present regardless of the total discretization steps. The blue boxes represent interquartile ranges of the data, while the green and orange dashed lines indicate inne… view at source ↗
Figure 2
Figure 2. Analysis of robust loss: Pseudo-Huber, Cauchy, and Geman-McClure [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Model convergence plot on different c schedule. (Left) Our proposed c values. Perfor￾mance on FID (Middle) and Recall (Right) of our proposed c in comparison with different choices. where k is current training iteration, K is total training iteration and s0 = 10, s1 = 640. During training, we notice that the variance of TD is significantly reduced as doubling total discretization steps using eq. (10). Since the more… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Our qualitative results using 1-NFE at resolution [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: iLCT qualitative results using 1-NFE at resolution [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: One-step samples on CelebA-HQ 256 × 256 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Two-step samples on CelebA-HQ 256 × 256 15 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: One-step samples on LSUN Church 256 × 256 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Two-step samples on LSUN Church 256 × 256 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: One-step samples on FFHQ 256 × 256 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Two-step samples on FFHQ 256 × 256 17 [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: One-step samples on CelebA-HQ 256 × 256 (L2 loss) [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: One-step samples on CelebA-HQ 256 × 256 (E-LatentLPIPS loss) 18 [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Continuous-Time Consistency Model for 3D Point Cloud Generation

    cs.CV 2025-09 reject novelty 5.0 of 10

    ConTiCoM-3D trains a continuous-time consistency-style model directly on raw 3D point clouds using flow matching plus Chamfer distance, with one- to two-step generation.

Reference graph

Works this paper leans on

31 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [3]

    Self-corrected flow distillation for consistent one-step and few-step text-to-image generation

    Quan Dao, Hao Phung, Trung Dao, Dimitris Metaxas, and Anh Tran. Self-corrected flow distillation for consistent one-step and few-step text-to-image generation. arXiv preprint arXiv:2412.16906, 2024a. Quan Dao, Binh Ta, Tung Pham, and Anh Tran. A high-quality robust diffusion framework for corrupted dataset. In European Conference on Computer Vision, pp. 1...

    work page arXiv

  2. [6]

    Multistep consistency models

    Jonathan Heek, Emiel Hoogeboom, and Tim Salimans. Multistep consistency models. arXiv preprint arXiv:2403.06807,

    arXiv

  3. [11]

    ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models

    Fei Kong, Jinhao Duan, Lichao Sun, Hao Cheng, Renjing Xu, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, and Kaidi Xu. Act: Adversarial consistency models. arXiv preprint arXiv:2311.14097,

    work page Pith review arXiv

  4. [12]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 1931–1941,

    work page 1931

  5. [13]

    Minimizing trajectory curvature of ode-based gen- erative models

    Sangyun Lee, Beomsu Kim, and Jong Chul Ye. Minimizing trajectory curvature of ode-based gen- erative models. arXiv preprint arXiv:2301.12003,

    arXiv

  6. [14]

    Sdedit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073,

    arXiv

  7. [15]

    Reliable fidelity and diversity metrics for generative models.ArXiv, abs/2002.09797,

    Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models.ArXiv, abs/2002.09797,

    arXiv 2002

  8. [16]

    12 Published as a conference paper at ICLR 2025 Dogyun Park, Sihyeon Kim, Sojin Lee, and Hyunwoo J Kim

    URL https: //api.semanticscholar.org/CorpusID:211259260. 12 Published as a conference paper at ICLR 2025 Dogyun Park, Sihyeon Kim, Sojin Lee, and Hyunwoo J Kim. Ddmi: Domain-agnostic latent diffusion models for synthesizing high-quality implicit neural representations. arXiv preprint arXiv:2401.12517,

    arXiv 2025

Show all 31 references
  1. [17]

    Dimsum: Diffu- sion mamba–a scalable and unified spatial-frequency method for image generation.arXiv preprint arXiv:2411.04168,

    Hao Phung, Quan Dao, Trung Dao, Hoang Phan, Dimitris Metaxas, and Anh Tran. Dimsum: Diffu- sion mamba–a scalable and unified spatial-frequency method for image generation.arXiv preprint arXiv:2411.04168,

    arXiv

  2. [18]

    Multisample flow matching: Straightening flows with minibatch cou- plings

    Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lip- man, and Ricky TQ Chen. Multisample flow matching: Straightening flows with minibatch cou- plings. arXiv preprint arXiv:2304.14772,

    arXiv

  3. [19]

    Hyper-sd: Trajectory segmented consistency model for efficient image synthesis

    Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. arXiv preprint arXiv:2404.13686,

    arXiv

  4. [20]

    Adversarial diffusion dis- tillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. arXiv preprint arXiv:2311.17042,

    arXiv

  5. [22]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456,

    2011 arXiv

  6. [23]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469,

    arXiv

  7. [24]

    Relay diffusion: Unifying diffusion process across resolutions for image synthesis.arXiv preprint arXiv:2309.03350,

    Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang, and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis.arXiv preprint arXiv:2309.03350,

    arXiv

  8. [25]

    Improving and generalizing flow-based generative models with minibatch optimal transport

    Alexander Tong, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482,

    arXiv

  9. [26]

    Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    13 Published as a conference paper at ICLR 2025 Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Sy...

    2025

  10. [27]

    Tackling the generative learning trilemma with denoising diffusion gans

    Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion gans. arXiv preprint arXiv:2112.07804,

    arXiv

  11. [28]

    Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop

    Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop. ArXiv, abs/1506.03365,

    arXiv

  12. [29]

    Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu

    URL https://api.semanticscholar.org/CorpusID:8317437. Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu. The emergence of reproducibility and consistency in diffusion models. In Forty-first International Conference on Machine Learning, 2023a. L...

    2024

  13. [30]

    Trajectory consistency distillation

    Jianbin Zheng, Minghui Hu, Zhongyi Fan, Chaoyue Wang, Changxing Ding, Dacheng Tao, and Tat-Jen Cham. Trajectory consistency distillation. arXiv preprint arXiv:2402.19159,

    arXiv

  14. [31]

    We also provide additional uncurated samples of our models on CelebaA-HQ trained with L2 loss (12) and E-LatentLPIPS loss (13)

    14 Published as a conference paper at ICLR 2025 A A PPENDIX We provide additional uncurated samples of our models for three datasets: CelebaA-HQ (6, 7), LSUN Church (8, 9), and FFHQ (10, 11). We also provide additional uncurated samples of our models on CelebaA-HQ trained with...

    2025

  15. [1986]

    Consistency models made easy

    Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. arXiv preprint arXiv:2406.14548,

    arXiv

  16. [2015]

    Improved techniques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310.14189,

    arXiv

  17. [2018]

    Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli

    URL https://proceedings.neurips.cc/paper_files/paper/2018/ file/093f65e080a295f8076b1c5722a46aa2-Paper.pdf. Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conferen...

    2018

  18. [2019]

    Tract: Denoising diffusion models with transitive closure time-distillation

    David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap, Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.04248,

    arXiv

  19. [2020]

    Introvae: In- trospective variational autoencoders for photographic image synthesis

    11 Published as a conference paper at ICLR 2025 Huaibo Huang, zhihang li, Ran He, Zhenan Sun, and Tieniu Tan. Introvae: In- trospective variational autoencoders for photographic image synthesis. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garne...

    2025

  20. [2021]

    Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279,

    arXiv

  21. [2022]

    Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation

    Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. arXiv preprint arXiv:2106.05527,

    arXiv

  22. [2023]

    Flow matching in latent space

    Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint arXiv:2307.08698,

    arXiv

  23. [2024]

    Dice: Discrete inversion enabling controllable editing for multinomial diffusion and masked generative models.arXiv preprint arXiv:2410.08207,

    Xiaoxiao He, Ligong Han, Quan Dao, Song Wen, Minhao Bai, Di Liu, Han Zhang, Martin Renqiang Min, Felix Juefei-Xu, Chaowei Tan, et al. Dice: Discrete inversion enabling controllable editing for multinomial diffusion and masked generative models.arXiv preprint arXiv:2410.08207,

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.