Pith. sign in

REVIEW 3 major objections 6 minor 60 references

Continuous Semi-Implicit Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CoSIM trains a continuous transition kernel as a consistency function, letting a few sampling steps approach the diffusion teacher's quality.

desk verdict CoSIM is a solid, modest advance in diffusion distillation with strong FD-DINOv2 numbers, but its headline multistep theory rests on an unverified assumption about the trained consistency function. read the letter →

arxiv 2506.06778 v1 pith:BSOKT3TP submitted 2025-06-07 stat.ML cs.LG

Longlin Yu , Jiajun Zha , Tong Yang , Tianyu Xie , Xiangyu Zhang , S.-H. Gary Chan , Cheng Zhang This is my paper
classification stat.MLcs.LG
keywords continuoussemi-implicitmodelsdiffusionmodeldistillationconsistencyfunctionsscorematchingmultistepsamplinggenerativemodelingimagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CoSIM is a training procedure that takes a pretrained diffusion model and distills it into a fast stochastic sampler. It does this by learning a continuous transition kernel that maps the noisy distribution at one time to the distribution at an earlier time; when the kernel is parameterized as a consistency function, the paper proves that the trained map also sends every noisy distribution back to the clean data distribution. The central theoretical result is that iterating the map over a few steps shrinks the Wasserstein error from the one-step error at the largest time scale to a smaller error at an intermediate time scale, with a multiplicative decay factor of $(3/4)^{K-1}$. On image benchmarks the method reaches, with 2-4 function evaluations, quality comparable to the teacher diffusion model and the best FD-DINOv2 values in its comparisons.

What carries the argument

The load-bearing object is the continuous transition kernel, parameterized as a consistency function (a map that sends each noisy marginal $p(x_t;t)$ back to the clean data distribution $p(x_0;0)$): $q_\phi(x_s|x_t;s,t)$ is realized by sampling $x_s = a(s)G_\phi(x_t,t)+\sigma(s)\epsilon$ with $\epsilon\sim\mathcal{N}(0,I)$, so one network $G_\phi$ defines the kernel for every pair $0<s<t\le T$. The training objective is the two-stage score-matching problem (15), in which an auxiliary function $f_\psi$ is fitted to the score of the variational distribution and regularized toward the pretrained score network; the paper proves that the Nash equilibrium of this two-stage game makes $G_\phi$ a consistency function. This machinery gives simulation-free training, because samples $x_t$ come directly from the Gaussian perturbation $x_t=a(t)x_0+\sigma(t)\epsilon$, and it enables multistep sampling by iterating the kernel over a chosen schedule.

What would settle it

Take a synthetic data distribution with a known score, train CoSIM, and measure $\mathbb{E}\lVert \nabla\log p(x_s;s)-\nabla\log q_{\hat\phi}(x_s;s,t)\rVert^2$ for several values of $t$. If the measured error is flat in $t$ while multistep sampling still fails to reduce Wasserstein error, the scaling assumption behind Proposition 3.8 would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central claim is that a continuous semi-implicit model, CoSIM, can be trained at the distributional level so that its transition kernel becomes a consistency function: a map that sends each noisy marginal $p(x_t;t)$ back to the clean data distribution $p(x_0;0)$. The paper parameterizes the kernel as $x_s = a(s)G_\phi(x_t,t)+\sigma(s)\epsilon$, where $G_\phi$ is the consistency map and $\epsilon\sim\mathcal{N}(0,I)$, and trains it with the two-stage score-matching objective (15), regularizing the auxiliary function toward the pretrained score network. Proposition 3.2 shows that the optimal $G_\phi$ for this objective maps $p(x_t;t)$ back to $p(x_0;0)$, so the continuous kernel is exactly a consistency function. Proposition 3.8 then bounds the Wasserstein error of $K$-step sampling by $\delta^2 d + (3/4)^{K-1}E^{1/2}_{W_2^2(T)} + E^{1/2}_{W_2^2(t_{\mathrm{mid}})}$, where $E_{W_2^2}(t)$ is the one-step Wasserstein bound between the CoSIM estimate and $p(\cdot;\delta)$; because this bound grows with $t$, multistep sampling replaces the one-step error at time $T$ with a smaller error at an intermediate time. Empirically CoSIM reports FID 1.97 on CIFAR-10 with 4 steps, FID 1.46 on ImageNet 64 with 4 steps, and FD-DINOv2 41.79 on ImageNet 512 with 4 steps.

Load-bearing premise

The multistep benefit rests on the assumption that the consistency map's approximation error grows as the time gap grows: the error at time $t$ is bounded by a constant times the divergence between the forward-process distributions at $t+\delta$ and $\delta$. If the trained map's error does not actually grow with $t$, or if the pretrained score network misrepresents the true score, the guarantee that more steps reduce the sampling error does not follow.

Editorial extensions

If this is right

  • CoSIM's 4-step sampler reaches FID 1.97 on CIFAR-10, matching the 35-step teacher, and FID 1.46 on ImageNet 64, so a few function evaluations can replace hundreds of solver steps for a fixed pretrained diffusion model.
  • Because the kernel is trained at the distributional level, distillation does not require matching the deterministic reverse trajectory sample by sample, which the paper argues reduces training iterations relative to sequential hierarchical training.
  • FD-DINOv2 results improve monotonically as the number of steps grows from 2 to 4 on CIFAR-10, matching the theoretical prediction that multistep sampling reduces approximation error.
  • The method scales from 56M-parameter models to 778M-parameter models, and on ImageNet 512 the 4-step model's FD-DINOv2 of 41.79 beats the largest teacher configuration tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exponential decay factor $(3/4)^{K-1}$ implies that most of the quality gain from multistep sampling is realized within the first few steps; practitioners should expect diminishing returns beyond 4-6 function evaluations.
  • Because CoSIM trains in the latent space of a variational autoencoder for the largest benchmarks, the same continuous-kernel construction should transfer to latent text-to-image pipelines that already operate on such codes.
  • The bound's dependence on the dimension $d$ through the logarithmic Sobolev constant suggests the error floor rises with data dimension; a testable extension is whether larger consistency-map networks lower the $\varepsilon_c$ term faster than the dimension-dependent constant raises it.
  • The paper's general version of the equilibrium-shifting reformulation holds for any strictly convex penalty beyond the squared norm, so the same two-stage training scheme could be adapted to other variational divergences.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper introduces CoSIM, a continuous-time extension of hierarchical semi-implicit variational inference for distilling pretrained diffusion models. The model defines a transition kernel q_φ(x_s|x_t;s,t) = N(x_s; a(s)G_φ(x_t,t), σ(s)²I) and trains G_φ and an auxiliary function f_ψ with a two-stage score-matching objective (15). The authors prove that the two-stage objective is equivalent to a scaled Fisher divergence (Theorem 3.1), that a perfectly optimized G_φ is a consistency map (Proposition 3.2), and provide one-step and multistep error bounds (Propositions 3.4 and 3.8). Empirically, CoSIM reaches teacher-level FID with 4 NFE on CIFAR-10 (1.97) and ImageNet 64 (1.46), and reports state-of-the-art FD-DINOv2 on CIFAR-10 and ImageNet 512.

Significance. The paper addresses an important problem: accelerating diffusion models with few-step stochastic sampling while maintaining diversity. The main contributions are the continuous semi-implicit formulation, the two-stage training objective with regularization, and strong empirical results, including new state-of-the-art FD-DINOv2 scores on several benchmarks. The theoretical analysis is a useful step toward understanding multistep distillation at the distributional level, but it rests on assumptions that are only partially validated. The empirical claims are supported by comparisons with a range of recent methods and by ablations, and the code is released. The lack of error bars and the reliance on idealized assumptions temper the theoretical conclusions, but the core method appears sound and of interest to the generative modeling community.

major comments (3)
  1. [§3.4, Assumption 3.5, Proposition 3.8] The claimed multistep benefit is a comparison of upper bounds: the bound is (3/4)^{K-1} E^{1/2}_{W2²}(T) + E^{1/2}_{W2²}(tmid). The reduction from E^{1/2}(T) to E^{1/2}(tmid) requires E_{W2²}(t) to grow with t, and this growth is enforced entirely by Assumption 3.5 through ε_g(t)² = 2ε_c² FI(p(·;t+δ)||p(·;δ)). The assumption is validated only by the identity map, which realizes the bound with ε_c=1; it is not shown to hold for the trained G_φ. If the trained map has roughly constant accuracy across t, or if the ε_f term in (46) dominates, then E_{W2²}(t) is flat and the theorem yields no multistep advantage. I recommend adding an empirical check of Assumption 3.5 or a direct measurement of the one-step error E_{W2²}(t) at several t values for the trained model, and adjusting the theoretical claim accordingly.
  2. [§3.2 and Algorithm 2; Propositions 3.4 and 3.8] The theory assumes access to the true score ∇log p(x_s;s), while training and inference use the teacher score S_θ*(x_s;s). No bound on the discrepancy between S_θ* and the true score appears in Propositions 3.4 or 3.8, so the guarantees are relative to the teacher's marginal, not the data distribution. Since the empirical goal is to generate from p(x_0), this gap is material. I suggest either adding an assumption of the form E‖S_θ*(x;s) − ∇log p(x;s)‖² ≤ κ and propagating κ through the bounds, or explicitly restating the results as bounds against the teacher's distribution.
  3. [§3.4, Proposition 3.8 vs Algorithm 1] The sampling schedule analyzed in Proposition 3.8 has t_1 = ... = t_{K-1} = t_mid, i.e., repeated time points, whereas Algorithm 1 describes a strictly decreasing sequence T = t_0 > t_1 > ... > t_k = 0. The proof iteratively applies G_φ at the same intermediate time; this is a different inference procedure from the one stated in Algorithm 1. Please clarify whether the experimental multistep schedule is the one covered by the theorem, and if so, update Algorithm 1 and the theorem statement to use non-increasing sequences; otherwise, the guarantee does not apply to the reported sampling procedure.
minor comments (6)
  1. [Equations (10) and (12)] The second-stage regression target is written as ∥f_ψ − log q_φ(x_s|x_t;s,t)∥²; this should be ∥f_ψ − ∇_{x_s} log q_φ(x_s|x_t;s,t)∥², as used in Theorem 3.1 and Algorithm 2. The missing gradient makes the displayed objective inconsistent with the rest of the paper.
  2. [Table 3 and Section 4.2] The text reports an FD-DINOv2 of 56.66 for CoSIM at 4 NFE, while the table entry is 58.66; please reconcile.
  3. [Table 1] The row 'CIFAR:Test Split' appears to have a typo in the FID column ('-3.15'); the value should presumably be 3.15.
  4. [Table 6] Several entries are misformatted, e.g., 'coef = 0.5309.77' should be 'coef = 0.5' with value 309.77; please fix the table formatting.
  5. [Proposition 3.8] The main-text bound omits the constant (1 − a(t_mid)L_vp)^{-1} that multiplies E^{1/2}_{W2²}(tmid) in the proof; for the stated choice t_mid = O(log L_f) this constant is 4, so the bound should read δ²d + (3/4)^{K-1}E^{1/2}(T) + 4E^{1/2}(tmid) (or the proof should be adjusted).
  6. [Experiments] No error bars or multiple-seed results are reported; given that several comparisons differ by less than 0.1 FID, a statement about run-to-run variability would strengthen the empirical claims.

Circularity Check

1 steps flagged · score 4.0 of 10

Multistep-benefit theorem restates Assumption 3.5's monotone error scaling; the central empirical results remain external.

  1. self definitional [Section 3.4, Assumption 3.5; Proposition 3.8; Eq. (46) in Appendix B.4]
    "Assumption 3.5: '... the error term is scaled by εg(t)² := 2εc² FI(p(·;t+δ)∥p(·;δ)) ... We note that the divergence FI(p(·;t+δ)∥p(·,δ)) grows as t increases to T. Furthermore, when {Gφ|φ∈Φ} contains only the identity mapping, the aforementioned upper bound holds with ε²c = 1, validating the reasonableness of assumption 3.5.' Proposition 3.8: 'Intuitively, since the Wasserstein distance bound EW²₂(t) is increasing with t, the benefit of multistep sampling lies in reducing the error bound from E^{1/2}_{W²₂(T)} of the one-step model to a smaller one E^{1/2}_{W²₂(tmid)}.'"

    The advertised multistep advantage is built into Assumption 3.5 rather than derived for the trained Gφ. Setting εg(t)² ∝ FI(p(·;t+δ)∥p(·;δ)) makes the one-step bound EW²₂(t) := εc²ΔFI(p(·;t+δ)∥p(·;δ)) + Δεf² an increasing function of t by construction. The only validation offered is that the identity map realizes the bound, which says nothing about the learned network. Proposition 3.8 then predicts K-step sampling beats one-step because EW²₂(tmid) < EW²₂(T); that inequality is exactly the assumed monotone scaling. If εg(t) were independent of t, the same argument would yield no multistep benefit. Thus the proposition's central prediction is equivalent to Assumption 3.5's scaling, not an independent property of the trained model.

full rationale

The paper's core empirical claims are external: FID and FD-DINOv2 numbers on CIFAR-10, ImageNet 64, and ImageNet 512 are measured against public teacher models and published baselines, so they do not reduce to the paper's own assumptions. Proposition 3.2, which derives consistency from the two-stage objective, is a genuine mathematical consequence of the Gaussian reparameterization rather than a circular input; the proof in Appendix B.2 is self-contained. Theorem 3.1 and Proposition 3.4 are likewise proved in the appendix rather than merely imported. The one significant circular-adjacent step is the theoretical multistep guarantee: Proposition 3.8's claimed benefit from many steps is, by construction, a restatement of Assumption 3.5's monotone error scaling, and that assumption is justified only by an identity-map upper bound. Because this reduces one theoretical prediction to its own input while the central empirical content remains independent, the score is 4 rather than higher.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The central theoretical claims rest on standard score-matching assumptions plus a capacity assumption (3.5), an LSI condition (3.6), a Lipschitz condition (3.7), and an approximate optimality assumption on the auxiliary function (3.3). The method also assumes the pretrained score model equals the true score. No new physical entities are introduced.

free parameters (4)
  • α = 1.2
    Scaling of the squared penalty in the Fisher-divergence objective; taken from SiD (Zhou et al., 2024b) rather than derived.
  • coef = α(1+λ) = 1.0
    Regularization strength; chosen for simplicity and ablated in Table 6, where coef=1.0 gives the best FD-DINOv2 at the largest training budget.
  • R (time-schedule hyperparameter) = 4 or 8
    Controls the distribution of time gaps γ in the (s,t) sampling schedule in Appendix A.1; different values are used per dataset.
  • Inference time scale parameter = not reported
    Selected by a greedy algorithm for the sampling time sequence in Algorithm 1 (Section 3.3); no formula or final value is given, so the reported 2 to 4 step numbers include this tuning choice.
assumptions (7)
  • domain assumption The pretrained score model Sθ*(xs;s) accurately approximates ∇log p(xs;s) for all s in [δ,T].
    Used throughout Section 3.2 to replace the target score, including in equations (10), (15), and Algorithm 2; any teacher error enters the objective and the error bounds.
  • domain assumption The variational family Q is rich enough that the global minimizer of (38) attains zero Fisher divergence to p(xs;s) for each s,t, i.e., qφ(xs;s,t)=p(xs;s).
    Proposition 3.2's proof assumes exact score equality, which requires both adequate network capacity and exact optimization; in practice the match is approximate.
  • domain assumption Assumption 3.3: the estimated auxiliary function fψ is εf-accurate for all φ, s, t.
    Controls the one-step Fisher divergence bound in Proposition 3.4; it is a convergence and accuracy assumption on the inner optimization.
  • domain assumption Assumption 3.5: the consistency-map approximation gap scales as ε_g(t)² = 2ε_c² FI(p(·;t+δ)||p(·;δ)).
    Used in the proof of Proposition 3.4 to convert capacity error into a term that grows with t; this growth is what makes multistep sampling beneficial in Proposition 3.8.
  • standard math Assumption 3.6: p(xδ;δ) satisfies a logarithmic Sobolev inequality with constant L_LSI.
    Standard functional-inequality condition used to convert Fisher divergence to KL and Wasserstein bounds via Talagrand, following Lee et al. (2023) and Chen et al. (2021).
  • standard math Assumption 3.7: the network Fφ(x,t) is Lf-Lipschitz in x for all t in [δ,T].
    Needed in Proposition 3.8 to propagate Wasserstein error across steps; the authors note it is equivalent to a Lipschitz condition on Gφ under the VP scheme.
  • domain assumption p(x0;0) is supported on the hypercube [-1,1]^d, so p(xδ;δ) satisfies the stated LSI constant bound with the given σ(δ).
    Invoked in Appendix B.4 to derive the explicit LSI constant and early-stopping condition; it follows from image preprocessing rather than from the method itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continuous Semi-Implicit Models." pith.science (2026). https://pith.science/paper/BSOKT3TP

@misc{pith2026250606778,
  author       = {Pith},
  title        = {Pith review of: Continuous Semi-Implicit Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BSOKT3TP}},
  note         = {Machine review of arXiv:2506.06778}
}
read the original abstract

Semi-implicit distributions have shown great promise in variational inference and generative modeling. Hierarchical semi-implicit models, which stack multiple semi-implicit layers, enhance the expressiveness of semi-implicit distributions and can be used to accelerate diffusion models given pretrained score networks. However, their sequential training often suffers from slow convergence. In this paper, we introduce CoSIM, a continuous semi-implicit model that extends hierarchical semi-implicit models into a continuous framework. By incorporating a continuous transition kernel, CoSIM enables efficient, simulation-free training. Furthermore, we show that CoSIM achieves consistency with a carefully designed transition kernel, offering a novel approach for multistep distillation of generative models at the distributional level. Extensive experiments on image generation demonstrate that CoSIM performs on par or better than existing diffusion model acceleration methods, achieving superior performance on FD-DINOv2.

Figures

Figures reproduced from arXiv: 2506.06778 by the authors.

Figure 1
Figure 1. Selected generated images on Imagenet 512 × 512 using L model from Section 4.2. expressive family of variational posteriors, leading to im￾proved approximation accuracy (Yin & Zhou, 2018; Tit￾sias & Ruiz, 2019; Moens et al., 2021; Yu & Zhang, 2023; Cheng et al., 2024). Beyond variational inference, semi￾implicit architectures have been successfully integrated into deep generative models, including variational autoen… view at source ↗
Figure 2
Figure 2. Conditionally generated images on ImageNet (512 × 512). The two batches of images are generated from 4-step (left) and 2-step (right) CoSIM L model respectively, with identical initial noise and class labels [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Unconditionally generated 32 × 32 images on CIFAR10 using 2-step sampling. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Unconditionally generated 32 × 32 images on CIFAR10 using 4-step sampling. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: Conditionally generated 64 × 64 images on Imagenet using 2-step sampling. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Conditionally generated 64 × 64 images on Imagenet using 4-step sampling. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Class-conditioned 512 × 512 images generated by CoSIM with different steps on Imagenet using M model, starting from identical noise. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: Class-conditioned 512 × 512 images generated by CoSIM with different steps on Imagenet using L model, starting from identical noise. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 30 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Functional inequalities for gaussian convolutions of compactly supported measures: explicit bounds and dimension dependence

    Bardet, J.-B., Gozlan, N., Malrieu, F., and Zitt, P.-A. Functional inequalities for gaussian convolutions of compactly supported measures: explicit bounds and dimension dependence. Bernoulli, 24 0 (1): 0 333--353, 2018

    work page 2018

  3. [3]

    and Guillin, A

    Cattiaux, P. and Guillin, A. Functional inequalities for perturbed measures with applications to log-concave measures and to some bayesian problems. Bernoulli, 28 0 (4): 0 2294--2321, 2022

    work page 2022

  4. [4]

    Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions, 2023

    Chen, H., Lee, H., and Lu, J. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions, 2023. URL https://arxiv.org/abs/2211.01916

    arXiv 2023

  5. [5]

    Dimension-free log-Sobolev inequalities for mixture distributions

    Chen, H.-B., Chewi, S., and Niles-Weed, J. Dimension-free log-sobolev inequalities for mixture distributions, 2021. URL https://arxiv.org/abs/2102.11476

    work page Pith review arXiv 2021

  6. [6]

    Particle-based variational inference with generalized wasserstein gradient flow

    Cheng, Z., Zhang, S., Yu, L., and Zhang, C. Particle-based variational inference with generalized wasserstein gradient flow. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    work page 2023

  7. [7]

    Kernel semi-implicit variational inference

    Cheng, Z., Yu, L., Xie, T., Zhang, S., and Zhang, C. Kernel semi-implicit variational inference. In Forty-First International Conference on Machine Learning, 2024

    work page 2024

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

Show all 60 references
  1. [9]

    and Brox, T

    Dosovitskiy, A. and Brox, T. Generating images with perceptual similarity metrics based on deep networks, 2016. URL https://arxiv.org/abs/1602.02644

    2016 arXiv

  2. [10]

    Geng, Z., Pokle, A., Luo, W., Lin, J., and Kolter, J. Z. Consistency models made easy, 2024. URL https://arxiv.org/abs/2406.14548

    2024 arXiv

  3. [11]

    J., and Wierstra, D

    Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., and Wierstra, D. Draw: A recurrent neural network for image generation, 2015. URL https://arxiv.org/abs/1502.04623

    2015 arXiv

  4. [12]

    Gronwall, T. H. Note on the derivatives with respect to a parameter of the solutions of a system of differential equations. Annals of Mathematics, 20: 0 292, 1919

    1919

  5. [13]

    Multistep consistency models, 2024

    Heek, J., Hoogeboom, E., and Salimans, T. Multistep consistency models, 2024. URL https://arxiv.org/abs/2403.06807

    2024 arXiv

  6. [14]

    GAN s trained by a two time-scale update rule converge to a local N ash equilibrium

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. GAN s trained by a two time-scale update rule converge to a local N ash equilibrium. In Advances in Neural Information Processing Systems, pp.\ 6626--6637, 2017

    2017

  7. [15]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33: 0 6840--6851, 2020

    2020

  8. [16]

    Elucidating the design space of diffusion-based generative models

    Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=k7FuTOWMOc7

    2022

  9. [17]

    Analyzing and improving the training dynamics of diffusion models

    Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., and Laine, S. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 24174--24184, 2024

    2024

  10. [18]

    Consistency trajectory models: Learning probability flow ode trajectory of diffusion

    Kim, D., Lai, C.-H., Liao, W.-H., Murata, N., Takida, Y., Uesaka, T., He, Y., Mitsufuji, Y., and Ermon, S. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023

    arXiv

  11. [19]

    Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In ICLR, 2014

    2014

  12. [20]

    P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M

    Kingma, D. P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M. Improving variational inference with inverse autoregressive flow, 2017. URL https://arxiv.org/abs/1606.04934

    2017 arXiv

  13. [21]

    and Hinton, G

    Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. https://www.cs.toronto.edu/ kriz/cifar.html, 2009

    2009

  14. [22]

    Convergence for score-based generative modeling with polynomial complexity, 2023

    Lee, H., Lu, J., and Tan, Y. Convergence for score-based generative modeling with polynomial complexity, 2023. URL https://arxiv.org/abs/2206.06227

    2023 arXiv

  15. [23]

    Towards a mathematical theory for consistency training in diffusion models

    Li, G., Huang, Z., and Wei, Y. Towards a mathematical theory for consistency training in diffusion models. ArXiv, abs/2402.07802, 2024

    2024 arXiv

  16. [24]

    and Song, Y

    Lu, C. and Song, Y. Simplifying, stabilizing and scaling continuous-time consistency models, 2024. URL https://arxiv.org/abs/2410.11081

    2024 arXiv

  17. [25]

    DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps

    Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, J. DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022

    2022

  18. [26]

    Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models

    Luo, W., Hu, T., Zhang, S., Sun, J., Li, Z., and Zhang, Z. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=MLIs5iRq4w

    2023

  19. [27]

    Sampling is as easy as keeping the consistency: convergence guarantee for consistency models

    Lyu, J., Chen, Z., and Feng, S. Sampling is as easy as keeping the consistency: convergence guarantee for consistency models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=ZPiEIhQpos

    2024

  20. [28]

    Efficient semi-implicit variational inference

    Moens, V., Ren, H., Maraval, A., Tutunov, R., Wang, J., and Ammar, H. Efficient semi-implicit variational inference. arXiv preprint arXiv:2101.06070, 2021

    2021 arXiv

  21. [29]

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y., Li, S.-W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Maira...

    2024

  22. [30]

    and Villani, C

    Otto, F. and Villani, C. Generalization of an inequality by talagrand and links with the logarithmic sobolev inequality. Journal of Functional Analysis, 173: 0 361--400, 2000

    2000

  23. [31]

    Ranganath, R., Tran, D., and Blei, D. M. Hierarchical variational models, 2016. URL https://arxiv.org/abs/1511.02386

    2016 arXiv

  24. [32]

    Generating diverse high-fidelity images with vq-vae-2, 2019

    Razavi, A., van den Oord, A., and Vinyals, O. Generating diverse high-fidelity images with vq-vae-2, 2019. URL https://arxiv.org/abs/1906.00446

    2019 arXiv

  25. [33]

    Rezende, D. J. and Mohamed, S. Variational inference with normalizing flows, 2016. URL https://arxiv.org/abs/1505.05770

    2016 arXiv

  26. [34]

    J., Mohamed, S., and Wierstra, D

    Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models, 2014. URL https://arxiv.org/abs/1401.4082

    2014 arXiv

  27. [35]

    Improved techniques for training gans

    Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016

    2016

  28. [36]

    Multistep distillation of diffusion models via moment matching

    Salimans, T., Mensink, T., Heek, J., and Hoogeboom, E. Multistep distillation of diffusion models via moment matching. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=C62d2nS3KO

    2024

  29. [37]

    N., Weiss, E

    Sohl-Dickstein, J. N., Weiss, E. A., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. ArXiv, abs/1503.03585, 2015

    2015 arXiv

  30. [38]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021

    2021

  31. [39]

    and Dhariwal, P

    Song, Y. and Dhariwal, P. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=WNzy9bRDvG

    2024

  32. [40]

    and Ermon, S

    Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In Neural Information Processing Systems, 2019

    2019

  33. [41]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  34. [42]

    Consistency models

    Song, Y., Dhariwal, P., Chen, M., and Sutskever, I. Consistency models. arXiv preprint arXiv:2303.01469, 2023

    2023 arXiv

  35. [43]

    L., Taylor, E., and Loaiza-Ganem, G

    Stein, G., Cresswell, J., Hosseinzadeh, R., Sui, Y., Ross, B., Villecroze, V., Liu, Z., Caterini, A. L., Taylor, E., and Loaiza-Ganem, G. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Proces...

    2024

  36. [44]

    Rethinking the inception architecture for computer vision

    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 2818--2826, 2016

    2016

  37. [45]

    K., Raiko, T., Maaløe, L., Sønderby, S

    Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., and Winther, O. Ladder variational autoencoders, 2016. URL https://arxiv.org/abs/1602.02282

    2016 arXiv

  38. [46]

    Titsias, M. K. and Ruiz, F. J. R. Unbiased implicit variational inference. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.\ 167--176. PMLR, 2019

    2019

  39. [47]

    and Kautz, J

    Vahdat, A. and Kautz, J. Nvae: A deep hierarchical variational autoencoder, 2021. URL https://arxiv.org/abs/2007.03898

    2021 arXiv

  40. [48]

    A connection between score matching and denoising autoencoders

    Vincent, P. A connection between score matching and denoising autoencoders. Neural Computation, 23 0 (7): 0 1661--1674, 2011. doi:10.1162/NECO_a_00142

    2011 doi

  41. [49]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Wang, Z., Lu, C., Wang, Y., Bao, F., Li, C., Su, H., and Zhu, J. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In Advances in Neural Information Processing Systems (NeurIPS), 2023 a

    2023

  42. [50]

    Diffusion- GAN : Training GAN s with diffusion

    Wang, Z., Zheng, H., He, P., Chen, W., and Zhou, M. Diffusion- GAN : Training GAN s with diffusion. In The Eleventh International Conference on Learning Representations, 2023 b . URL https://openreview.net/forum?id=HZf7UbpWHuA

    2023

  43. [51]

    Pfgm++: Unlocking the potential of physics-inspired generative models

    Xu, Y., Liu, Z., Tian, Y., Tong, S., Tegmark, M., and Jaakkola, T. Pfgm++: Unlocking the potential of physics-inspired generative models. In International Conference on Machine Learning, pp.\ 38566--38591. PMLR, 2023

    2023

  44. [52]

    and Zhou, M

    Yin, M. and Zhou, M. Semi-implicit variational inference. In International Conference on Machine Learning, pp.\ 5646--5655, 2018

    2018

  45. [53]

    T., and Park, T

    Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W. T., and Park, T. One-step diffusion with distribution matching distillation. arXiv preprint arXiv:2311.18828, 2023

    2023 arXiv

  46. [54]

    and Zhang, C

    Yu, L. and Zhang, C. Semi-implicit variational inference via score matching. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=sd90a2ytrt

    2023

  47. [55]

    Hierarchical semi-implicit variational inference with application to diffusion model acceleration

    Yu, L., Xie, T., Zhu, Y., Yang, T., Zhang, X., and Zhang, C. Hierarchical semi-implicit variational inference with application to diffusion model acceleration. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=ghIBaprxsV

    2023

  48. [56]

    Truncated diffusion probabilistic models and diffusion-based adversarial auto-encoders

    Zheng, H., He, P., Chen, W., and Zhou, M. Truncated diffusion probabilistic models and diffusion-based adversarial auto-encoders. In The Eleventh International Conference on Learning Representations, 2023 a . URL https://openreview.net/forum?id=HDxgaKk956l

    2023

  49. [57]

    Learning stackable and skippable LEGO bricks for efficient, reconfigurable, and variable-resolution diffusion modeling, 2023 b

    Zheng, H., Wang, Z., Yuan, J., Ning, G., He, P., You, Q., Yang, H., and Zhou, M. Learning stackable and skippable LEGO bricks for efficient, reconfigurable, and variable-resolution diffusion modeling, 2023 b

    2023

  50. [58]

    Long and short guidance in score identity distillation for one-step text-to-image generation

    Zhou, M., Wang, Z., Zheng, H., and Huang, H. Long and short guidance in score identity distillation for one-step text-to-image generation. ArXiv 2406.01561, 2024 a . URL https://arxiv.org/abs/2406.01561

    2024 arXiv

  51. [59]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation

    Zhou, M., Zheng, H., Wang, Z., Yin, M., and Huang, H. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In International Conference on Machine Learning, 2024 b

    2024

  52. [60]

    Adversarial score identity distillation: Rapidly surpassing the teacher in one step

    Zhou, M., Zheng, H., Gu, Y., Wang, Z., and Huang, H. Adversarial score identity distillation: Rapidly surpassing the teacher in one step. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=lS2SGfWizd

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.