Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Denoising diffusion models provably reach the optimal √D/ε sampling rate in Wasserstein-2 distance, even when score evaluations are corrupted by noise.

desk verdict A real advance on W2 rates for DDPMs, but the optimality claim hides log factors and the key score-bound assumption is a strong sup-norm condition that the authors themselves flag. read the letter →

arxiv 2506.09681 v1 pith:VXNULYFF submitted 2025-06-11 stat.ML cs.LGmath.STstat.TH

Vahan Arsenyan , Elen Vardanyan , Arnak Dalalyan This is my paper
classification stat.MLcs.LGmath.STstat.TH MSC 60J6562M0562G05
keywords denoisingdiffusionprobabilisticmodelsWasserstein-2distancescoreestimationnoiseoptimalsamplingratesscore-basedgenerativediscretizationerrorstronglog-concavitysemi-log-concavedistributions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes finite-sample guarantees for denoising diffusion probabilistic models (DDPMs) in Wasserstein-2 distance when the score function is evaluated through a randomized estimator with bounded bias and variance. The main theorems place an upper bound of order √D hmax on the sampling error, where hmax is the largest time step of the discretization, matching the known lower bound for Gaussian targets and making the number of score queries needed for accuracy ε scale as √D/ε up to logarithmic factors. The same bound shows that additive noise in the score evaluations only contributes a term of order √D $hmax^{{1/2}}$ εv_score, which vanishes as the number of steps grows, explaining the empirically observed robustness. Experiments on CIFAR-10, CelebA-HQ, and LSUN-Churches confirm that sample quality is nearly unaffected by constant-scale noise in the score until a certain noise level.

What carries the argument

The load-bearing machinery is a recursion on the squared Wasserstein distance between the idealized reverse diffusion and the discretized, score-estimated process. The contraction factor at each step is controlled by Assumption 1 through the second-order Tweedie formula, which expresses the Hessian of the log-density of the smoothed target as (α²/β⁴)Var(X|Y=y) − $β^{{-2}}$I_D; this gives a factor 1 − m_k h_k on the arithmetic part of the grid. Assumption 2 converts the localized per-step bias and variance of the score estimator into global constants εb_score and εv_score, and Lemma 11 unfolds the recursion to give the exponential factor $e^{{(4/3)bM}}$. The two-stage time grid (arithmetic then geometric) keeps the discretization bias and variance small while keeping the total number of queries at 2K0.

What would settle it

Take a compactly supported target and a score estimator with small L2 error under the smoothed target but large sup-norm error on a small-probability region; if the resulting DDPM's Wasserstein-2 error exceeds the bound of Theorem 2, the uniform-norm assumption is essential. A concrete check is to compute the empirical W2 error for a mixture target with a far-away low-mass component and compare it to the predicted √D hmax scaling as the step size decreases.

Watch

Extended reading notes

Core claim

The central claim is that, for targets satisfying Assumption 1 with a conditional-variance function φ of one of two explicit forms, the distribution generated by Algorithm 1 after K = 2K0 queries satisfies W2(P*, P^DDPM) ≤ $e^{{(4/3)bM}}$ {$2e^{{-T1}}$ + 7√(6a) hmax + 4√(6a)(2εb_score + $hmax^{{1/2}}$ εv_score)}√D, and an analogous bound for semi-log-concave targets with bounded support. Here √D is the dimension scaling, hmax is the maximal step of an arithmetic-then-geometric time grid, and εb_score, εv_score are the uniform sup-norm bounds on the bias and standard deviation of the randomized score estimator. Because the Gaussian target already requires an error of order √D hmax, the paper concludes that the rate is optimal, improving on earlier rates of order D/ε² or poorer in Wasserstein distance. The proof obtains the linear hmax dependence by running the recursive error analysis on the squared Wasserstein distance rather than the distance itself, using the second-order Tweedie formula to relate the conditional variance of the target given its noisy observation to the Hessian of the smoothed log-density.

Load-bearing premise

The argument collapses if the score estimator is only controlled in an L2 sense with respect to the smoothed target: the proof requires bounds on the bias and variance that hold uniformly over every input x in R^D and every time on the grid, as stated in Assumption 2.

Editorial extensions

If this is right

  • For strongly log-concave targets, the bound simplifies to W2 ≲ √(LD) (logL logK/K + εb_score + √(logL logK/K) εv_score), so O(√D/ε) queries suffice for accuracy ε.
  • Constant-variance noise in the score has a vanishing effect on sample quality as K grows, since its contribution is O(√D hmax^{1/2} εv_score), unlike previous bounds where noise contributed a constant-order term.
  • The √D hmax rate matches the Gaussian lower bound, so the query complexity √D/ε is optimal up to constants and logarithmic factors for the considered class of targets.
  • The guarantees cover compactly supported semi-log-concave distributions living on low-dimensional subspaces, without requiring the target to be absolutely continuous on R^D.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same squared-distance recursion may yield improved step-size dependence for kinetic Langevin samplers and randomized-midpoint discretizations, where the error currently scales with a fractional power of the step.
  • If Assumption 2 could be relaxed to an L2 bound with respect to the DDPM's own intermediate laws, the results would apply directly to score estimators trained by empirical score matching, which only control integrated squared error.
  • The empirical FID degradation threshold around σ≈1–2 could be quantitatively predicted from the exponential factor e^{(4/3)bM} and the score Lipschitz constants; measuring this threshold across targets with different diameters would test that prediction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies denoising diffusion probabilistic models (DDPMs) with randomized and noisy score estimators. It states two finite-sample Wasserstein-2 guarantees, Theorems 2 and 3, under Assumption 1 on the target distribution and Assumption 2 requiring uniform sup-norm control of the bias and variance of the score estimator. The bounds are of the form sqrt(D) times a sum of a discretization error, a score-bias term, and a sqrt(h_max)-weighted score-variance term, and the paper claims that this matches the Gaussian lower bound sqrt(D) h_max, giving an optimal query complexity of order sqrt(D)/epsilon. The paper also reports CIFAR-10, CelebA-HQ, and LSUN-Church experiments showing empirical robustness to added score noise.

Significance. If the stated bounds are correct, this is a meaningful advance over previous DDPM analyses: it improves the Wasserstein-2 rate from D/epsilon^2 to sqrt(D)/epsilon for a substantially broader class than strongly log-concave targets, and it provides a quantitative explanation of robustness to score-evaluation noise. The appendix gives a detailed proof skeleton with explicit lemmas, and I did not find an internal contradiction. The main caveat, which the authors themselves acknowledge in Section 5, is that the theorems rely on the strong uniform sup-norm condition in Assumption 2 rather than the standard L2(P^*_t) score-error condition; this narrows the practical scope of the advertised guarantees.

major comments (3)
  1. [Section 2.3 and Appendix C.1] Assumption 2 is load-bearing and stronger than the standard score-estimation condition. In the proof, the localized error quantities epsilon^b_k and epsilon^v_k are L2 norms evaluated under the conditional law of Z_k, which follows the DDPM path, and the passage to global epsilon^b_score and epsilon^v_score uses the uniform sup bound over x in R^D. If only an L2 bound with respect to P^*_t is available, the stated Theorems 2 and 3 do not follow. The manuscript explicitly notes this in Section 5, but the abstract and introduction advertise finite-sample guarantees for noisy score estimates in a way that could mislead readers about the scope. The revision should either prove a version with an L2 bound along the DDPM output law, or state prominently that the results apply only to estimators satisfying the sup-norm condition and that standard L2(P^*_t) score estimators are not covered.
  2. [Section 2.1, Example 2] The compressed score estimator is incompatible with Assumption 2 for targets with unbounded score. For the estimator s_tilde(t,x) = D times the selected coordinate basis vector, the variance is (D-1) times the squared norm of the estimated score, which is unbounded over x whenever the score is unbounded. Thus one of the paper's motivating examples falls outside the theorem's hypotheses. Please either restrict Example 2 to bounded-score targets, explicitly flag it as outside the applicability of Theorems 2 and 3, or replace it with an example that satisfies the sup-norm condition.
  3. [Remark 1, Section 4.1] The optimality claim should be qualified carefully. The Gaussian lower bound from [GNZ25] gives a sqrt(D) h_max rate, while the upper bound in (7) involves prefactors such as exp((4/3)bM), sqrt(6a), and terms depending on m, M, and b. If 'optimal up to a multiplicative constant' means a constant independent of the target-distribution parameters, this is not established; if the constant is allowed to depend on those parameters, the statement should say so explicitly.
minor comments (4)
  1. [Theorem 2] The theorem states m, M, b >= 0, but the definition a = 1/m + b requires m > 0. Please either assume m > 0 or define a separately for the case m = 0.
  2. [Appendix B, Proof of Lemma 1] The displayed inequality '1 <= sup ... <= 1' near the end is confusing as written; the lower bound follows from the limit r(sigma) -> 1 and the upper bound from the contraction result, and the two should be stated separately.
  3. [Section 6, Figure 2] The subcaptions contain duplicated 'Noiseless score FID' values (e.g., repeated entries for CIFAR-10 and LSUN-Churches). Please verify that each value corresponds to the correct dataset and noise level.
  4. [Remark 4, Section 4.1] The remark says 'after K queries' and later writes K in the rate, but the theorem's K is 2K0. Please clarify the exact relationship between K and K0 so readers can translate the bound into a query complexity statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorems 2–3 are conditional upper bounds derived from Assumptions 1–2, and the optimality benchmark is the external Gaussian lower bound of [GNZ25], not a fitted or self-imported quantity.

full rationale

The derivation chain is self-contained. The main theorems (Theorem 2, Eq. (7); Theorem 3, Eq. (9)) upper-bound W2(P*, PDDPM) by a sum of an initialization term, a discretization term proportional to h_max, and score-bias and score-variance terms εb_score and εv_score. These constants are inputs specified in Assumption 2, not parameters fitted to the target quantity; the proof in Appendix C converts the pathwise quantities εb_k and εv_k into the global constants only through the stated uniform sup-norm assumption, which is a legitimate, if strong, hypothesis. The claimed optimality is benchmarked against the Gaussian lower bound of [GNZ25], an external result, and the paper does not invoke an author-specific uniqueness theorem or an ansatz smuggled in by self-citation. Self-citations (e.g., [Dal17a, DM17, DK19]) appear only as background for the recursive proof strategy; the actual recursion, Lemma 11, and the contraction bound, Lemma 10, are proved in the appendix. The Section 5 note that the uniform-norm condition 'could be relaxed' is a stated limitation of scope, not a circular reduction. No equation or claim reduces to its own input by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted: all constants are explicit functions of the assumed problem parameters (m, M, b, D_X, T1, K0). No new entities are introduced. The main auxiliary object is Assumption 1's conditional variance bound, which is a modeling condition rather than a fitted quantity.

assumptions (4)
  • domain assumption Target distribution P* satisfies Assumption 1 with φ(σ)=σ²/(1+mσ²)+bM²σ⁴/(1+Mσ²)² (Theorem 2) or φ(σ)=b∧σ²/(1-Mσ²)+ (Theorem 3).
    This variance bound on the conditional covariance controls the Hessian of the log-density via the Tweedie formula (Proposition 1) and is used in Lemma 10 to obtain contraction.
  • domain assumption Score estimator satisfies Assumption 2 with uniform sup-norm bounds on bias and variance over all x∈R^D and grid times.
    The proof converts local L2 errors εb_k and εv_k into global εb_score and εv_score using this uniform bound; the paper states that weaker L2(P*_t) bounds would not suffice.
  • domain assumption E[||X||²] ≤ D and the schedule parameters K0, T1, δ satisfy the inequalities in Theorems 2 and 3.
    Normalizes the bound and guarantees the two-stage grid satisfies the contraction conditions of Lemmas 10 and 12.
  • standard math Standard inequalities: Tweedie's formula, Brascamp-Lieb, Popoviciu, data-processing inequality, Girsanov, and Cramér-Rao.
    Used throughout the appendix proofs; standard results in the literature and not the paper's contribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds." pith.science (2026). https://pith.science/paper/VXNULYFF

@misc{pith2026250609681,
  author       = {Pith},
  title        = {Pith review of: Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VXNULYFF}},
  note         = {Machine review of arXiv:2506.09681}
}
read the original abstract

Generative modeling aims to produce new random examples from an unknown target distribution, given access to a finite collection of examples. Among the leading approaches, denoising diffusion probabilistic models (DDPMs) construct such examples by mapping a Brownian motion via a diffusion process driven by an estimated score function. In this work, we first provide empirical evidence that DDPMs are robust to constant-variance noise in the score evaluations. We then establish finite-sample guarantees in Wasserstein-2 distance that exhibit two key features: (i) they characterize and quantify the robustness of DDPMs to noisy score estimates, and (ii) they achieve faster convergence rates than previously known results. Furthermore, we observe that the obtained rates match those known in the Gaussian case, implying their optimality.

Figures

Figures reproduced from arXiv: 2506.09681 by the authors.

Figure 1
Figure 1. Generated images obtained by DDPM with a constant-level noise added to the estimated [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. FID as a function of noise level for four distributions and different standard deviations. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Notations corresponding to the discretization schedule. [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: FID as a function of time steps. Blue: standard DDPM inference. Orange: same sampler [PITH_FULL_IMAGE:figures/full_fig_p036_4.png]
Figure 5
Figure 5. Figure 5: A single example of CIFAR-10 (top), CelebA-HQ (middle) and LSUN-Churches (bottom) [PITH_FULL_IMAGE:figures/full_fig_p037_5.png]
Figure 6
Figure 6. Figure 6: Additional CIFAR-10 generations for 3 noise families (rows) and 2 noise levels (columns). [PITH_FULL_IMAGE:figures/full_fig_p038_6.png]
Figure 7
Figure 7. Figure 7: Additional LSUN-Church generations for 3 noise families (rows) and 2 noise levels [PITH_FULL_IMAGE:figures/full_fig_p039_7.png]
Figure 8
Figure 8. Figure 8: Additional CelebA-HQ generations for 3 noise families (rows) and 2 noise levels (columns). [PITH_FULL_IMAGE:figures/full_fig_p040_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 8 canonical work pages

  1. [6]

    Rotskoff

    [CRYR24] Haoxuan Chen, Yinuo Ren, Lexing Ying, and Grant M. Rotskoff. Accelerating diffusion models with parallel sampling: Inference at sub-linear time complexity. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024,

    work page 2024

  2. [10]

    Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality.CoRR, arXiv:2410.18784,

    [HWC24] Zhihan Huang, Yuting Wei, and Yuxin Chen. Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality.CoRR, arXiv:2410.18784,

    arXiv

  3. [11]

    Linear convergence of diffusion models under the manifold hypothesis.CoRR, arXiv:2410.09046,

    [PAD24] Peter Potaptchik, Iskander Azangulov, and George Deligiannidis. Linear convergence of diffusion models under the manifold hypothesis.CoRR, arXiv:2410.09046,

    arXiv

  4. [12]

    Beyond log-concavity and score regularity: Improved convergence bounds for score-based generative models in w2-distance.CoRR, arXiv:2501.02298,

    [SO25] Marta Gentiloni Silveri and Antonio Ocello. Beyond log-concavity and score regularity: Improved convergence bounds for score-based generative models in w2-distance.CoRR, arXiv:2501.02298,

    arXiv

  5. [14]

    Score-based diffusion models via stochastic differential equations - a technical tutorial.CoRR, arXiv:2402.07487,

    [TZ24] Wenpin Tang and Hanyang Zhao. Score-based diffusion models via stochastic differential equations - a technical tutorial.CoRR, arXiv:2402.07487,

    arXiv

  6. [15]

    [VCK25] Adrien Vacher, Omar Chehab, and Anna Korba. Polynomial time sampling from log-smooth distributions in fixed dimension under semi-log-concavity of the forward diffusion with application to strongly dissipative distributions.CoRR, arXiv:2501.00565,

    arXiv

  7. [16]

    Stochastic runge-kutta methods: Provable acceleration of diffusion models.CoRR, arXiv:2410.04760,

    [WCW24] Yuchen Wu, Yuxin Chen, and Yuting Wei. Stochastic runge-kutta methods: Provable acceleration of diffusion models.CoRR, arXiv:2410.04760,

    arXiv

  8. [17]

    Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem

    [Wib18] Andre Wibisono. Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors,Proceedings of the 31st Conference On Learning Theory, volume 75 ofProceedings of Machine Learning Research, pages 2093–3027. PMLR, 06–09 Jul

    work page

Show all 21 references
  1. [18]

    Optimal score estimation via empirical bayes smoothing

    [WWY24] Andre Wibisono, Yihong Wu, and Kaylee Yingxi Yang. Optimal score estimation via empirical bayes smoothing. In Shipra Agrawal and Aaron Roth, editors,The Thirty Seventh Annual Confer- ence on Learning Theory, June 30 - July 3, 2023, Edmonton, Canada, volume 247 ofProcee...

    2023

  2. [19]

    Convergence in KL and rényi divergence of the unadjusted langevin algorithm using estimated score

    [YW22] Kaylee Yingxi Yang and Andre Wibisono. Convergence in KL and rényi divergence of the unadjusted langevin algorithm using estimated score. InNeurIPS 2022 Workshop on Score-Based Methods,

    2022

  3. [20]

    Advancing wasserstein convergence analysis of score-based models: Insights from discretization and second-order acceleration.CoRR, arXiv:2502.04849,

    [YY25] Yifeng Yu and Lu Yu. Advancing wasserstein convergence analysis of score-based models: Insights from discretization and second-order acceleration.CoRR, arXiv:2502.04849,

    arXiv

  4. [21]

    Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop.arXiv preprint arXiv:1506.03365,

    [YZS+15] Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop.arXiv preprint arXiv:1506.03365,

    arXiv

  5. [1982]

    Nearly d-linear convergence bounds for diffusion models via stochastic localization

    [BBDD24] Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d-linear convergence bounds for diffusion models via stochastic localization. InThe Twelfth International Conference on Learning Representations, ICLR 2024,

    2024

  6. [2014]

    Adaptivity of diffusion models to manifold structures

    [TY24] Rong Tang and Yun Yang. Adaptivity of diffusion models to manifold structures. In Sanjoy Das- gupta, Stephan Mandt, and Yingzhen Li, editors,International Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain, volume 23...

    2024

  7. [2018]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

    [CCL+23] Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. InThe Eleventh International Conference on Learning Representations, ICLR 2023,

    2023

  8. [2019]

    Faster diffusion-based sampling with randomized midpoints: Sequential and parallel.CoRR, arXiv:2406.00924,

    [GCC24] Shivam Gupta, Linda Cai, and Sitan Chen. Faster diffusion-based sampling with randomized midpoints: Sequential and parallel.CoRR, arXiv:2406.00924,

    arXiv

  9. [2020]

    Denoising diffusion probabilistic models

    [HJA20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InNeurIPS 2020, December 6-12, 2020, virtual,

    2020

  10. [2022]

    On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates.CoRR, arXiv:2311.13584,

    [BZL+23] Stefano Bruno, Ying Zhang, Dong-Young Lim, Ömer Deniz Akyildiz, and Sotirios Sabanis. On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates.CoRR, arXiv:2311.13584,

    arXiv

  11. [2023]

    An overview of diffusion models: Applications, guided generation, statistical rates and optimization.CoRR, arXiv:2404.07771,

    [CMFW24] Minshuo Chen, Song Mei, Jianqing Fan, and Mengdi Wang. An overview of diffusion models: Applications, guided generation, statistical rates and optimization.CoRR, arXiv:2404.07771,

    arXiv

  12. [2024]

    Convergence of diffusion models under the manifold hypothesis in high-dimensions.CoRR, arXiv:2409.18804,

    [ADR24] Iskander Azangulov, George Deligiannidis, and Judith Rousseau. Convergence of diffusion models under the manifold hypothesis in high-dimensions.CoRR, arXiv:2409.18804,

    arXiv

  13. [2025]

    Convergence analysis for general probability flow odes of diffusion models in wasserstein distances.arXiv:2401.17958,

    [GZ24] Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow odes of diffusion models in wasserstein distances.arXiv:2401.17958,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.