Pith. sign in

REVIEW 3 major objections 5 minor 64 references

Diffusion models recover mixture weights accurately even when the target score is insensitive to them: annealing across noise levels keeps the score matching loss sensitive, and the paper proves this for Gaussian mixtures in any dimension.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 23:13 UTC pith:S74A53PO

load-bearing objection The DSSI is a genuinely useful way to think about why annealing helps with mixture weights, but the paper's central bridge from score error to generated-sample accuracy is assumed, not proved. the 3 major comments →

arxiv 2607.15485 v1 pith:S74A53PO submitted 2026-07-16 cs.LG math.PRstat.ML

Diffusion models recover accurate mixture weights despite score function insensitivity

Andrew Dennehy , Ramchandran Muthukumar , Rebecca Willett , Nisha Chandramoorthy This is my paper
classification cs.LG math.PRstat.ML MSC 62B1062C20
keywords diffusion modelsscore matchingmixture weightsdiffusion score sensitivity indexminimax estimationGaussian mixturesmode amplificationnoise schedules
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper resolves an apparent paradox in score-based generative modeling: diffusion models appear to cover all modes of a multimodal target, yet are suspected of getting the relative mode weights wrong. The right quantity, the paper argues, is not the sensitivity of the target score but the sensitivity of the diffusion score matching (DSM) loss, which anneals scores across all noise levels. The paper defines the diffusion score sensitivity index (DSSI) as the curvature of the DSM loss with respect to the mixture weight, and proves that the squared weight-estimation error is bounded by the DSM loss divided by the DSSI. For bimodal Gaussian mixtures in any dimension, the paper proves the DSSI is non-negligible, so a well-trained diffusion model recovers mixture weights accurately even at extreme weights and large mode separation, where classical score matching and gradient-based sampling from the target score alone fail. The same index predicts mixture-weight recovery on real data, and noise schedules that shrink it produce mode amplification.

Core claim

The paper's central claim is that diffusion models recover mixture weights accurately even when the target score is insensitive to them, because the diffusion score matching (DSM) loss, annealed across noise levels, stays sensitive to the weights. The load-bearing identity: (γ̂−γ*)² ≤ ℓDSM / L(γ*), where ℓDSM is the diffusion score matching loss and L(γ*) — the diffusion score sensitivity index — is its curvature in the weight. Theorem 4.1 bounds L(γ*) away from zero for bimodal Gaussian mixtures in any dimension, so a well-trained diffusion model recovers weights to within a multiple of the DSM loss, a rate that target-score-based estimation cannot match.

What carries the argument

The central object is the diffusion score sensitivity index (DSSI), L(γ*) := inf_γ ℓDSM(p(γ), p(γ*))/(γ−γ*)², a ratio of loss curvature to parameter distance that converts diffusion training loss into parameter-estimation error. Three mechanisms carry the argument: (1) mixture closure under the variance-preserving forward SDE — each noised marginal is a γ-mixture of noised components with the same γ — so the noised score decomposes into responsibility-weighted component scores; (2) a Gaussian white-noise score-oracle model, under which the KL divergence between score-observation laws at time t equals half the score matching loss at t, allowing classical minimax lower-bound arguments (packing

Load-bearing premise

The load-bearing premise is that mixture-weight recovery from generated diffusion samples can be modeled as estimation from the true score corrupted by Gaussian noise at a single diffusion time: the minimax theorem applies to this score oracle, not to the actual reverse-time sampler that integrates learned scores over all times with discretization error, and the loss-to-error bound additionally assumes the trained model's implied generative distribution lies in the same mixtu

What would settle it

Run the reverse-SDE sampler with the exact analytic score of a well-separated bimodal Gaussian mixture (separation ≥ 10) and extreme weight γ* = 0.1, using a coarse time discretization; estimate the weight γ̂ from generated samples by expectation-maximization and compare with the predicted bound (γ̂ − γ*)² ≤ ℓDSM/L(γ*) computed from the paper's formulas. If the observed weight error exceeds the prediction by an amount that grows as the time step increases, the oracle-level analysis is missing the sampler's discretization error; if the error tracks the bound, the mechanism is confirmed. Repeati

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For Gaussian mixtures in any dimension, a diffusion model whose trained score is close to the true score in L² generates samples whose mixture weights are accurate to within a factor of the DSM loss, even for extreme weights and large mode separation, where the target score carries almost no weight information.
  • Noise schedules and coarse-discretization samplers that shrink the DSSI can amplify some modes over others while preserving each mode's geometry, so sample-fidelity metrics will not flag the distortion; the bias shows up only in recovered weights, not in per-mode appearance.
  • The framework governs any scalar parameter, not just weights: the paper demonstrates the same sensitivity analysis for component means and for a covariance interpolation parameter, so parameter-recovery guarantees extend across qualitative properties of the target.
  • Annealing has a provable minimax benefit for parameter recovery, not only for density estimation: estimators with access to an intermediate-noise score oracle achieve strictly better worst-case weight error than estimators restricted to the target score.
  • Any training procedure that drives the DSM loss to zero recovers the target weight, because the bound forces (γ̂−γ*)² toward zero, provided the trained model's implied generative distribution stays within the mixture family.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The DSSI is effectively a Fisher-information-type curvature along the diffusion path; extended to multi-parameter targets it becomes a matrix, and the paper's responsibility-weighted score decomposition suggests cross-parameter identifiability will be governed by the same log-odds functions analyzed here for the weight.
  • A testable extension is whether the dimension-independent lower bound survives unequal covariances, anisotropic components, or more than two modes; the log-odds structure suggests the bound should degrade gracefully rather than vanish, but the theorem itself covers only the two-component isotropic case.
  • The cleanest practical confirmation of the mechanism would measure, for a single trained network, the achieved DSM loss, the DSSI of its noise schedule, and the weight estimated from its generated samples; the paper's Figure 2c is an early version of this experiment, and a version resolved over seeds and checkpoints would separate score error from time-discretization error.
  • Deploying the DSSI as an evaluation metric on real data requires knowing the true parameter and the true score, which the paper's real-data experiment sidesteps with surrogate trained models; a surrogate-free estimator of L(γ*), for instance from model pairs trained on resampled data, would make the framework usable where the target is unknown.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the diffusion score sensitivity index (DSSI), L(γ*) = inf_γ ℓDSM(p(γ), p(γ*))/(γ−γ*)², and argues that it governs the accuracy with which mixture weights (and, more generally, scalar parameters) can be recovered from generated samples of a diffusion model. The authors prove a minimax separation between score-restricted estimators that observe a Gaussian white-noise score oracle at an intermediate time t and estimators that observe only the target score (Theorem 3.5), and a dimension-independent lower bound on L(γ*) for bimodal isotropic Gaussian mixtures under a linear noise schedule (Theorem 4.1). In the Gaussian setting this gives the formal inequality (γ−γ*)² ≤ ℓDSM(p(γ),p(γ*))/L(γ*), so that a small DSM loss for a score that lies in the mixture family forces accurate weight recovery. Empirically, the paper estimates L(γ*) for GMMs and MNIST latent mixtures, shows that modified noise schedules reduce L(γ*) and worsen weight recovery, and argues that intermediate noise levels restore score sensitivity that is absent at time t=0.

Significance. If the central transfer statement were fully established, the paper would make a valuable conceptual contribution: it provides a concrete, computable diagnostic — the DSSI — for whether a diffusion model can recover qualitative parameters such as mixture weights, and it explains the beneficial role of annealing in terms of a quantity that is not fitted to data and is provably non-negligible for Gaussian mixtures in arbitrary dimension (Theorem 4.1). The empirical finding that noise schedules with smaller L(γ*) produce worse mixture-weight recovery, even when sample appearance is similar, is interesting and potentially practical. The lower-bound theorem is nontrivial and dimension-independent, and the paper does not fit constants to make its predictions; these are genuine strengths. However, the main theoretical gap — connecting DSM loss of a trained score to the distribution of mixture weights estimated from reverse-SDE samples — currently prevents the paper's central claim from being fully supported.

major comments (3)
  1. [Theorem 4.1 vs. Remark 6] The paper's central claim — that a small DSM loss implies accurate mixture-weight recovery from generated samples — is not established. Theorem 3.5 is a minimax statement about estimators that observe a Gaussian-white-noise version of the score at a fixed time t, not about the distribution produced by integrating learned scores over all t with the reverse SDE (4), nor about the mixture weight estimated from those samples. Theorem 4.1 lower-bounds the ratio ℓDSM(p(γ'), p(γ*))/(γ'−γ*)² for exact Gaussian-mixture score fields, but this constrains γ' only if the learned score is exactly the score of a mixture in the family. Remark 2 states: 'any score model which closely approximates the true score ... will have small ℓDSM, and thus its sample distribution will accurately reflect γ*.' That 'thus' is the load-bearing inference, and no theorem or cited result supplies it. Since p(θ) is not ass
  2. [Appendix C, Theorem 3.5 proof] The proof of Theorem 4.1 uses λ_t = 1, as Remark 6 acknowledges, whereas the DSM loss in (8), the DSSI in (1), and the numerical results use λ_t = 1−α_t. Since 1−α_t ≤ 1, a lower bound on the λ=1 integral does not imply the same lower bound for the λ=1−α_t integral; the factor may depend on t and shrink the constant. The theorem statement in Section 4 does not specify that λ_t=1, so Eq. (12) is not literally proved for the quantity defined in (8). Remark 6 says the analysis 'can be performed' for other weightings, but the manuscript does not provide the needed bound (the integral ∫ τ(1−τ²)/√(−log τ) dτ is mentioned only as a future challenge). This must be fixed: either state Theorem 4.1 with λ_t=1 and adjust the definitions, or extend the proof to the actual weighting λ_t=1−α_t.
  3. [Definitions 3.3–3.4] The proof of Theorem 3.5 is not complete as written. The Gaussian measures q_t^γ in Definition 3.1 are indexed by L²(p_t^γ), so q_t^γ and q_t^γ' are not defined on a common measurable space; the Cameron–Martin computation in Remark 5 requires a common Hilbert space and hence needs justification. In addition, Definitions 3.3 and 3.4 are nearly identical, but the proof text says 'using ... an appropriate choice of δ as a function of ε in Definition 3.4,' while Definition 3.4 contains no δ. The theorem itself invokes 'ϵ-diffusion sensitivity' with ε=ε_t, but the proof of Lemma C.1 and the lower bound seem to use the (ε,δ) version. The minimax separation in Theorem 3.5 is a listed main contribution, so this gap is load-bearing. The authors should either give a rigorous proof with the correct definitions and common Hilbert-space construction, or present the theorem as a conditional result.
minor comments (5)
  1. [Theorem 3.5] There is a sign error in the definition of α_t in the theorem statement: it should be α_t = exp(−∫_0^t β_s ds), matching Eq. (2)–(3), not exp(+∫ β_s ds). Lemma D.5 also states d/dt √α_t = ½β_t√α_t, while the surrounding algebra uses d/dt √α_t = −½β_t√α_t; this is confusing even if the displayed computation is correct.
  2. [Eq. (12)] The constants in the minimax bounds are 2 A_t² ε_t² (upper) and 4 A_t² ε_t² (lower). A separation factor of 2 is fine, but given that the target-score lower bound is formally the minimum over Γ_score,0 and the intermediate-score upper bound is the minimum over Γ_score,t, the reader would benefit from a sentence explaining why the two minimax values are directly comparable (same parameter space, same loss, same class of measurable estimators).
  3. [Remark 4] The definition of k in the theorem statement is garbled by notation. It should be k := |log((1−a)/a)| + |log((1−γ*)/γ*)|. The current display appears to contain nested norms and is not readable.
  4. [Definitions 3.3–3.4] The second sentence of Remark 4 appears to have a typo: it says 'the estimators in Γ_score,0 must include sampling (denoising) steps that access score data from q_t^γ,' but in context this should be Γ_score,t. As written, the remark is internally inconsistent and obscures the intended extension of the estimator classes.
  5. [General] Definition 3.3 (ε,δ)-diffusion sensitivity is never used in any theorem and adds confusion. If it is not needed, it should be removed; if it is used in the proof of Theorem 3.5, the theorem should refer to it explicitly.

Circularity Check

1 steps flagged

DSSI bound is a definitional rearrangement, but the independent lower bound in Theorem 4.1 carries the recovery claim; no load-bearing self-citation.

specific steps
  1. self definitional [Eq. (1), Section 1; Remark 2]
    "L(γ∗) = inf_{γ∈Γ} ℓDSM(p(γ), p(γ∗))/(γ−γ∗)² ... Recall that if ℓDSM(p(γ), p(γ∗)) < δ then (γ−γ∗)² ≤ δ/L(γ∗) by the definition of L(γ∗) in (1)."

    L(γ*) is defined as the infimum of the ratio ℓDSM(p(γ),p(γ*))/(γ−γ*)², so the claimed bound on (γ−γ*)² is the defining inequality rearranged. The statement that the DSSI 'governs' mixture-weight error is therefore true by construction rather than by an independent derivation. The paper's substantive content is the separate lower bound on L(γ*) in Theorem 4.1; absent that bound the inequality is vacuous. This is a transparent definitional identity, not a hidden circularity, and it does not by itself invalidate the recovery theorem.

full rationale

The paper's central derivation chain is not circular in a damaging sense. The DSSI bound is literally the definition of L(γ*) in (1), and the paper explicitly labels it as such in Remark 2; it becomes a useful statement only because Theorem 4.1 independently lower-bounds L(γ*) for Gaussian mixtures in any dimension. Theorem 3.5 is a minimax separation proved by applying external results (Yang and Barron [29], Fano's method [30]) to the score-oracle model; its definitions, while complex, do not assume the conclusion. The ANN-to-DSM equivalence (Theorem D.3) is a standard score-matching argument. The only self-citation by an author ([53]) is used in Section 5 as supporting context for geometry learning and is not load-bearing for the main theorems. The λt = 1 versus λt = 1−αt discrepancy noted in Remark 6 is a correctness/consistency gap in the proof of Theorem 4.1, not a circularity. Overall, the recovery result rests on independent mathematical content, so the circularity score is low.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 1 invented entities

The central derivation rests on modeling diffusion sampling as a score-oracle experiment, on a stability assumption for the time-t score family, and on the empirical fact that target scores are insensitive to mixture weights. These are reasonable but not fully proved for the actual sampling procedure.

axioms (5)
  • ad hoc to paper Score-oracle equivalence (Definition 3.1 and Remark 4): diffusion sampler parameter recovery is represented by estimation from GWN observations of the score at a single intermediate t.
    This abstraction is not derived from the reverse-SDE sampling process; it is the main modeling bridge and the most fragile step.
  • domain assumption Stable score matching at time t (Assumption 3.2): (γ−γ′)² ≤ A_t² KL(q_t^γ‖q_t^γ′) for some A_t.
    Needed to define packing sets in Theorem 3.5; explicit A_t estimates for GMMs are promised but not shown.
  • domain assumption ε-diffusion sensitivity of the target score (Definitions 3.3/3.4): target-score observations are KL-insensitive at parameter separation ≥4√2 A_t ε.
    Encodes the empirical 'target score insensitivity' phenomenon; demonstrated in Fig. 2d, not proven for GMMs.
  • standard math Standard VP-SDE and Anderson reverse-time SDE formulas (Eqs. 2–4).
    Background from Song et al. [23] and Anderson [24]; accepted infrastructure.
  • domain assumption Trained score model is L2-accurate and its implied generative distribution lies in the mixture family (Remark 2).
    Needed to connect ℓNN to ℓDSM to weight error; not guaranteed in practical trained models, only approximately true.
invented entities (1)
  • Gaussian white-noise score oracle q_t^γ (Definition 3.1) no independent evidence
    purpose: Models the information available for parameter recovery from an intermediate-time score; enables minimax bounds and the claimed separation between target-score and diffusion-score estimation.
    A mathematical abstraction without a direct falsifiable handle; its equivalence to generated-sample parameter recovery is assumed, not proved.

pith-pipeline@v1.3.0-alltime-deepseek · 31805 in / 18928 out tokens · 193349 ms · 2026-08-01T23:13:22.430064+00:00 · methodology

0 comments
read the original abstract

Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights. We resolve this apparent paradox by relating the diffusion score matching (DSM) loss to the error in estimating mixture weights from generated samples. We show that, even when the target score is insensitive to mixture weights, generated samples can recover the weights accurately if the scores at intermediate noise levels are informative about the weights. Accordingly, we define the diffusion score sensitivity index (DSSI) as the variation in the DSM loss relative to changes in a parameter. We then show that the DSSI governs the accuracy with which the parameter of the target distribution can be estimated from generated samples. For Gaussian mixtures in arbitrary dimensions, we prove that the mixture weight estimation errors are on the same order as the DSM loss under mild conditions. Empirically, we show the emergence of sensitivity during the noising process of benchmark data distributions under typical noise schedules, and that these sensitivity values predict how well a well-trained model recovers mixture weights. Furthermore, we show that the choice of noise schedule can reduce diffusion sensitivity, leading to mode amplification. Although we focus on mixture weights, the proposed sensitivity framework governs the recovery of any qualitative parameter of the target distribution.

Figures

Figures reproduced from arXiv: 2607.15485 by Andrew Dennehy, Nisha Chandramoorthy, Ramchandran Muthukumar, Rebecca Willett.

Figure 1
Figure 1. Figure 1: Score sensitivity to mixture weights at different diffusion times. At [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Score sensitivity for Gaussian mixture models in dimension [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Effect of Gaussian mixture model parameters on score sensitivity in dimension [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Influence of noise schedule on mixture weight recovery in MNIST. (a) Linear noise [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between different sampling approaches on a high-separation Gaussian mixture [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Sensitivity of ℓDSM to means of mixture components. (a) ℓSM(p (δ) t , p (δ ∗) t ) as a function of t and δ. (b) ℓDSM(p (δ) ,p(δ ∗) ) (δ−δ ∗) 2 as a function of δ. We generate all samples used in [PITH_FULL_IMAGE:figures/full_fig_p036_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Sensitivity of ℓDSM to covariance matrix of mixture components. (a) ℓSM(p (κ) t , p (κ ∗) t ) as a function of t and κ. (b) ℓDSM(p (κ) ,p(κ ∗) ) (κ−κ∗) 2 as a function of κ. in dimension d = 10 where we range δ between 0 and 1. Here, we define Σκ := Σ 1/2 0  Σ −1/2 0 Σ1Σ −1/2 0 κ Σ 1/2 0 to be the geodesic interpolating between Σ0 and Σ1. Figure 7a shows ℓSM(p (κ) t , p (κ ∗) t ) as a function of κ and t… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 2 canonical work pages

  1. [1]

    Deep generative models for galaxy image simulations.Monthly Notices of the Royal Astronomical Society, 504(4):5543–5555, 2021

    Fran¸ cois Lanusse, Rachel Mandelbaum, Siamak Ravanbakhsh, Chun-Liang Li, Peter Freeman, and Barnab´ as P´ oczos. Deep generative models for galaxy image simulations.Monthly Notices of the Royal Astronomical Society, 504(4):5543–5555, 2021

    2021

  2. [2]

    Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson

    Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Probabilistic weather forecasting with machine learning.Nature, Dec 2024. doi: https://doi.org/10.1038/s41586-024-08252-9

    doi:10.1038/s41586-024-08252-9 2024

  3. [3]

    Gardel, and Vincenzo Vitelli

    Ming Han, John Devany, Michel Fruchart, Margaret L. Gardel, and Vincenzo Vitelli. Learning noisy tissue dynamics across time scales, 2025. URLhttps://arxiv.org/abs/2510.19090

    arXiv 2025

  4. [4]

    A review on generative ai models for synthetic medical text, time series, and longitudinal data.npj Digital Medicine, 8(1):281, 2025

    Mohammad Loni, Fatemeh Poursalim, Mehdi Asadi, and Arash Gharehbaghi. A review on generative ai models for synthetic medical text, time series, and longitudinal data.npj Digital Medicine, 8(1):281, 2025

    2025

  5. [5]

    The generative era of medical ai.Cell, 188(14):3648–3660, 2025

    L John Fahrner, Emma Chen, Eric Topol, and Pranav Rajpurkar. The generative era of medical ai.Cell, 188(14):3648–3660, 2025

    2025

  6. [6]

    Generative artificial intelligence in medicine.Nature medicine, 31(10): 3270–3282, 2025

    Zhen Ling Teo, Arun James Thirunavukarasu, Kabilan Elangovan, Haoran Cheng, Pras- anth Moova, Brian Soetikno, Christopher Nielsen, Andreas Pollreisz, Darren Shu Jeng Ting, Robert JT Morris, et al. Generative artificial intelligence in medicine.Nature medicine, 31(10): 3270–3282, 2025

    2025

  7. [7]

    Machine learning as a tool for hypothesis generation

    Jens Ludwig and Sendhil Mullainathan. Machine learning as a tool for hypothesis generation. The Quarterly Journal of Economics, 139(2):751–827, 2024. 12 A preprint - July 20, 2026

    2024

  8. [8]

    Posner and Shivam Saran

    Eric A. Posner and Shivam Saran. Judge ai: A case-study of large language models as judges. Journal of Law & Empirical Analysis, 0(0):2755323X261433614, 2026. doi: 10.1177/2755323X 261433614. URLhttps://doi.org/10.1177/2755323X261433614

    work page doi:10.1177/2755323x 2026

  9. [9]

    Statistical efficiency of score matching: The view from isoperimetry

    Frederic Koehler, Alexander Heckett, and Andrej Risteski. Statistical efficiency of score matching: The view from isoperimetry. InThe Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview.net/forum?id=TD7AnQjNzR6

    2023

  10. [10]

    Dimension-free log-sobolev inequalities for mixture distributions, 2021

    Hong-Bin Chen, Sinho Chewi, and Jonathan Niles-Weed. Dimension-free log-sobolev inequalities for mixture distributions, 2021. ISSN 0022-1236. URL https://www.sciencedirect.com/sc ience/article/pii/S0022123621003189

    2021

  11. [11]

    Soft mixture denoising: Beyond the expressive bottleneck of diffusion models

    Yangming Li, Boris van Breugel, and Mihaela van der Schaar. Soft mixture denoising: Beyond the expressive bottleneck of diffusion models. InThe Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=aaBnFAyW9O

    2024

  12. [12]

    Alspach and H

    D. Alspach and H. Sorenson. Nonlinear bayesian estimation using gaussian sum approximations. IEEE Transactions on Automatic Control, 17(4):439–448, 1972. doi: 10.1109/TAC.1972.11000 34

    arXiv 1972

  13. [13]

    Yakowitz and John D

    Sidney J. Yakowitz and John D. Spragins. On the identifiability of finite mixtures.The Annals of Mathematical Statistics, 39(1):209–214, 1968. doi: 10.1214/aoms/1177698520

    arXiv 1968

  14. [14]

    On the collapse errors induced by the deterministic sampler for diffusion models, 2025

    Yi Zhang, Zhenyu Liao, Jingfeng Wu, and Difan Zou. On the collapse errors induced by the deterministic sampler for diffusion models, 2025. URL https://arxiv.org/abs/2508.16154

    Pith/arXiv arXiv 2025

  15. [15]

    How i met your bias: Investigating bias amplification in diffusion models

    Nathan Roos, Ekaterina Iakovleva, Ani Gjergji, Vito Paolo Pastore, and Enzo Tartaglione. How i met your bias: Investigating bias amplification in diffusion models. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5374–5383, 2026

    2026

  16. [16]

    Deeper diffusion models amplify bias.arXiv preprint arXiv:2505.17560, 2025

    Shahin Hakemi, Naveed Akhtar, Ghulam Mubashar Hassan, and Ajmal Mian. Deeper diffusion models amplify bias.arXiv preprint arXiv:2505.17560, 2025

    arXiv 2025

  17. [17]

    Rare event sampling methods.Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(8):080402, 2019

    Freddy Bouchet, Joran Rolland, and Jeroen Wouters. Rare event sampling methods.Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(8):080402, 2019. doi: 10.1063/1.5120509

    doi:10.1063/1.5120509 2019

  18. [18]

    Henry Addison, Elizabeth Kendon, Suman Ravuri, Laurence Aitchison, and Peter A. G. Watson. Machine learning emulation of precipitation from km-scale UK regional climate simulations using a diffusion model.Journal of Advances in Modeling Earth Systems, 2026. doi: 10.1029/2025MS005140. arXiv:2407.14158

    arXiv 2026

  19. [19]

    Finkbeiner

    Nayantara Mudur, Carolina Cuesta-Lazaro, and Douglas P. Finkbeiner. Cosmological field emulation and parameter inference with diffusion models. InNeurIPS Workshop on Machine Learning and the Physical Sciences, 2023. arXiv:2312.07534

    Pith/arXiv arXiv 2023

  20. [20]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associate...

    2017

  21. [21]

    Assessing generative models via precision and recall.Advances in neural information processing systems, 31, 2018

    Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall.Advances in neural information processing systems, 31, 2018

    2018

  22. [22]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. InAdvances in Neural Information Processing Systems, volume 32, pages 11895– 11907, 2019

    2019

  23. [23]

    Score-based generative modeling through stochastic differential equations, 2021

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations, 2021. URLhttps://openreview.net/forum?id=PxTIG12RRHS

    2021

  24. [24]

    Anderson

    Brian D.O. Anderson. Reverse-time diffusion equation models.Stochastic Processes and their Applications, 12(3):313–326, 1982. ISSN 0304-4149. doi: https://doi.org/10.1016/0304-4149(82 )90051-5. URL https://www.sciencedirect.com/science/article/pii/03044149829005 15

    arXiv 1982

  25. [25]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    2022

  26. [26]

    Estimation of non-normalized statistical models by score matching.J

    Aapo Hyv¨ arinen. Estimation of non-normalized statistical models by score matching.J. Mach. Learn. Res., 6:695–709, December 2005. ISSN 1532-4435

    2005

  27. [27]

    Cambridge university press, 2000

    Sara A Geer.Empirical Processes in M-estimation, volume 6. Cambridge university press, 2000

    2000

  28. [28]

    Nonparametric estimators

    Alexandre B Tsybakov. Nonparametric estimators. InIntroduction to Nonparametric Estimation, pages 1–76. Springer, 2008

    2008

  29. [29]

    Information-theoretic determination of minimax rates of convergence.The Annals of Statistics, 27(5):1564–1599, 1999

    Yuhong Yang and Andrew Barron. Information-theoretic determination of minimax rates of convergence.The Annals of Statistics, 27(5):1564–1599, 1999

    1999

  30. [30]

    Springer New York, New York, NY, 1997

    Bin Yu.Assouad, Fano, and Le Cam, pages 423–435. Springer New York, New York, NY, 1997. ISBN 978-1-4612-1880-7. doi: 10.1007/978-1-4612-1880-7 29. URL https: //doi.org/10.1007/978-1-4612-1880-7_29

    work page doi:10.1007/978-1-4612-1880-7 1997

  31. [31]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https://openreview.net /forum?id=St1giarCHLP

    2021

  32. [32]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynk¨ a¨ anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. InAdvances in Neural Information Processing Systems, volume 32, 2019

    2019

  33. [33]

    Precision-recall curves using information divergence frontiers

    Josip Djolonga, Mario Lucic, Marco Cuturi, Olivier Bachem, Olivier Bousquet, and Sylvain Gelly. Precision-recall curves using information divergence frontiers. InProceedings of the 23rd International Conference on Artificial Intelligence and Statistics, volume 108 ofProceedings of Machine Learning Research, pages 2550–2559, 2020

    2020

  34. [34]

    Reliable fidelity and diversity metrics for generative models

    Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. InProceedings of the 37th 14 A preprint - July 20, 2026 International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pages 7176–7185, 2020

    2026

  35. [35]

    Alaa, Boris van Breugel, Evgeny S

    Ahmed M. Alaa, Boris van Breugel, Evgeny S. Saveliev, and Mihaela van der Schaar. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. InProceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, pages 290–306, 2022. arXiv:2102.08921

    Pith/arXiv arXiv 2022

  36. [36]

    Position: All current generative fidelity and diversity metrics are flawed

    Ossi R¨ ais¨ a, Boris van Breugel, and Mihaela van der Schaar. Position: All current generative fidelity and diversity metrics are flawed. InForty-second International Conference on Machine Learning Position Paper Track, 2025. URL https://openreview.net/forum?id=DMRrbb36r5

    2025

  37. [37]

    Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L

    George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URL ...

    2023

  38. [38]

    Learning mixtures of gaussians using the DDPM objective

    Kulin Shah, Sitan Chen, and Adam Klivans. Learning mixtures of gaussians using the DDPM objective. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=aig7sgdRfI

    2023

  39. [39]

    Learning mixtures of gaussians using diffusion models

    Khashayar Gatmiry, Jonathan Kelner, and Holden Lee. Learning mixtures of gaussians using diffusion models. In Nika Haghtalab and Ankur Moitra, editors,Proceedings of Thirty Eighth Conference on Learning Theory, volume 291 ofProceedings of Machine Learning Research, pages 2403–2456. PMLR, 30 Jun–04 Jul 2025. URL https://proceedings.mlr.press/v291 /gatmiry25b.html

    2025

  40. [40]

    Learning general gaussian mixtures with efficient score matching

    Sitan Chen, Vasilis Kontonis, and Kulin Shah. Learning general gaussian mixtures with efficient score matching. In Nika Haghtalab and Ankur Moitra, editors,Proceedings of Thirty Eighth Conference on Learning Theory, volume 291 ofProceedings of Machine Learning Research, pages 1029–1090. PMLR, 30 Jun–04 Jul 2025. URL https://proceedings.mlr.press/v291 /che...

    2025

  41. [41]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. InThe Eleventh International Conference on Learning Representations, 2023. URL https: //openreview.net/forum?id=zyLVMgsZ0U_

    2023

  42. [42]

    Convergence of score-based generative modeling for general data distributions

    Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. InProceedings of the 34th International Conference on Algorithmic Learning Theory, volume 201 ofProceedings of Machine Learning Research, pages 946–985. PMLR, 2023. arXiv:2209.12381

    Pith/arXiv arXiv 2023

  43. [43]

    Radford M. Neal. Annealed importance sampling.Statistics and Computing, 11(2):125–139, April 2001. ISSN 1573-1375. doi: 10.1023/A:1008923215028. URL https://doi.org/10.102 3/A:1008923215028

    doi:10.1023/a:1008923215028 2001

  44. [44]

    Dynamical regimes of diffusion models.Nature Communications, 15:9957, 2024

    Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Marc M´ ezard. Dynamical regimes of diffusion models.Nature Communications, 15:9957, 2024. doi: 10.1038/s41467-024-54281-3. 15 A preprint - July 20, 2026

    doi:10.1038/s41467-024-54281-3 2024

  45. [45]

    Spontaneous symmetry breaking in generative diffusion models

    Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion models. InAdvances in Neural Information Processing Systems, volume 36, 2023. URL https://arxiv.org/abs/2305.19693

    Pith/arXiv arXiv 2023

  46. [46]

    Fit like you sample: sample-efficient generalized score matching from fast mixing diffusions

    Yilong Qin and Andrej Risteski. Fit like you sample: sample-efficient generalized score matching from fast mixing diffusions. InThe Thirty Seventh Annual Conference on Learning Theory, pages 4413–4457. PMLR, 2024

    2024

  47. [47]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. arXiv:2010.02502

    Pith/arXiv arXiv 2021

  48. [48]

    DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. InAdvances in Neural Information Processing Systems, volume 35, 2022. arXiv:2206.00927

    Pith/arXiv arXiv 2022

  49. [49]

    Mnist handwritten digit database.ATT Labs [Online]

    Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database.ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010

    2010

  50. [50]

    Zhang, Shaoqing Ren, and Jian Sun

    Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2015. URLhttps://api.semanticscholar.org/CorpusID:206594692

    2016

  51. [51]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang, editors,Proceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pages 8162–8171. PMLR, 18–24 Jul 2021. URL https://proceedings.mlr.press/v139/nic hol21a.html

    2021

  52. [52]

    Your diffusion model secretly knows the dimension of the data manifold.arXiv preprint arXiv:2212.12611, 2022

    Jan Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Sch¨ onlieb. Your diffusion model secretly knows the dimension of the data manifold.arXiv preprint arXiv:2212.12611, 2022

    Pith/arXiv arXiv 2022

  53. [53]

    de Clercq

    Nisha Chandramoorthy and Adriaan A. de Clercq. When and how can inexact generative models still sample from the data manifold? InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL https://openreview.net/forum?id=QBjuYL4g AX

    2026

  54. [54]

    Beyond log-concavity: Provable guarantees for sampling multi-modal distributions using simulated tempering langevin monte carlo

    Holden Lee, Andrej Risteski, and Rong Ge. Beyond log-concavity: Provable guarantees for sampling multi-modal distributions using simulated tempering langevin monte carlo. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018...

    2018

  55. [55]

    Mcmc for multi- modal distributions.arXiv preprint arXiv:2501.05908, 2025

    Krzysztof Latuszy´ nski, Matthew T Moores, and Timoth´ ee Stumpf-F´ etizon. Mcmc for multi- modal distributions.arXiv preprint arXiv:2501.05908, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    Bishop.Pattern Recognition and Machine Learning (Information Science and Statistics)

    Christopher M. Bishop.Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg, 2006. ISBN 0387310738

    2006

  57. [57]

    A connection between score matching and denoising autoencoders.Neural Computation, 23(7):1661–1674, 2011

    Pascal Vincent. A connection between score matching and denoising autoencoders.Neural Computation, 23(7):1661–1674, 2011. doi: 10.1162/NECO a 00142. 16 A preprint - July 20, 2026

    doi:10.1162/neco 2011

  58. [58]

    Improved noise schedule for diffusion training, 2024

    Tiankai Hang, Shuyang Gu, Xin Geng, and Baining Guo. Improved noise schedule for diffusion training, 2024. URLhttps://arxiv.org/abs/2407.03297

    Pith/arXiv arXiv 2024

  59. [59]

    Adabelief optimizer: Adapting stepsizes by the belief in observed gradients.Advances in Neural Information Processing Systems, 33, 2020

    Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar C Tatikonda, Nicha Dvornek, Xenophon Papademetris, and James Duncan. Adabelief optimizer: Adapting stepsizes by the belief in observed gradients.Advances in Neural Information Processing Systems, 33, 2020

    2020

  60. [60]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Yash Katariya, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URLhttp://github.com/jax-ml/jax

    2018

  61. [61]

    Equinox: neural networks in JAX via callable PyTrees and filtered transformations.Differentiable Programming workshop at Neural Information Processing Systems 2021, 2021

    Patrick Kidger and Cristian Garcia. Equinox: neural networks in JAX via callable PyTrees and filtered transformations.Differentiable Programming workshop at Neural Information Processing Systems 2021, 2021

    2021

  62. [62]

    Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, St´ efan J

    Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, St´ efan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, ˙Ilhan Polat, Yu Feng, Eric W....

    2020

  63. [63]

    J. D. Hunter. Matplotlib: A 2d graphics environment.Computing in Science & Engineering, 9 (3):90–95, 2007. doi: 10.1109/MCSE.2007.55

    doi:10.1109/mcse.2007.55 2007

  64. [64]

    score data

    Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https://www.wandb. com/. Software available from wandb.com. 17 A preprint - July 20, 2026 A Notation Symbol Definition In words P(p 0, p1) {p(γ) :p (γ) =γ p1 + (1−γ)p 0, γ∈(0,1)} Family of bimodal mixtures of p0 and p1 pt dxt = − βt 2 xtdt + √βt dWt and x0 ∼p =⇒ xt ∼p t Distribution of ...

    2020