On Forgetting and Stability of Score-based Generative models

Antonio Ocello; Gabriel Cardoso; Stanislas Strasman; Sylvain Le Corff; Vincent Lemaire

REVIEW 2 major objections 5 minor 2 references

On Forgetting and Stability of Score-based Generative models

T0 review · 2 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Score-based generative samplers are stable because their reverse-time diffusion forgets its past: under a dissipativity condition on the data score, each backward step contracts a weighted total-variation metric by a factor strictly below 1

desk verdict A genuinely new Harris-theoretic error decomposition for SGMs under dissipativity, but the uniform contraction factor that makes geometric discounting quantitative is asserted rather than proved, and the explicit constants suggest it may die on refined grids. read the letter →

arxiv 2601.21868 v2 pith:GNMT4XLH submitted 2026-01-29 stat.ML cs.LG

Stanislas Strasman , Gabriel Cardoso , Sylvain Le Corff , Vincent Lemaire , Antonio Ocello This is my paper · ORCID

classification stat.MLcs.LG MSC 60J0560J6065C30

keywords score-basedgenerativemodelsdiffusionMarkovchainstabilityHarrisergodictheoremweightedtotalvariationforgettingsamplingerrorboundsdissipativity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that score-based generative models (SGMs) are intrinsically stable because the reverse-time diffusion that turns noise into data is a contracting Markov process. Under a dissipativity condition on the data score and a polynomial growth bound on its Jacobian, each transition of the backward chain shrinks a weighted total-variation metric by a factor strictly below 1. As a consequence, the final sampling error is bounded by a telescoping sum in which initialization, discretization, and score-approximation errors are all geometrically discounted along the trajectory. This puts a formal backbone under the practical heuristic that accuracy near the end of the reverse path matters most, and it covers nonconvex, multimodal targets and targets with super-Gaussian tails, without requiring global Lipschitz or one-sided Lipschitz score assumptions.

What carries the argument

The engine is the classical Harris ergodic theorem for Markov chains, applied locally in time to the time-inhomogeneous backward diffusion. Its two ingredients are (i) a Lyapunov drift inequality for the quadratic function V2(x)=||x||^2, which follows because the dissipativity of the data score propagates along the forward diffusion and forces the reverse drift to pull states back toward the origin; and (ii) a localized Doeblin minorization on a centered ball, where the backward kernel dominates a state-independent measure with probability decaying like exp(-r^2/sigma^2). Together these give a one-step contraction of the weighted total-variation metric rho_b(mu1,mu2)=integral(1+b||x||^2)|mu1

What would settle it

Run the paper's initialization-bias experiment on a dissipative target with gamma0 just above alpha/2 and a very long horizon T, measuring the output error as a function of the time t_bias where a fixed perturbation is injected. Theorem 4.3 predicts the effect decays exponentially in the number of remaining steps (like alpha-bar^(N-k)); if the measured decay is polynomial rather than exponential, the uniform contraction factor used in the proof does not exist.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.3: if the initial data score satisfies Assumption 3.1 and the discretized generative chain has finite moments, then the output distribution satisfies a weighted total-variation bound of the form rho_b(pi_data, pi_hat_theta_N) <= alpha-bar^N * Lambda(T) * C_mix + sum_{k=1}^N alpha-bar^(N-k) (Delta_k C_k^disc + sqrt(Delta_k) C_k^net ||E_{k-1}||_{L2}). The three error sources—wrong initialization, discretization, and score mismatch—enter linearly, but each is discounted by the contraction factor alpha-bar raised to the number of remaining steps, so early errors vanish exponentially as the horizon grows. The proof obtains this by establishing a Lyapunov drift inequ

Load-bearing premise

The proof assumes that every short segment of the reverse-time diffusion contracts with the same uniform contraction factor, even though the classical Harris theorem it invokes is stated for time-homogeneous chains; if that uniform contraction does not exist, the geometric discounting in the main bound collapses.

Editorial extensions

If this is right

Initialization error from replacing the intractable noised-data law p_T by the Gaussian reference decays like alpha-bar^N, so a long forward horizon exponentially suppresses the choice of the starting distribution.
Discretization and score errors incurred at step k have influence alpha-bar^(N-k), so a local error very early in the reverse trajectory contributes almost nothing to the final sample when N is large.
For a uniform grid, the accumulated error is controlled by Delta/(1-alpha-bar) times a worst-case local discretization constant plus sqrt(Delta)/(1-alpha-bar) times the worst-case L2 score error, so the bound remains finite even as the number of steps grows, provided alpha-bar stays bounded away from 1.
In the Gaussian case the contraction factor is explicit: the conditional mean of the backward kernel is a strict contraction in Euclidean norm in the variance-exploding setting, and in the variance-preserving setting under lambda_max(Sigma) < alpha^(-1), matching the dissipativity threshold gamma0 > alpha/2.
Because rho_b controls total variation and 2-Wasserstein distance up to multiplicative constants, the same geometrically discounted decomposition transfers to those metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The geometric discounting suggests a principled rule for designing discretization grids and allocating score-network capacity: choose step sizes and score accuracy so that Delta_k C_k^disc and sqrt(Delta_k) C_k^net scale like alpha-bar^(-(N-k)), equalizing the discounted contributions across steps. This is a testable prescription the paper does not formulate.
A practical prediction, consistent with the experiments, is that fine-tuning only the late-stage (low-noise) part of a trained score network should recover most of the performance lost to a badly trained early part; one could benchmark this by deliberately corrupting early score weights and measuring output degradation.
The contraction mechanism also invites an extension to deterministic samplers such as the probability-flow ODE: the same weighted-total-variation analysis might be replaced by a Lyapunov argument on the ODE flow, and the analogous question is whether the geometric discount survives without stochastic noise.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this paper gives a structural stability theorem for score-based generative models using Harris theory. Under a dissipativity condition on the data score plus polynomial growth of its Jacobian, the reverse-time semigroup contracts in a weighted TV metric, and the sampling error becomes a telescoping sum where early errors are discounted by \bar{\alpha}^{N-k}. That is a real departure from the log-concavity or bounded-support assumptions in the prior literature. The propagation of dissipativity along the forward flow (Prop D.1) and the one-step Girsanov estimates (Prop H.3) are serious work, and the Gaussian explicit contraction is a nice sanity check. If the main theorem holds, it gives a principled explanation for why fine steps near the data matter more than early ones.

The soft spot is exactly where the reader and the stress-test point. The jump from interval-wise contraction (Prop 3.4) to a single uniform pair (b, \bar{\alpha}) for every interval is one sentence: "Since these constants exist ... there exist ..." No proof is given, and Theorem 4.3 needs that uniformity because it iterates the same metric and the same contraction factor N times. The explicit VE constants in Appendix F make the problem concrete: the minorization constant \epsilon^{(r)} decays like exp(-r^2/\sigma^2_{T-s|T-t}) with \sigma^2 ~ \beta \Delta, so as the grid refines, \epsilon \to 0 and the one-step contraction factor \bar{\alpha} \to 1. If \bar{\alpha} = 1 - c exp(-C/\Delta), then \bar{\alpha}^N \to 1 rather than decaying geometrically. In other words, the advertised forgetting mechanism is not quantitatively established on refined grids. This is not a contradiction of the theorem, but it means the central quantitative claim rests on an unverified uniform-constants assumption.

The paper also has smaller issues: the proof of Proposition 3.4 hand-waves the adaptation of Hairer-Mattingly to time-inhomogeneous kernels, the appendix contains an unresolved placeholder ("the scheduler from ?"), and the replication counts are inconsistent (20 vs 5). No code is released. None of these kill the framework, but they reinforce the need for a careful revision.

Verdict: this deserves a serious referee. The framework is original and the components are credible, but the central uniform-contraction claim must either be proved under explicit conditions or the theorem should be restated with step-dependent metrics and a clear statement of which quantities actually decay. I would not cite the quantitative bound as it stands, but I would bring the paper to a reading group to probe the Harris-theory gap.

Referee Report

2 major / 5 minor

Summary. This paper claims quantitative stability bounds for score-based generative models by analyzing the reverse-time diffusion as a time-inhomogeneous Markov semigroup. Under a dissipativity condition on the data score and polynomial growth of its Jacobian, the authors establish a Lyapunov drift inequality and a localized Doeblin minorization condition. Combining these through Harris-type contraction in a weighted total variation distance, Theorem 4.3 asserts that the final sampling error is controlled by a telescoping sum in which initialization, discretization, and score-approximation errors are discounted geometrically by powers of a uniform contraction factor. The paper also provides explicit constants for VE and VP schedules, a closed-form Gaussian analysis, and numerical experiments illustrating qualitative forgetting behavior.

Significance. If Theorem 4.3 is valid, the result is significant: it offers a structural explanation of why early errors in the reverse sampling trajectory are forgotten, and it places SGM stability in the classical Harris-theoretic framework with weaker assumptions than global log-concavity or global Lipschitz score regularity. The paper contains substantial technical work: explicit propagation of dissipativity along the forward flow, a localized minorization argument, a Girsanov-based one-step error estimate, and a telescoping decomposition that cleanly separates initialization, discretization, and score errors. These ingredients are valuable and largely coherent. The main caveat is that the central contraction argument requires a uniform pair (b, alpha) across all discretization intervals, and this uniformity is asserted rather than proved. Because the advertised geometric discounting depends on that uniformity, the main theorem is not yet established as stated.

major comments (2)

[Section 4, paragraph before Theorem 4.3; Proposition 3.4] The sentence "Since these constants exist for all 0≤k≤N−1, there exist α*∈(0,1) and b*>0 such that Proposition 3.4 holds uniformly" is a non sequitur. Proposition 3.4 supplies a pair (b^r_{s,t}, α_{t|s}) for each interval, with both parameters depending on s,t,r and on λ, K, ε from Propositions 3.2–3.3. Existence for each interval does not imply that a single weighted metric ρ_b and a single contraction factor α work simultaneously for all intervals. The proof of Theorem 4.3 requires exactly this uniformity: Step 1 iterates the same ρ_b and α over all N transitions to obtain α^N, and Step 2 uses the same α for the tail factors α^{N−k}. Without a proof of uniformity, the telescoping bound is not established. Section 6's remark that a uniform pair "can lead to pessimistic global bounds" addresses pessimism, not existence.
[Appendix F and Proposition 3.3] The explicit constants suggest that the uniformity assumption may fail quantitatively as the grid is refined. Proposition 3.3 gives ε^{(r)}_{t|s} proportional to exp(−r²/σ²_{T−s|T−t}), and Proposition 3.4 defines α as roughly 1 − (ε^{(r)}_{t|s} − α0) (up to the metric-weight term). On a refined grid, σ²_{T−s|T−t} ≈ β Δ, while the required radius satisfies r_c² = 2K/(1−λ) = O(1) from the VE formula (43). Hence ε→0 and α→1 as Δ→0, so α^N need not decay. This does not disprove the theorem, but it shows that the advertised geometric discounting rests on an unverified uniformity input. The authors should either prove the uniform pair under explicit sufficient conditions or reformulate the theorem with interval-dependent metrics and quantify the resulting bound.

minor comments (5)

[Section 5] The text refers to "the scheduler from?, Equation 5" without a citation. Please replace the placeholder with the actual reference.
[Section 3.1 / Notation] There are several typos: "Froebenius" should be "Frobenius", "inequlity" should be "inequality", and "Gaussien" in Appendix I should be "Gaussian".
[Appendix A.1, Lemma A.1 proof] In the displayed integral, the term e^{−λ||x||²} appears after reducing to a Gaussian integral; the sign seems inconsistent with the previous line, which should be e^{λ||x||²}. Please check and correct.
[Figure 6 caption] The caption says "Algorithm 1" but the experiment is described by Algorithm 2. This is a minor cross-reference error.
[Appendix H.2, after Proposition H.3] The phrase "absorbing 1/4 in D^{(0)} and D^{(1)}" is imprecise: the constants in the displayed bound of Proposition H.3 do not exactly match the earlier definitions D^{(0)}, D^{(1)}. Please make the absorption explicit or adjust the constants.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the contraction argument is derived from independently stated assumptions and an external Harris theorem; the only self-citation is supporting, not definitional.

full rationale

The central derivation is not circular. Assumption 3.1 states dissipativity and polynomial-growth conditions on the data score independently of the target theorem, and the drift and minorization conditions (Propositions 3.2 and 3.3) are proved from these assumptions in Appendices D and E. Proposition 3.4 then invokes the external Hairer & Mattingly (2008) theorem, and Theorem 4.3 applies the resulting contraction as a telescoping, geometrically weighted bound on local errors. The final bound contains the local score error E_k as a term rather than redefining it as the prediction, so no fitted input is renamed as a prediction. The numerical experiments are illustrative, not fitted to the claimed bound. The main genuine weakness is the asserted uniformity paragraph before Theorem 4.3: 'Since these constants exist for all 0≤k≤N−1, there exist ¯α∗∈(0,1) and b∗>0 such that Proposition 3.4 holds uniformly.' No proof is given, and Section 6 concedes that a single uniform pair 'can lead to pessimistic global bounds.' That is a correctness/proof gap, not circularity: the theorem does not assume the uniform contraction as an input, it asserts it without proof. The only proof-relevant self-citation is in the VP part of Lemma H.1: 'It follows from Lemma B.1 of Strasman et al. (2025a), after a suitable time change and an adjustment of the invariant measure, that (47) holds.' This supplies the VP initialization term but is a supporting KL-decay estimate from a separate published TMLR paper, not the mechanism of geometric forgetting itself; it is not load-bearing for the paper's main contraction argument. Overall, the derivation chain does not reduce to its inputs by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper does not fit parameters to data, but the final theorem depends on assumed structural constants (gamma0, kappa0, C0, p) and on existential constants (b, alpha_bar) that are asserted rather than explicitly computed.

free parameters (2)

weighted-TV metric weight b
The theorem asserts existence of b>0 but provides no explicit value or selection rule; all constants and the contraction norm depend on b.
uniform contraction factor alpha_bar
Defined only through an existence argument after Appendix F; no closed-form expression is given, yet the geometric discounting in Theorem 4.3 depends on alpha_bar being uniformly less than 1.

assumptions (5)

domain assumption Data density p0 is C^2 and satisfies Assumption 3.1(I)(II): dissipativity of the score outside a compact set and polynomial growth of its Jacobian, with gamma0 > alpha/2.
This is the structural assumption on the data distribution that powers the propagation of dissipativity along the forward flow and the Lyapunov drift for the reverse semigroup.
domain assumption Assumption 4.1: finite 4p+4 polynomial moments of the discretized chain (or polynomial growth of the learned score).
Assumed so that the one-step error constants and the moment terms in Theorem 4.3 are finite; Remark 4.2 argues it is mild but it is still an extra condition.
ad hoc to paper Hairer & Mattingly Theorem 1.3 applies to time-inhomogeneous transition kernels interval-by-interval with a uniform pair (b, alpha_bar).
Proposition 3.4 asserts this adaptation but does not prove it in detail; this is the key premise behind the geometric discounting of errors.
standard math Girsanov-type path-measure equivalence and finite KL divergence between the exact backward diffusion and the continuous-time interpolation of the Euler discretization.
Used in Lemma H.5 and Corollary H.6 to control the one-step discretization and score-approximation error.
standard math Tweedie-type identities and integration by parts for Gaussian convolutions, valid under the sub-Gaussian tails implied by Assumption 3.1(I).
Used in Section C/D to propagate dissipativity and polynomial-growth bounds from the data score to the time-marginal scores S_t.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On Forgetting and Stability of Score-based Generative models." pith.science (2026). https://pith.science/paper/GNMT4XLH

@misc{pith2026260121868,
  author       = {Pith},
  title        = {Pith review of: On Forgetting and Stability of Score-based Generative models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GNMT4XLH}},
  note         = {Machine review of arXiv:2601.21868}
}

read the original abstract

Understanding the stability and long-time behavior of generative models is a fundamental problem in modern machine learning. This paper provides quantitative bounds on the sampling error of score-based generative models by leveraging stability and forgetting properties of the Markov chain associated with the reverse-time dynamics. Under weak assumptions, we provide the two structural properties to ensure the propagation of initialization and discretization errors of the backward process: a Lyapunov drift condition and a Doeblin-type minorization condition. A practical consequence is quantitative stability of the sampling procedure, as the reverse diffusion dynamics induces a contraction mechanism along the sampling trajectory. Our results clarify the role of stochastic dynamics in score-based models and provide a principled framework for analyzing propagation of errors in such approaches.

Figures

Figures reproduced from arXiv: 2601.21868 by the authors.

**Figure 3.** Max SW as a function of the noise level and λ in the case of initialization perturbation. We use the forward-time convention (t = 0 is the data distribution). Local score approximation error. A key qualitative prediction of our stability theory is that local errors are discounted when they occur early along the reverse trajectory. 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Forward noise level t 0 5 10 15 20 25 M… view at source ↗

**Figure 4.** Max SW as a function of the noise level and perturbation magnitude λ for the score-perturbation experiment. We use the forward-time convention (t = 0 is the data distribution). 7 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 1.** Initialization bias in the GMM case for several noise levels σt and with λ = 20. Red points are samples from πdata and blue points the output of the biased initialization experiment [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

Figures from the paper (3 more)

**Figure 2.** Figure 2: Local perturbation of the score in the GMM case for several noise levels σt and with λ = 50. Red points are samples from πdata and blue points the output of the perturbed score experiment. To probe this local-to-global propagation mechanism, we run the reverse-time sam…

**Figure 5.** Figure 5: Sensitivity to initialization for isotropic, heteroscedastic, and correlated Gaussian targets. Curves report the mean over 5 independent replications of Algorithm 1 (T = 1, M = 30,000, h = 2.5 × 10−3 ); shaded regions indicate ±1 standard deviation across replications.…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p046_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 1 linked inside Pith

[3]

Gaussian Mixture Model.We consider a Gaussian mixture model inR 50 defined by pdata (x) = 25X i=1 ωiN(µ i,Σ i), where •{ω i}25 i=1 are sampled i.i.d

Correlated.We use a dense covariance matrix with unit marginal variances and slowly decaying off-diagonal correlations: Σ(corr) jj := 1,Σ (corr) jj ′ := 1p |j−j ′|+ 1 forj̸=j ′, for1≤j, j ′ ≤d. Gaussian Mixture Model.We consider a Gaussian mixture model inR 50 defined by pdata (x) = 25X i=1 ωiN(µ i,Σ i), where •{ω i}25 i=1 are sampled i.i.d. from aξ 2 dis...

2019
[2015]

1 σ4 t|0 (x− − →X0)(x− − →X0) T − 1 σ2 t|0 Id ! Xt =x # . Finally, sinceE[x−X 0|Xt =x] =x−E[X 0|Xt =x], a direct expansion yields E

URL https://eloecherbach.u-cergy. fr/cours.pdf. Meyn, S. P. and Tweedie, R. L. Stability of markovian processes iii: Foster–lyapunov criteria for continuous- time processes.Advances in Applied Probability, 25(3): 518–548, 1993. Meyn, S. P. and Tweedie, R. L.Markov Chains and Stochas- tic Stability. Cambridge University Press, 2 edition, 2009. ISBN 978-0-5...

arXiv 1993

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [3]

Gaussian Mixture Model.We consider a Gaussian mixture model inR 50 defined by pdata (x) = 25X i=1 ωiN(µ i,Σ i), where •{ω i}25 i=1 are sampled i.i.d

Correlated.We use a dense covariance matrix with unit marginal variances and slowly decaying off-diagonal correlations: Σ(corr) jj := 1,Σ (corr) jj ′ := 1p |j−j ′|+ 1 forj̸=j ′, for1≤j, j ′ ≤d. Gaussian Mixture Model.We consider a Gaussian mixture model inR 50 defined by pdata (x) = 25X i=1 ωiN(µ i,Σ i), where •{ω i}25 i=1 are sampled i.i.d. from aξ 2 dis...

2019

[2] [2015]

1 σ4 t|0 (x− − →X0)(x− − →X0) T − 1 σ2 t|0 Id ! Xt =x # . Finally, sinceE[x−X 0|Xt =x] =x−E[X 0|Xt =x], a direct expansion yields E

URL https://eloecherbach.u-cergy. fr/cours.pdf. Meyn, S. P. and Tweedie, R. L. Stability of markovian processes iii: Foster–lyapunov criteria for continuous- time processes.Advances in Applied Probability, 25(3): 518–548, 1993. Meyn, S. P. and Tweedie, R. L.Markov Chains and Stochas- tic Stability. Cambridge University Press, 2 edition, 2009. ISBN 978-0-5...

arXiv 1993