Rao-Blackwellized Score Matching on Manifolds

Divit Rawal

REVIEW 3 major objections 3 minor 1 cited by

Rao-Blackwellized Score Matching on Manifolds

T0 review · 3 major / 3 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Ambient Gaussian denoising score matching on manifold-supported data, after conditioning on the nearest-point projection, estimates the true Riemannian score to first order in noise, with an explicit σ² curvature bias that vanishes on S².

desk verdict Solid canonicality and variance-collapse core, but the headline sigma-squared formula is false as stated: it is missing a third-derivative extrinsic term. read the letter →

arxiv 2605.25567 v3 pith:K5NPMOM3 submitted 2026-05-25 stat.ML cs.LG

Divit Rawal This is my paper · ORCID

classification stat.MLcs.LG MSC 62R3053B25

keywords denoisingscorematchingmanifoldhypothesisRao-BlackwellizationRiemanniannearest-pointprojectioncurvaturebiasWeingartenoperatorsphereS^2

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Denoising score matching is well-posed on manifold-supported data only at positive noise, and the object it learns has been unclear. This paper identifies the canonical object: the conditional expectation of the tangent denoising target given the nearest-point projection π(X), the unique L²-optimal predictor among all estimators that see the data only through π(X). Conditioning removes a singular normal-fiber noise channel whose variance diverges as d/σ², replacing it with a target of bounded variance. The paper then computes the small-noise expansion of this target: it equals the intrinsic Riemannian score up to first order, with an explicit second-order bias combining an intrinsic smoothing term and an extrinsic curvature operator built from the Weingarten and Ricci tensors. On the 2-sphere the extrinsic term cancels exactly, giving a quantitative explanation for why ambient denoising score matching has worked well on spherical scientific data despite the manifold singularity.

What carries the argument

The load-bearing object is rσ(z) = E[ P_{T_z M}(Z−X)/σ² | π(X)=z ], the conditional expectation of the tangent denoising target given the nearest-point projection π(X) onto M. This Rao-Blackwellized target is the unique L²-optimal predictor among estimators depending on X through π(X), and it removes the raw target's irreducible d/σ² variance floor. The expansion is derived by changing to tubular coordinates around M and normal coordinates at z, obtaining a fiber posterior that is a centered Gaussian in tangent displacements times smooth geometric corrections, and applying a manifold Stein identity. The curvature operator ½ W_H − Ric^# — mean-curvature Weingarten minus Ricci endomorphism — c

What would settle it

Compute rσ by high-accuracy quadrature on a compact embedded manifold with known intrinsic score, such as the flat torus T² embedded in R³ with a wrapped-Gaussian density, at several small σ, and form the ratio (rσ − ∇_M log q − σ² b_q)/(σ² ∇_M log q). Theorem 5.2 predicts the constant ½ W_H − Ric^# (on T², +½ for the chosen embedding); any measurable σ-dependence at leading order, or any value inconsistent with the Weingarten and Ricci computation, would refute the expansion.

Watch

Extended reading notes

Core claim

The paper's central discovery is Theorem 5.2: for a compact C^5 embedded manifold of positive reach, with strictly positive latent density q and isotropic Gaussian corruption, the Rao-Blackwellized tangent target satisfies rσ(z) = ∇_M log q(z) + σ²( b_q(z) + g_ext_M(z) ) + o(σ²), where b_q is the intrinsic Tweedie correction ½∇_M(Δ_M log q + ‖∇_M log q‖²) and g_ext_M = (½ W_H − Ric^#_z) ∇_M log q, with W_H the Weingarten operator in the mean-curvature direction and Ric^# the Ricci endomorphism. The first term recovers the true intrinsic Riemannian score; the second-order correction is the finite-σ bias introduced specifically by ambient Gaussian corruption on curved supports. On spheres the

Load-bearing premise

The load-bearing premise is that the latent distribution sits exactly on a compact C^5 manifold with positive reach and is corrupted by isotropic Gaussian noise; if the data only approximate the manifold or the noise is anisotropic, the d/σ² variance floor and the curvature coefficient ½ W_H − Ric^# need not hold.

Editorial extensions

If this is right

Provides a population-level answer to what ambient denoising score matching learns on manifold-supported data: the intrinsic Riemannian score, up to an explicit order-σ² curvature bias.
Rao-Blackwellization is not a constant-factor improvement: it removes the singular d/σ² variance channel, shown to be an irreducible Bayes-risk floor for any estimator using only a fiber-collapsing summary.
The ambient-versus-intrinsic bias is computable from the Weingarten and Ricci tensors and the learned score, so practitioners can accept, subtract, or avoid it depending on target accuracy.
On S² the extrinsic bias is exactly zero, so ambient denoising score matching there matches intrinsic methods up to the intrinsic smoothing bias; on S¹, S³, and higher spheres the bias is nonzero and grows with dimension.
Flat cases reduce exactly to standard d-dimensional Gaussian denoising score matching, pinning down the baseline against which curvature effects must be measured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the expansion holds at moderate σ, the same formula suggests a practical debiasing step for ambient score models on product manifolds and rotation groups such as SO(3), where the dimension-dependent coefficient 1 − d/2 would predict systematic shrinkage or amplification of the learned score.
The S² cancellation is a coincidence of the Einstein identity at dimension two, not a sign that ambient denoising score matching is generally unbiased; using only S² to validate manifold-aware score matching may understate the bias present on other manifolds.
A natural testable extension is anisotropic Gaussian corruption: replacing the isotropic covariance in the noise model should change both the variance floor to a weighted trace of the covariance and the curvature operator, and the same quadrature approach could verify the modified formula.
The finite-sample bandwidth rate in the appendix, roughly (σ⁻²N)^(-2/(d+2)), suggests Rao-Blackwellized preprocessing translates directly into improved sample complexity; direct empirical comparison on non-spherical product manifolds would quantify the gain.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Divit has a genuinely good idea here: the Rao-Blackwellized target is the right object for ambient DSM on manifold data, the d/sigma^2 variance floor and collapse theorem are clean, and the leading-order identification r_sigma = grad_M log q + O(sigma^2) is solid. The flat-case reduction is also exact and useful. But the paper's main theorem, the explicit sigma^2 coefficient, is not correct as stated. In the graph-coordinate proof (Appendix F.3, Proposition F.6), log M_sigma(s) is expanded as const - 1/2 + O(||s||^3), and the paper then asserts grad laplacian m_sigma(0) = 0. That is false for typical embeddings: the O(||s||^3) remainder has a cubic part, and grad laplacian of that cubic part is generally a nonzero constant that feeds directly into the sigma^2 term via the Gaussian-moment expansion.

A concrete check: take M as the graph of y = x^2 + x^3 in R^2, z = 0, uniform q. Then J_gr(s) = 1 + 2s^2 + 6s^3 + ..., F_sigma(s) = 1 - 2s^2 - 2s^3, so M_sigma(s) = 1 + 4s^3 + ..., and the exact conditional expectation gives r_sigma(0) = 12 sigma^2 + O(sigma^4). Theorem 5.2 predicts 0 because the intrinsic score is zero. So Eq. (16) is missing a third-derivative extrinsic term. The S^d and T^2 cases are symmetric enough that the cubic term vanishes, which is why the numerical checks look good, but the general formula does not hold.

The rest of the paper should not be judged by this mistake. The variance-collapse theorem and the Rao-Blackwell canonicality calculation are independent and appear correct; the finite-sample rate is only sketched, and there is no code or error bars, but those are minor compared with the missing term. I would send this to peer review—the core ideas deserve referee time, and the error is fixable by adding the missing term and rechecking. But the current version is not acceptable. A reader who wants the S^2/T^2 coefficients should be directed to the symmetric cases, not to the general theorem.

Referee Report

3 major / 3 minor

Summary. This paper studies ambient Gaussian denoising score matching (DSM) when the latent distribution is supported on a compact smooth submanifold M ⊂ R^D. It defines the Rao-Blackwellized tangent target rσ(z) = E[Tσ | π(X) = z], proves that among estimators depending on the observation only through the nearest-point projection π(X), rσ is the unique L2-optimal predictor, and shows that the raw tangent target Tσ has conditional variance d/σ^2 + O(1) while rσ has bounded variance. The paper proves the leading-order identification rσ(z) = ∇_M log q(z) + O(σ^2) and claims an explicit second-order expansion with an intrinsic Tweedie term and an extrinsic curvature term g_ext = (1/2 W_H − Ric^#) ∇_M log q. It specializes to spheres, observes that the extrinsic term vanishes on S^2, and provides a finite-sample local-averaging rate.

Significance. If the second-order expansion were correct, the paper would give a sharp and useful account of what ambient DSM learns on manifold-supported data, with a computable bias and a theoretical explanation for the benign behavior on S^2. The variance-collapse theorem (Thm 4.2) and the leading-order identification (Eq. 10) are solid and of clear value: the d/σ^2 risk floor is an exact, parameter-free statement, and the flat-case reduction (Prop 5.1) is clean. The paper also makes falsifiable coefficient predictions (Fig. 2), which is a strength. However, the central second-order formula (Thm 5.2) is incomplete: a cubic term in the geometric weight contributes to the σ^2 coefficient through ∇Δ m(0), as a simple graph example shows. Thus the paper's main quantitative claim is not reliable as stated, although the leading-order and variance results appear unaffected.

major comments (3)

[Appendix F.3, Prop F.6 and Eq. (64)] The proof asserts ∇Δmσ(0)=0 on the grounds that the leading s-dependent term in log Mσ is quadratic. This is false: the O(||s||^3) remainder generally contains a cubic term C(s), and ∇ΔC(0) is a nonzero constant that contributes to rσ(z) at order σ^2 through Lemma F.7. Concretely, take a compact extension of the graph y = x^2 + x^3 at z=0 with uniform q. Then Jgr(s)=1+2s^2+6s^3+... and Fσ(s)=1−2s^2−2s^3 (since W=2), so log Mσ(s)=4s^3+...; hence ∇Δm(0)=24 and the exact posterior mean gives rσ(0)=12σ^2+O(σ^4), whereas Eq. (16) predicts 0. Thus Theorem 5.2 is missing a third-derivative term.
[Theorem 5.2 / Corollary 5.4] The stated second-order expansion is not correct in general; the missing term involves third derivatives of the embedding and is not captured by W_H or Ric^#. The numerical checks in Figure 2 are on S^d and T^2, which are even/odd symmetric at the evaluation point so the cubic term averages to zero; they therefore do not detect the error. The manuscript should give a corrected expansion with a rigorous remainder and test it on a non-symmetric manifold (e.g., an ellipsoid or the graph example).
[Section F.3, Lemmas F.4-F.6] The proof strategy expands Jgr and Fσ only to quadratic order and then, in Prop F.6, concludes that the remainder 'contains no term linear in s at order σ^2.' That condition does not imply ∇Δmσ(0)=0. A valid second-order derivation must expand the geometric weight to cubic order and include the σ^2/2 ∇(Δm)(0) term in the Gaussian-moment expansion (Lemma F.7). The algebra needs to be redone; the stated operator (1/2 W_H − Ric^#) is at most part of the correct coefficient.

minor comments (3)

[Appendix F.3] There are several reference inconsistencies: the proof of Prop F.6 refers to 'Theorem F.4 and Theorem F.5' where these are Lemmas F.4 and F.5, and in the application of Lemma F.7 it is called 'Theorem F.7'. These should be corrected.
[Section 4.3 and Figure 1] The text states the variance is d/σ^2 + O(1), but Figure 1 displays the second moment E||Tσ||^2. Since E||Tσ||^2 = Var(Tσ) + ||E[Tσ]||^2 and the mean is O(1), the plot is consistent; the caption should clarify which quantity is shown.
[Remark F.3 / Eq. (59)] The claim for the product of two circles S^1(R1)×S^1(R2) in R^4 that the operator is ½ diag(1/R1^2, 1/R2^2) is stated without derivation. A short verification would improve readability and trust in the frame formula.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main expansion is a self-contained asymptotic derivation with no fitted parameters, self-citation load-bearing steps, or definitional reductions.

full rationale

The paper's central claim, Theorem 5.2, is a population-level asymptotic expansion of rσ(z) = E[PT(π(X))(Z−X)/σ² | π(X)=z] under the explicit corruption model (1). The derivation proceeds through a fiber posterior normal form (Proposition B.2), a Stein identity (Lemma B.3), and a graph-coordinate expansion of the geometric weight (Section F). The target rσ is defined from the true joint law, and the small-σ limit is computed, not assumed. No parameter is fitted to data and then renamed as a prediction; the coefficients bq and g_ext(M) are explicit functions of q and the manifold geometry. The numerical experiments in Figures 2 and 3 are independent verification of the formula, not inputs to it. There are no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation. The flat-case reduction to ordinary Gaussian DSM is a proved equivalence and a sanity check, not a renaming that carries the curved-case claim. The skeptical objection about the alleged vanishing of ∇∆mσ(0) in Proposition F.6 is a potential mathematical correctness concern, not a circularity: it does not exhibit any equation that reduces to its own inputs by construction. Therefore, under the requested circularity standard, the paper is not circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted; the expansion is parameter-free. The axioms are the standard geometric and domain assumptions listed above.

assumptions (4)

domain assumption M⊂R^D is a compact C^5 embedded submanifold with positive reach and q∈C^5(M) strictly positive.
Needed for Taylor expansions and tubular neighborhood calculus used throughout; stated in Section 3.
domain assumption Gaussian corruption X=Z+σξ with isotropic ξ∼N(0,I_D).
Defines the ambient DSM problem; all variance and expansion computations use isotropy (Section 3, Eq. (1)).
standard math Federer-Gray tube formula and Gauss equation Ric^# = W_H − S for submanifolds.
Used in Section C and Appendix F to compute tube Jacobian and the geometric weight; standard results (Federer 1959; do Carmo 1992).
standard math Regular conditional expectation given π(X)=z is well-defined and admits a density.
Justifies Definition 3.1 and the fiber posterior calculation in Proposition B.2; smoothness of π and positivity of q are needed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Rao-Blackwellized Score Matching on Manifolds." pith.science (2026). https://pith.science/paper/K5NPMOM3

@misc{pith2026260525567,
  author       = {Pith},
  title        = {Pith review of: Rao-Blackwellized Score Matching on Manifolds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K5NPMOM3}},
  note         = {Machine review of arXiv:2605.25567}
}

abstract

We study denoising score matching (DSM) when data are drawn from an embedded manifold $M \subset \mathbb{R}^D$. We show that under ambient Gaussian corruption, the target has variance that diverges as the noise scale decreases and correct for it by regressing against the conditional expectation given the nearest point projection on the manifold: the $L^2$-optimal Rao-Blackwellized target. We then compute the small-noise expansion of this target and show that it recovers the true intrinsic Riemannian score to first order, with a second-order bias from a Tweedie term and two geometric terms dependent on how the manifold is embedded in ambient space: a curvature operator acting on the intrinsic score, and an additive drift generated by the spatial variation of the embedding's second fundamental form. On hyperspheres, we derive a simplified formula and show that both geometric terms vanish exactly on $S^2$, offering a theoretical explanation for why ambient DSM performs comparably to intrinsic methods on real Earth science spherical data in prior work.

Figures

Figures reproduced from arXiv: 2605.25567 by the authors.

**Figure 1.** Variance collapse on S 2 under vMF(µ, κ=2). Second moment of the raw target Tσ (black, slope −2 in log σ, matching d/σ2 with d = 2) versus the Rao-Blackwellized target rσ(π(X)) (blue, flat at the theoretical E ∥∇M log q∥ 2 ). The gap is the irreducible d/σ2 Bayes-risk floor of Section 4.3. are independent. Let pT .= q ∗ ϕ (d) σ be the d-dimensional Gaussian convolution of q on V . Proposition 5.1 (Flat Case: Tweedi… view at source ↗

**Figure 2.** Extrinsic coefficient αext across manifolds. Quadrature estimates (one dot per σ ∈ {0.05, 0.06, 0.08}) are computed by Gauss-Hermite quadrature of rσ(z) and compared to the predicted coefficients αd = 1 − d/2 on S d and α = +1 2 on T 2 . Latent density q is von Mises-Fisher with κ = 2 on each S d and a wrapped Gaussian with κ = 1.5 on T 2 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** (a) Densities of z · µ at the equilibrium of three closed-form Langevin drifts (σ=0.3): the intrinsic score ∇M log q (black), the raw ambient-DSM drift (1 + σ 2αd)∇M log q with αd=1 − d/2 (orange), and its Theorem 5.4 debias (1 − σ 2αd)(1 + σ 2αd)∇M log q (blue). (b) Score MSE of the same MLP architecture and training budget regressing on the raw Tσ,i target (orange) versus the RaoBlackwellized rbσ,i target (blue).… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Asymptotic Preservation and Uniform Accuracy of Diffusion and Flow-Matching Samplers
cs.LG 2026-07 conditional novelty 7.5 of 10

DDIM (σ-clock Euler) is the unique layer-exact fixed-step sampler; deterministic residual budgets stay O(1) with no log(1/σ_min), while stochastic path-KL scales as Λ²/N from the Itô term alone.

Reference graph

Works this paper leans on

2 extracted references · 1 linked inside Pith · cited by 1 Pith paper

[1992]

org/CorpusID:53587425

URL https://api.semanticscholar. org/CorpusID:53587425. Federer, H. Curvature measures.Transactions of the Amer- 7 Rao-Blackwellized Score Matching on Manifolds ican Mathematical Society, 93(3):418–491, 1959. doi: 10.1090/S0002-9947-1959-0110078-1. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion prob- abilistic models, 2020. URL https://arxiv.org/ ab...

arXiv 1959
[2011]

org/CorpusID:23284154

URL https://api.semanticscholar. org/CorpusID:23284154. Fan, J. Design-adaptive nonparametric regression.Journal of the American Statistical Association, 87:998–1004,

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1992]

org/CorpusID:53587425

URL https://api.semanticscholar. org/CorpusID:53587425. Federer, H. Curvature measures.Transactions of the Amer- 7 Rao-Blackwellized Score Matching on Manifolds ican Mathematical Society, 93(3):418–491, 1959. doi: 10.1090/S0002-9947-1959-0110078-1. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion prob- abilistic models, 2020. URL https://arxiv.org/ ab...

arXiv 1959

[2] [2011]

org/CorpusID:23284154

URL https://api.semanticscholar. org/CorpusID:23284154. Fan, J. Design-adaptive nonparametric regression.Journal of the American Statistical Association, 87:998–1004,