Pith. sign in

REVIEW 4 major objections 5 minor 60 references

Density Ratio Estimation with Conditional Probability Paths

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Density ratio estimation can be made faster and more accurate by learning the time score through conditional probability paths; the paper proves the conditional objective is equivalent to the original up to a constant.

desk verdict A useful DRE trick with an overreaching theory: the CTSM objective works in practice, but the stated equivalence theorem needs regularity conditions it does not have. read the letter →

arxiv 2502.02300 v3 pith:GFWYJB5C submitted 2025-02-04 cs.LG

Hanlin Yu , Arto Klami , Aapo Hyvärinen , Anna Korba , Omar Chehab This is my paper · ORCID
classification cs.LG MSC 62G0768T07
keywords densityratioestimationtimescorematchingconditionalprobabilitypathsflowmutualinformationenergy-basedmodelingtheoreticalerrorbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Density ratios between two distributions can be written as the time integral of a time score, $\partial_t \log p_t(x)$, along a probability path connecting them. The paper establishes that this time score can be learned by regressing on conditional probability paths: the Conditional Time Score Matching loss equals the original Time Score Matching loss up to an additive constant, so the intractable marginal score is replaced by a closed-form conditional target. A vectorized variant learns the score componentwise and is several times faster per optimization step while matching or improving accuracy, including on MNIST in pixel space. The paper also proves a general error bound for the resulting density ratio estimator, decomposing error into discretization error and finite-sample estimation error.

What carries the argument

The load-bearing object is the conditioning variable $z$ with Gaussian conditional paths $p_t(x \mid z) = \mathcal{N}(x; \mu_t(z), k_t I)$, for example the variance-preserving path $x = \alpha_t x_1 + \sqrt{1-\alpha_t^2}\,x_0$. The identity $\partial_t \log p_t(x) = \mathbb{E}_{p_t(z \mid x)}[\partial_t \log p_t(x \mid z)]$ lets the intractable marginal score be replaced by a tractable conditional score inside a squared loss; expanding the loss shows the cross-term matches because of this expectation identity, leaving only a constant difference. A vectorized version sums per-dimension conditional scores, and a novel time-score normalization weighting $\lambda(t)$ stabilizes training.

What would settle it

Construct $p_0$ and $p_1$ with a known ratio and choose a path that is not a mixture of Gaussian conditionals, such as $p_t(x) = (1-t)p_0(x) + t p_1(x)$; train the CTSM objective, integrate the resulting score, and compare with the true $\log(p_1/p_0)$. A systematic discrepancy would show the mixture identity is load-bearing.

Watch

Extended reading notes

Core claim

The central discovery is that the marginal time score $\partial_t \log p_t(x)$ equals the posterior expectation of the conditional time score $\partial_t \log p_t(x \mid z)$ when the marginal path is a mixture of conditional paths, and that square-loss regression against the two targets produces the same minimizer up to an additive constant (Theorems 1-3). This holds for scalar time scores and for the vectorized autoregressive decomposition, so a network can be trained with a closed-form Gaussian target and then the estimated score integrated over time, $\log(p_1(x)/p_0(x)) = \int_0^1 \hat{s}(x,t)\,dt$ approximately, to recover the density ratio. The paper further derives an explicit KL error bound (Theorem 4) and an asymptotic sample-size rate (Proposition 5) for integrating an estimated time score.

Load-bearing premise

The method assumes the marginal probability path is exactly a mixture of the chosen conditional paths with the conditioning variable drawn independently of time; if that decomposition does not hold, the conditional loss is no longer equivalent to the time score loss.

Editorial extensions

If this is right

  • The TSM objective can be optimized without double automatic differentiation, because CTSM never evaluates $\partial_t \log p_t(x)$ or its parameter derivatives; this is why each iteration is several times cheaper.
  • The learned time score, integrated over time, yields the density ratio and hence an unnormalized model for $p_1$ when $p_0$ is known; the paper demonstrates this by modeling MNIST directly in pixel space.
  • The error bound shows that accuracy scales as $1/K^2$ in the number of integration steps and $1/N$ in sample size, so both discretization and optimization contribute in a controlled way.
  • Vectorized CTSM is essential in practice: the non-vectorized CTSM was essentially untrainable on MNIST, while CTSM-v achieved competitive or better BPD than TSM at a fraction of the per-step cost.
  • The general regression equivalence (Theorem 3) applies to any intractable function that is a conditional expectation of a tractable one, not only time scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mixture identity fails, for a path not representable as $p_t(x) = \int p_t(x\mid z)\,p(z)\,dz$ with $z$ independent of $t$, the CTSM objective regresses a different target and the integrated ratio becomes biased; testing this on non-Gaussian or deterministic paths would quantify the method's robustness.
  • The same conditional-expectation trick could be applied to other intractable functionals, such as the Stein score or velocities, unifying conditional score matching and flow matching objectives under one theorem; the paper notes related versions but does not push this unification.
  • The vectorized objective suggests an autoregressive view of time scores; in very high dimensions, learning per-dimension components may interact with network inductive biases differently than learning the scalar score, which could be studied by ablating the vectorization.
  • Weighting by the inverse variance of the conditional time score is derived under an assumed constant $c$ set to 1; estimating $c$ from data is a natural tuning knob that the paper reports sometimes helps and sometimes hurts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes new objectives, Conditional Time Score Matching (CTSM) and its vectorized variant (CTSM-v), for estimating the time score ∂t log pt(x) along a probability path connecting two densities, and uses the integrated time score to estimate the density ratio log p1(x)/p0(x). The objectives are obtained by conditioning on a latent variable z so that the conditional time score is available in closed form; the paper proves that the TSM and CTSM losses have the same minimizer up to an additive constant (Theorems 1--3), gives a general error bound and an asymptotic M-estimation bound (Theorem 4 and Proposition 5), and reports experiments on Gaussians, Gaussian mixtures, mutual information estimation, and MNIST energy-based modeling, claiming substantial speedups and competitive or better accuracy relative to TSM.

Significance. The core idea---regressing conditional time scores rather than the marginal time score---is a natural and useful extension of denoising score matching and flow matching to density ratio estimation, and Theorem 3 generalizes the known conditional-score identity to arbitrary functions. If the theoretical statements are made rigorous with appropriate regularity conditions, and if the empirical comparison is properly controlled, the contribution would be valuable: it offers a tractable objective that avoids the double backpropagation required by TSM, provides closed-form weighting functions for VP and Schrödinger bridge paths, and supplies formal error guarantees that prior work in this line (Rhodes et al., 2020; Choi et al., 2022) lacked. The release of code and the detailed appendices are strengths. However, as stated, the theoretical results are incomplete and partly not applicable to the paper's own VP path, and the main experimental comparison does not isolate the proposed objective from the concurrent change of weighting function.

major comments (4)
  1. [§3.1, Theorem 1 / Appendix D.1--D.2] The equality between the TSM loss (Eq. 4) and the CTSM loss (Eq. 8) requires the conditional time score to have finite second moments, but for the VP path with αt = t used throughout the paper this fails. Equation (65) gives Var[∂t log pt(x|z)] = O((1−t)^{-2}) as t → 1, so with uniform t and λ(t)=1 the CTSM loss is infinite for every θ. As a concrete counterexample, for p0 = p1 = N(0, I) the marginal time score is identically zero and the TSM loss is finite, while the CTSM loss is infinite, so the claimed finite additive constant in Theorem 1 cannot exist; the gradient proof in D.2 is not valid for infinite losses. The subsequent theorems that rely on the same identity (Theorems 2 and 3, and through them Proposition 5) inherit this issue. The paper's practical safeguards (ε-clipping, time score normalization Eq. 20--21, importance sampling with t1 = 0.9) avoid the singularity, but the theorems are stated without these restrictions. The statements should be revised to hold on a truncated interval t ∈ [0, 1−ε], or under a weighting λ(t) that vanishes sufficiently fast near t=1, or under explicit finite-moment assumptions, and the proofs should justify differentiating under the integral in Eq. (78).
  2. [§5, Proposition 5 / Eq. (114)] Proposition 5 is an M-estimation result, but its validity requires the population loss to be finite and the gradient covariance G(θ*) in Eq. (114) to be finite. As noted above, for the VP path the residual w(x,z,t) = ∂t log pt(x|z) − ∂t log pt(x) has infinite second moment near t=1 under uniform weighting, so G(θ*) is infinite and the asymptotic expansion √N(θ̂ − θ*) ⇒ N(0, H^{-1} G H^{-1}) is not justified. Even if one restricts to a truncated time interval, the proposition should explicitly state the regularity conditions (finite second moments of the conditional time score, interchange of differentiation and integration, well-specified model, and standard M-estimation conditions) rather than leaving them implicit in the proof. Without these, the claimed o(1/N) rate for the final density-ratio error is not established for the paper's own running example.
  3. [§5, Theorem 4 / Eq. (24) and Appendix D.3] The theorem is stated as a bound on KL(p1, p̂1), but Eq. (23) defines log p̂1(x) as an integral of an estimated time score plus log p0(x), which does not in general produce a normalized density. For unnormalized p̂1, the quantity E_{p1}[log p1 − log p̂1] is not the KL divergence; the proof in D.3 actually bounds the squared pointwise error of the log-density ratio, E_{p1}[(log p1(x) − log p̂1(x))^2]. The theorem should either be restated for this squared log-ratio error, or a normalization-constant term should be carried through. Separately, the proof of the discretization term uses the left rectangular rule with K equal subintervals, but Eq. (23) does not specify that the ti form a uniform grid; if the ti are arbitrary or sampled (e.g., from an importance distribution as in §4), the bound L(x)/(2K) in Eq. (24) does not follow. Both issues affect Proposition 5, which invokes Theorem 4.
  4. [§6, Tables 2--3 and Figure 2] The empirical comparison is confounded by a concurrent change of weighting function. Throughout Section 6, TSM is trained with Stein score normalization (Eq. 19, λ(t) ∝ kt) while CTSM and CTSM-v are trained with time score normalization (Eq. 20--21), and in the EBM experiments the CTSM-v architecture additionally differs from TSM's (Fourier feature embeddings vs. linear embeddings, and a different U-Net in the ambient-space experiments). Since the authors declare time score normalization to be an integral part of CTSM, the observed accuracy differences cannot be attributed solely to the proposed objective. Appendix Table 4 provides a partial TSM-with-time-normalization comparison only for the Gaussian toy problem, which helps but is not the main text comparison. The main claims would be supported by reporting TSM+Time normalization in the main figures, or by adding an ablation in which CTSM uses Stein normalization, and by separating the effect of the vectorized objective from the effect of the weighting and architecture changes.
minor comments (5)
  1. [Appendix B.1, Eq. (65)] The variance formula in Eq. (65) is missing a factor of D that follows from Lemma 6: the variance of the scalar time score is proportional to D. Since D does not depend on t, the proportional weighting in Eq. (21) is unaffected, but the displayed formula is incorrect as a standalone identity.
  2. [§3.2, Eq. (9)] The notation xi and x<i in the definition of the vectorized conditional time score is introduced informally; for the VP path the components do not depend on x<i, but this should be stated explicitly to avoid confusion about the ordering of the autoregressive decomposition.
  3. [§5, Proposition 5] Proposition 5 should state explicitly that the model is assumed to be well-specified (existence of θ* with s_{θ*}(x,t) = ∂t log pt(x)) and that standard M-estimation regularity conditions hold; these are currently only mentioned inside the proof.
  4. [§6.4] The statement that 'in practice, CTSM could not be used for this task' is reported without supporting results; since the paper advocates the vectorized variant, a brief explanation or a reference to preliminary experiments would help the reader assess the limitation of the non-vectorized objective.
  5. [§4, Eq. (21)] The definition of the time score normalization for CTSM-v is said to assume each dimension has equal scale; the manuscript would benefit from stating what is done when this assumption is violated and how the scalar weighting λ̂(t) is applied to the vectorized objective.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CTSM objective is derived from a standard conditioning identity and the central claims are not reductions of fitted inputs; self-citations are background only.

full rationale

The central equivalence (Theorem 1; general version Theorem 3) is not a fitting or definitional identity. The proof expands both losses and uses Eq. 7, ∂t log pt(x) = E_{pt(z|x)}[∂t log pt(x|z)], which follows from the mixture construction pt(x)=∫pt(x|z)p(z)dz (Appendix D.1). Both sides are defined independently: the TSM target is the marginal time score, the CTSM target is the conditional time score, and the theorem shows their squared-error objectives differ by the constant E[f^2]−E[g^2]. No parameter is fitted to the right-hand side and then called a prediction; the learned sθ is evaluated against the density-ratio integral Eq. 2 on held-out data. Theorem 4 and Proposition 5 are standard decompositions (Jensen, Lipschitz discretization, M-estimator asymptotics via Bach and van der Vaart) and do not use the paper's fitted values. The paper honestly credits earlier versions of Theorem 3 (Vincent 2011; Lipman et al. 2023; Pooladian et al. 2023; Tong et al. 2024a). Self-citations (Chehab et al. 2023a,b) appear only as background for NCE variance and normalizing-constant estimation, not as load-bearing justification for the central derivation. The substantive weakness is a regularity gap, not circularity: Eq. 7 requires differentiating under the integral and finite variance of ∂t log pt(x|z), and for the VP path with αt=t the conditional time-score variance diverges like D/(2(1−t)^2) as t→1 (Eq. 65), so with λ=1 the CTSM loss is not finite and the 'additive constant' in Theorem 1 is not well-defined; the paper's ε-clipping and time-score normalization are practical fixes not reflected in the stated theorems. This affects soundness of the theoretical statements but is not a reduction of the result to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The free parameters are practical design choices, not part of the mathematical derivation; they influence reported performance. No new physical entities, unobserved variables, forces, or mediators are introduced. The conditioning variable z is the data point x1 or the pair (x0,x1), which are observed quantities in the sampling process.

free parameters (3)
  • c in time score normalization = 1 in most experiments (Eq. 21)
    The weighting formula depends on c=(Tr Σ + ||μ||^2)/D, the mean/covariance of the target data. Instead of estimating c, the paper sets c=1 because 'setting c = 1 often yields better results' (Section 4). This hand-chosen constant affects the weighting and the reported performance.
  • σ in Schrodinger bridge path = 1.0 for GMM experiments
    For the Gaussian mixture experiment, the path variance parameter σ is chosen empirically: 'we set σ = 1.0 due to strong empirical results' (Appendices B.2 and F.3). The variance-preserving choice would be σ = √2, and different values of σ change both TSM and CTSM results.
  • t1 in importance sampling = 0.9
    The importance sampling scheme truncates the inverse-CDF sampling at t1=0.9 (Appendix C.1). This heuristic is used in the EBM experiments and affects the weighting over time.
assumptions (4)
  • domain assumption Mixture identity: pt(x)=∫ pt(x|z)p(z)dz with p(z) independent of t, and interchanging ∂t with the integral is valid.
    Used to derive Eq. 7 and Theorems 1 and 3 (Section 3, Appendix D.1). For the Gaussian VP and SB paths this holds by construction, but the paper does not state the regularity conditions that justify differentiating through the integral.
  • domain assumption Well-specified model: there exists θ* such that s_{θ*}(x,t)=∂t log pt(x).
    Proposition 5 assumes 'the model is well-specified' (Appendix D.4). Neural networks are generally misspecified, so the stated O(1/N) rate does not directly apply to the practical estimator.
  • domain assumption Lipschitz regularity: t ↦ s(x,t) is L(x)-Lipschitz.
    Theorem 4 bounds the discretization error by L(x)/(2K). If the time score is not Lipschitz, for example near α_t → 1 where the variance of the time score explodes, the discretization term may blow up (Section 5, Appendix D.3).
  • standard math Standard M-estimator regularity conditions from van der Vaart (2000, Thm 5.23) and Bach (2024, Sec 4.7).
    Used in Appendix D.4 to obtain asymptotic normality of θ̂; these conditions are imported without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Density Ratio Estimation with Conditional Probability Paths." pith.science (2026). https://pith.science/paper/GFWYJB5C

@misc{pith2026250202300,
  author       = {Pith},
  title        = {Pith review of: Density Ratio Estimation with Conditional Probability Paths},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GFWYJB5C}},
  note         = {Machine review of arXiv:2502.02300}
}
read the original abstract

Density ratio estimation in high dimensions can be reframed as integrating a certain quantity, the time score, over probability paths which interpolate between the two densities. In practice, the time score has to be estimated based on samples from the two densities. However, existing methods for this problem remain computationally expensive and can yield inaccurate estimates. Inspired by recent advances in generative modeling, we introduce a novel framework for time score estimation, based on a conditioning variable. Choosing the conditioning variable judiciously enables a closed-form objective function. We demonstrate that, compared to previous approaches, our approach results in faster learning of the time score and competitive or better estimation accuracies of the density ratio on challenging tasks. Furthermore, we establish theoretical guarantees on the error of the estimated density ratio.

Figures

Figures reproduced from arXiv: 2502.02300 by the authors.

Figure 1
Figure 1. Densities are shown in blue. Left: A bi-modal probability path transitioning from a Gaussian distribution (t = 0) to a mixture of Diracs (t = 1). This path is esti￾mated using “time scores”, which are not available in closed form in general; they are depicted by arrows, with magni￾tudes ranging from low (gray) to high (red). Right: A useful decomposition of the probability path and time scores is obtained by conditi… view at source ↗
Figure 2
Figure 2. For estimating the density ratio between two Gaus [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Mutual information estimation. Left: Time per iteration. Middle: Estimated and true (in dashed black) Mutual Information for different dimensions, where we directly report the estimates obtained after a few thousand iterations (see Appendix, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: We report the samples obtained using ambient [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Left: samples drawn from a model trained with TSM, Gaussian flows; right: samples drawn from a model trained [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 52 canonical work pages

  1. [1]

    Albergo, M. S. and Vanden-Eijnden, E. Building Normalizing Flows with Stochastic Interpolants . In The Eleventh International Conference on Learning Representations , 2023

    work page 2023

  2. [2]

    Learning Theory from First Principles

    Bach, F. Learning Theory from First Principles. MIT Press, 2024

    work page 2024

  3. [3]

    D., Hutchinson, M., Wirnsberger, P., and Doucet, A

    Bortoli, V. D., Hutchinson, M., Wirnsberger, P., and Doucet, A. Target Score Matching , 2024. \_eprint: 2402.08667

    arXiv 2024

  4. [4]

    Accurate and conservative estimates of MRF log-likelihood using reverse annealing

    Burda, Y., Grosse, R., and Salakhutdinov, R. Accurate and conservative estimates of MRF log-likelihood using reverse annealing . In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, volume 38 of Proceedings of Machine Learning Research, pp.\ 102--110, San Diego, California, USA, 09--12 May 2015. PMLR

    work page 2015

  5. [5]

    Optimizing the Noise in Self - Supervised Learning : from Importance Sampling to Noise - Contrastive Estimation , 2023 a

    Chehab, O., Gramfort, A., and Hyvarinen, A. Optimizing the Noise in Self - Supervised Learning : from Importance Sampling to Noise - Contrastive Estimation , 2023 a . \_eprint: 2301.09696

    arXiv 2023

  6. [6]

    Provable benefits of annealing for estimating normalizing constants: Importance Sampling , Noise - Contrastive Estimation , and beyond

    Chehab, O., Hyvarinen, A., and Risteski, A. Provable benefits of annealing for estimating normalizing constants: Importance Sampling , Noise - Contrastive Estimation , and beyond. In Advances in Neural Information Processing Systems , volume 36, pp.\ 45945--45970. Curran Associates, Inc., 2023 b

    work page 2023

  7. [7]

    The probability flow ODE is provably fast

    Chen, S., Chewi, S., Lee, H., Li, Y., Lu, J., and Salim, A. The probability flow ODE is provably fast. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    work page 2023

  8. [8]

    On the importance of noise scheduling for diffusion models, 2023

    Chen, T. On the importance of noise scheduling for diffusion models, 2023

    work page 2023

Show all 60 references
  1. [9]

    Density ratio estimation via infinitesimal classification

    Choi, K., Meng, C., Song, Y., and Ermon, S. Density ratio estimation via infinitesimal classification. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pp.\ 2552--2573. PMLR, ...

    2022

  2. [10]

    Fast and Accurate Deep Network Learning by Exponential Linear Units ( ELUs )

    Clevert, D.-A., Unterthiner, T., and Hochreiter, S. Fast and Accurate Deep Network Learning by Exponential Linear Units ( ELUs ). In Bengio, Y. and LeCun, Y. (eds.), 4th International Conference on Learning Representations , ICLR 2016, San Juan , Puerto Rico , May 2-4, 2016, C...

    2016

  3. [11]

    W., and Doucet, A

    De Bortoli, V., Mathieu, E., Hutchinson, M., Thornton, J., Teh, Y. W., and Doucet, A. Riemannian Score - Based Generative Modelling . In Advances in Neural Information Processing Systems , volume 35, pp.\ 2406--2422. Curran Associates, Inc., 2022

    2022

  4. [12]

    Dormand, J. R. and Prince, P. J. A family of embedded Runge - Kutta formulae. Journal of Computational and Applied Mathematics, 6 0 (1): 0 19--26, 1980

    1980

  5. [13]

    B., Dieleman, S., Fergus, R., Sohl-Dickstein, J

    Du, Y., Durkan, C., Strudel, R., Tenenbaum, J. B., Dieleman, S., Fergus, R., Sohl-Dickstein, J. N., Doucet, A., and Grathwohl, W. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. In International Conference on Machine Learning, 2023

    2023

  6. [14]

    Random fields and diffusion processes

    F \"o llmer, H. Random fields and diffusion processes. Ecole d'Ete de probabilities de Saint-Flour XV-XVII,1985-87. LNM 1362. Springer-Verlag, Berlin, 1988

    1985

  7. [15]

    P., Xu, Z., Dai, A

    Gao, R., Nijkamp, E., Kingma, D. P., Xu, Z., Dai, A. M., and Wu, Y. N. Flow contrastive estimation of energy-based models. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 7515--7525, 2019

    2020

  8. [16]

    Gao, Y., Huang, J., and Jiao, a. Y. Gaussian Interpolation Flows . Journal of Machine Learning Research, 25 0 (253): 0 1--52, 2024

    2024

  9. [17]

    Generative Adversarial Networks

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative Adversarial Networks . Commun. ACM, 63 0 (11): 0 139--144, October 2020. Place: New York, NY, USA Publisher: Association for Computing Machinery

    2020

  10. [18]

    Provable Benefit of Annealed Langevin Monte Carlo for Non -log-concave Sampling

    Guo, W., Tao, M., and Chen, Y. Provable Benefit of Annealed Langevin Monte Carlo for Non -log-concave Sampling . In The Thirteenth International Conference on Learning Representations , ICLR 2025, Singapore , April 24-28, 2025 . OpenReview.net, 2025

    2025

  11. [19]

    Gutmann, M. U. and Hyv \"a rinen, A. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13 0 (11): 0 307--361, 2012

    2012

  12. [20]

    Deep Residual Learning for Image Recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep Residual Learning for Image Recognition . In 2016 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ) , pp.\ 770--778, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  13. [21]

    Denoising Diffusion Probabilistic Models

    Ho, J., Jain, A., and Abbeel, P. Denoising Diffusion Probabilistic Models . In Advances in Neural Information Processing Systems , volume 33, pp.\ 6840--6851. Curran Associates, Inc., 2020

    2020

  14. [22]

    Estimation of non-normalized statistical models by score matching

    Hyv \"a rinen, A. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6 0 (24): 0 695--709, 2005

    2005

  15. [23]

    High- Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation

    Izbicki, R., Lee, A., and Schafer, C. High- Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation . In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics , volume 33 of Proceedings of Machine L...

    2014

  16. [24]

    P., and Mallat, S

    Kadkhodaie, Z., Guth, F., Simoncelli, E. P., and Mallat, S. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In The Twelfth International Conference on Learning Representations , 2024

    2024

  17. [25]

    and Ganguli, S

    Kamb, M. and Ganguli, S. An analytic theory of creativity in convolutional diffusion models, 2024. \_eprint: 2412.20292

    2024 arXiv

  18. [26]

    Multi-task learning using uncertainty to weigh losses for scene geometry and semantics

    Kendall, A., Gal, Y., and Cipolla, R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7482--7491, 2017

    2018

  19. [27]

    and Bondell, H

    Kimura, M. and Bondell, H. Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifolds . In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , volume 258 of Proceedings of Machine Learning Research , pp.\...

    2025

  20. [28]

    and Gao, R

    Kingma, D. and Gao, R. Understanding diffusion objectives as the elbo with simple data augmentation. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 65484--65516. Curran Ass...

    2023

  21. [29]

    MNIST handwritten digit database

    LeCun, Y., Cortes, C., and Burges, C. MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010

    2010

  22. [30]

    P., and Risteski, A

    Lee, H., Pabbaraju, C., Sevekari, A. P., and Risteski, A. Pitfalls of gaussians as a noise distribution in NCE . In The Eleventh International Conference on Learning Representations, 2023

    2023

  23. [31]

    Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023

    2023

  24. [32]

    K., and Risteski, A

    Liu, B., Rosenfeld, E., Ravikumar, P. K., and Risteski, A. Analyzing and improving the optimization landscape of noise-contrastive estimation. In International Conference on Learning Representations, 2022

    2022

  25. [33]

    Minimizing f -divergences by interpolating velocity fields

    Liu, S., Yu, J., Simons, J., Yi, M., and Beaumont, M. Minimizing f -divergences by interpolating velocity fields. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on ...

    2024

  26. [34]

    Flow Straight and Fast : Learning to Generate and Transfer Data with Rectified Flow

    Liu, X., Gong, C., and liu, q. Flow Straight and Fast : Learning to Generate and Transfer Data with Rectified Flow . In The Eleventh International Conference on Learning Representations , 2023

    2023

  27. [35]

    Autoregressive Score Matching

    Meng, C., Yu, L., Song, Y., Song, J., and Ermon, S. Autoregressive Score Matching . In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp.\ 6673--6683. Curran Associates, Inc., 2020

    2020

  28. [36]

    Annealed importance sampling

    Neal, R. Annealed importance sampling. Statistics and Computing, 11: 0 125--139, 1998

    1998

  29. [37]

    Nichol, A. Q. and Dhariwal, P. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.\ 8162--8171. PMLR, 18--24 Jul 2021

    2021

  30. [38]

    Pooladian, A.-A., Ben-Hamu, H., Domingo-Enrich, C., Amos, B., Lipman, Y., and Chen, R. T. Q. Multisample flow matching: Straightening flows with minibatch couplings. In International Conference on Machine Learning, 2023

    2023

  31. [39]

    Rhodes, B., Xu, K., and Gutmann, M. U. Telescoping Density - Ratio Estimation . In Advances in Neural Information Processing Systems , volume 33, pp.\ 4905--4916. Curran Associates, Inc., 2020

    2020

  32. [40]

    An overview of multi-task learning in deep neural networks

    Ruder, S. An overview of multi-task learning in deep neural networks. ArXiv, abs/1706.05098, 2017

    2017 arXiv

  33. [41]

    Scarvelis, C., Borde, H. S. d. O., and Solomon, J. Closed- Form Diffusion Models , 2024

    2024

  34. [42]

    Sinha, A., O' Kelly, M., Tedrake, R., and Duchi, J. C. Neural Bridge Sampling for Evaluating Safety - Critical Autonomous Systems . In Advances in Neural Information Processing Systems , volume 33, pp.\ 6402--6416. Curran Associates, Inc., 2020

    2020

  35. [43]

    Deep Unsupervised Learning using Nonequilibrium Thermodynamics

    Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep Unsupervised Learning using Nonequilibrium Thermodynamics . In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pp.\ 2256--226...

    2015

  36. [44]

    and Ermon, S

    Song, J. and Ermon, S. Understanding the Limitations of Variational Mutual Information Estimators . In International Conference on Learning Representations , 2020

    2020

  37. [45]

    Maximum Likelihood Training of Score - Based Diffusion Models

    Song, Y., Durkan, C., Murray, I., and Ermon, S. Maximum Likelihood Training of Score - Based Diffusion Models . In Advances in Neural Information Processing Systems , volume 34, pp.\ 1415--1428. Curran Associates, Inc., 2021 a

    2021

  38. [46]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021 b

    2021

  39. [47]

    Srivastava, A., Han, S., Xu, K., Rhodes, B., and Gutmann, M. U. Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression . Transactions on Machine Learning Research, 2023. ISSN 2835-8856

    2023

  40. [48]

    Density ratio estimation: A comprehensive review

    Sugiyama, M., Suzuki, T., and Kanamori, T. Density ratio estimation: A comprehensive review. In Statistical Experiment and Its Related Topics , Research Institute for Mathematical Sciences Kokyuroku , volume 1703, pp.\ 10--31, 2010. Presented at Research Institute for Mathemat...

    2010

  41. [49]

    Fourier features let networks learn high frequency functions in low dimensional domains

    Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., and Ng, R. Fourier features let networks learn high frequency functions in low dimensional domains. In Advances in Neural Information Processing Systems, vol...

    2020

  42. [50]

    Improving and generalizing flow-based generative models with minibatch optimal transport

    Tong, A., Fatras, K., Malkin, N., Huguet, G., Zhang, Y., Rector-Brooks, J., Wolf, G., and Bengio, Y. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024 a . ISSN 2835-8856. Expert Certification

    2024

  43. [51]

    Y., Malkin, N., Fatras, K., Atanackovic, L., Zhang, Y., Huguet, G., Wolf, G., and Bengio, Y

    Tong, A. Y., Malkin, N., Fatras, K., Atanackovic, L., Zhang, Y., Huguet, G., Wolf, G., and Bengio, Y. Simulation-free S chrödinger bridges via score and flow matching. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of ...

    2024

  44. [52]

    Asymptotic Statistics

    van der Vaart , A. Asymptotic Statistics. Asymptotic Statistics. Cambridge University Press, 2000

    2000

  45. [53]

    A connection between score matching and denoising autoencoders

    Vincent, P. A connection between score matching and denoising autoencoders. Neural Computation, 23: 0 1661--1674, 2011

    2011

  46. [54]

    E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S

    Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J....

    2020

  47. [55]

    Learning Calibrated Uncertainties for Domain Shift : A Distributionally Robust Learning Approach

    Wang, H., Yu, Z., Yue, Y., Anandkumar, A., Liu, A., and Yan, J. Learning Calibrated Uncertainties for Domain Shift : A Distributionally Robust Learning Approach . In Proceedings of the Thirty - Second International Joint Conference on Artificial Intelligence , IJCAI -23 , pp.\...

    2023

  48. [56]

    J., Wang, L., Ying, Q., Liu, S., and Kolar, M

    Williams, D. J., Wang, L., Ying, Q., Liu, S., and Kolar, M. High- Dimensional Differential Parameter Inference in Exponential Family using Time Score Matching . In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , volume 258 of Procee...

    2025

  49. [57]

    and Xie, Y

    Wu, D. and Xie, Y. Annealing Flow Generative Models Towards Sampling High - Dimensional and Multi - Modal Distributions , 2025. \_eprint: 2409.20547

    2025 arXiv

  50. [58]

    Computing high-dimensional optimal transport by flow neural networks

    Xu, C., Cheng, X., and Xie, Y. Computing high-dimensional optimal transport by flow neural networks. In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , volume 258 of Proceedings of Machine Learning Research , pp.\ 2872--2880. PMLR, May 2025

    2025

  51. [59]

    Classification Diffusion Models : Revitalizing Density Ratio Estimation

    Yadin, S., Elata, N., and Michaeli, T. Classification Diffusion Models : Revitalizing Density Ratio Estimation . In The Thirty -eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  52. [60]

    and Michaeli, T

    Yair, O. and Michaeli, T. Thinking fourth dimensionally: Treating time as a random variable in EBM s, 2023

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.