Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coverage

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Data-free diffusion samplers can be trained to cover every mode of a multi-modal target by reweighting the score matching loss with self-normalized importance sampling estimates, turning a forward-KL mode-covering objective into a…

desk verdict A useful empirical recipe for mode coverage in data-free diffusion samplers, but the theory section analyzes a different estimator than the algorithm actually uses. read the letter →

arxiv 2505.19431 v1 pith:TYIYJ4JE submitted 2025-05-26 cs.LG

Chenguang Wang , Xiaoyu Zhang , Kaiyuan Cui , Weichen Zhao , Yongtao Guan , Tianshu Yu This is my paper · ORCID
classification cs.LG
keywords importanceweightedscorematchingdiffusionsamplersmodecoverageforwardKLdivergenceself-normalizedsamplingdata-freeLennard-Jonesbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural samplers trained from an unnormalized energy function, with no target samples available, tend to optimize reverse-KL objectives and therefore collapse onto a few modes. This paper claims that the objective choice is the root cause, and that targeting the forward KL between diffusion path measures, which penalizes putting zero probability where the target has mass, is the principled fix. The contribution is a method, Importance Weighted Score Matching, that makes such a forward-KL-type objective tractable in the data-free setting by reweighting the score matching loss with self-normalized importance sampling estimates computed from the sampler's own replay buffer and the known energy function. If the claim holds, mode coverage becomes a direct property of the training objective rather than a hoped-for byproduct of heuristics, and data-free samplers could be applied more reliably to multi-modal problems in molecular and statistical physics.

What carries the argument

The load-bearing object is the importance-weighted score matching objective $\mathbb{E}_{t}\mathbb{E}_{x_t\sim p_t^B}[w(x_t)\lVert s_\theta(x_t,t)-\nabla\log p_t(x_t)\rVert^2]$ with $w=p_t/p_t^B$, which converts an expectation under the intractable true marginal into one under an accessible proposal that adaptively improves as the replay buffer is refreshed. Three estimator components carry the argument. First, the score target $S_L(x_t,t)=\nabla_{x_t}\log\sum_{i=1}^{L}\exp(-E(x_{0|t}^{(i)}))$ with $x_{0|t}^{(i)}\sim\mathcal{N}(x_0;x_t,\sigma_t^2 I)$, whose validity rests on the kernel symmetry $\mathcal{N}(x_t;x_0,\sigma_t^2 I)=\mathcal{N}(x_0;x_t,\sigma_t^2 I)$ that turns the intractable marginal into a computable expectation. Second, the unnormalized weight estimator $\tilde{w}=N_K/D_M$, combining averaged energy exponentials in the numerator with averaged forward-kernel evaluations over buffer samples in the denominator, which is proportional to the true weight up to the unknown normalizing constant $Z$. Third, self-normalization of these weights over a batch, which cancels $Z$ and yields the practical loss of Eq. (13) and Eq. (15). The theory rides on Proposition 1 (path forward KL equals score matching), Proposition 2 (bias and variance of $S_L$), and Proposition 3 (bias and MSE of the self-normalized loss).

What would settle it

On a two-mode Gaussian mixture, compute the importance-weighted score matching loss three ways: with per-parent self-normalization exactly as in Eq. (15), with batch-level self-normalization over the marginal proposal as assumed in Proposition 3, and with a high-sample-count direct Monte Carlo estimate of the true forward-KL objective, then track all three through training. If the per-parent loss converges to a different value or yields different trained scores than the batch-level loss while the batch-level version tracks the true objective, the implemented estimator is not the one the theory analyzes.

Watch

Extended reading notes

Core claim

The paper's central claim is that the forward path-KL objective $\mathrm{KL}(\mathbb{P}_r \,\|\, \mathbb{P}_\theta)$ between the true and learned reverse diffusion measures is both desirable and optimizable without target data. Proposition 1 identifies this objective with the score matching loss $\mathbb{E}_{t\sim U(0,1)}\mathbb{E}_{x_t\sim p_t}[\lVert s_\theta(x_t,t)-\nabla\log p_t(x_t)\rVert^2]$, and the method then rewrites the expectation under the intractable true marginal $p_t$ as an expectation under the sampler-induced proposal $p_t^B$ with importance weight $w(x_t)=p_t(x_t)/p_t^B(x_t)$. The weight is estimated by Monte Carlo: the numerator uses the Gaussian symmetry of the variance-exploding (VE-SDE) kernel to write the unnormalized marginal as an expectation over $x_0\sim\mathcal{N}(x_0;x_t,\sigma_t^2 I)$ of $\exp(-E(x_0))$, and the denominator averages the forward kernel over replay-buffer samples; self-normalization over a batch then cancels the unknown normalizing constant. On these estimates the paper proves a score-estimator bias and variance of order $1/\sqrt{L}$ (Proposition 2) and bias and mean-squared-error bounds for the self-normalized loss that shrink with the importance-weight variability and with the score-estimator sample size (Proposition 3). The paper reports that the resulting sampler matches or beats the compared baselines on every distributional metric across GMM-40/80/120, DW-4, LJ-13, and LJ-55, and that the reweighting itself, not the base score matching objective, is what drives the improvement.

Load-bearing premise

The training loss normalizes the estimated importance weights separately for each base sample drawn from the replay buffer, while the theory's bias and variance bounds are derived for normalization over a full batch of proposal samples; the paper offers no proof that per-base-sample normalization preserves the weighted expectation that the objective is claimed to estimate.

Editorial extensions

If this is right

  • Mode coverage becomes a property of the training objective itself: regions of the target that the current sampler under-represents receive the largest importance weights and therefore the strongest gradient pressure, so training automatically concentrates on the modes it is missing.
  • The reported benchmarks imply the approach scales: 120-mode Gaussian mixtures and the 165-dimensional LJ-55 cluster are handled without the divergence or breakdown the compared baselines suffer at that complexity.
  • Small SNIS sample counts suffice, since the ablations show $S=5$ to $S=10$ essentially saturates the benefit at modest training-time cost, so the correction is cheap to add to an existing score-based sampler.
  • Direct sampling with the estimated score alone is shown in the paper's Appendix H to be orders of magnitude slower at matched quality, so amortized learning plus importance weighting, not a larger Monte Carlo batch, is the practical route to accurate sampling.
  • If Proposition 3's bounds are taken at face value, the bias and variance of the trained loss are controlled by the importance-weight variability, giving a concrete lever for future work: improve the proposal distribution and the bound tightens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Whether the per-parent normalization used in Eq. (15) is equivalent to the batch-level normalization the theory analyzes is an empirical question; a controlled comparison of the two on a low-mode GMM would show whether the implemented estimator matches the one whose bias and variance are bounded.
  • The importance weights themselves could serve as a diagnostic: tracking their variability over training would measure how far the replay-buffer proposal has drifted from the true marginal, giving a principled mode-collapse monitor and stopping rule that the paper does not propose.
  • The forward-KL mechanism suggests a stress test the paper does not run: freeze the replay buffer mid-training; if coverage relies on the adaptive proposal as the argument claims, a stale buffer should degrade performance toward the unweighted baseline, isolating the importance weighting as the active ingredient.
  • Because the objective is agnostic to the sampler architecture, the same importance weighting could attach to other score-based estimators, for instance proposals built from annealed or sequentially refined densities, extending mode coverage to settings where maintaining one replay buffer is impractical.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes Importance Weighted Score Matching (IWSM), a data-free training objective for diffusion samplers that reweights a score-matching loss by the density ratio pt/pB_t, where pB_t is the marginal obtained by diffusing samples from a replay buffer of the model's own past samples. The paper derives a path-KL/score-matching equivalence (Proposition 1), proves bias/variance bounds for the proposed Monte Carlo estimators (Propositions 2 and 3), and reports experiments on GMM-40/80/120, DW-4, LJ-13, and LJ-55 that outperform PIS, FAB, DiKL, and iDEM on several distributional metrics. The central difficulty is that the theoretical estimator in Section 3.3 normalizes SNIS weights over a batch from the marginal proposal pB_t, whereas Algorithm 1 and Eq. (15) normalize weights within per-parent conditional batches {x_{t|0}^{(s)}} ~ p_{t|0}(·|x0); these two estimators are not the same, and the paper does not prove that the per-parent normalization preserves the intended expectation.

Significance. The underlying idea—using the sampler's own replay buffer to build an adaptive proposal and correcting the score-matching loss by importance weights—is attractive and, if properly analyzed, would be a meaningful step toward principled mode-covering samplers. The experiments cover a good range of benchmarks, and the ablations in Section 5.3 and Appendix H provide useful empirical evidence about the role of SNIS sample counts and the cost of direct estimated-score sampling. However, the manuscript's central theoretical claim does not currently cover the implemented loss: the per-parent normalization in Eq. (15) estimates a different functional that need not converge to the forward-KL score-matching objective. The theoretical support is therefore load-bearing and currently broken, although the empirical findings may survive a corrected analysis.

major comments (3)
  1. [Section 3.4, Eq. (15); Algorithm 1] The loss actually optimized is not the estimator analyzed in Proposition 3. In Eq. (12) the SNIS weights are normalized over S samples {x_t^(s)} drawn from the marginal proposal pB_t, and Proposition 3 bounds that estimator. In Algorithm 1 (lines 4–7) and Eq. (15), for each buffer sample x0 the S samples are drawn conditionally from p_{t|0}(·|x0), and the weights are normalized inside that per-parent batch. For fixed x0, self-normalization estimates E_{p_{t|0}(·|x0)}[w f] / E_{p_{t|0}(·|x0)}[w], not E_{p_{t|0}(·|x0)}[w f]. Since E_{p_{t|0}(·|x0)}[w] is generally not 1 and varies with x0, the batch-averaged loss in Eq. (16) is not an estimator of Eq. (14) even as S grows; the per-parent normalizer is data-dependent through w = pt/pB_t whenever the buffer misrepresents a mode. The paper provides no argument that this denominator cancels or equals one, so Proposition 3's bias and MSE bounds do not apply to the trained objective. This gap is load-bearing because the abstract and Section 3.4 claim that the practical loss is theoretically analyzed.
  2. [Appendix E.3, Assumptions A1–A2; Appendix F.1] The regularity assumptions behind Propositions 2 and 3 are not satisfied by the Lennard-Jones benchmarks. Proposition 2 assumes ∇E is Lipschitz with bounded norm, while the LJ energy (Appendix F.1, Eq. (33)) has unbounded gradients as the interparticle distance dij tends to zero; the score estimator SL in Eq. (7) contains terms of the form -exp(-E)∇E, so its variance is not uniformly bounded. Assumptions A1 and A2 in Appendix E.3 require uniform bounds on E_{S_L}[||sθ - S_L||^2] and on ||sθ - ∇log pt||, which are not verified and are implausible for LJ-13 and LJ-55. Since Table 2 and the claims in Section 5.2 include these systems, the theoretical analysis does not extend to the full experimental scope. The paper should either restrict its theoretical claims to settings where the assumptions hold or supply a separate argument for singular potentials.
  3. [Section 3.4, Eq. (11) and Algorithm 1] The sample-reuse strategy for the denominator estimate D_M is not covered by Proposition 3. In Algorithm 1, for a parent x0^(b), the denominator D_M in the weight ilde w(x_{t|0}^{(b,s)}) sums p_{t|0}(x_{t|0}^{(b,s)} | x0^(j)) over j = 1,...,B including j = b, while the numerator N_K uses fresh samples from N(x0; x_t, σ_t^2 I). Proposition 3 treats the buffer draws for the denominator as independent of the state x_t, but here the state x_t is generated from one of the same buffer draws that appear in the denominator. This coupling can bias the estimated weights in a way that the current bounds do not control, and the paper does not analyze it. Please either sample the denominator from a separate buffer batch or prove that the reuse leaves the bias and MSE bounds unchanged.
minor comments (4)
  1. [Section 3.3, Eq. (9)] The notation {x_{0|t}^{(i)}} is used for samples from N(x0; xt, σ_t^2 I), but the conditional distribution p_{0|t}(x0|xt) is never formally defined in the main text; please introduce it explicitly or use a neutral symbol such as q(x0|xt).
  2. [Table 2] The DiKL row for LJ-55 reads `∗ ∗ ∗ ∗iDEM` with no separator; this formatting glitch makes the table difficult to parse and should be corrected.
  3. [Appendix H] The text alternates between 'SwES' and 'DwES' (Tables 3 and 4, Figure 7, and the surrounding discussion); please use one consistent name for the direct estimated-score sampling baseline.
  4. [References] Reference [78] contains a typo in the page range ('279–29 Q–learning'); please fix the citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained and the claimed objective is not equivalent to its inputs by construction.

full rationale

The derivation chain is self-contained rather than circular. Proposition 1 obtains the score-matching objective from the Girsanov-based path-KL identity, an external mathematical fact; Eq. (5) is not defined in terms of the model's outputs. Eq. (14) then applies the standard importance-sampling identity E_{pB_t}[w f] = E_{pt}[f] with w = pt/pB_t, so the weighted objective is algebraically the same as the ideal objective by construction—this is a legitimate equivalence, not a circular one. The Monte Carlo estimators in Eqs. (7), (9)-(11) use only the known energy E and the forward kernel, not the trained score. Proposition 3 bounds the SNIS loss using external SNIS results plus Proposition 2's score-estimator bounds; the assumptions do not include the proposition's conclusion. There is no load-bearing self-citation chain: the cited score estimator from [2] is used as a component and is re-analyzed, not invoked to forbid alternatives. The one substantive gap is that the implemented per-parent normalization in Eq. (15) is not exactly the marginal-batch SNIS estimator analyzed in Proposition 3; that is a correctness or soundness risk about whether the theory covers the algorithm, not a case of a prediction being equivalent to its inputs by construction. Since no fitted parameter is renamed as a prediction and no claimed result reduces by definition to an input, no circular step is established.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the path-KL equivalence (standard), on bounded-energy and bounded-gradient assumptions that fail for the Lennard-Jones benchmark, and on an unstated equivalence between per-parent and per-batch SNIS normalization. No new physical entities are introduced.

free parameters (4)
  • SNIS sample count S = 2 to 10 (5 on GMMs, 2 on DW-4)
    Chosen per benchmark without a fitting procedure; affects the bias-variance tradeoff in Proposition 3.
  • Score estimator sample count L = 500 (GMM), 1000 (DW-4, LJ-13), 100 (LJ-55)
    Number of Monte Carlo samples for the score target SL; the bias bound in Proposition 2 depends on 1/sqrt(L).
  • Replay buffer size = 10k/20k/10k for GMM-40/80/120
    Buffer size controls proposal quality pB_0; chosen manually.
  • VE-SDE noise schedule (sigma_min, sigma_max) = e.g., (1e-5, 1) for GMM, (0.5, 4) for LJ-55
    Noise schedule is a design choice for each benchmark; the theory assumes a fixed VE-SDE.
assumptions (5)
  • standard math Girsanov theorem and Novikov condition for path measure equivalence (Proposition 1).
    Section E.1 assumes the exponential martingale is a true martingale; standard in diffusion theory.
  • domain assumption E(x) bounded below and ∇E(x) Lipschitz with bounded norm (Proposition 2).
    Used to bound the score estimator; not satisfied by Lennard-Jones potentials with singularities.
  • ad hoc to paper Uniform boundedness of the inner loss and model error (Assumptions A1/A2 in Appendix E.3).
    These assumptions are introduced to make Proposition 3 hold; they are not verified in experiments.
  • ad hoc to paper Finite second moment of the estimated importance weights (Assumption A3).
    Needed for self-normalized importance sampling bounds; may fail when the buffer poorly covers a mode.
  • ad hoc to paper The batch samples in Eq. (15) are treated as i.i.d. from pB_t for the purpose of SNIS normalization.
    Section 3.4 normalizes weights across samples conditioned on a single x0, which is not equivalent to sampling from pB_t; this assumption is unstated and likely false.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coverage." pith.science (2026). https://pith.science/paper/TYIYJ4JE

@misc{pith2026250519431,
  author       = {Pith},
  title        = {Pith review of: Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coverage},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYIYJ4JE}},
  note         = {Machine review of arXiv:2505.19431}
}
read the original abstract

Training neural samplers directly from unnormalized densities without access to target distribution samples presents a significant challenge. A critical desideratum in these settings is achieving comprehensive mode coverage, ensuring the sampler captures the full diversity of the target distribution. However, prevailing methods often circumvent the lack of target data by optimizing reverse KL-based objectives. Such objectives inherently exhibit mode-seeking behavior, potentially leading to incomplete representation of the underlying distribution. While alternative approaches strive for better mode coverage, they typically rely on implicit mechanisms like heuristics or iterative refinement. In this work, we propose a principled approach for training diffusion-based samplers by directly targeting an objective analogous to the forward KL divergence, which is conceptually known to encourage mode coverage. We introduce \textit{Importance Weighted Score Matching}, a method that optimizes this desired mode-covering objective by re-weighting the score matching loss using tractable importance sampling estimates, thereby overcoming the absence of target distribution data. We also provide theoretical analysis of the bias and variance for our proposed Monte Carlo estimator and the practical loss function used in our method. Experiments on increasingly complex multi-modal distributions, including 2D Gaussian Mixture Models with up to 120 modes and challenging particle systems with inherent symmetries -- demonstrate that our approach consistently outperforms existing neural samplers across all distributional distance metrics, achieving state-of-the-art results on all benchmarks.

Figures

Figures reproduced from arXiv: 2505.19431 by the authors.

Figure 1
Figure 1. Visualization of samples from different methods across GMM benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Energy distributions for different molecular systems. Further visualizations for interatomic [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of training and testing per￾formance on the GMM-40. Left: Training loss tra￾jectories for different importance sampling strate￾gies. Right: Evaluation of trained models using 1-Wasserstein (circles) and 2-Wasserstein (trian￾gles) distance metrics. In the left panel, all SNIS variants (2, 5, 10 pro￾posal samples) achieve significantly lower train￾ing losses than the baseline (e.g., SNIS-5/10 around 2.2, SN… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Energy and Interatomic Distance distributions for different molecular systems. [PITH_FULL_IMAGE:figures/full_fig_p029_4.png]
Figure 5
Figure 5. Figure 5: Comparison of training and testing performance on the GMM-80 and GMM-120. Left: [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]
Figure 6
Figure 6. Figure 6: Illustration of Forward vs. Reverse KL divergence behavior on a 1D bimodal Gaussian [PITH_FULL_IMAGE:figures/full_fig_p031_6.png]
Figure 7
Figure 7. Figure 7: Visualization of samples from direct sampling with estimated scores across GMM bench [PITH_FULL_IMAGE:figures/full_fig_p033_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Adaptive External Communication in Autonomous Vehicles: A Conceptual Design Framework

    cs.HC 2025-08 unverdicted novelty 5.0 of 10

    A three-layer framework (input, processing, output) for adaptive external human-machine interfaces in autonomous vehicles is introduced to systematize design and analysis.

Reference graph

Works this paper leans on

84 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [1]

    Impor- tance sampling: Intrinsic dimension and computational cost

    Sergios Agapiou, Omiros Papaspiliopoulos, Daniel Sanz-Alonso, and Andrew M Stuart. Impor- tance sampling: Intrinsic dimension and computational cost. Statistical Science, pages 405–431, 2017

    2017

  2. [2]

    Iterated denoising energy matching for sampling from boltzmann densities

    Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, et al. Iterated denoising energy matching for sampling from boltzmann densities. arXiv preprint arXiv:2402.06121, 2024

    arXiv 2024

  3. [3]

    Flow-based generative models for markov chain monte carlo in lattice field theory

    Michael S Albergo, Gurtej Kanwar, and Phiala E Shanahan. Flow-based generative models for markov chain monte carlo in lattice field theory. Physical Review D, 100(3):034515, 2019

    2019

  4. [4]

    Nets: A non-equilibrium transport sampler

    Michael S Albergo and Eric Vanden-Eijnden. Nets: A non-equilibrium transport sampler. arXiv preprint arXiv:2410.02711, 2024

    arXiv 2024

  5. [5]

    Reverse-time diffusion equation models

    Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982

    1982

  6. [6]

    Gflownet foundations

    Yoshua Bengio, Tristan Deleu, Edward J Hu, Salem Lahlou, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. arXiv preprint arXiv:2111.09266, 2021

    arXiv 2021

  7. [7]

    Nearly d- linear convergence bounds for diffusion models via stochastic localization

    Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d- linear convergence bounds for diffusion models via stochastic localization. arXiv preprint arXiv:2308.03686, 2023

    arXiv 2023

  8. [8]

    An optimal control perspective on diffusion- based generative modeling

    Julius Berner, Lorenz Richter, and Karen Ullrich. An optimal control perspective on diffusion- based generative modeling. arXiv preprint arXiv:2211.01364, 2022

    arXiv 2022

Show all 84 references
  1. [9]

    Pattern recognition and machine learning, volume 4

    Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  2. [10]

    SE(3)-stochastic flow matching for protein backbone generation

    Avishek Joey Bose, Tara Akhound-Sadegh, Kilian Fatras, Guillaume Huguet, Jarrid Rector- Brooks, Cheng-Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, and Alexander Tong. SE(3)-stochastic flow matching for protein backbone generation. International Conferenc...

    2024

  3. [11]

    Bayesian inference in statistical analysis

    George EP Box and George C Tiao. Bayesian inference in statistical analysis. John Wiley & Sons, 2011

    2011

  4. [12]

    Introduction to Rare Event Simulation

    James A Bucklew. Introduction to Rare Event Simulation. Springer Science & Business Media, 2004

    2004

  5. [13]

    Adaptive importance sampling in general mixture classes

    Olivier Cappé, Randal Douc, Arnaud Guillin, Jean-Michel Marin, and Christian P Robert. Adaptive importance sampling in general mixture classes. Statistics and Computing, 18:447– 459, 2008

    2008

  6. [14]

    Br-snis: bias reduced self-normalized importance sampling

    Gabriel Cardoso, Sergey Samsonov, Achille Thin, Eric Moulines, and Jimmy Olsson. Br-snis: bias reduced self-normalized importance sampling. Advances in Neural Information Processing Systems, 35:716–729, 2022

    2022

  7. [15]

    Sequential controlled langevin diffusions

    Junhua Chen, Lorenz Richter, Julius Berner, Denis Blessing, Gerhard Neumann, and Anima Anandkumar. Sequential controlled langevin diffusions. arXiv preprint arXiv:2412.07081, 2024

    2024 arXiv

  8. [16]

    The probability flow ode is provably fast.Advances in Neural Information Processing Systems, 36:68552–68575, 2023

    Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast.Advances in Neural Information Processing Systems, 36:68552–68575, 2023

    2023

  9. [17]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. arXiv preprint arXiv:2209.11215, 2022. 11

    2022 arXiv

  10. [18]

    Adaptive multiple importance sampling

    JEAN-MARIE CORNUET, JEAN-MICHEL MARIN, Antonietta Mira, and Christian P Robert. Adaptive multiple importance sampling. Scandinavian Journal of Statistics, 39(4):798–812, 2012

    2012

  11. [19]

    Sequential Monte Carlo samplers

    Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3):411–436, 2006

    2006

  12. [20]

    Density estimation using Real NVP

    Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using Real NVP. International Conference on Learning Representations (ICLR), 2017

    2017

  13. [21]

    Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H

    Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, Ievgen Redko, Antoine Rolet, Ant...

    2021

  14. [22]

    Understanding molecular simulation: from algorithms to applications

    Daan Frenkel and Berend Smit. Understanding molecular simulation: from algorithms to applications. Elsevier, 2023

    2023

  15. [23]

    MCMC variational inference via uncorrected Hamiltonian annealing

    Tomas Geffner and Justin Domke. MCMC variational inference via uncorrected Hamiltonian annealing. Neural Information Processing Systems (NeurIPS), 2021

    2021

  16. [24]

    Sampling-based approaches to calculating marginal densities

    Alan E Gelfand and Adrian FM Smith. Sampling-based approaches to calculating marginal densities. Journal of the American statistical association, 85(410):398–409, 1990

    1990

  17. [25]

    Neural adaptive sequential monte carlo

    Shixiang Shane Gu, Zoubin Ghahramani, and Richard E Turner. Neural adaptive sequential monte carlo. Advances in neural information processing systems, 28, 2015

    2015

  18. [26]

    Training neural samplers with reverse diffusive kl divergence.arXiv preprint arXiv:2410.12456, 2024

    Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and José Miguel Hernández-Lobato. Training neural samplers with reverse diffusive kl divergence.arXiv preprint arXiv:2410.12456, 2024

    2024 arXiv

  19. [27]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Neural Information Processing Systems (NeurIPS), 2020

    2020

  20. [28]

    The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo

    Matthew D Hoffman, Andrew Gelman, et al. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res., 15(1):1593–1623, 2014

    2014

  21. [29]

    Schrödinger-föllmer sampler

    Jian Huang, Yuling Jiao, Lican Kang, Xu Liao, Jin Liu, and Yanyan Liu. Schrödinger-föllmer sampler. IEEE Transactions on Information Theory, 2024

    2024

  22. [30]

    Highly accurate protein structure prediction with alphafold

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021

    2021

  23. [31]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems , 35:26565–26577, 2022

    2022

  24. [32]

    Equivariant flow matching

    Leon Klein, Andreas Krämer, and Frank Noé. Equivariant flow matching. Neural Information Processing Systems (NeurIPS), 2023

    2023

  25. [33]

    Rigid body flows for sampling molecular crystal structures

    Jonas Köhler, Michele Invernizzi, Pim De Haan, and Frank Noé. Rigid body flows for sampling molecular crystal structures. International Conference on Machine Learning (ICML), 2023

    2023

  26. [34]

    Equivariant flows: exact likelihood generative learning for symmetric densities

    Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: exact likelihood generative learning for symmetric densities. International Conference on Machine Learning (ICML), 2020

    2020

  27. [35]

    A theory of continuous generative flow networks

    Salma Lahlou, Tristan Deleu, Pablo Lemos, Dinghuai Zhang, Anastasiia V olokhova, Alexandre Hernández-Garcia, Lounes N Ezzine, Yoshua Bengio, and Nikolay Malkin. A theory of continuous generative flow networks. International Conference on Machine Learning (ICML), 2023. 12

    2023

  28. [36]

    Markov chains and mixing times , volume 107

    David A Levin and Yuval Peres. Markov chains and mixing times , volume 107. American Mathematical Soc., 2017

    2017

  29. [37]

    Towards non-asymptotic convergence for diffusion-based generative models

    Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  30. [38]

    Gradient estimators for implicit models

    Yingzhen Li and Richard E Turner. Gradient estimators for implicit models. arXiv preprint arXiv:1705.07107, 2017

    2017 arXiv

  31. [39]

    Entropy-based training methods for scalable neural implicit samplers

    Weijian Luo, Boya Zhang, and Zhihua Zhang. Entropy-based training methods for scalable neural implicit samplers. Advances in Neural Information Processing Systems, 36:7137–7157, 2023

    2023

  32. [40]

    Learning gflownets from partial episodes for improved convergence and stability

    Kanika Madan, Jarrid Rector-Brooks, Maksym Korablyov, Emmanuel Bengio, Moksh Jain, Andreea Nica, Tristan Bosc, Yoshua Bengio, and Nikolay Malkin. Learning gflownets from partial episodes for improved convergence and stability. InInternational Conference on Machine Learning (IC...

    2022

  33. [41]

    Gflownet

    Chris J Maddison, Laurent Dinh, Etienne Paulus, and Yoshua Bengio. Gflownet. arXiv preprint arXiv:2106.04399, 2021

    2021 arXiv

  34. [42]

    Trajectory balance: Improved credit assignment in gflownets

    Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chenghao Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems (NeurIPS), 35:14651–14664, 2022

    2022

  35. [43]

    Gflownets and variational inference

    Nikolay Malkin, Salma Lahlou, Tristan Deleu, Xuezhou Ji, Edward Hu, Kyle Everett, Dinghuai Zhang, and Yoshua Bengio. Gflownets and variational inference. International Conference on Learning Representations (ICLR), 2023

    2023

  36. [44]

    Independent doubly adaptive rejection metropolis sampling within gibbs sampling

    Luca Martino, Jesse Read, and David Luengo. Independent doubly adaptive rejection metropolis sampling within gibbs sampling. IEEE Transactions on Signal Processing, 63(12):3123–3138, 2015

    2015

  37. [45]

    Equation of state calculations by fast computing machines

    Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953

    1953

  38. [46]

    SE(3) equivariant augmented coupling flows

    Laurence I Midgley, Vincent Stimper, Javier Antorán, Emile Mathieu, Bernhard Schölkopf, and José Miguel Hernández-Lobato. SE(3) equivariant augmented coupling flows. Neural Information Processing Systems (NeurIPS), 2023

    2023

  39. [47]

    Flow annealed importance sampling bootstrap

    Laurence Illing Midgley, Vincent Stimper, Gregor NC Simm, Bernhard Schölkopf, and José Miguel Hernández-Lobato. Flow annealed importance sampling bootstrap. International Conference on Learning Representations (ICLR), 2023

    2023

  40. [48]

    Playing atari with deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  41. [49]

    Neural importance sampling

    Thomas Müller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. Neural importance sampling. ACM Transactions on Graphics (ToG), 38(5):1–19, 2019

    2019

  42. [50]

    Annealed importance sampling

    Radford M Neal. Annealed importance sampling. Statistics and computing, 11:125–139, 2001

    2001

  43. [51]

    MCMC using Hamiltonian dynamics

    Radford M Neal et al. MCMC using Hamiltonian dynamics. Handbook of Markov chain Monte Carlo, 2(11):2, 2011

    2011

  44. [52]

    Asymptotically unbiased estimation of physical observables with neural samplers

    Kim A Nicoli, Shinichi Nakajima, Nils Strodthoff, Wojciech Samek, Klaus-Robert Müller, and Pan Kessel. Asymptotically unbiased estimation of physical observables with neural samplers. Physical Review E, 101(2):023304, 2020

    2020

  45. [53]

    Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning

    Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457):eaaw1147, 2019. 13

    2019

  46. [54]

    Machine learning for molecular simulation

    Frank Noé, Alexandre Tkatchenko, Klaus-Robert Müller, and Cecilia Clementi. Machine learning for molecular simulation. Annual review of physical chemistry, 71(1):361–390, 2020

    2020

  47. [55]

    Adaptive importance sampling in monte carlo integration

    Man-Suk Oh and James O Berger. Adaptive importance sampling in monte carlo integration. Journal of statistical computation and simulation, 41(3-4):143–168, 1992

    1992

  48. [56]

    Art B. Owen. Monte Carlo theory, methods and examples. https://artowen.su.domains/ mc/, 2013

    2013

  49. [57]

    Monte Carlo theory, methods and examples, volume 1

    Art B Owen. Monte Carlo theory, methods and examples, volume 1. Springer, 2013

    2013

  50. [58]

    Statistical aspects of wasserstein distances

    Victor M Panaretos and Yoav Zemel. Statistical aspects of wasserstein distances. Annual review of statistics and its application, 6(1):405–431, 2019

    2019

  51. [59]

    Parallel markov chain monte carlo simulations

    Ruichao Ren and G Orkoulas. Parallel markov chain monte carlo simulations. The Journal of chemical physics, 126(21), 2007

    2007

  52. [60]

    Variational inference with normalizing flows

    Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. Interna- tional Conference on Machine Learning (ICML), 2015

    2015

  53. [61]

    Monte Carlo Statistical Methods

    Christian P Robert and George Casella. Monte Carlo Statistical Methods. Springer, 2013

    2013

  54. [62]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022

    2022 arXiv

  55. [63]

    Scalable discrete diffusion samplers: Combinatorial opti- mization and statistical physics

    Sebastian Sanokowski, Wilhelm Berghammer, Martin Ennemoser, Haoyu Peter Wang, Sepp Hochreiter, and Sebastian Lehner. Scalable discrete diffusion samplers: Combinatorial opti- mization and statistical physics. arXiv preprint arXiv:2502.08696, 2025

    2025 arXiv

  56. [64]

    Improved off-policy training of diffusion samplers

    Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, and Nikolay Malkin. Improved off-policy training of diffusion samplers. Advances in Neural Information Processing Systems, 37:81016–81045, 2024

    2024

  57. [65]

    Kernel implicit variational inference

    Jiaxin Shi, Shengyang Sun, and Jun Zhu. Kernel implicit variational inference. arXiv preprint arXiv:1705.10119, 2017

    2017 arXiv

  58. [66]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023

    2023

  59. [67]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019

    2019

  60. [68]

    Sliced score matching: A scalable approach to density and score estimation

    Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in artificial intelligence , pages 574–584. PMLR, 2020

    2020

  61. [69]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations (ICLR), 2021

    2021

  62. [70]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  63. [71]

    Monte Carlo variational auto-encoders

    Achille Thin, Nikita Kotelevskii, Arnaud Doucet, Alain Durmus, Eric Moulines, and Maxim Panov. Monte Carlo variational auto-encoders. International Conference on Machine Learning (ICML), 2021

    2021

  64. [72]

    Importance sampling: a review

    Surya T Tokdar and Robert E Kass. Importance sampling: a review. Wiley Interdisciplinary Reviews: Computational Statistics, 2(1):54–60, 2010

    2010

  65. [73]

    Denoising diffusion samplers

    Francisco Vargas, Will Grathwohl, and Arnaud Doucet. Denoising diffusion samplers. Interna- tional Conference on Learning Representations (ICLR), 2023

    2023

  66. [74]

    Transport meets variational inference: Controlled monte carlo diffusions

    Francisco Vargas, Shreyas Padhy, Denis Blessing, and Nikolas Nüsken. Transport meets variational inference: Controlled monte carlo diffusions. arXiv preprint arXiv:2307.01050, 2023. 14

    2023 arXiv

  67. [75]

    The wasserstein distances

    Cédric Villani and Cédric Villani. The wasserstein distances. Optimal transport: old and new, pages 93–111, 2009

    2009

  68. [76]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674, 2011

    2011

  69. [77]

    High-dimensional statistics: A non-asymptotic viewpoint , volume 48

    Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge university press, 2019

    2019

  70. [78]

    Q-learning

    Christopher JCH Watkins. Q-learning. Machine learning, 8(3-4):279–29 Q–learning, 1992

    1992

  71. [79]

    Targeted free energy estimation via learned mappings

    Peter Wirnsberger, Andrew J Ballard, George Papamakarios, Stuart Abercrombie, Sébastien Racanière, Alexander Pritzel, Danilo Jimenez Rezende, and Charles Blundell. Targeted free energy estimation via learned mappings. The Journal of Chemical Physics, 153(14), 2020

    2020

  72. [80]

    Stochastic normalizing flows

    Hao Wu, Jonas Köhler, and Frank Noé. Stochastic normalizing flows. Neural Information Processing Systems (NeurIPS), 2020

    2020

  73. [81]

    Diffusion generative flow samplers: Improving learning signals through partial trajectory optimization

    Dinghuai Zhang, Ricky TQ Chen, Cheng-Hao Liu, Aaron Courville, and Yoshua Bengio. Diffusion generative flow samplers: Improving learning signals through partial trajectory optimization. International Conference on Learning Representations (ICLR), 2024

    2024

  74. [82]

    Variational f- divergence minimization

    Mingtian Zhang, Thomas Bird, Raza Habib, Tianlin Xu, and David Barber. Variational f- divergence minimization. arXiv preprint arXiv:1907.11891, 2019

    1907 arXiv

  75. [83]

    Path integral sampler: a stochastic control approach for sampling

    Qinsheng Zhang and Yongxin Chen. Path integral sampler: a stochastic control approach for sampling. International Conference on Learning Representations (ICLR), 2022. 15 A Related Works Neural Samplers. Several neural sampling approaches have emerged for approximating complex ...

    2022

  76. [84]

    Future Work,

    (this follows from E[X 2] = Var(X) + (E[X])2 for each component). ˆσ2∗ = σ2 + 0.5(µ2 1 + µ2 2) − µ1 + µ2 2 2 = σ2 + (µ1 − µ2)2 4 . 31 Reverse KL Optimization (KL(q || p)). Minimizing KL(q || p) = R q(x; µ, ˆσ2) log q(x;µ,ˆσ2) p(x) dx with respect to µ and ˆσ2. The optimal cond...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.