Pith. sign in

REVIEW 3 major objections 5 minor 13 references

Particle-based Generalised Stochastic Optimisation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single coupled mean-field system unifies particle-based stochastic optimisers and, under one joint contractivity condition, contracts exponentially to the optimum with an explicit finite-particle error bound.

desk verdict A clean, honest unifying framework for coupled sampling-optimisation; the theory contract is conditional and unverified for the paper's own algorithms, but the contribution merits refereeing. read the letter →

arxiv 2608.02844 v1 pith:T4WKDRXD submitted 2026-08-03 stat.ML cs.LGstat.CO

Jiechen Jackie Zhang , O. Deniz Akyildiz This is my paper · ORCID
classification stat.MLcs.LGstat.CO
keywords interactingparticlesystemsmean-fielddynamicsstochasticoptimisationintractablegradientsenergy-basedmodelsmaximummarginallikelihoodestimationjointcontractivitynon-asymptoticerrorbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a family of stochastic optimisation algorithms—those whose gradient is an expectation under a parameter-dependent distribution, $\nabla_\theta \ell(\theta)=\mathbb{E}_{\pi_\theta}[F(\theta,X)]$—can be analysed as one coupled mean-field system in which an optimiser state and a cloud of sampling particles evolve together. The intended payoff is practical: instead of running an inner MCMC loop at every optimisation step, a single interacting particle system persists through training, and existing methods such as particle gradient descent, momentum particle gradient descent, and persistent contrastive divergence appear as special cases of one mechanism. Under a joint contractivity condition coupling the optimiser map, the gradient observation, and the sampler's drift and diffusion, the mean-field law contracts exponentially to the stationary pair $(\vartheta^\star,\nu_{\theta^\star})$, and the $N$-particle approximation carries an explicit finite-time error of order $N^{-1/2}$. The paper also instantiates the framework with a higher-order Langevin sampler for image deblurring and a momentum particle optimiser for energy-based models, reporting faster convergence than the respective baselines.

What carries the argument

The load-bearing object is the coupled mean-field system (3)–(4) together with the joint contractivity Assumption A3. A3 states that for every pair of optimiser states and sampler laws, the combined discrepancy—the optimiser update disagreement plus the sampler drift and diffusion disagreement, measured in weighted Euclidean and Wasserstein-like norms—is bounded above by $-\rho$ times the squared distance between the two states. This condition is genuinely joint: the paper's Appendix B.1 gives a counterexample where the exact-gradient optimiser and each frozen sampler are separately dissipative yet the coupled first moments grow exponentially, so cross-coupling must be controlled. Appendix B then supplies verifiable routes to A3: a separated small-gain criterion combining individual dissipativity and Lipschitz constants, and a projected-field criterion under strong monotonicity of $\Phi=(F,\nabla_xE)$, with a concrete hypocoercive verification for heavy-ball optimisers on underdamped samplers.

What would settle it

Take any instance satisfying Assumptions A1–A5, numerically verify inequality (10) with an estimated $\rho>0$, then simulate the particle system and check whether $\mathbb{E}\|\theta^N_t-\theta^\star\|$ stays below the Theorem 4 bound over a long horizon with, say, $N=10^3$; a single instance where the inequality holds but the observed error exceeds the bound would point to a gap in the proof or an unstated regularity condition, and an instance where the error grows despite the inequality would refute the theorem.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the optimiser and the sampler need not be analysed separately: the pair is a single contracting dynamical system. The authors define the mean-field system $d\vartheta_t=\Psi_t(\vartheta_t,G_{\nu_t}(\vartheta_t))\,dt$ and $dZ_t=b_{\Pi_\theta(\vartheta_t)}(Z_t)\,dt+\sigma_{\Pi_\theta(\vartheta_t)}(Z_t)\,dB_t$, where $G_{\nu_t}$ is the expectation of $F$ under the current sampler law. When the optimiser map has a fixed point $\vartheta^\star$ at zero gradient and the frozen sampler has invariant law $\nu_{\theta^\star}$ with X-marginal $\pi_{\theta^\star}$, the product law $\delta_{\vartheta^\star}\otimes\nu_{\theta^\star}$ is stationary (Theorem 1). The key result (Theorem 2) is that Assumption A3—a joint contractivity inequality in twisted metrics with rate $\rho>0$—makes every pair of solutions contract exponentially, so the stationary pair is unique among product laws with finite second moment. Theorems 3 and 4 translate this into a bound on the particle system: $\mathbb{E}\|\theta^N_t-\theta^\star\| \le \sqrt{\kappa}e^{-\rho t}d_0 + (L_\Psi/\rho)\sqrt{\kappa V_F/N}(1-e^{-\rho t})$, which implies the double limit $\lim_{N\to\infty}\limsup_{t\to\infty}\mathbb{E}\|\theta^N_t-\theta^\star\|=0$.

Load-bearing premise

The entire convergence argument rests on Assumption A3, a joint contractivity condition requiring the optimiser, the gradient estimate, and the sampler to dissipate together at a rate $\rho>0$; separate stability of each component does not suffice, and the paper does not verify A3 for the neural-network energy or the total-variation posterior used in its experiments.

Editorial extensions

If this is right

  • Any existing or new algorithm that fits (3)–(4) inherits exponential mean-field contraction and the $N^{-1/2}$ particle error bound as soon as A3 is verified, so particle gradient descent, momentum particle gradient descent, and related persistent-particle schemes receive a unified convergence certificate.
  • The parameter error bound shows that for fixed $N$ the error does not vanish below $O(L_\Psi/\rho)\sqrt{\kappa V_F/N}$, so a user can trade particle count against a target tolerance without waiting for an inner sampler to equilibrate.
  • Because the optimiser map and the sampler diffusion enter only through the joint inequality, optimiser choices such as Adam-style auxiliary states and sampler choices such as underdamped or higher-order Langevin dynamics are interchangeable components of one theorem rather than separate analyses.
  • For the two new instantiations, the reported numerical evidence indicates that higher-order Langevin sampling improves image deblurring reconstructions and that momentum acceleration lowers Fisher divergence faster than persistent contrastive divergence under matched gradient budgets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to localise A3: if contractivity holds only inside a basin around $\theta^\star$, the same argument should yield exponential attraction from that basin, which would cover the non-convex objectives the paper explicitly leaves open.
  • The discretisation gap is the next bottleneck: the paper bounds the continuous-time system and only sketches how a step-size error would enter, so a combined discretisation-aware bound is the direct route from theorem to implemented algorithm.
  • The Appendix B.1 counterexample suggests a practical diagnostic: before using any coupled particle optimiser, one can compute the small-gain matrix from the separated criterion and check its positive definiteness; a negative result warns that the coupled system may diverge even when each component looks stable.
  • Since the data distribution enters only through the positive-phase average, the same machinery could extend to score-based and latent-variable training objectives beyond maximum marginal likelihood and energy-based models, including the fine-tuning setting mentioned in the introduction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a continuous-time mean-field framework, Eqs. (3)-(4), for stochastic optimisation problems whose loss gradient is an expectation under a parameter-dependent distribution. An optimiser map Ψ_t is coupled to a sampler whose invariant X-marginal is π_θ, and the framework is shown to contain PGD, MPGD, ImpDiff, and related methods as special cases. Under Assumptions A1-A3, the authors prove exponential contraction of the mean-field system to a stationary law δ_{ϑ⋆}⊗ν_{θ⋆} (Theorem 2). For the interacting particle system (8)-(9), they prove a finite-particle interaction error bound (Theorem 3) and a parameter error bound (Theorem 4) under A4-A5. The paper then proposes two instantiations: HOL-MYPGD for MMLE with a total-variation prior, and M-MFPO for training energy-based models, with numerical experiments on image deblurring and synthetic 2D EBMs. The limitations section (6.1) explicitly acknowledges that the theory relies on global contractivity and that discretisation analysis is left for future work.

Significance. If the conditional theorems are correct, the framework is a useful unification: it abstracts the optimiser-sampler coupling used across PGD, MPGD, and ImpDiff, and it provides clean continuous-time contraction and finite-particle bounds under a joint contractivity condition A3. The proofs in the appendix are self-contained and the statements of Theorems 2-4 are explicit. The paper also delivers concrete new algorithm variants and an honest limitations section. However, the significance is currently limited by the fact that A3 is not verified for any nontrivial instantiation, and the implemented algorithms are discrete-time splittings that are not covered by the continuous-time guarantees. The value of the paper at this stage is primarily the framework and its conditional analysis; the experimental claims remain heuristic until the gap between assumptions and practice is closed or explicitly reframed.

major comments (3)
  1. [Section 4, Eq. (10); Appendix B] Assumption A3 is the load-bearing condition for Theorems 2-4, but it is never verified for any nontrivial problem. The sufficient conditions in Appendix B require strong monotonicity of the projected field Φ=(F,∇_x E) with rate ρ_Φ>0 and global Lipschitz continuity (A9, Corollary 2, Proposition B.1). For the TV-prior MMLE objective in Section 5.1, TV is only 1-homogeneous and non-differentiable, and the quadratic term ∥y-Bx∥^2 may be weak or rank-deficient, so neither global Lipschitzness nor strong monotonicity of Φ is established; for the neural EBM in Section 5.2.1, no verification is attempted. Since Theorems 2-4 and the claim in the abstract that 'a large class of algorithms inherit these guarantees' depend on A3, the theoretical results do not currently cover HOL-MYPGD or M-MFPO. The paper should either verify A3 in at least one concrete non-trivial setting (e.g., a strongly convex quadratic with Gaussian posterior) and clearly state that the experimental objectives lie outside this regime, or narrow the stated claims accordingly.
  2. [Section 4 (after Theorem 4) and Section 5] The numerical algorithms are defined only through discrete-time splittings: the Mou et al. (2021) higher-order scheme in Eq. (13) and the OBABO-like scheme (24) for M-MFPO. The paper's discussion of time discretisation after Theorem 4 assumes an abstract error bound E∥θ̂_k^{N,h}-θ_{t_k}^N∥ ≤ ε_disc(h,t_k,N) and notes that establishing such rates is left to future work. Consequently, Theorems 2-4 do not directly justify the plotted convergence curves or the comparison results in Figures 1, 3, and 6-10. The authors should either provide a discretisation error and stability analysis for the specific splittings used, or explicitly label all experimental claims as heuristic demonstrations outside the scope of the theoretical guarantees. The current wording in Sections 5.1-5.2 does not make this distinction sufficiently clear.
  3. [Section 1, contribution (C2)] The introduction claims that 'Under an additional conservative, jointly strongly convex structure, the stationary parameter is the unique global minimiser of the loss.' No theorem, proof, or formal statement of this claim appears in the main text or the appendix. Either add a precise statement with assumptions and proof, or remove this contribution from the summary of results.
minor comments (5)
  1. [Section 2.2] Typo: 'continous-time' should be 'continuous-time' in the sentence 'we formulate a continous-time mean-field optimisation-sampling system'.
  2. [Section 5.2.1] Typo: 'Kullback-Liebler' should be 'Kullback-Leibler' in the phrase 'minimising the (forward) Kullback-Liebler divergence'.
  3. [Acknowledgements] Typo: 'EPRSC studentship' should be 'EPSRC studentship'.
  4. [Section 4, Eq. (10)] The notation ∥D_z^{1/2}(σ_{Π_θ(ϑ)}(z)-σ_{Π_θ(˜ϑ)}(˜z))∥_HS is dense; a short clarification that the HS norm is taken on the linear-map space and that the factor 1/2 arises from the Itô correction would improve readability.
  5. [Appendix C.2.3] The text 'we use a set of10, 000particles' and 'ForM-MFPO' have spacing issues; unify the formatting for numbers and method names.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorems 2-4 are conditional derivations from explicit contractivity and variance assumptions; self-citations appear only as special cases and baselines.

full rationale

The paper's central claim is explicitly conditional: if Assumptions A1-A5 hold, then exponential contraction and finite-particle error bounds follow. Assumption A3 (equation (10)) is a joint contractivity condition, and Theorem 2's proof applies it directly via synchronous coupling and Gronwall's lemma, so the exponential contraction conclusion is a derivation from the stated hypothesis rather than a disguised input. Theorems 3 and 4 use Assumptions A4 and A5 together with synchronous couplings to propagate an independent variance bound into an N^{-1/2} interaction error; no parameter is fitted to data and then reported as a prediction. The numerical sections (MMLE deblurring and EBM training) are empirical demonstrations, and Section 6.1 explicitly concedes that the theory is not verified for those non-convex settings: 'The theory relies on global contractivity and uniform-moment assumptions, which can be restrictive for the non-convex objectives encountered in large neural models.' Self-citations such as Akyildiz et al. (2025), Lim et al. (2024), and Encinar et al. (2025) are used to identify special cases and baselines, not to supply the convergence argument; uniqueness of the stationary law is proved inside Theorem 2 rather than imported. The gap that Assumption A3 and the Appendix B sufficient conditions are not checked for the specific experimental instantiations is a correctness or applicability risk, not circularity.

Assumptions & free parameters 6 free parameters · 9 assumptions · 0 invented entities

The central theoretical results rest on unproved regularity and contractivity conditions (A1-A5) that are standard in the interacting-particle and stochastic-approximation literature; they are stated as assumptions rather than derived. In addition, the sampler construction inherits the complete-recipe conditions of Ma et al. (2015), and the deblurring and EBM experiments postulate finite normalisation constants (Cθ < ∞) without verification. The tuned hyperparameters (step sizes, frictions, higher-order Langevin coefficients) are fit to the target problems and affect the reported comparisons, though they are not part of the theorems.

free parameters (6)
  • HOL-MYPGD friction γ = 0.6 (setup A), 0.7 (setup B)
    Grid-searched per setup to maximize SSIM after K iterations on an example image (Appendix C.1.3).
  • HOL-MYPGD higher-order rate α = 4.0 (setup A), 4.2 (setup B)
    Grid-searched jointly with γ; see Appendix C.1.3.
  • M-MFPO step-size h = 0.19 (rings), 0.1 (beads), 0.075 (lattice)
    Tuned per dataset to optimize AOC/FD∞ trade-off (Appendix C.2.3, Table 1).
  • M-MFPO friction γ = 5.5 (rings), 8.0 (beads), 6.0 (lattice)
    Tuned per dataset; see Table 1.
  • lattice learning-rate schedule (h0, hT, L, T) = hT = h0/10, L = 6000, T = 400000, h0 = 0.075
    Linear decay schedule used only for the lattice dataset to damp fluctuations; eq. (27).
  • PCD step-size (baseline tuning) = 0.016 (rings), 0.01 (beads), 0.004 (lattice)
    Tuned for the baseline; affects the fairness of the comparison.
assumptions (9)
  • domain assumption A1: the mean-field system (3)-(4) admits a pathwise unique strong solution for all finite times with P2 initial laws.
    Stated in Section 3.1; standard Lipschitz and linear-growth conditions suffice but are not proven.
  • domain assumption A2: there exists an optimizer fixed point ϑ⋆ with ∇θℓ(θ⋆)=0 and Ψt(ϑ⋆,0)=0, and the frozen sampler at θ⋆ has invariant law νθ⋆ with X-marginal πθ⋆.
    Assumed in Section 3.1; used in Theorem 1 and the θ⋆ part of Theorem 4.
  • domain assumption A3: joint contractivity of the coupled optimizer-sampler system in twisted metrics with rate ρ>0.
    Central assumption of Theorem 2; not verified for the experimental problems.
  • domain assumption A4: Ψt is uniformly Lipschitz in its gradient argument with constant LΨ.
    Assumed in Section 4 for the finite-particle bound.
  • domain assumption A5: uniform bounded variance VF of the stochastic gradient along the mean-field path.
    Assumed in Section 4 for the N^{-1/2} rate.
  • standard math Ma et al. (2015) complete-recipe conditions for the generalized Langevin sampler to have invariant law νθ∝e^{-H}.
    Imported in Section 3.1; the stationary Fokker-Planck, boundary, and regularity conditions are not re-derived.
  • domain assumption Finite normalisation constants Cθ<∞ for the EBM models and TV prior throughout training.
    Assumed in Sections 5.1 and 5.2.3; not checked during training.
  • standard math TV scaling C(θ)∝e^{dxθ} from one-homogeneity of TV and the change of variables u=e^θx.
    Used in Appendix C.1 to derive ∇θℓ(θ) = E[e^θ TV(X)] - dx.
  • standard math Fisher's identity for the MMLE gradient.
    Used in Section 5.1 to write the loss gradient as an expectation under the posterior πθ.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Particle-based Generalised Stochastic Optimisation." pith.science (2026). https://pith.science/paper/T4WKDRXD

@misc{pith2026260802844,
  author       = {Pith},
  title        = {Pith review of: Particle-based Generalised Stochastic Optimisation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T4WKDRXD}},
  note         = {Machine review of arXiv:2608.02844}
}
read the original abstract

We develop a class of diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provides a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate it by developing momentum and higher-order Langevin variants and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.

Figures

Figures reproduced from arXiv: 2608.02844 by the authors.

Figure 1
Figure 1. Evolution of MSE (left) and SSIM (right) of the particle cloud mean across iterations, [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Ground truth (left), blurred image (middle-left), and comparisons to single particle recon [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Comparison of Fisher divergence over iterations for the [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparison of learnt densities and fresh samples from the model for the [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Setup B deblurring image comparison 1. 0 2000 4000 6000 Iteration 14 12 10 8 6 4 estimate 0 2000 4000 6000 Iteration 10 3 MSE Particle-cloud MSE 0 2000 4000 6000 Iteration 0.2 0.4 0.6 SSIM Particle-cloud SSIM MYPGD HOL-MYPGD [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]
Figure 6
Figure 6. Figure 6: Setup B deblurring convergence comparison 1. [PITH_FULL_IMAGE:figures/full_fig_p030_6.png]
Figure 7
Figure 7. Figure 7: Setup B deblurring image comparison 2. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]
Figure 8
Figure 8. Figure 8: Setup B deblurring convergence comparison 2. [PITH_FULL_IMAGE:figures/full_fig_p031_8.png]
Figure 9
Figure 9. Figure 9: Setup B deblurring image comparison 3. 0 2000 4000 6000 Iteration 14 12 10 8 6 4 estimate 0 2000 4000 6000 Iteration 10 3 10 4 MSE Particle-cloud MSE 0 2000 4000 6000 Iteration 0.1 0.2 0.3 0.4 0.5 0.6 0.7 SSIM Particle-cloud SSIM MYPGD HOL-MYPGD [PITH_FULL_IMAGE:figur…
Figure 10
Figure 10. Figure 10: Setup B deblurring convergence comparison 3. [PITH_FULL_IMAGE:figures/full_fig_p031_10.png]
Figure 11
Figure 11. Figure 11: Effect of step-size on convergence (AOC) and converged-value (F D∞) for PCD (left) and contour plot of Fisher divergence across iterations for a range of step-sizes (right) We see in figure 11 increasing step-size leads to a worse convergence and decreasing step-size …
Figure 12
Figure 12. Figure 12: Effect of step-size and friction on the AOC (left) and F D∞ (right). Values in black are lower than the lowest observed corresponding value in the overdamped PCD across all step-sizes (lower is better). hyperparameters for both methods from the sweeps whilst also ensu…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

  1. [1]

    A Unifying and Canonical Description of Measure-Preserving Diffusions

    Akyildiz, Ö. D., Crucinio, F. R., Girolami, M., Johnston, T., and Sabanis, S. (2025). Interacting particle Langevin algorithm for maximum marginal likelihood estimation.ESAIM: Probability and Statistics. Andrieu, C. and Vihola, M. (2014). Markovian stochastic approximation with expanding projections. Bernoulli, 20(2):545–585. Atchadé, Y. F., Fort, G., and...

    work page Pith review arXiv 2025

  2. [2]

    Corollary 2 therefore verifiesA3 in the displayed hypocoercive metrics

    Finally, Aθ +Ax = 2βL2 Φ (1−ε)γ 2 <βρ Φ. Corollary 2 therefore verifiesA3 in the displayed hypocoercive metrics. C Experimental Setting C.1MMLEwith Higher-Order Langevin Dynamics Here we discuss the Bayesian image-deblurring experiments introduced in Section 5.1. Given a blurred and noisy observationy = Bx +ε, obtained through an ill-conditioned blurring ...

    work page 2025

  3. [3]

    0 2000 4000 6000 Iteration 14 12 10 8 6 4 estimate 0 2000 4000 6000 Iteration 103 104 MSE Particle-cloud MSE 0 2000 4000 6000 Iteration 0.1 0.2 0.3 0.4 0.5 0.6 0.7SSIM Particle-cloud SSIM MYPGD HOL-MYPGD Figure 10: Setup B deblurring convergence comparison

    work page 2000

  4. [6]

    21 Proof.Fix arbitraryt≥0,ϑ, ˜ϑ∈Θ×M,ν,˜ν∈P 2(X×V), andΛ∈Γ(ν,˜ν)

    A simpler sufficient condition is ρΨ > κL2 σ 2 + √κ 2 (LΨLF +Lb), ρ b > κL2 σ 2 + √κ 2 (LΨLF +Lb).(19) Consequently, ifA1 also holds, Theorem 2 gives exponential contraction. 21 Proof.Fix arbitraryt≥0,ϑ, ˜ϑ∈Θ×M,ν,˜ν∈P 2(X×V), andΛ∈Γ(ν,˜ν). Define a:=∥ϑ− ˜ϑ∥Dϑ, b:= Z ∥z−˜z∥2 Dz Λ(dz,d˜z) 1/2 . Also write λ− :=λ min(diag(Dϑ,Dz)), λ + :=λ max(diag(Dϑ,Dz)), s...

    work page 2023

  5. [8]

    The gradient is given for allx∈R d by ∇xEλ(θ,x) =∇ xg1(x) + 1 λ(x−prox λ g2(θ,x)), 27 where we treatθ as fixed in the minimisation overRdx

    We denote the fullMYapproximation with Eλ(θ,x ) =g1(θ,x ) +gλ 2 (θ,x ), which is always differentiable and converges pointwise toE(θ,x)in the limitλ→0. The gradient is given for allx∈R d by ∇xEλ(θ,x) =∇ xg1(x) + 1 λ(x−prox λ g2(θ,x)), 27 where we treatθ as fixed in the minimisation overRdx. SinceE(θ,x )is continuously differentiable with respect toθ, we m...

    work page 2025

  6. [9]

    0 2000 4000 6000 Iteration 14 12 10 8 6 4 estimate 0 2000 4000 6000 Iteration 103 MSE Particle-cloud MSE 0 2000 4000 6000 Iteration 0.2 0.4 0.6SSIM Particle-cloud SSIM MYPGD HOL-MYPGD Figure 6: Setup B deblurring convergence comparison

    work page 2000

  7. [10]

    30 0 2000 4000 6000 Iteration 14 12 10 8 6 4 estimate 0 2000 4000 6000 Iteration 103 MSE Particle-cloud MSE 0 2000 4000 6000 Iteration 0.1 0.2 0.3 0.4 0.5 0.6 0.7SSIM Particle-cloud SSIM MYPGD HOL-MYPGD Figure 8: Setup B deblurring convergence comparison

    work page 2000

  8. [12]

    C.2.1M-MFPODiscretisation We begin by constructing a discretisation to provide a concrete algorithm for our framework

    31 C.2 Momentum Mean-Field Particle Optimiser Here, we discuss theM-MFPOalgorithm in Section 5.2, motivate a discretisation and present details on theEBMtraining experiment. C.2.1M-MFPODiscretisation We begin by constructing a discretisation to provide a concrete algorithm for our framework. A naive Euler-Maruyama discretisation ofULDcan have poor stabili...

    work page 2021

Show all 13 references
  1. [13]

    In this limit, bXi n+1 = bXi n− h2 2∇xE(bθn, bXi n) +hξXi,1 n , bθn+1 =bθn− h2 2 ¯F(bθn, bX[N] n )

    Conversely, asγh→∞ , one has η→ 0: the particle momenta are replaced by independent standard Gaussian draws, whereas the parameter momentum is set to zero because theLflow has no noise. In this limit, bXi n+1 = bXi n− h2 2∇xE(bθn, bXi n) +hξXi,1 n , bθn+1 =bθn− h2 2 ¯F(bθn, bX...

    2025

  2. [28]

    Marion, P., Korba, A., Bartlett, P., Blondel, M., De Bortoli, V., Doucet, A., Llinares-López, F., Paquette, C., and Berthet, Q

    Curran Associates, Inc. Marion, P., Korba, A., Bartlett, P., Blondel, M., De Bortoli, V., Doucet, A., Llinares-López, F., Paquette, C., and Berthet, Q. (2025). Implicit diffusion: Efficient optimization through stochastic sampling. InInternational Conference on Artificial Inte...

    2025

  3. [32]

    Oliva, P. F. V. and Akyildiz, O. D. (2024). Kinetic interacting particle Langevin Monte Carlo.arXiv preprint arXiv:2407.05790. Oliva, P. F. V., Akyildiz, O. D., and Duncan, A. (2025). Uniform-in-time convergence bounds for persistent contrastive divergence algorithms.arXiv pre...

    2024 arXiv

  4. [407]

    and Kingma, D

    Song, Y. and Kingma, D. P. (2021). How to train your energy-based models.arXiv preprint arXiv:2101.03288. Tieleman, T. (2008). Training restricted Boltzmann machines using approximations to the likelihood gradient. InProceedings of the 25th international conference on Machine ...

    2021 arXiv

  5. [1259]

    Glyn-Davies, A., Duffin, C., Kazlauskaite, I., Girolami, M., and Akyildiz, Ö. D. (2025). Statistical finite elements via interacting particle Langevin dynamics.SIAM/ASA Journal on Uncertainty Quantification, 13(3):1200–1227. Goldman, J. V., Sell, T., and Singh, S. S. (2022). G...

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.