Pith. sign in

REVIEW 3 major objections 4 minor 29 references

Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read At the same wall-clock cost as a deep ensemble, SBMC matches its accuracy and supplies epistemic uncertainty that deep ensembles miss.

desk verdict Solid empirical contribution with a misleading wall-clock claim: SBMC's cost is roughly double deep ensembles, but the UQ gains on IMDb are real. read the letter →

arxiv 2505.13585 v2 pith:5327WUS4 submitted 2025-05-19 stat.ML cs.LGstat.CO

Xinzhu Liang , Joseph M. Lukens , Sanjaya Lohani , Brian T. Kirby , Thomas A. Searles , Xin Qiu , Kody J. H. Law This is my paper · ORCID
classification stat.MLcs.LGstat.CO MSC 62F1565C0568T07
keywords BayesiandeeplearninguncertaintyquantificationsequentialMonteCarlosamplersHamiltoniananchoredposteriorensemblesepistemicout-of-distributiondetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Scalable Bayesian Monte Carlo (SBMC), a recipe for Bayesian uncertainty in deep networks at the same wall-clock budget as a deep ensemble. Instead of sampling the true posterior, SBMC targets an anchored distribution pulled toward the maximum-a-posteriori weights by a scalar parameter $s$, which makes the target easy to sample when $s$ is small while preserving accuracy. In experiments on MNIST, CIFAR, and IMDb, short parallel MCMC and SMC chains on this anchored target match or beat deep-ensemble accuracy and produce epistemic-entropy scores that separate correct from incorrect and out-of-domain predictions. The paper thereby argues that the gap between principled but slow samplers and fast but heuristic uncertainty estimates can be closed.

What carries the argument

The load-bearing object is the anchored posterior $\pi_s(\theta) \propto L(\theta)\,\mathcal{N}(\theta;\theta_{\mathrm{MAP}}, s v I_d)$, with scalar interpolation parameter $s\in[0,1]$ (the paper uses $\alpha(s)=\mathbf{1}_{s<1/2}$ and $\Sigma(s)=s v I_d$ in the isotropic case). It turns an intractable posterior into a family of targets interpolating between a point mass at the MAP and the full posterior; small $s$ flattens the likelihood relative to the anchored prior, improves Hessian conditioning, and lets short parallel chains mix. The companion machinery is the parallel BMC layer: $P$ independent executions of SMC$_{\parallel}$ or MCMC$_{\parallel}$, each with $N$ particles or chains, combined through a weighted ratio estimator without inter-process communication.

What would settle it

On a small problem where a long gold-standard HMC run is feasible, compute the rank correlation between SBMC's per-input epistemic entropy at the recommended default $s=0.1$ and the gold-standard posterior's over correct, incorrect, and OOD inputs; a weak or negative correlation on the incorrect/OOD subset would falsify the claim that the anchored posterior recovers useful epistemic uncertainty.

Watch

Extended reading notes

Core claim

The paper's central claim is that, for a fixed time budget measured in epochs, targeting the MAP-anchored posterior $\pi_s(\theta) \propto L(\theta)\mathcal{N}(\theta;\theta_{\mathrm{MAP}}, s v I_d)$ with parallel SMC or MCMC recovers essentially the accuracy of the point estimator while supplying an epistemic-uncertainty signal that deep ensembles systematically underestimate. In the reported numbers, SBMC at roughly 160 to 200 epochs reaches accuracy comparable to deep ensembles and, when its posterior features are fed to a confidence meta-classifier, achieves higher F1 and AUC-ROC for detecting misclassifications and out-of-distribution inputs; the long-run gold-standard HMC posterior remains the stated ceiling. The interpolation parameter $s$ is the control knob: $s=0$ collapses to the MAP, $s=1$ is the true posterior, and intermediate values trade a small bias for much faster mixing.

Load-bearing premise

The load-bearing premise is that a single scalar anchoring strength, chosen per dataset and justified only by a heuristic sketch, makes the approximate posterior's epistemic-entropy ranking of mistakes and out-of-distribution inputs trustworthy.

Editorial extensions

If this is right

  • At equal per-chain epoch cost, SBMC matches deep-ensemble accuracy on MNIST7, IMDb, and CIFAR10 while delivering markedly higher epistemic entropy, especially on out-of-distribution inputs.
  • Epistemic entropy from the SBMC posterior ranks individual predictions: incorrect and OOD examples receive higher scores, so the signal can drive abstention decisions via a confidence meta-classifier.
  • Because the sampler is consistent for the anchored target, any Bayesian Monte Carlo algorithm, such as SMC$_{\parallel}$, HMC$_{\parallel}$, or future stochastic-gradient variants, can be dropped into SBMC without changing the estimator.
  • Scaling the number of parallel processes $P$ improves uncertainty estimates, with S-SMC$_{\parallel}$ improving more sharply than S-HMC$_{\parallel}$ as $P$ grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own experiments tune $s$ per dataset (0.05 to 0.35), so a fully automatic selection rule for $s$, for example empirical Bayes on the marginal likelihood or a cross-validated grid, is the natural next step; without it, the scalable claim carries an extra tuning cost.
  • The anchored-posterior trick is a post-hoc add-on: once a MAP or deep ensemble exists, short chains on the last layer or on LoRA adapters could give the same UQ benefit to frozen pretrained models, which is the direction the GPT-2 appendix gestures at.
  • A minimal deployment test would ask whether a single epistemic-entropy threshold, without the seven-feature meta-classifier, already separates OOD from in-domain inputs; if it does, SBMC's practical overhead drops further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Scalable Bayesian Monte Carlo (SBMC), a method for Bayesian deep learning that combines a MAP-anchored Gaussian prior with an interpolation parameter s, and parallel SMC or MCMC samplers that target the resulting anchored posterior. The authors claim that, for the same wall-clock time as deep ensembles (DE), SBMC delivers comparable or better accuracy and substantially improved epistemic uncertainty quantification, demonstrated on MNIST7, IMDb, CIFAR-10, and a preliminary GPT-2 experiment. The paper includes extensive experiments with 5 realizations, ablations over s and P, and comparisons with MAP, DE, MC Dropout, Laplace approximation, and SWA, as well as long-run HMC as a gold standard.

Significance. If the headline claim were fully supported, SBMC would be a practically valuable contribution: it offers a sampling-based method with formal consistency for its target, while providing epistemic UQ at a cost comparable to a popular heuristic (DE). The empirical study is unusually thorough for this line of work: standard errors from 5 realizations are reported, many baselines are included, and the sensitivity to the tuning parameters s and P is examined. The authors are also transparent about the approximate nature of the target and explicitly flag the heuristic character of the theoretical sketch. However, the central 'same wall-clock time' claim is not supported by the paper's own cost accounting, which uses worker-epochs rather than wall-clock time. This is a load-bearing issue that must be resolved before the paper's main contribution can be accepted as stated.

major comments (3)
  1. [Section 2, Table 1, Abstract] The claim that SBMC matches deep ensembles 'for the same wall-clock time' is contradicted by the paper's own accounting. Section 2 defines time cost as 'simulation time per chain/particle' in epochs, a hardware-agnostic proxy, and the Total Cost column in Table 1 counts MAP epochs plus (sampling epochs × N × P). On identical parallel hardware, the wall-clock critical path for S-HMC∥ (P=1, N=10) is 160 epochs for the MAP phase plus 160 epochs for the parallel sampling phase, i.e. 320 epochs, versus 176 epochs for a deep ensemble whose members run in parallel. The text near Table 1 even concedes 'SBMC methods require the MAP estimator, so their total time cost is roughly double,' which directly contradicts the abstract's phrasing. The comparison should be re-framed as equal worker-epochs or total compute, and the abstract and title-level claims should be adjusted accordingly.
  2. [Table 1, HMC (GS) row] The NLL for the 200,000-epoch HMC gold standard is listed as 1.94±0.004, whereas Table 8 reports 1.942e-1 (0.1942) for the same setting. The decimal error in Table 1 makes the gold-standard look dramatically worse and is inconsistent with the supporting tables. This should be corrected, and the surrounding text should be checked for similar transcription errors.
  3. [Appendix C and Section 5] The theoretical justification is explicitly heuristic: Appendix C states 'let's suppose we wave our hands' in deriving the link to a Laplace approximation, and Section 5 recommends treating convergence heuristically because the target is not the posterior for s<1. The empirical UQ gain is thus an empirical observation for an approximate, data-dependent target, not a consequence of a formal theory. The paper should state this limitation more prominently in the abstract and introduction, so that readers do not infer a principled derivation of the reported UQ improvement.
minor comments (4)
  1. [Section 2, Eq. (2)] The symbol π0 is used both for the original prior N(0,V) and for the anchored prior N(α θ_MAP, Σ(s)), which causes confusion. Please use distinct notation, e.g. π_0^orig and π_0^anch.
  2. [Figure 1 caption] The caption says 'every method runs for 25 epochs except the Gold-Standard,' which is inconsistent with Table 1 where MNIST7 runs are 160–178 epochs. Please clarify the dataset or setting to which the 25-epoch statement applies.
  3. [Tables with bolding] The phrase 'bold the first 30% data in mean' (e.g. in Tables 4–7) is unclear. Please explain what the bolding indicates and why the 30% cutoff is used.
  4. [References] Some references are incomplete, e.g. 'Qiu et al.' appears without a year or venue. Please normalize all citations.

Circularity Check

1 steps flagged · score 2.0 of 10

No circular derivation: the anchored posterior is a transparent approximation; the only flagged issue is a minor non-load-bearing self-citation, while the wall-clock claim is a correctness concern rather than circularity.

  1. other [Section 2, paragraph after Algorithm 1 (parallel speed-up motivation)]
    "Parallel implementations of consistent BMC algorithms like SMC∥ and HMC∥ improve time cost with near linear speed-up [Liang et al., 2025], but each process still needs to run for a long time, as seen in Figure 1 (c)."

    This is a self-citation by overlapping authors for the near-linear parallel speed-up claim used in the scalability motivation. It is flagged only as a minor self-citation, not as a load-bearing circular step: the paper's own ablations over P (Appendix H) independently show that increasing P improves accuracy and UQ at fixed per-chain cost, and the anchored-posterior construction does not reduce to the cited speed-up.

full rationale

The paper's central target is the anchored posterior in (3), π(θ) ∝ L(θ) N(θ; α(s)θ_MAP, s v I), with θ_MAP estimated from the same data. This makes the epistemic entropy a property of a data-dependent approximate target, not the true posterior, but the paper explicitly and repeatedly disclaims that it targets the posterior for s < 1: 'since the method no longer targets the posterior for any s < 1, we would recommend adopting a heuristic approach to convergence as with other SOTA methods, rather than chasing more rigorous convergence guarantees.' The sampler consistency for the anchored target is standard, and the UQ claims are benchmarked against gold-standard HMC and held-out OOD/meta-training labels, so no fitted parameter is renamed as a prediction. The one same-author citation, [Liang et al., 2025], supports near-linear parallel speed-up; it is not machine-checked here, but the paper's own P-ablation results provide independent empirical support, so it is not load-bearing for the central method. The abstract's 'same wall-clock time' statement is not supported by Table 1, which counts worker-epochs and adds a serial MAP phase, as the paper itself concedes that 'SBMC methods require the MAP estimator, so their total time cost is roughly double'; however, this is an accounting and correctness issue, not a circular reduction of a derivation to its inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two empirical facts: that the anchored posterior is a useful proxy and that short parallel chains are near-stationary. Neither is derived rigorously; both are supported by experiments on three datasets. The key fitted numbers are s and v, which are dataset-dependent.

free parameters (3)
  • s (interpolation parameter) = 0.1 (MNIST7), 0.35 (IMDb with v=1), 0.05 (CIFAR10)
    Controls the spread of the anchored prior around the MAP; tuned per dataset to balance accuracy and UQ; authors recommend 0.1 as default.
  • v (prior variance) = 0.1 (MNIST7), 0.025 or 1 (IMDb), 0.2 (CIFAR10)
    Prior variance of the original Gaussian prior; chosen a priori per dataset and affects both accuracy and entropy values.
  • P (number of parallel chains/particles) = 1 or 8 in experiments; N=10 particles
    Increasing P improves UQ metrics (especially for S-SMC); chosen to match deep ensemble cost.
assumptions (3)
  • standard math MCMC and SMC samplers are asymptotically unbiased for their target distribution; the parallel estimators in (4)-(5) converge to the target pi_s as N goes to infinity.
    The consistency of SMC and MCMC is standard; the paper uses them as building blocks.
  • domain assumption Short chains with the chosen hyperparameters (leapfrog L=1, adaptive mutation steps) mix fast enough to make the estimators reliable at the reported compute budget.
    The paper verifies with 5 chains and standard error but provides no formal mixing diagnostics for all setups; the gold-standard HMC runs much longer.
  • domain assumption The MAP estimator theta_MAP, computed on the same training data, is a suitable center for the posterior, so an anchored prior around it yields useful epistemic uncertainty.
    This is the core modeling assumption of SBMC; it is tested empirically but not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles." pith.science (2026). https://pith.science/paper/5327WUS4

@misc{pith2026250513585,
  author       = {Pith},
  title        = {Pith review of: Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5327WUS4}},
  note         = {Machine review of arXiv:2505.13585}
}
abstract

This work introduces a new method designed for Bayesian deep learning called scalable Bayesian Monte Carlo (SBMC). The method is comprised of a model and an algorithm. The model interpolates between a point estimator and the posterior. The algorithm is a parallel implementation of sequential Monte Carlo sampler (SMC$_\parallel$) or Markov chain Monte Carlo (MCMC$_\parallel$). We collectively refer to these consistent (asymptotically unbiased) algorithms as Bayesian Monte Carlo (BMC), and any such algorithm can be used in our SBMC method. The utility of the method is demonstrated on practical examples: MNIST, CIFAR, IMDb. A systematic numerical study reveals that for the same wall-clock time as state-of-the-art (SOTA) methods like deep ensembles (DE), SBMC achieves comparable or better accuracy and substantially improved uncertainty quantification (UQ)--in particular, epistemic UQ. This is demonstrated on the downstream task of estimating the confidence in predictions, which can be used for reliability assessment or abstention decisions.

Figures

Figures reproduced from arXiv: 2505.13585 by the authors.

Figure 1
Figure 1. Left panels: IMDb sentiment classification. (a) SBMC provides a good balance of accuracy and UQ (quantified by epistemic entropy on OOD data), for the same cost as deep ensembles (every method runs for 25 epochs except the Gold-Standard (GS) BMC solution, which runs for 8000 epochs). (c) Standard implementation of HMC and HMC∥. BMC methods typically deliver high accuracy for high cost (GS) and low accuracy for low c… view at source ↗
Figure 2
Figure 2. Average total and epistemic entropy over four OOD [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Accuracy, UQ, and confidence meta-classifier abstention (Abst) metrics (re-normalized so [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (27 more)
Figure 4
Figure 4. Figure 4: 2-level estimator (using confidence meta [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Dual axis Accuracy and Hep(OOD) ablations over s in column 1 and P in column 2 for MNIST7 (row 1), IMDb (row 2), and CIFAR10 (row 3). Ablations are considered by varying s and P. Small s improves mixing, as shown in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Left: SBMC for various s. Middle: Cold posterior for various T. Right: Autocorrelation functions using NUTS sampler, showing that SBMC improves mixing, while CP hinders mixing. for an illustration of the mixing behavior, and Appendix C for further discussion, including…
Figure 7
Figure 7. Figure 7: Comparison of entropy across groups for MNIST7. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Comparison of average total and epistemic entropy over four out-of-domain classes and [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Comparison of entropy across groups for IMDb. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Comparison of average total and epistemic entropy over four out-of-domain classes and [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Comparison of entropy across groups for IMDb. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Comparison of average total and epistemic entropy over three out-of-domain classes and [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Averaged confusion rate matrices for OOD prediction on MNIST7, with default decision [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Averaged curve plots for OOD detection on MNIST7. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Averaged confusion rate matrices for OOD prediction on IMDb, with optimal [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Averaged curve plots for OOD detection in IMDb. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Averaged confusion rate matrices for OOD prediction on IMDb, with optimal [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]
Figure 18
Figure 18. Figure 18: Averaged curve plots for OOD detection in IMDb. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: Averaged confusion rate matrices for OOD prediction on CIFAR10, with default decision [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: Averaged curve plots for OOD detection in CIFAR10. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 160, M = 7, v = 0.1 and s = 0.25, on MNIST7 (5 realizations) [PITH_FULL_IMAGE:figures/full_fig_p029_21.png]
Figure 22
Figure 22. Figure 22: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 160, M = 10, v = 0.1 and s = 0.1, on MNIST7 (5 realizations). H.2 IMDB The experiments in this section are tested on the IMDb dataset with the model setting st…
Figure 23
Figure 23. Figure 23: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed L = 1, B = 25, M = 1, v = 0.025, s = 0.1, on IMDb (5 realizations) Experiments with v = 1. Summary metrics of IMDb dataset with v = 15 and s = 0.35 are shown in the spider-plot in [PITH_FU…
Figure 24
Figure 24. Figure 24: Summary metrics for IMDb in all methods. S-SMC [PITH_FULL_IMAGE:figures/full_fig_p031_24.png]
Figure 25
Figure 25. Figure 25: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 26, M = 1, v = 1 and s = 0.35, on IMDb (5 realizations and ±s.e. in accuracy ) [PITH_FULL_IMAGE:figures/full_fig_p031_25.png]
Figure 26
Figure 26. Figure 26: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 26, M = 2, v = 1 and s = 0.25, on IMDb (5 realizations and ± s.e. in accuracy) [PITH_FULL_IMAGE:figures/full_fig_p031_26.png]
Figure 27
Figure 27. Figure 27: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 26, M = 2, v = 1 and s = 0.1, on IMDb (5 realizations). 31 [PITH_FULL_IMAGE:figures/full_fig_p031_27.png]
Figure 28
Figure 28. Figure 28: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 200, M = 4, v = 0.2 and s = 0.05, on CIFAR10 (5 realizations) [PITH_FULL_IMAGE:figures/full_fig_p032_28.png]
Figure 29
Figure 29. Figure 29: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 200, M = 4, v = 0.2 and s = 0.1, on CIFAR10 (5 realizations) [PITH_FULL_IMAGE:figures/full_fig_p033_29.png]
Figure 30
Figure 30. Figure 30: Comparison of S-SMC∥ (P chains with N = 10) and S-HMC∥ (NP chains), with fixed number of leapfrog L = 1, B = 200, M = 4, v = 0.2 and s = 0.2, on CIFAR10 (5 realizations). I All-inclusive data tables [PITH_FULL_IMAGE:figures/full_fig_p033_30.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 13 canonical work pages

  1. [1]

    We train with SGD (momentum = 0.9 ) for a 25-epoch warm-up, then perform SW A weight averaging with 1 sample per epoch, at a fixedswa_lr = 0.0005

    • SW A. We train with SGD (momentum = 0.9 ) for a 25-epoch warm-up, then perform SW A weight averaging with 1 sample per epoch, at a fixedswa_lr = 0.0005. • MC Dropout. A fixed 30% dropout in the fully-connected layer;10samples are used. • Laplace. We fit a Laplace approximation with a Kronecker-factored approximation of the Hessian8Daxberger et al. [2021...

    work page 2021

  2. [2]

    Johnathan M

    URLhttps://arxiv.org/abs/2502.13923. Johnathan M. Bardsley, Antti Solonen, Heikki Haario, and Marko Laine. Randomize–then–optimize: A method for sampling from posterior distributions in nonlinear inverse problems.SIAM Journal on Scientific Computing, 36(4):A1895–A1910,

    arXiv

  3. [3]

    Table 3: Comparison of methods on test accuracy, NLL, and various entropy metrics for next-token prediction with GPT2 on tiny Shakespeare. Methods Accuracy (%) NLL Htot correct Htot incorrect Hep correct Hep incorrect MAP 38.66 3.166 1.554 3.605 0 0 S-HMC 39.36 3.083 1.571 3.612 0.047 0.077 A.2 Overcoming other computational bottlenecks Our sampler relies...

    work page 2017

  4. [6]

    Pierre Del Moral.Feynman-kac formulae

    URLhttps://arxiv.org/abs/2501.12948. Pierre Del Moral.Feynman-kac formulae. Springer,

    arXiv

  5. [9]

    Introduces Bernoulli-thinning (random inclusion) unbiased-likelihood estimators inside SMC

    doi: 10.1214/20-BA1220. Introduces Bernoulli-thinning (random inclusion) unbiased-likelihood estimators inside SMC. Fredrik K Gustafsson, Martin Danelljan, and Thomas B Schon. Evaluating scalable Bayesian deep learning methods for robust computer vision. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 318–319,

    work page doi:10.1214/20-ba1220

  6. [10]

    Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952,

    Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952,

    arXiv

  7. [12]

    Yanping Huang et al

    URL https://arxiv.org/abs/ 2503.03734. Yanping Huang et al. GPipe: Efficient training of giant neural networks using pipeline parallelism. InNeurIPS,

    arXiv

  8. [13]

    Averaging weights leads to wider optima and better generalization

    11 Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. In34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, pages 876–885. Association For Uncertainty in Artificial Intelligence (AUAI),

    work page 2018

Show all 29 references
  1. [14]

    S-SMC∥ (P= 8 chain withN= 10 ), S-HMC∥ (NP chains), DE (N models) and MAP, with fixed number of leapfrog L= 1 ,B= 26 ,M= 2 , v= 1ands= 0.35(5realizations)

    Figure 10: Comparison of average total and epistemic entropy over four out-of-domain classes and correct/incorrect predictions in-domain for IMDb. S-SMC∥ (P= 8 chain withN= 10 ), S-HMC∥ (NP chains), DE (N models) and MAP, with fixed number of leapfrog L= 1 ,B= 26 ,M= 2 , v= 1a...

  2. [15]

    Comparison of parallel SMC and MCMC for Bayesian deep learning.arXiv preprint arXiv:2402.06173,

    Xinzhu Liang, Joseph M Lukens, Sanjaya Lohani, Brian T Kirby, Thomas A Searles, Xin Qiu, and Kody JH Law. Comparison of parallel SMC and MCMC for Bayesian deep learning.arXiv preprint arXiv:2402.06173,

    arXiv

  3. [18]

    URL https://openai.com/ index/gpt-oss-model-card/. Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David Dunson, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, Aliaksandr Hubin, et al. Po- sition paper: Bayesian deep learning ...

    arXiv

  4. [21]

    Optimised annealed Sequential Monte Carlo samplers.arXiv preprint arXiv:2408.12057,

    13 Saifuddin Syed, Alexandre Bouchard-Côté, Kevin Chern, and Arnaud Doucet. Optimised annealed Sequential Monte Carlo samplers.arXiv preprint arXiv:2408.12057,

  5. [23]

    A Future Directions The most obvious next step is UQ for modern large language models (LLMs) [DeepSeek-AI, 2025], where robustness and hallucination detection are crucial pain points [Vashurin et al., 2024]. It has been recently shown that high-quality entropy metrics are valu...

    2025

  6. [25]

    use EB for fitting prior and likelihood variance in the context of transfer learning for regression, and then they extended this idea for building estimators from an ensemble of pre-trained models You et al. [2022]. The latter could naturally be combined with other ensemble ap...

    2022

  7. [26]

    tune away

    prior, and now it is time to incorporate the rest of the data (1−a)ℓ . The Hessian of our Laplace approximation is a∇2ℓ(θ) + 1 2(v/a) Id. This could be carried through rigorously, but for the sake of the argument, let’s suppose we wave our hands and swap outNtrainId for∇2ℓ(θ) ...

    2025

  8. [28]

    The NN model and parameter prior for IMDb10 are built as follows • NN is followed by (i) no hidden layer, (ii) ReLU activation, (iii) a final linear layer, and (iv) softmax output

    In other words, frozen weights from all-mpnet-base-v2 until the 768 dimensional [CLS] output. The NN model and parameter prior for IMDb10 are built as follows • NN is followed by (i) no hidden layer, (ii) ReLU activation, (iii) a final linear layer, and (iv) softmax output. • ...

    2023

  9. [29]

    Lorem ipsum

    11, containing customer reviews and 8https://github.com/aleximmer/Laplace 9https://huggingface.co/sentence-transformers/all-mpnet-base-v2 10https://huggingface.co/datasets/stanfordnlp/imdb 11https://amazon-reviews-2023.github.io/ 19 product metadata. Two datasets directly used...

    2023

  10. [30]

    not in CIFAR-10

    E.3.3 CIFAR-10 Classification Example Here, the architecture is ResNet-50 pre-trained from ImageNet with all parameters frozen until the final pooled 2048 dimensional features. The NN model and parameter prior for CIFAR10 are as follows. • NN is followed by (i) no hidden layer...

  11. [31]

    There are also unconnected AMD “Genoa” compute nodes, with 2×84-core AMD EPYC 9634 CPUs and 1.5TB RAM

    E.4 Hardware description The main CPU cluster we access has nodes with 2 × 16-core Intel Skylake Gold 6130 CPU @ 2.10GHz, 192GB RAMwithout communicationin between, so it can only run N/P= 32 particles in parallel with one particle per core. There are also unconnected AMD “Geno...

    2000

  12. [1953]

    Evaluating Bayesian deep learning for radio galaxy classifica- tion.arXiv preprint arXiv:2405.18351,

    Devina Mohan and Anna MM Scaife. Evaluating Bayesian deep learning for radio galaxy classifica- tion.arXiv preprint arXiv:2405.18351,

    arXiv

  13. [1977]

    Mixtures of laplace ap- proximations for improved post-hoc uncertainty in deep learning.arXiv preprint arXiv:2111.03577,

    Runa Eschenhagen, Erik Daxberger, Philipp Hennig, and Agustinus Kristiadi. Mixtures of laplace ap- proximations for improved post-hoc uncertainty in deep learning.arXiv preprint arXiv:2111.03577,

    arXiv

  14. [1992]

    Firefly Monte Carlo: Exact MCMC with subsets of data

    Dougal Maclaurin and Ryan P Adams. Firefly Monte Carlo: Exact MCMC with subsets of data. arXiv preprint arXiv:1403.5693,

    arXiv

  15. [2006]

    Sid Black et al

    ISBN 978-0-387-31073-2. Sid Black et al. GPT-NeoX-20B: An open-source autoregressive language model.arXiv 2204.06745,

    arXiv

  16. [2010]

    Accurate, large minibatch sgd: Training imagenet in 1 hour.arXiv 1706.02677,

    Priya Goyal et al. Accurate, large minibatch sgd: Training imagenet in 1 hour.arXiv 1706.02677,

    arXiv

  17. [2014]

    9 Carlo Berzuini and Walter Gilks

    doi: 10.1137/140964023. 9 Carlo Berzuini and Walter Gilks. Resample-move filtering with cross-model jumps.Sequential Monte Carlo Methods in Practice, pages 117–138,

    doi

  18. [2019]

    Lewis J Rendell, Adam M Johansen, Anthony Lee, and Nick Whiteley

    URL https://arxiv.org/ abs/1908.10084. Lewis J Rendell, Adam M Johansen, Anthony Lee, and Nick Whiteley. Global consensus Monte Carlo.Journal of Computational and Graphical Statistics, 30(2):249–259,

    1908 arXiv

  19. [2020]

    Nils Reimers and Iryna Gurevych

    URLhttps://arxiv.org/abs/1910.02054. Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using siamese BERT- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11

    1910 arXiv

  20. [2023]

    Andreas Krause and Jonas Hübotter

    doi: 10.1145/3571730. Andreas Krause and Jonas Hübotter. Probabilistic Artificial Intelligence.arXiv preprint arXiv:2502.05244,

    arXiv

  21. [2024]

    Benchmarking uncertainty quantification methods for large language models with LM-polygraph

    Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Lyudmila Rvanova, Akim Tsvigun, Daniil Vasilev, Rui Xing, Abdelrahman Boda Sadallah, Kirill Grishchenkov, Sergey Petrakov, et al. Benchmarking uncertainty quantification methods for large language models with LM-polygraph. a...

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.