Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators

Michael D. Shields; Ponkrshnan Thiagarajan; Tamer A. Zaki

REVIEW 4 major objections 4 minor 72 references

Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hybrid VI-HMC method makes full-batch HMC tractable for neural networks by sampling only the parameters that drive predictive uncertainty.

desk verdict A practical VI-HMC hybrid with a clear computational benefit, but the core freezing approximation needs calibration evidence before I would trust its uncertainty estimates. read the letter →

arxiv 2507.14652 v2 pith:53EENCC3 submitted 2025-07-19 stat.ML cs.CEcs.LGphysics.data-an

Ponkrshnan Thiagarajan , Tamer A. Zaki , Michael D. Shields This is my paper · ORCID

classification stat.MLcs.CEcs.LGphysics.data-an MSC 62F1565C0568T07

keywords HamiltonianMonteCarlovariationalinferenceBayesianneuralnetworksoperatorsuncertaintyquantificationsensitivityanalysisDeepONetdimensionalityreduction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that Hamiltonian Monte Carlo, normally too expensive for large Bayesian neural networks, can be made practical by first running cheap variational inference and then measuring how much each network parameter contributes to predictive variance. It claims that this contribution is sparse: a small fraction of parameters carries most of the output uncertainty, so HMC only needs to sample those, while the rest are frozen at their variational means. If true, the method would give HMC-grade posterior samples and uncertainty estimates at a fraction of the cost, on networks with tens to hundreds of thousands of parameters. The paper demonstrates the idea on small Bayesian networks and on DeepONet surrogates for Burgers' equation and for wall-pressure in hypersonic flow over a cone.

What carries the argument

The carrying object is the per-parameter sensitivity score $S_i^2$, which multiplies the variational posterior variance $\sigma_i^2$ by the average squared gradient of the network output with respect to parameter $\theta_i$, evaluated at the VI mean. A threshold $\tau$ on the cumulative fraction of explained variance selects the active subset $\Theta_s$; the conditional posterior $P(\Theta_s \mid D, \Theta_{\sim s} = \mu_{\sim s})$ is then sampled by HMC. The computational argument is that HMC cost scales roughly as $O(D^{5/4})$ in parameter dimension, so removing inert coordinates reduces leapfrog error accumulation, raises acceptance rates, and permits larger step sizes.

What would settle it

Run full HMC and VI-HMC on a linear-Gaussian regression network where the posterior is analytically known and compute the exact predictive variance; if fixing the low-sensitivity parameters at their VI means changes the predictive variance by more than the tolerance implied by the chosen threshold $\tau$, the central assumption behind the method is false.

Watch

Extended reading notes

Core claim

The central claim is that posterior predictive uncertainty in a Bayesian neural network or neural operator can be accurately recovered by sampling, with HMC, only the parameter subset with the largest sensitivity scores $S_i^2$, defined from a first-order Taylor expansion of the network output around the variational mean as $S_i^2 = \frac{\sigma_i^2}{N_d}\sum_{j=1}^{N_d}\left(\frac{\partial F_\mu(x_j)}{\partial \theta_i}\right)^2$. The remaining parameters are conditioned on their variational means, so HMC draws from $P(\Theta_s \mid D, \Theta_{\sim s} = \mu_{\sim s})$ instead of the full posterior. The paper reports that this reduced-space HMC matches full HMC uncertainty estimates on small networks where both can be run, captures parameter correlations that mean-field VI misses, and makes inference feasible for DeepONets with 16,321 to 172,401 parameters.

Load-bearing premise

The load-bearing premise is that parameters judged insensitive by a local first-order sensitivity analysis at the variational mean can be frozen at their VI means without materially changing the posterior predictive distribution, an assumption the paper does not bound quantitatively.

Editorial extensions

If this is right

Full-batch HMC becomes applicable to networks with tens to hundreds of thousands of parameters, not just the small networks where it was previously feasible.
With a fixed step size, reduced-space HMC achieves substantially higher acceptance rates than full HMC, and with acceptance fixed at 80 percent it tolerates step sizes up to several orders of magnitude larger.
VI-HMC uncertainty estimates track HMC more closely than VI alone does, including the correlation structure between paired parameters that mean-field VI discards.
The same recipe extends from functions to operators, producing uncertainty estimates for DeepONet surrogates of PDE solutions, including Burgers' equation and hypersonic cone flow.
The computational advantage should grow with network size because the cost saving from dimension reduction scales with the exponent in the $O(D^{5/4})$ HMC cost estimate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural stress test the paper does not run is to compare VI-HMC predictive intervals against full HMC on a moderately sized network across many datasets and thresholds, checking coverage rather than visual agreement.
Because Appendix E shows reported uncertainty shrinks as the threshold $\tau$ decreases, a practitioner should treat the threshold as a tunable calibration knob and validate interval coverage on held-out data.
The sensitivity ranking is computed on training inputs averaged over the data set; out-of-distribution or extrapolation queries could reorder the ranking, so the active subset may not transfer to new input regions.
The same sensitivity screen could be replaced by second-order variance propagation or by gradients evaluated at several posterior draws, which would test whether the local first-order ranking at the VI mean is stable.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you work on scalable Bayesian inference for neural operators. The idea is simple: train VI on the full network, rank parameters by their contribution to predictive variance via a first-order sensitivity analysis, freeze the low-ranking ones at VI means, and run HMC only on the small active set. They demonstrate this on BNNs and on DeepONets for Burgers and hypersonic cone flow, with code and a cost comparison showing roughly 2x speedups and much better acceptance rates than full HMC. The exposition is clear, and the sensitivity ranking is verified against analytic derivatives in the small Case I. That is real work and it ships code, which I credit.

The soft spot is the freezing step. The paper shows visual agreement with full HMC only in the two small BNN examples. In the larger DeepONet problems, full HMC is not run at all, so there is no calibration or coverage check. Appendix E shows that lowering the threshold tau shrinks reported uncertainty, which is expected if you are discarding variance by construction, but the paper does not quantify how much of that discarded variance matters for the predictive distribution. The stress-test note about Case I is fair: the frozen parameter omega2 has VI mean 2.94 while the true value is -3, and full HMC recovers -3. The predictive curve may still look acceptable because VI means as a set reparameterize the function, but that is exactly why freezing can bias predictions. Also, in the Burgers example the mean relative L2 error for VI-HMC is 7.52% versus 3.8% for VI, which is a red flag for the claim that the hybrid improves mean predictions. Section 5 does acknowledge some limitations, but the central assumption deserves more scrutiny than it gets.

That said, the paper does not oversell the method; it frames the sparsity as a hypothesis. The contribution is a useful heuristic with a clear computational benefit. Whether it produces calibrated uncertainties is an empirical question the paper does not yet answer. This is a solid candidate for peer review, not a desk reject. I would ask the authors to add a medium-scale comparison with full HMC (coverage of predictive intervals, not just visual agreement), to report calibration metrics on all examples, and to state more precisely under what conditions freezing low-sensitivity parameters is valid. As it stands, I would not cite it as a validated method, but I would send it to review and would be interested to see the revised version.

Referee Report

4 major / 4 minor

Summary. The paper proposes a hybrid VI-HMC method for Bayesian neural networks and neural operators. The method first trains a mean-field variational posterior on the full network, then ranks parameters by a local first-order sensitivity score S_i^2 = sigma_i^2 times the average squared gradient at the VI mean. Parameters whose cumulative sensitivity reaches a threshold tau are sampled with HMC, while the remaining parameters are fixed at their VI means. The authors report experiments on two small BNN benchmarks, a Burgers-equation DeepONet, and a hypersonic-flow cone DeepONet, and claim that the reduced-space HMC closely reproduces full-HMC posterior predictive uncertainties at substantially lower cost.

Significance. If the central approximation is valid, the contribution is significant: it would make full-batch HMC feasible for networks with tens to hundreds of thousands of parameters, and the application to neural operators for hypersonic flow is timely and practically motivated. The paper is commendably explicit about its approximations, provides code, and includes a limitations section. However, the current validation is mostly visual and qualitative, the core freezing assumption is not supported by a calibration test or error bound, and the operator-network examples lack any full-HMC benchmark. The idea is promising, but the evidence as presented is not yet sufficient for the strength of the claims made.

major comments (4)

[Section 3.1, Eq. (18)] The definition of the reduced parameter count N_hat is inconsistent with the stated goal. The text says tau is the proportion of variance to be captured and the experiments describe selecting the first parameters whose cumulative sensitivity reaches tau. The displayed equation uses 'max' and '<= tau', which for a nondecreasing cumulative ratio selects the largest N_T with cumulative ratio no greater than tau and would typically pick a set capturing slightly less than tau, not the four parameters claimed for case I. The inequality direction and the optimization should be reversed: one needs the smallest N_T such that the cumulative ratio is at least tau.
[Section 3.2, Eq. (20), Algorithm 2, Appendix E] The central approximation—fixing low-sensitivity parameters at their VI means—is not supported by a quantitative error bound or calibration test. The sensitivity score in Eq. (17) is local and first-order, evaluated at the VI mean, so parameters with small VI variance, small gradient, or strong correlation with sensitive parameters can still materially affect the posterior predictive distribution when the full posterior is considered. Appendix E explicitly shows that the reported uncertainty shrinks as tau decreases, confirming that the freezing step discards genuine posterior uncertainty, and no criterion for choosing tau or measuring the induced bias is provided.
[Sections 4.2.1 and 4.2.2, Table 2] The main claim that VI-HMC reproduces HMC uncertainty is not quantitatively validated for the operator problems. For Burgers, full HMC is not run and the only reported accuracy numbers are mean relative L2 errors: VI 3.8% versus VI-HMC 7.52%, so the VI-HMC mean prediction is worse than VI. No coverage, interval score, or other calibration metric is reported. For the cone problem there is no full-HMC baseline at all, so statements such as 'VI overestimates uncertainties when compared to the hybrid VI-HMC approach' compare two approximations rather than validating against the target posterior.
[Section 4.1.1, Table 1] The statement that 'the parameters sampled through VI-HMC closely match those obtained using the full HMC approach' is not supported by Table 1. The frozen parameters omega_2 and phi_1 are fixed at VI means 2.94 and -0.072, whereas full HMC gives -3.00 and 6.29, and VI-HMC samples around a different mode (-4.00) for omega_1 than full HMC (4.00). The predictive agreement in Figure 3 appears to rely on the functional symmetry of the sinusoidal representation, not on the method recovering the full posterior. The text should state clearly that VI-HMC targets a conditional posterior anchored at the VI mode and justify under what conditions predictive quantities, rather than parameter values, are the objects of inference.

minor comments (4)

[Section 4.2.3, Table 2] The 'Required step size' column mixes values for HMC and VI-HMC with footnotes in a way that is hard to read; please reformat and also report effective sample sizes in addition to acceptance rates so that the computational gains are measured per effective sample.
[Section 5 and Appendix C] The statement in Appendix C that 'Both the prior and likelihood are assumed to be the same for VI, HMC, and VI-HMC approaches' is difficult to reconcile with the discussion in Section 5, which describes using a small likelihood standard deviation for VI and a large one for HMC in Case II; please clarify whether the final reported experiments use identical likelihoods for all three methods.
[Figure 10 caption] The caption lists 'samples, mean prediction, true solution, initial condition' without explaining which quantity each curve or band represents or how the 'best' and 'worst' cases are defined; please clarify.
[Introduction, Section 1] The sentence 'Cobb and Jalaian [25] proposed a new integration scheme by splitting the data into batches that preserve the Hamiltonian' appears to misstate the method; the cited work uses a symmetric splitting of the Hamiltonian dynamics, not a splitting of data batches. Please verify and correct the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reduced-space HMC target and its full-HMC comparisons are external to the sensitivity-based selection.

full rationale

The paper's derivation chain is not circular. The sensitivity scores S_i^2 in Eq. (17) are defined from a first-order Taylor expansion of the network at the VI mean together with VI variances; this is an approximation used to select a parameter subspace, not the quantity that is ultimately predicted. The final predictive uncertainty is obtained by HMC sampling from the conditional posterior P(Theta_s | D, Theta_~s = mu_~s) in Eq. (20), which is a well-defined target independent of the selection criterion. In Cases I and II (Sections 4.1.1 and 4.1.2) this target is validated against full HMC on the full parameter space, and the reported parameter moments and joint distributions are benchmarked externally. The operator-network demonstrations (Burgers, cone) do not include full-HMC baselines; the paper explicitly states 'Without this reduction, HMC is computationally prohibitive, and therefore, results from full HMC are not shown here' in Section 4.2.1. That is a missing benchmark and a correctness risk, but not circularity, because the reduced-space HMC still samples from an explicit conditional posterior rather than from a quantity defined by the sensitivity measure. Appendix E shows that lowering tau reduces predictive uncertainty; this is an acknowledged limitation of the freezing heuristic, and the random-subset comparison is an external control, not a tautology. Self-citations (e.g., [27], [43], [57], and the hypersonic-data and flow-physics references such as [71]) are contextual, data-source, or background citations; none is used as the sole justification for the load-bearing claim that low-sensitivity parameters can be frozen. No uniqueness theorem is imported from the authors' prior work, and no fitted quantity is renamed as a prediction. The validity of the method is therefore a question of approximation error and calibration, not of definitional circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central approximations are a first-order local sensitivity, reliance on the VI posterior for subset selection, and unconditional freezing of low-sensitivity parameters. The first two are common modeling assumptions; the third is the load-bearing approximation of the proposed method. The threshold tau is a hand-chosen free parameter.

free parameters (2)

Variance threshold tau = 0.9 (default; varied from 0.30 to 0.90 in Appendix E)
Controls how many parameters are retained for HMC via Eq. 18; central cost and accuracy results depend on this hand-chosen threshold, and the paper shows behavior changes with tau.
HMC step size = 1E-5 to 1E-3 depending on problem (Table C.4)
Manually tuned per experiment; acceptance rates and computational cost comparisons in Table 2 depend on it.

assumptions (3)

domain assumption First-order Taylor expansion of the network output at the VI mean is sufficient for ranking parameter influence on predictive variance (Eqs. 13-17).
Sensitivity is local around the VI mean; second-order and nonlocal effects are neglected.
domain assumption VI posterior means and variances provide a reliable proxy for the true posterior when selecting the sensitive subset.
Subset selection is performed using the approximate VI posterior; if VI is far from the true posterior, the ranking may be wrong.
ad hoc to paper Fixing insensitive parameters at VI means does not materially change posterior predictive uncertainty (Eq. 20, Algorithm 2).
This is the central approximation of the proposed method; the paper provides no error bound, and Appendix E shows uncertainty shrinks with tau.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators." pith.science (2026). https://pith.science/paper/53EENCC3

@misc{pith2026250714652,
  author       = {Pith},
  title        = {Pith review of: Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/53EENCC3}},
  note         = {Machine review of arXiv:2507.14652}
}

read the original abstract

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network's parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. We show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Figures

Figures reproduced from arXiv: 2507.14652 by the authors.

**Figure 2.** Case I: Ratio of variance captured for increasing number of BNN parameters. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Case I: BNN predictions and their uncertainties from (a) VI, (b) HMC, and (c) VI-HMC. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: Joint distributions of two parameters (ϕ2 and a) obtained by (a) VI, (b) VI-HMC, (c) HMC. that better match the uncertainties from HMC. They are qualitatively better in their lower uncertainties near existing training data, while also yielding larger and more realistic…

**Figure 5.** Figure 5: Case II: Evolution of (a) ELBO loss function and (b) Mean square error in VI training. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Case II: BNN parameter sensitivities (a) Histogram of sensitivity values for the BNN parameters [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Case II: BNN predictions and their uncertainties from (a) VI, (b) HMC, and (c) VI-HMC. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 8.** Figure 8: Burgers’ Equation: (a) ELBO loss and (b) MSE during DeepONet training using VI. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

**Figure 9.** Figure 9: Burgers’ Equation: Bayesian DeepONet sensitivities. (a) Histogram of sensitivity values for the [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

**Figure 10.** Figure 10: Comparing predictions between VI (left) and VI-HMC (right) for the Burgers equation for 4 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

**Figure 11.** Figure 11: Flow over a cone, (a) ELBO loss and (b) MSE during DeepONet training using VI. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

**Figure 12.** Figure 12: Flow over cone: Bayesian DeepONet sensitivities. (a) Histogram of sensitivity values for the [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

**Figure 13.** Figure 13: Flow over cone: Comparing uncertainties quantified by (a-d) VI and (e-h) VI-HMC. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 60 canonical work pages

[1]

H. D. Kabir, A. Khosravi, M. A. Hosen, S. Nahavandi, Neural network-based uncertainty quantification: A survey of methodologies and applications, IEEE access 6 (2018) 36218– 36234

work page 2018
[2]

A. F. Psaros, X. Meng, Z. Zou, L. Guo, G. E. Karniadakis, Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons, Journal of Compu- tational Physics 477 (2023) 111902

2023
[3]

W. He, Z. Jiang, T. Xiao, Z. Xu, Y. Li, A survey on uncertainty quantification methods for deep learning, arXiv preprint arXiv:2302.13425 (2023)

arXiv 2023
[4]

Graves, Practical variational inference for neural networks, Advances in neural in- formation processing systems 24 (2011)

A. Graves, Practical variational inference for neural networks, Advances in neural in- formation processing systems 24 (2011)

work page 2011
[5]

Blundell, J

C. Blundell, J. Cornebise, K. Kavukcuoglu, D. Wierstra, Weight uncertainty in neural network, in: International conference on machine learning, Proceedings of Machine Learning Research, 2015, pp. 1613–1622

work page 2015
[6]

M. D. Hoffman, D. M. Blei, C. Wang, J. Paisley, Stochastic variational inference, Journal of Machine Learning Research (2013)

work page 2013
[7]

R. M. Neal, et al., Mcmc using hamiltonian dynamics, Handbook of markov chain monte carlo 2 (11) (2011) 2

work page 2011
[8]

T. Chen, E. Fox, C. Guestrin, Stochastic gradient hamiltonian monte carlo, in: Inter- national conference on machine learning, PMLR, 2014, pp. 1683–1691

work page 2014

Show all 72 references

[9]

Zhang, C

R. Zhang, C. Li, J. Zhang, C. Chen, A. G. Wilson, Cyclical stochastic gradient mcmc for bayesian deep learning, in: 8th International Conference on Learning Representations, ICLR 2020, 2020

2020
[10]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting, The journal of machine learning research 15 (1) (2014) 1929–1958

2014
[11]

Y. Gal, Z. Ghahramani, Dropout as a bayesian approximation: Representing model uncertainty in deep learning, in: international conference on machine learning, PMLR, 2016, pp. 1050–1059

2016
[12]

Y. Gal, J. Hron, A. Kendall, Concrete dropout, Advances in neural information pro- cessing systems 30 (2017)

2017
[13]

D. J. MacKay, A practical bayesian framework for backpropagation networks, Neural computation 4 (3) (1992) 448–472

1992
[14]

Ritter, A

H. Ritter, A. Botev, D. Barber, A scalable laplace approximation for neural networks, in: 6th international conference on learning representations, ICLR 2018-conference track proceedings, Vol. 6, International Conference on Representation Learning, 2018. 30

2018
[15]

Lakshminarayanan, A

B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and scalable predictive uncer- tainty estimation using deep ensembles, Advances in neural information processing sys- tems 30 (2017)

2017
[16]

S. Fort, H. Hu, B. Lakshminarayanan, Deep ensembles: A loss landscape perspective, arXiv preprint arXiv:1912.02757 (2019)

2019 arXiv
[17]

Welling, Y

M. Welling, Y. W. Teh, Bayesian learning via stochastic gradient langevin dynamics, in: Proceedings of the 28th international conference on machine learning (ICML-11), Citeseer, 2011, pp. 681–688

2011
[18]

S. Ahn, A. Korattikara, M. Welling, Bayesian posterior sampling via stochastic gradient fisher scoring, in: 29th International Conference on Machine Learning, ICML 2012, 2012, pp. 1591–1598

2012
[19]

S. Ahn, B. Shahbaba, M. Welling, Distributed stochastic gradient mcmc, in: Interna- tional conference on machine learning, PMLR, 2014, pp. 1044–1052

2014
[20]

Y.-A. Ma, T. Chen, E. Fox, A complete recipe for stochastic gradient mcmc, Advances in neural information processing systems 28 (2015)

2015
[21]

N. Ding, Y. Fang, R. Babbush, C. Chen, R. D. Skeel, H. Neven, Bayesian sampling using stochastic gradient thermostats, Advances in neural information processing systems 27 (2014)

2014
[22]

Zhang, A

R. Zhang, A. F. Cooper, C. De Sa, Amagold: Amortized metropolis adjustment for efficient stochastic gradient mcmc, in: International conference on artificial intelligence and statistics, PMLR, 2020, pp. 2142–2152

2020
[23]

Garriga-Alonso, V

A. Garriga-Alonso, V. Fortuin, Exact langevin dynamics with stochastic gradients, in: Third Symposium on Advances in Approximate Bayesian Inference, 2021. URLhttps://openreview.net/forum?id=Rprd8aVUYkE

2021
[24]

Izmailov, S

P. Izmailov, S. Vikram, M. D. Hoffman, A. G. G. Wilson, What are bayesian neural network posteriors really like?, in: International conference on machine learning, PMLR, 2021, pp. 4629–4640

2021
[25]

A. D. Cobb, B. Jalaian, Scaling hamiltonian monte carlo inference for bayesian neural networks with symmetric splitting, in: Uncertainty in Artificial Intelligence, PMLR, 2021, pp. 675–685

2021
[26]

Wenzel, K

F. Wenzel, K. Roth, B. S. Veeling, J. ´Swikatkowski, L. Tran, S. Mandt, J. Snoek, T. Salimans, R. Jenatton, S. Nowozin, How good is the bayes posterior in deep neural networks really?, in: Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 10248–10259

2020
[27]

G. D. Pasparakis, L. Graham-Brady, M. D. Shields, Bayesian neural networks for pre- dicting uncertainty in full-field material response, Computer Methods in Applied Me- chanics and Engineering 433 (2025) 117486. 31

2025
[28]

G. E. Hinton, D. Van Camp, Keeping the neural networks simple by minimizing the description length of the weights, in: Proceedings of the sixth annual conference on Computational learning theory, 1993, pp. 5–13

1993
[29]

Barber, C

D. Barber, C. M. Bishop, Ensemble learning in bayesian neural networks, Nato ASI Series F Computer and Systems Sciences 168 (1998) 215–238

1998
[30]

J. M. Hern´ andez-Lobato, R. Adams, Probabilistic backpropagation for scalable learning of bayesian neural networks, in: International conference on machine learning, Proceed- ings of Machine Learning Research, 2015, pp. 1861–1869

2015
[31]

D. P. Kingma, T. Salimans, M. Welling, Variational dropout and the local reparame- terization trick, Advances in neural information processing systems 28 (2015)

2015
[32]

Molchanov, A

D. Molchanov, A. Ashukha, D. Vetrov, Variational dropout sparsifies deep neural net- works, in: International conference on machine learning, PMLR, 2017, pp. 2498–2507

2017
[33]

Louizos, M

C. Louizos, M. Welling, Multiplicative normalizing flows for variational bayesian neural networks, in: International Conference on Machine Learning, PMLR, 2017, pp. 2218– 2227

2017
[34]

M. Khan, D. Nielsen, V. Tangkaratt, W. Lin, Y. Gal, A. Srivastava, Fast and scalable bayesian deep learning by weight-perturbation in adam, in: International conference on machine learning, PMLR, 2018, pp. 2611–2620

2018
[35]

Zhang, S

G. Zhang, S. Sun, D. Duvenaud, R. Grosse, Noisy natural gradient as variational infer- ence, in: International conference on machine learning, PMLR, 2018, pp. 5852–5861

2018
[36]

A. Wu, S. Nowozin, E. Meeds, R. E. Turner, J. M. Hernandez-Lobato, A. L. Gaunt, Deterministic variational inference for robust bayesian neural networks, in: International Conference on Learning Representations, 2019

2019
[37]

Osawa, S

K. Osawa, S. Swaroop, M. E. E. Khan, A. Jain, R. Eschenhagen, R. E. Turner, R. Yokota, Practical deep learning with bayesian principles, Advances in neural in- formation processing systems 32 (2019)

2019
[38]

Dusenberry, G

M. Dusenberry, G. Jerfel, Y. Wen, Y. Ma, J. Snoek, K. Heller, B. Lakshminarayanan, D. Tran, Efficient and scalable bayesian neural nets with rank-1 factors, in: International conference on machine learning, PMLR, 2020, pp. 2782–2792

2020
[39]

Salimans, D

T. Salimans, D. Kingma, M. Welling, Markov chain monte carlo and variational infer- ence: Bridging the gap, in: International conference on machine learning, PMLR, 2015, pp. 1218–1226

2015
[40]

C. Wolf, M. Karl, P. van der Smagt, Variational inference with hamiltonian monte carlo, arXiv preprint arXiv:1609.08203 (2016)

2016 arXiv
[41]

D. V. Patel, J. Lee, M. W. Farthing, P. K. Kitanidis, E. F. Darve, Multi-fidelity hamil- tonian monte carlo, arXiv preprint arXiv:2405.05033 (2024). 32

2024 arXiv
[42]

J.-K. Wang, A. Wibisono, Accelerating hamiltonian monte carlo via chebyshev inte- gration time, in: The Eleventh International Conference on Learning Representations, 2023

2023
[43]

S. L. Dhulipala, Y. Che, M. D. Shields, Efficient bayesian inference with latent hamil- tonian neural networks in no-u-turn sampling, Journal of Computational Physics 492 (2023) 112425

2023
[44]

T. Cui, J. Martin, Y. M. Marzouk, A. Solonen, A. Spantini, Likelihood-informed dimen- sion reduction for nonlinear inverse problems, Inverse Problems 30 (11) (2014) 114015

2014
[45]

P. G. Constantine, C. Kent, T. Bui-Thanh, Accelerating markov chain monte carlo with active subspaces, SIAM Journal on Scientific Computing 38 (5) (2016) A2779–A2805

2016
[46]

Tripathy, I

R. Tripathy, I. Bilionis, Deep active subspaces: A scalable method for high-dimensional uncertainty propagation, in: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 59179, American So- ciety of Mechanical Eng...

2019
[47]

Izmailov, W

P. Izmailov, W. J. Maddox, P. Kirichenko, T. Garipov, D. Vetrov, A. G. Wilson, Sub- space inference for bayesian deep learning, in: Uncertainty in Artificial Intelligence, PMLR, 2020, pp. 1169–1179

2020
[48]

Daxberger, E

E. Daxberger, E. Nalisnick, J. U. Allingham, J. Antor´ an, J. M. Hern´ andez-Lobato, Bayesian deep learning via subnetwork inference, in: International Conference on Ma- chine Learning, PMLR, 2021, pp. 2510–2521

2021
[49]

Schuster, P

I. Schuster, P. G. Constantine, T. Sullivan, Exact active subspace metropolis-hastings, with applications to the lorenz-96 system, arXiv preprint arXiv:1712.02749 (2017)

2017 arXiv
[50]

Ripoli, R

L. Ripoli, R. G. Everitt, Sequential monte carlo with active subspaces, arXiv preprint arXiv:2411.05935 (2024)

2024 arXiv
[51]

Jantre, N

S. Jantre, N. M. Urban, X. Qian, B.-J. Yoon, Learning active subspaces for effective and scalable uncertainty quantification in deep neural networks, in: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 5330–5334

2024
[52]

T. Cui, X. T. Tong, A unified performance analysis of likelihood-informed subspace methods, Bernoulli 28 (4) (2022) 2788–2815

2022
[53]

Y. Li, R. E. Turner, R´ enyi divergence variational inference, Advances in neural infor- mation processing systems 29 (2016)

2016
[54]

A. B. Dieng, D. Tran, R. Ranganath, J. Paisley, D. Blei, Variational inference via χupper bound minimization, Advances in Neural Information Processing Systems 30 (2017). 33

2017
[55]

N. Wan, D. Li, N. Hovakimyan, F-divergence variational inference, Advances in neural information processing systems 33 (2020) 17370–17379

2020
[56]

Deasy, N

J. Deasy, N. Simidjievski, P. Li` o, Constraining variational inference with geometric jensen-shannon divergence, Advances in Neural Information Processing Systems 33 (2020) 10647–10658

2020
[57]

Thiagarajan, S

P. Thiagarajan, S. Ghosh, Jensen–shannon divergence based novel loss functions for bayesian neural networks, Neurocomputing (2024) 129115

2024
[58]

M. D. Hoffman, A. Gelman, et al., The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo., J. Mach. Learn. Res. 15 (1) (2014) 1593–1623

2014
[59]

Paszke, S

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems 32 (2019)

2019
[60]

Shridhar, F

K. Shridhar, F. Laumann, M. Liwicki, A comprehensive guide to bayesian convolutional neural network with variational inference, arXiv preprint arXiv:1901.02731 (2019)

2019 arXiv
[61]

L. Lu, P. Jin, G. Pang, Z. Zhang, G. E. Karniadakis, Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nature machine intelligence 3 (3) (2021) 218–229

2021
[62]

Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. liu, K. Bhattacharya, A. Stuart, A. Anand- kumar, Fourier neural operator for parametric partial differential equations, in: Inter- national Conference on Learning Representations, 2021

2021
[63]

J. Park, T. A. Zaki, Sensitivity of high-speed boundary-layer stability to base-flow distortion, J. Fluid Mech. 859 (2019) 476–515.doi:10.1017/jfm.2018.819

2019 doi
[64]

Jahanbakhshi, T

R. Jahanbakhshi, T. A. Zaki, Nonlinearly most dangerous disturbance for high-speed boundary-layer transition, J. Fluid Mech. 876 (2019) 87–121.doi:10.1017/jfm.2019. 527

2019 doi
[65]

D. A. Buchta, T. A. Zaki, Observation-infused simulations of high-speed boundary-layer transition, J. Fluid Mech. 916 (2021) A44.doi:10.1017/jfm.2021.172

2021 doi
[66]

Buchta, S

D. Buchta, S. Laurence, T. A. Zaki, Assimilation of wall-pressure measurements in high-speed flow over a cone, J. of Fluid Mech. (Rapids) (2022)

2022
[67]

Q. Wang, T. A. Zaki, Domain of dependence for wall-pressure measurements in high- speed boundary layers, Journal of Fluid Mechanics 1009 (2025) A67.doi:10.1017/ jfm.2025.224

2025
[68]

Z. Mao, L. Lu, O. Marxen, T. A. Zaki, G. E. Karniadakis, DeepM&Mnet for hyper- sonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators, Journal of computational physics 447 (2021) 110698. 34

2021
[69]

Clark Di Leoni, L

P. Clark Di Leoni, L. Lu, C. Meneveau, G. E. Karniadakis, T. A. Zaki, Neural op- erator prediction of linear instability waves in high-speed boundary layers, Journal of Computational Physics 474 (2023) 111793.doi:10.1016/j.jcp.2022.111793

2023
[70]

Y. Hao, P. C. Di Leoni, O. Marxen, C. Meneveau, G. E. Karniadakis, T. A. Zaki, Instability-wave prediction in hypersonic boundary layers with physics-informed neural operators, Journal of computational science 73 (2023) 102120

2023
[71]

Morra, C

P. Morra, C. Meneveau, T. A. Zaki, ML for fast assimilation of wall-pressure measure- ments from hypersonic flow over a cone, Scientific Reports 14 (1) (2024) 12853

2024
[72]

T. A. Zaki, Turbulence from an observer perspective, Annual Review of Fluid Mechanics 57 (2025) 311–334.doi:10.1146/annurev-fluid-030424-114735. 35

2025 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

H. D. Kabir, A. Khosravi, M. A. Hosen, S. Nahavandi, Neural network-based uncertainty quantification: A survey of methodologies and applications, IEEE access 6 (2018) 36218– 36234

work page 2018

[2] [2]

A. F. Psaros, X. Meng, Z. Zou, L. Guo, G. E. Karniadakis, Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons, Journal of Compu- tational Physics 477 (2023) 111902

2023

[3] [3]

W. He, Z. Jiang, T. Xiao, Z. Xu, Y. Li, A survey on uncertainty quantification methods for deep learning, arXiv preprint arXiv:2302.13425 (2023)

arXiv 2023

[4] [4]

Graves, Practical variational inference for neural networks, Advances in neural in- formation processing systems 24 (2011)

A. Graves, Practical variational inference for neural networks, Advances in neural in- formation processing systems 24 (2011)

work page 2011

[5] [5]

Blundell, J

C. Blundell, J. Cornebise, K. Kavukcuoglu, D. Wierstra, Weight uncertainty in neural network, in: International conference on machine learning, Proceedings of Machine Learning Research, 2015, pp. 1613–1622

work page 2015

[6] [6]

M. D. Hoffman, D. M. Blei, C. Wang, J. Paisley, Stochastic variational inference, Journal of Machine Learning Research (2013)

work page 2013

[7] [7]

R. M. Neal, et al., Mcmc using hamiltonian dynamics, Handbook of markov chain monte carlo 2 (11) (2011) 2

work page 2011

[8] [8]

T. Chen, E. Fox, C. Guestrin, Stochastic gradient hamiltonian monte carlo, in: Inter- national conference on machine learning, PMLR, 2014, pp. 1683–1691

work page 2014