Efficient Bayesian Deep Ensembles via Analytic Predictive Inference

Jaesung Lee; Marie Maros; Sina Aghaee Dabaghan Fard

REVIEW 3 major objections 6 minor 13 references

Independently trained neural predictors, aggregated by closed-form Bayesian linear regression, yield calibrated Student-t uncertainty at deep-ensemble cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 21:32 UTC pith:4MOV3ZZQ

load-bearing objection Solid engineering synthesis: independent NN predictors as basis functions plus conjugate Bayesian aggregation, correctly derived and competitive on UCI under two training regimes. the 3 major comments →

arxiv 2607.06776 v1 pith:4MOV3ZZQ submitted 2026-07-07 cs.LG stat.ME

Efficient Bayesian Deep Ensembles via Analytic Predictive Inference

Sina Aghaee Dabaghan Fard , Marie Maros , Jaesung Lee This is my paper

classification cs.LG stat.ME

keywords Bayesian deep ensemblesanalytic predictive inferenceBayesian last layerdeep kernel learningStudent-t processuncertainty calibrationfinite-rank Gaussian processregression

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that a small set of neural networks trained separately can serve as basis functions for a Bayesian linear model, producing an exact Student-t posterior predictive without variational or sampling approximations. The method keeps the training cost of ordinary deep ensembles while replacing uniform averaging with posterior weights that downweight poorly supported members. The resulting model is also a finite-rank Gaussian process whose kernel is learned by the ensemble, so uncertainty is principled yet cheap. On standard UCI regression benchmarks the approach matches or beats deep ensembles and Bayesian last-layer baselines in both accuracy and log-likelihood under both aggressive and conservative training regimes. A sympathetic reader cares because many safety-critical and sequential-decision systems need reliable uncertainty that is still fast enough to train and easy to interpret as weights on concrete predictors.

Core claim

When several neural networks are trained independently on the same regression data, their scalar mean outputs form a low-dimensional feature matrix that can be treated as fixed. Placing a conjugate Gaussian–Jeffreys prior on the linear combination of those features and the noise variance yields a closed-form Student-t posterior predictive whose only extra cost is a small H-by-H Bayesian linear regression. The posterior weights automatically reweight ensemble members according to data support, giving calibrated uncertainty and interpretability without approximate inference.

What carries the argument

Bayesian Deep Kernel Networks (BDKN): independent neural mean functions induce an N-by-H design matrix; conjugate Bayesian linear regression under a Jeffreys noise prior produces the analytic Student-t predictive of Eq. (5) at cost O(N H² + H³).

Load-bearing premise

The independently trained neural mean functions already span a rich enough finite-dimensional space that all remaining uncertainty can safely sit only on their linear weights and a single noise scale.

What would settle it

On a controlled distribution-shift regression task, compare BDKN’s predictive coverage and NLL against a full Bayesian neural network and a single-representation Bayesian last-layer model; if BDKN becomes systematically over-confident while the full BNN does not, the finite-span premise fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Uncertainty quantification for regression can keep deep-ensemble training pipelines and still obtain exact posterior weights and Student-t predictive distributions.
Ensemble size H becomes an explicit knob on function-space rank and expressivity that can be increased in parallel without cubic GP cost.
Poorly optimized ensemble members are automatically down-weighted, reducing the fragility of uniform averaging under aggressive learning rates.
The same closed-form predictive can be dropped into larger systems (e.g., model-based RL or physics-informed nets) that need calibrated regression components.
Interpretability is immediate: each posterior coefficient is the contribution of one concrete trained network.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same aggregation layer could be applied after heterogeneous or diversified ensembles (different architectures or activations) to turn structural diversity into a finite-rank Bayesian kernel.
Because the predictive is Student-t with closed-form moments, it is a natural plug-in for acquisition functions in Bayesian optimization that currently rely on Gaussian approximations.
Extending the conjugate head to non-Gaussian likelihoods while preserving analytic tractability remains open; success there would transfer the method to classification and count data with the same interpretability benefits.
The finite-rank GP view suggests that theoretical results on posterior contraction for finite-rank kernels can be used to diagnose when H is too small for a given data regime.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid engineering synthesis: independent NN predictors as basis functions plus conjugate Bayesian aggregation, correctly derived and competitive on UCI under two training regimes.

read the letter

The one thing worth knowing is that this is a clean, usable recipe rather than a foundational result. They train H independent neural nets (each a full predictor with its own heteroscedastic head), freeze the mean functions as basis maps, then put ordinary conjugate Bayesian linear regression on top under a Jeffreys noise prior. The posterior predictive is a closed-form Student-t (Eq. 5), the finite-rank GP view is correctly derived, and the extra cost is O(N H^{2} + H^{3}) with H small. That is the whole method.

What is new is the specific combination: treating the ensemble members themselves as the feature maps rather than a single last-layer representation, then letting the posterior automatically down-weight weak members. That is a legitimate engineering improvement over uniform deep-ensemble averaging and over single-representation BLL. The math is standard BLR and checks out; the two-regime UCI tables (high and low learning rate, 20 splits) are thorough and include the right baselines. BDKN stays stable when DE blows up under aggressive optimization, which is a real practical point. Runtime tables show the Bayesian step is essentially free relative to training the nets.

Soft spots are modest and already flagged by the modeling premise itself. All residual uncertainty lives on the linear coefficients and a scalar noise once the maps are frozen; there is no strong distribution-shift stress test, and no code release. Ensemble size H and the empirical-Bayes σ_β^{2} are free parameters, but that is ordinary. The paper does not over-claim these points. Citations are appropriate and the related-work framing is honest about the ingredients.

This is for people who already use deep ensembles or BLL-style models and want a drop-in Bayesian aggregation layer with interpretable weights and analytic predictive distributions. It is not for someone hunting a new theory of uncertainty. I would send it to peer review; a methods venue should give it a serious referee. Worth reading if you care about practical UQ components.

Referee Report

3 major / 6 minor

Summary. The paper proposes Bayesian Deep Kernel Networks (BDKN): H independently trained neural networks produce scalar mean predictors that are treated as fixed basis functions; a conjugate Bayesian linear regression layer with Jeffreys prior on noise variance then yields closed-form posterior weights and a Student-t posterior predictive (Eq. 5). The construction is interpreted as a finite-rank GP whose kernel is the inner product of the ensemble features. Complexity of the aggregation step is O(N H^{2} + H^{3}). On standard UCI regression benchmarks, under both a high-learning-rate (40-epoch) and a low-learning-rate (dataset-specific) regime, BDKN reports RMSE and NLL competitive with deep ensembles, Bayesian last-layer variants, DKL, MFVI-BNN, VBLL, and GP (where feasible), with improved stability relative to uniform deep ensembles under aggressive optimization.

Significance. If the results hold, the work offers a practical middle ground between deep ensembles and Bayesian last-layer / finite-rank GP models: exact predictive inference at ensemble scale, interpretable posterior weights over members, and negligible overhead beyond training H networks. The finite-rank GP derivation (Appendix A.2–A.3), Jeffreys conjugacy, and empirical-Bayes σ_β^{2} step are standard but cleanly assembled; the dual optimization-regime evaluation and runtime tables strengthen the efficiency claim. The contribution is incremental rather than foundational—essentially Bayesian stacking of independently trained predictors—but it is useful for practitioners who want calibrated aggregation without variational or MCMC machinery, and it is a natural building block for RL or PINN-style pipelines that already use ensembles.

major comments (3)

[§1.1, Abstract; Experiments §3] Abstract and §1.1 list “interpretable posterior weights” as a core contribution, yet §3 and the appendix report only RMSE/NLL. No table, figure, or analysis shows the posterior mean/covariance of β, downweighting of weak members, or correlation with member quality. Without this evidence the interpretability claim is unsupported; at minimum, include posterior-weight diagnostics on a few datasets (e.g., Yacht/Power under Setting 1 where DE is unstable).
[§2.1] §2.1 asserts that heteroscedastic variance heads are used only during training “to induce heterogeneous learned feature maps.” This is a load-bearing design choice for component (iii), but there is no ablation against homoscedastic (or MSE-only) ensemble training. If diversity is essential to the calibration/robustness gains over DE and single-representation BLL, that should be demonstrated; otherwise the claim should be softened.
[§1; §2.1–2.2; §3] Intro and related discussion motivate better OOD/extrapolative uncertainty relative to BLL and DE, yet all experiments use random in-distribution UCI splits. The modeling premise that span{ϕ_h} is rich enough for residual uncertainty to live only in β and σ_ε² is therefore not stress-tested where the paper’s motivation is strongest. Either add a simple shift/extrapolation protocol or clearly scope the empirical claims to in-distribution calibration and optimization robustness.

minor comments (6)

[§2.1, Eq. (2)] Notation: Φ is written both as stacking ϕ_h(x_i) over i and as a collection of maps; in §2.1 the displayed Φ construction mixes per-point and per-member indexing. Clarify that each column of Φ is the vector of predictions of one ensemble member on the training set.
[Table 1] Table 1a: DE RMSE on Power (3424±6918) and Sarcos (53±157) under Setting 1 indicates training collapse; a brief note on clipping, early stopping, or failed runs would help readers interpret the “robustness” narrative.
[Appendix A.4] Algorithm 1 and Appendix A.4: state explicitly that σ_h^{2} heads are discarded after training and that only the mean outputs enter Φ.
[Figures 2–3] Figure 2–3 captions are clear, but axis labels and method colors are hard to parse in grayscale; consider distinct line styles.
[§4] Related work could more explicitly position BDKN against classical Bayesian model averaging / stacking of ensembles and against “ensemble of BLLs,” to sharpen the novelty claim.
[Title/front matter; References] Typos: “Deep Kernel Networks” vs title “Bayesian Deep Ensembles”; “arXiv:2607.06776v1 [cs.LG] 7 Jul 2026” date looks off; “V olpp” spacing in references.

Circularity Check

0 steps flagged

No significant circularity: closed-form Student-t predictive follows from conjugacy after independently trained features are frozen; empirical Bayes on σ²_β is ordinary hyperparameter tuning.

full rationale

The derivation chain is self-contained and non-circular. Independently trained neural networks produce fixed feature maps {ϕ_h} (Section 2.1); a conjugate Normal-Jeffreys prior is then placed on the linear coefficients β and noise variance σ²_ε (Eqs. 3–4); posterior updates and the Student-t predictive (Eq. 5) follow by standard conjugacy (Appendix A.1–A.4). The sole fitted scalar σ²_β is obtained by maximizing the closed-form marginal likelihood (Eq. 10), which is ordinary empirical Bayes and does not redefine the target predictive distribution. The finite-rank GP interpretation (Section 2.3) is a direct consequence of the linear model, not a renaming of an external result. Empirical RMSE/NLL comparisons on UCI benchmarks are external evaluations, not fitted-input-as-prediction. No self-definitional loop, no load-bearing self-citation uniqueness claim, and no ansatz smuggled via citation appear in the derivation. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central claim rests on standard conjugate Bayesian linear regression applied to independently trained neural predictors, plus a small set of modeling and hyper-parameter choices. No new physical entities are postulated; free parameters are the usual neural-network weights plus one scalar prior variance.

free parameters (4)

ensemble size H = 5 (main experiments)
Fixed to 5 in main tables (varied 5–50 in Yacht ablation); controls rank of the induced feature space and therefore model capacity.
prior variance σ_β^{2} = optimized per split
Initialized at 1 and optimized by maximizing the closed-form marginal likelihood (5 Adam steps, lr 0.1); empirical-Bayes hyper-parameter of the isotropic Gaussian prior on β.
neural-network weights θ_h and heteroscedastic variance heads
Trained by maximum-likelihood on each ensemble member; the learned mean functions become the fixed basis for subsequent Bayesian aggregation.
learning rates and epoch budgets = see Tables 4–5
Two hand-chosen regimes (0.1/40 epochs vs. dataset-specific low rates up to 500–100000 epochs) that materially affect relative ranking of baselines.

axioms (4)

standard math Conjugate Normal–Inverse-Gamma (Jeffreys limit) Bayesian linear regression yields a Student-t posterior predictive when the design matrix is fixed.
Invoked in Section 2.2 and Appendix A.1–A.4; classical result.
domain assumption Independently trained neural mean functions form a sufficiently expressive finite-dimensional basis for the regression function.
Core modeling premise of Sections 2.1–2.3 that allows all remaining uncertainty to be placed only on linear coefficients β.
domain assumption Isotropic Gaussian prior β|σ_ε^{2} ~ N(0, σ_ε^{2} σ_β^{2} I) is adequate; no structured prior over ensemble members is required.
Stated in Section 2.2 and Appendix A.3; simplifies inference but treats all members exchangeably a priori.
ad hoc to paper Heteroscedastic training loss induces useful diversity among ensemble members without harming the subsequent Bayesian aggregation.
Introduced in Section 2.1 as a training device; not derived from first principles.

invented entities (1)

Bayesian Deep Kernel Networks (BDKN) no independent evidence
purpose: Name for the overall pipeline that treats independently trained neural predictors as basis functions for conjugate Bayesian linear regression.
Terminological packaging of existing components; no new mathematical object beyond the finite-rank kernel already known in the literature.

pith-pipeline@v1.1.0-grok45 · 25398 in / 2853 out tokens · 26796 ms · 2026-07-10T21:32:10.174137+00:00 · methodology

0 comments

read the original abstract

We introduce an efficient Bayesian deep ensemble method for predictive regression designed to enhance interpretability while maintaining competitive predictive performance and computational efficiency. Our method combines the statistical rigor of Bayesian inference with the scalability of deep ensembles, providing calibrated uncertainty estimates that enable its use not only for standalone prediction but also as a component within broader learning systems. To achieve these goals, our work relies on three key design components: (i) low-dimensional ensemble representation: predictions are expressed as a combination of a small number of trained neural predictors, enabling scalable inference whose cost depends on ensemble size rather than dataset size; (ii) closed-form Bayesian aggregation: ensemble predictions are combined using Bayesian linear regression, yielding interpretable posterior weights and calibrated uncertainty without approximate inference; and (iii) Independent ensemble training: multiple neural networks are trained separately, producing diverse predictive representations that improve robustness and uncertainty calibration. Empirical results on standard regression benchmarks demonstrate that the proposed approach achieves competitive predictive performance while maintaining reliable uncertainty estimates across settings.

Figures

Figures reproduced from arXiv: 2607.06776 by Jaesung Lee, Marie Maros, Sina Aghaee Dabaghan Fard.

**Figure 1.** Figure 1: Proposed framework. An ensemble of neural networks induces a feature space representation {ϕh(xi ; θh)} H h=1. A Bayesian linear model is placed on top of these ensemble-induced features, and the linear weights β and noise variance σ 2 ε are integrated out with respect to their posteriors, yielding a closed-form posterior predictive distribution over targets y. 1 Introduction We consider predictive regres… view at source ↗

**Figure 2.** Figure 2: Boston dataset - Benchmarks test data performance over training epochs under two experimental settings. (a) Setting 1 (high learning rate). (b) Setting 2 (low learning rate). We show quartile plots of test RMSE (left) and test NLL (right) as functions of training epochs. Solid lines denote the median across 20 splits, while shaded regions indicate the interquartile range (25th–75th percentiles). Lower valu… view at source ↗

**Figure 3.** Figure 3: Yacht (Setting 1): effect of BDKN ensemble size on the test data performance. (a) Quartile plots of test RMSE (left) and test NLL (right) as functions of training epochs for ensembles with 5, 10, 30, and 50 feature maps. Solid lines denote the median across splits and shaded regions indicate the interquartile range (25th–75th percentiles). (b) Boxplots of test RMSE (left) and test NLL (right) across splits… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages · 9 internal anchors

[1]

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

Angelos Filos, Sebastian Farquhar, Aidan N Gomez, Tim GJ Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud De Kroon, and Yarin Gal. A systematic comparison of bayesian deep learning robustness in diabetic retinopathy tasks.arXiv preprint arXiv:1912.10481,

work page internal anchor Pith review Pith/arXiv arXiv 1912
[2]

Deep Ensembles: A Loss Landscape Perspective

Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspec- tive.arXiv preprint arXiv:1912.02757,

work page internal anchor Pith review Pith/arXiv arXiv 1912
[3]

Variational Bayesian Last Layers

James Harrison, John Willes, and Jasper Snoek. Variational bayesian last layers.arXiv preprint arXiv:2404.11599,

work page internal anchor Pith review Pith/arXiv arXiv
[4]

Deep Neural Networks as Gaussian Processes

Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes.arXiv preprint arXiv:1711.00165,

work page internal anchor Pith review Pith/arXiv arXiv
[5]

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

work page internal anchor Pith review Pith/arXiv arXiv 2005
[6]

A unifying view of sparse approximate gaussian process regression.Journal of machine learning research, 6(Dec):1939–1959,

Joaquin Quinonero-Candela and Carl Edward Rasmussen. A unifying view of sparse approximate gaussian process regression.Journal of machine learning research, 6(Dec):1939–1959,

work page 1939
[7]

DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation

Alexandre Rame and Matthieu Cord. Dice: Diversity in deep ensembles via conditional redundancy adversarial estimation.arXiv preprint arXiv:2101.05544,

work page internal anchor Pith review Pith/arXiv arXiv
[8]

A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference

Kumar Shridhar, Felix Laumann, and Marcus Liwicki. A comprehensive guide to bayesian convolu- tional neural network with variational inference.arXiv preprint arXiv:1901.02731,

work page internal anchor Pith review Pith/arXiv arXiv 1901
[9]

Functional Variational Bayesian Neural Networks

Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional variational bayesian neural networks.arXiv preprint arXiv:1903.05779,

work page internal anchor Pith review Pith/arXiv arXiv 1903
[10]

How Good is the Bayes Posterior in Deep Neural Networks Really?

Florian Wenzel, Kevin Roth, Bastiaan S Veeling, Jakub ´Swi ˛ atkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really?arXiv preprint arXiv:2002.02405,

work page internal anchor Pith review Pith/arXiv arXiv 2002
[11]

Deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm.Mathematical Problems in Engineering, 2020(1):4275623,

12 Junta Wu and Huiyun Li. Deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm.Mathematical Problems in Engineering, 2020(1):4275623,

work page 2020
[12]

The log-likelihood ofσ 2 ε is ℓ(σ2 ε) =− N 2 logσ 2 ε − 1 2σ2ε ∥y−Φβ∥ 2 +C, whereCdenotes a constant independent ofσ 2 ε

Consider the observation model y|β, σ 2 ε ∼ N Φβ, σ 2 εIN , whereΦ∈R N×H is the ensemble-induced design matrix. The log-likelihood ofσ 2 ε is ℓ(σ2 ε) =− N 2 logσ 2 ε − 1 2σ2ε ∥y−Φβ∥ 2 +C, whereCdenotes a constant independent ofσ 2 ε. Differentiating twice with respect toσ 2 ε yields ∂2ℓ ∂(σ 2ε)2 = N 2 1 (σ2ε)2 − ∥y−Φβ∥ 2 (σ2ε)3 . Taking the negative expec...

work page 2006
[13]

We adopt a zero-mean isotropic Gaussian prior over the linear coefficients, i.e., β∼ N(0, σ 2 βI), which is a standard and commonly used choice in Bayesian last-layer models Watson et al. [2021]. A.5.3 Runtime All experiments were conducted on a workstation running Windows 10 Enterprise (64-bit), equipped with a 12th-generation Intel Core i7-12700 CPU (12...

work page 2021

[1] [1]

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

Angelos Filos, Sebastian Farquhar, Aidan N Gomez, Tim GJ Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud De Kroon, and Yarin Gal. A systematic comparison of bayesian deep learning robustness in diabetic retinopathy tasks.arXiv preprint arXiv:1912.10481,

work page internal anchor Pith review Pith/arXiv arXiv 1912

[2] [2]

Deep Ensembles: A Loss Landscape Perspective

Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspec- tive.arXiv preprint arXiv:1912.02757,

work page internal anchor Pith review Pith/arXiv arXiv 1912

[3] [3]

Variational Bayesian Last Layers

James Harrison, John Willes, and Jasper Snoek. Variational bayesian last layers.arXiv preprint arXiv:2404.11599,

work page internal anchor Pith review Pith/arXiv arXiv

[4] [4]

Deep Neural Networks as Gaussian Processes

Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes.arXiv preprint arXiv:1711.00165,

work page internal anchor Pith review Pith/arXiv arXiv

[5] [5]

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

work page internal anchor Pith review Pith/arXiv arXiv 2005

[6] [6]

A unifying view of sparse approximate gaussian process regression.Journal of machine learning research, 6(Dec):1939–1959,

Joaquin Quinonero-Candela and Carl Edward Rasmussen. A unifying view of sparse approximate gaussian process regression.Journal of machine learning research, 6(Dec):1939–1959,

work page 1939

[7] [7]

DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation

Alexandre Rame and Matthieu Cord. Dice: Diversity in deep ensembles via conditional redundancy adversarial estimation.arXiv preprint arXiv:2101.05544,

work page internal anchor Pith review Pith/arXiv arXiv

[8] [8]

A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference

Kumar Shridhar, Felix Laumann, and Marcus Liwicki. A comprehensive guide to bayesian convolu- tional neural network with variational inference.arXiv preprint arXiv:1901.02731,

work page internal anchor Pith review Pith/arXiv arXiv 1901

[9] [9]

Functional Variational Bayesian Neural Networks

Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional variational bayesian neural networks.arXiv preprint arXiv:1903.05779,

work page internal anchor Pith review Pith/arXiv arXiv 1903

[10] [10]

How Good is the Bayes Posterior in Deep Neural Networks Really?

Florian Wenzel, Kevin Roth, Bastiaan S Veeling, Jakub ´Swi ˛ atkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really?arXiv preprint arXiv:2002.02405,

work page internal anchor Pith review Pith/arXiv arXiv 2002

[11] [11]

Deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm.Mathematical Problems in Engineering, 2020(1):4275623,

12 Junta Wu and Huiyun Li. Deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm.Mathematical Problems in Engineering, 2020(1):4275623,

work page 2020

[12] [12]

The log-likelihood ofσ 2 ε is ℓ(σ2 ε) =− N 2 logσ 2 ε − 1 2σ2ε ∥y−Φβ∥ 2 +C, whereCdenotes a constant independent ofσ 2 ε

Consider the observation model y|β, σ 2 ε ∼ N Φβ, σ 2 εIN , whereΦ∈R N×H is the ensemble-induced design matrix. The log-likelihood ofσ 2 ε is ℓ(σ2 ε) =− N 2 logσ 2 ε − 1 2σ2ε ∥y−Φβ∥ 2 +C, whereCdenotes a constant independent ofσ 2 ε. Differentiating twice with respect toσ 2 ε yields ∂2ℓ ∂(σ 2ε)2 = N 2 1 (σ2ε)2 − ∥y−Φβ∥ 2 (σ2ε)3 . Taking the negative expec...

work page 2006

[13] [13]

We adopt a zero-mean isotropic Gaussian prior over the linear coefficients, i.e., β∼ N(0, σ 2 βI), which is a standard and commonly used choice in Bayesian last-layer models Watson et al. [2021]. A.5.3 Runtime All experiments were conducted on a workstation running Windows 10 Enterprise (64-bit), equipped with a 12th-generation Intel Core i7-12700 CPU (12...

work page 2021