Sub-network Laplace approximations always underestimate full-model predictive variance, and two new gradient-based and greedy selection rules provide theoretically grounded improvements.
'In-Between' Uncertainty in Bayesian Neural Networks
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
abstract
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. This can lead to catastrophically overconfident predictions when testing on out-of-distribution data. Avoiding such overconfidence is critical for active learning, Bayesian optimisation and out-of-distribution robustness. We instead find that a classical technique, the linearised Laplace approximation, can handle 'in-between' uncertainty much better for small network architectures.
representative citing papers
Derives optimal low-rank subspace for Laplace approx in BNNs, provides scalable outperforming version, and new comparison metric.
Last-layer linearization for Bayesian GLMs in DNN uncertainty quantification matches full-network performance in UQ quality while improving efficiency, according to random matrix theory analysis and empirical tests across tasks.
citing papers explorer
-
Optimality of Sub-network Laplace Approximations: New Results and Methods
Sub-network Laplace approximations always underestimate full-model predictive variance, and two new gradient-based and greedy selection rules provide theoretically grounded improvements.
-
Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks
Derives optimal low-rank subspace for Laplace approx in BNNs, provides scalable outperforming version, and new comparison metric.
-
Is the Last Layer Sufficient for Uncertainty Quantification?
Last-layer linearization for Bayesian GLMs in DNN uncertainty quantification matches full-network performance in UQ quality while improving efficiency, according to random matrix theory analysis and empirical tests across tasks.