REVIEW 4 cited by
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been shown to improve generalization in deep learning. With SWAG, we fit a Gaussian using the SWA solution as the first moment and a low rank plus diagonal covariance also derived from the SGD iterates, forming an approximate posterior distribution over neural network weights; we then sample from this Gaussian distribution to perform Bayesian model averaging. We empirically find that SWAG approximates the shape of the true posterior, in accordance with results describing the stationary distribution of SGD iterates. Moreover, we demonstrate that SWAG performs well on a wide variety of tasks, including out of sample detection, calibration, and transfer learning, in comparison to many popular alternatives including MC dropout, KFAC Laplace, SGLD, and temperature scaling.
Forward citations
Cited by 4 Pith papers
-
Evaluation of Seismic Artificial Intelligence with Uncertainty
A seismic AI evaluation framework jointly estimates training and data uncertainty and learning efficiency, and shows no single PhaseNet training approach wins at all data budgets.
-
Bayesian Inference for Large Scale Image Classification
ATMC, a new adaptive-noise MCMC sampler, is reported to outperform SGD baselines in accuracy, log-likelihood, and calibration on Cifar10 and ImageNet, and is claimed to be the first MCMC method to train a neural netwo...
-
Solar image denoising with convolutional neural networks
A U-net trained on noisy image pairs alone, without clean targets, denoises solar Stokes images to about 6e-4 continuum residual, matching clean-target training on synthetic data.
-
MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support
A retraining-free uncertainty layer is claimed to reduce overconfident clinical QA errors, but the key derivation is invalid and the abstract and full text report different results.
Discussion (0). Continue with ORCID to comment.