REVIEW 2 cited by
An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Comparing Bayesian neural networks (BNNs) with different widths is challenging because, as the width increases, multiple model properties change simultaneously, and, inference in the finite-width case is intractable. In this work, we empirically compare finite- and infinite-width BNNs, and provide quantitative and qualitative explanations for their performance difference. We find that when the model is mis-specified, increasing width can hurt BNN performance. In these cases, we provide evidence that finite-width BNNs generalize better partially due to the properties of their frequency spectrum that allows them to adapt under model mismatch.
Forward citations
Cited by 2 Pith papers
-
Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
A replica/HCIZ theory predicts the Bayes-optimal generalization error of proportional-width MLPs near interpolation and discovers layer-wise specialization transitions that make deeper targets harder to learn.
-
A ZeNN architecture to avoid the Gaussian trap
ZeNNs, which replace the equal-weight average of MLP neurons with an index-weighted sum of frequency-scaled neurons, provably converge pointwise, retain non-Gaussian limits, and learn high-frequency features in low-di...
Discussion (0). Continue with ORCID to comment.