REVIEW 6 cited by
User-friendly introduction to PAC-Bayes bounds
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Aggregated predictors are obtained by making a set of basic predictors vote according to some weights, that is, to some probability distribution. Randomized predictors are obtained by sampling in a set of basic predictors, according to some prescribed probability distribution. Thus, aggregated and randomized predictors have in common that they are not defined by a minimization problem, but by a probability distribution on the set of predictors. In statistical learning theory, there is a set of tools designed to understand the generalization ability of such procedures: PAC-Bayesian or PAC-Bayes bounds. Since the original PAC-Bayes bounds of D. McAllester, these tools have been considerably improved in many directions (we will for example describe a simplified version of the localization technique of O. Catoni that was missed by the community, and later rediscovered as "mutual information bounds"). Very recently, PAC-Bayes bounds received a considerable attention: for example there was workshop on PAC-Bayes at NIPS 2017, "(Almost) 50 Shades of Bayesian Learning: PAC-Bayesian trends and insights", organized by B. Guedj, F. Bach and P. Germain. One of the reason of this recent success is the successful application of these bounds to neural networks by G. Dziugaite and D. Roy. An elementary introduction to PAC-Bayes theory is still missing. This is an attempt to provide such an introduction.
Forward citations
Cited by 6 Pith papers
-
Post-Cut Metadata Inference Attacks on Quantum Circuit Cutting Pipelines
Post-cut metadata from quantum circuit fragments enables high-accuracy inference of algorithm family, cut mechanism, and Hamiltonian structure via machine learning on fragment width, depth, and gate counts.
-
Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities
A unified DPI-based framework yields novel change-of-measure inequalities that produce tighter high-probability generalization bounds.
-
Generalization of Gibbs and Langevin Monte Carlo Algorithms in the Interpolation Regime
New PAC-Bayes bounds for the Gibbs posterior remain non-vacuous in the interpolation regime and can be approximated by Langevin Monte Carlo, but the tight experimental numbers rely on an unproved random-label calibrat...
-
Model Diffusion for Certifiable Few-shot Transfer Learning
STEEL samples a finite set of PEFT adapters from a diffusion model and selects the best on the downstream support set, producing non-vacuous PAC-Bayes generalization certificates for low-shot LLM and vision transfer learning.
-
Model Merging is Secretly Certifiable: Non-Vacuous Generalisation Bounds for Low-Shot Learning
First non-vacuous PAC-Bayes certificates for large vision and language models in the 100-example low-shot regime, obtained by reinterpreting model merging as a low-dimensional posterior.
-
Uncertainty Quantification for Misspecified Machine Learned Interatomic Potentials
POPS uncertainty bounds from misspecification-aware parameter sampling envelop DFT reference values for a broad range of tungsten properties and for MACE-MPA-0 energies.
Discussion (0). Continue with ORCID to comment.