Bernstein-von Mises Theorem for Sparse Generalized Linear Model
Pith reviewed 2026-07-01 15:51 UTC · model grok-4.3
The pith
The fractional posterior for sparse generalized linear models with spike-and-slab priors satisfies an oracle Bernstein-von Mises theorem.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The main result is an oracle Bernstein-von Mises theorem for the fractional posterior under supportwise likelihood assumptions. The proof develops sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, and combines them with fixed-prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-entry verifications are given for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions. The ordinary posterior is treated only
What carries the argument
Sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers under supportwise likelihood assumptions.
If this is right
- The posterior contracts around the true sparse parameter at the oracle rate.
- Support recovery occurs with high probability under the beta-min separation condition.
- The posterior approximates a Gaussian mixture that collapses to the oracle Gaussian law.
- The theorem applies to fractional posteriors in Gaussian, logistic, Poisson, probit, Gamma, and negative-binomial regressions under the stated conditions.
Where Pith is reading between the lines
- Credible intervals from the fractional posterior may achieve asymptotic frequentist coverage when the support is recovered.
- The approach could extend to other high-dimensional models where supportwise conditions can be verified.
- Model selection uncertainty becomes asymptotically negligible once the beta-min condition ensures support recovery.
Load-bearing premise
The supportwise likelihood assumptions must hold to enable local asymptotic normality and approximation around each possible support.
What would settle it
A simulation study where the fractional posterior for a sparse logistic regression model fails to concentrate around the oracle estimator or deviates from Gaussianity would falsify the central claim.
read the original abstract
We study spike-and-slab priors for generalized linear models with possible grouped sparsity. The main result is an oracle Bernstein--von Mises theorem for the fractional posterior under supportwise likelihood assumptions. The proof develops sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, and combines them with fixed-prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-entry verifications are given for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions. The ordinary posterior is treated only through restricted Gaussian and canonical-link extensions, with coverage under additional active-dimension and moment conditions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript establishes an oracle Bernstein-von Mises theorem for the fractional posterior under spike-and-slab priors in generalized linear models with grouped sparsity. The argument proceeds by establishing sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, then invoking fixed prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, exact support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-specific verifications are supplied for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions; the ordinary posterior receives only restricted treatment.
Significance. If the supportwise likelihood assumptions hold and close the LAN remainder uniformly, the result supplies a rigorous justification for asymptotic normality of fractional posteriors in sparse high-dimensional GLMs. This would strengthen the theoretical foundation for Bayesian inference procedures that rely on oracle properties and mixture collapse, particularly for the listed GLM families where explicit verifications are given.
major comments (1)
- The supportwise likelihood assumptions (invoked throughout the proof sketch in the abstract to control the sparse LAN expansion and enable collapse of the Gaussian mixture to the oracle law) are load-bearing yet non-standard. The manuscript must supply their precise statement and demonstrate that the model-specific sufficient conditions for the five GLM families uniformly bound the LAN remainder over the recovered supports; without this, the subsequent contraction and oracle BvM steps do not follow.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive comments on our manuscript. We address the single major comment below and agree that greater explicitness on the supportwise likelihood assumptions will improve clarity. The revised manuscript will incorporate the requested clarifications and verifications.
read point-by-point responses
-
Referee: The supportwise likelihood assumptions (invoked throughout the proof sketch in the abstract to control the sparse LAN expansion and enable collapse of the Gaussian mixture to the oracle law) are load-bearing yet non-standard. The manuscript must supply their precise statement and demonstrate that the model-specific sufficient conditions for the five GLM families uniformly bound the LAN remainder over the recovered supports; without this, the subsequent contraction and oracle BvM steps do not follow.
Authors: We agree that the supportwise likelihood assumptions are central and that their uniform control of the LAN remainder must be verified explicitly for the stated GLM families. These assumptions appear in the manuscript as Assumption 2.1 (Supportwise LAN), which requires that the log-likelihood ratio expansion holds with remainder o_p(1) uniformly over all supports of cardinality at most s_n. Sections 4.1–4.5 supply model-specific sufficient conditions (bounded design, beta-min separation, moment bounds) under which the LAN remainder is controlled for each of the five families. To address the referee’s concern directly, the revision will add a new Lemma 4.6 that aggregates these verifications and proves the uniform o_p(1) bound over the recovered supports, thereby closing the gap between the sufficient conditions and Assumption 2.1. This addition will be placed immediately before the main contraction and BvM arguments. revision: yes
Circularity Check
No circularity: derivation relies on external assumptions and standard expansions
full rationale
The claimed oracle BvM result is obtained by first establishing sparse LAN and Laplace approximation around support-specific centers, then applying fixed-prior mass, support penalization, recovery geometry and beta-min conditions to reach contraction, exact support recovery and collapse to the oracle Gaussian law. These steps invoke stated supportwise likelihood assumptions whose content is external to the derivation and are verified model-by-model under sufficient conditions; no equation or step is shown to reduce by construction to a fitted parameter, self-citation chain, or renamed input. The ordinary-posterior extensions are likewise handled under additional explicit conditions. The derivation chain is therefore self-contained against the listed assumptions.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption supportwise likelihood assumptions
Reference graph
Works this paper leans on
-
[1]
barticle [author] Abramovich , Felix F. Grinshtein , Vadim V. ( 2016 ). Model selection and minimax estimation in generalized linear models . IEEE Transactions on Information Theory 62 3721--3730 . 10.1109/TIT.2016.2555812 barticle
-
[2]
barticle [author] Alquier , Pierre P. Ridgway , James J. ( 2020 ). Concentration of tempered posteriors and of their variational approximations . The Annals of Statistics 48 1475--1497 . 10.1214/19-AOS1855 barticle
-
[3]
barticle [author] Bai , Ray R. ( 2025 ). Bayesian group regularization in generalized linear models with a continuous spike-and-slab prior . Annals of the Institute of Statistical Mathematics . Published online 18 October 2025 . 10.1007/s10463-025-00964-7 barticle
-
[4]
barticle [author] Bai , Ray R. , Moran , Gemma E G. E. , Antonelli , Joseph L J. L. , Chen , Yong Y. Boland , Mary R M. R. ( 2022 ). Spike-and-slab group lassos for grouped regression and sparse generalized additive models . Journal of the American Statistical Association 117 184--197 . 10.1080/01621459.2020.1765784 barticle
-
[5]
barticle [author] Belitser , Eduard E. Ghosal , Subhashis S. ( 2020 ). Empirical Bayes oracle uncertainty quantification for regression . The Annals of Statistics 48 3113--3137 . 10.1214/19-AOS1845 barticle
-
[6]
barticle [author] Bhattacharya , Anirban A. , Pati , Debdeep D. Yang , Yun Y. ( 2019 ). Bayesian fractional posteriors . The Annals of Statistics 47 39--66 . 10.1214/18-AOS1712 barticle
-
[7]
barticle [author] Carvalho , Carlos M C. M. , Polson , Nicholas G N. G. Scott , James G J. G. ( 2010 ). The horseshoe estimator for sparse signals . Biometrika 97 465--480 . 10.1093/biomet/asq017 barticle
-
[8]
barticle [author] Castillo , Isma \"e l I. , Schmidt-Hieber , Johannes J. van der Vaart , Aad A. ( 2015 ). Bayesian linear regression with sparse priors . The Annals of Statistics 43 1986--2018 . 10.1214/15-AOS1334 barticle
-
[9]
barticle [author] Ge , Changhao C. , Lin , Buyu B. Liu , Jun S J. S. ( 2025 ). A variational spike-and-slab approach for group variable selection . Bayesian Analysis . Advance publication . 10.1214/25-BA1537 barticle
-
[10]
barticle [author] George , Edward I E. I. McCulloch , Robert E R. E. ( 1993 ). Variable selection via G ibbs sampling . Journal of the American Statistical Association 88 881--889 . 10.1080/01621459.1993.10476353 barticle
-
[11]
( 1997 )
barticle [author] Ghosal , Subhashis S. ( 1997 ). Normal approximation to the posterior distribution for generalized linear models with many covariates . Mathematical Methods of Statistics 6 332--348 . 1475901 barticle
1997
-
[12]
barticle [author] Ghosal , Subhashis S. , Ghosh , Jayanta K. J. K. van der Vaart , Aad W. A. W. ( 2000 ). Convergence rates of posterior distributions . The Annals of Statistics 28 500--531 . 10.1214/aos/1016218228 barticle
-
[13]
barticle [author] Ghosal , Subhashis S. van der Vaart , Aad W. A. W. ( 2007 ). Convergence rates of posterior distributions for non-i.i.d. observations . The Annals of Statistics 35 192--223 . 10.1214/009053606000001172 barticle
-
[14]
barticle [author] Huang , Junzhou J. Zhang , Tong T. ( 2010 ). The benefit of group sparsity . The Annals of Statistics 38 1978--2004 . 10.1214/09-AOS778 barticle
-
[15]
barticle [author] Jeong , Seonghyun S. Ghosal , Subhashis S. ( 2021 ). Posterior contraction in sparse generalized linear models . Biometrika 108 367--379 . 10.1093/biomet/asaa074 barticle
-
[16]
barticle [author] Jiang , Wenxin W. ( 2007 ). Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities . The Annals of Statistics 35 1487--1511 . 10.1214/009053607000000019 barticle
-
[17]
barticle [author] Johnson , Valen E V. E. Rossell , David D. ( 2012 ). Bayesian model selection in high-dimensional settings . Journal of the American Statistical Association 107 649--660 . 10.1080/01621459.2012.682536 barticle
-
[18]
barticle [author] Komodromos , Michael M. , Evangelou , Marina M. , Filippi , Sarah S. Ray , Kolyan K. ( 2025 ). Group spike-and-slab variational Bayes . Bayesian Analysis . Advance publication . 10.1214/25-BA1542 barticle
-
[19]
bbook [author] Le Cam , Lucien L. ( 1986 ). Asymptotic Methods in Statistical Decision Theory . Springer , New York . 10.1007/978-1-4612-4946-7 bbook
-
[20]
barticle [author] Lee , Jeyong J. , Chae , Minwoo M. Martin , Ryan R. ( 2025 ). Advances in B ayesian model selection consistency for high-dimensional generalized linear models . The Annals of Statistics 53 2012--2039 . 10.1214/25-AOS2528 barticle
-
[21]
barticle [author] Liang , Faming F. , Song , Qifan Q. Yu , Kai K. ( 2013 ). Bayesian subset modeling for high-dimensional generalized linear models . Journal of the American Statistical Association 108 589--606 . 10.1080/01621459.2012.761942 barticle
-
[22]
barticle [author] Lounici , Karim K. , Pontil , Massimiliano M. , van de Geer , Sara S. Tsybakov , Alexandre B A. B. ( 2011 ). Oracle inequalities and optimal inference under group sparsity . The Annals of Statistics 39 2164--2204 . 10.1214/11-AOS896 barticle
-
[23]
barticle [author] Martin , Ryan R. , Mess , Raymond R. Walker , Stephen G S. G. ( 2017 ). Empirical Bayes posterior concentration in sparse high-dimensional linear models . Bernoulli 23 1822--1847 . 10.3150/15-BEJ797 barticle
-
[24]
bincollection [author] Martin , Ryan R. Syring , Nicholas N. ( 2022 ). Direct Gibbs posterior inference on risk minimizers: Construction, concentration, and calibration . In Handbook of Statistics , 47 1--41 . Elsevier . 10.1016/bs.host.2022.06.004 bincollection
-
[25]
barticle [author] Mitchell , Toby J T. J. Beauchamp , John J J. J. ( 1988 ). Bayesian variable selection in linear regression . Journal of the American Statistical Association 83 1023--1032 . 10.1080/01621459.1988.10478694 barticle
-
[26]
barticle [author] Narisetty , Naveen N N. N. He , Xuming X. ( 2014 ). Bayesian variable selection with shrinking and diffusing priors . The Annals of Statistics 42 789--817 . 10.1214/14-AOS1207 barticle
-
[27]
barticle [author] Negahban , Sahand N S. N. , Ravikumar , Pradeep P. , Wainwright , Martin J M. J. Yu , Bin B. ( 2012 ). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers . Statistical Science 27 538--557 . 10.1214/12-STS400 barticle
-
[28]
barticle [author] Nie , Lizhen L. Ro c kov \'a , Veronika V. ( 2023 ). Bayesian bootstrap spike-and-slab lasso . Journal of the American Statistical Association 118 2013--2028 . 10.1080/01621459.2022.2025815 barticle
-
[29]
barticle [author] Rigollet , Philippe P. ( 2012 ). Kullback- L eibler aggregation and misspecified generalized linear models . The Annals of Statistics 40 639--665 . 10.1214/11-AOS961 barticle
-
[30]
barticle [author] Ro c kov \'a , Veronika V. ( 2018 ). Bayesian estimation of sparse signals with a continuous spike-and-slab prior . The Annals of Statistics 46 401--437 . 10.1214/17-AOS1554 barticle
-
[31]
barticle [author] Ro c kov \'a , Veronika V. George , Edward I E. I. ( 2018 ). The spike-and-slab LASSO . Journal of the American Statistical Association 113 431--444 . 10.1080/01621459.2016.1260469 barticle
-
[32]
barticle [author] Spokoiny , Vladimir V. ( 2012 ). Parametric estimation. Finite sample theory . The Annals of Statistics 40 2877--2909 . 10.1214/12-AOS1054 barticle
-
[33]
barticle [author] Spokoiny , Vladimir V. ( 2017 ). Penalized maximum likelihood estimation and effective dimension . Annales de l'Institut Henri Poincar \'e , Probabilit \'e s et Statistiques 53 389--429 . 10.1214/15-AIHP720 barticle
-
[34]
barticle [author] Syring , Nicholas N. Martin , Ryan R. ( 2019 ). Calibrating general posterior credible regions . Biometrika 106 479--486 . 10.1093/biomet/asy054 barticle
-
[35]
bbook [author] Tadesse , Mahlet G M. G. Vannucci , Marina M. , eds. ( 2021 ). Handbook of Bayesian Variable Selection . Chapman and Hall/CRC , New York . 10.1201/9781003089018 bbook
-
[36]
barticle [author] Tang , Yiqi Y. Martin , Ryan R. ( 2024 ). Empirical B ayes inference in sparse high-dimensional generalized linear models . Electronic Journal of Statistics 18 3212--3246 . 10.1214/24-EJS2274 barticle
-
[37]
barticle [author] van de Geer , Sara A S. A. ( 2008 ). High-dimensional generalized linear models and the lasso . The Annals of Statistics 36 614--645 . 10.1214/009053607000000929 barticle
-
[38]
bbook [author] van der Vaart , Aad W A. W. ( 1998 ). Asymptotic Statistics . Cambridge University Press , Cambridge . 10.1017/CBO9780511802256 bbook
-
[39]
barticle [author] Yang , Yun Y. , Wainwright , Martin J M. J. Jordan , Michael I M. I. ( 2016 ). On the computational complexity of high-dimensional B ayesian variable selection . The Annals of Statistics 44 2497--2532 . 10.1214/15-AOS1417 barticle
-
[40]
barticle [author] Yuan , Ming M. Lin , Yi Y. ( 2006 ). Model selection and estimation in regression with grouped variables . Journal of the Royal Statistical Society Series B: Statistical Methodology 68 49--67 . 10.1111/j.1467-9868.2005.00532.x barticle
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.