pith. sign in

arxiv: 2605.27137 · v1 · pith:ORI2WQNUnew · submitted 2026-05-26 · 🧮 math.ST · stat.ME· stat.TH

Bernstein-von Mises Theorem for Sparse Generalized Linear Model

Pith reviewed 2026-07-01 15:51 UTC · model grok-4.3

classification 🧮 math.ST stat.MEstat.TH
keywords Bernstein-von Mises theoremsparse generalized linear modelsspike-and-slab priorsfractional posteriororacle propertysupport recoverygeneralized linear models
0
0 comments X

The pith

The fractional posterior for sparse generalized linear models with spike-and-slab priors satisfies an oracle Bernstein-von Mises theorem.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proves that under supportwise likelihood assumptions, the fractional posterior in sparse generalized linear models concentrates and approximates a Gaussian distribution centered at the oracle estimator that knows the true support in advance. This is established for spike-and-slab priors allowing grouped sparsity. A reader would care because it provides a theoretical justification for Bayesian inference in high-dimensional sparse settings, where the posterior can be used for uncertainty quantification as if the model were known to be sparse. The result covers several common generalized linear models including logistic and Poisson regression.

Core claim

The main result is an oracle Bernstein-von Mises theorem for the fractional posterior under supportwise likelihood assumptions. The proof develops sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, and combines them with fixed-prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-entry verifications are given for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions. The ordinary posterior is treated only

What carries the argument

Sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers under supportwise likelihood assumptions.

If this is right

  • The posterior contracts around the true sparse parameter at the oracle rate.
  • Support recovery occurs with high probability under the beta-min separation condition.
  • The posterior approximates a Gaussian mixture that collapses to the oracle Gaussian law.
  • The theorem applies to fractional posteriors in Gaussian, logistic, Poisson, probit, Gamma, and negative-binomial regressions under the stated conditions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Credible intervals from the fractional posterior may achieve asymptotic frequentist coverage when the support is recovered.
  • The approach could extend to other high-dimensional models where supportwise conditions can be verified.
  • Model selection uncertainty becomes asymptotically negligible once the beta-min condition ensures support recovery.

Load-bearing premise

The supportwise likelihood assumptions must hold to enable local asymptotic normality and approximation around each possible support.

What would settle it

A simulation study where the fractional posterior for a sparse logistic regression model fails to concentrate around the oracle estimator or deviates from Gaussianity would falsify the central claim.

read the original abstract

We study spike-and-slab priors for generalized linear models with possible grouped sparsity. The main result is an oracle Bernstein--von Mises theorem for the fractional posterior under supportwise likelihood assumptions. The proof develops sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, and combines them with fixed-prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-entry verifications are given for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions. The ordinary posterior is treated only through restricted Gaussian and canonical-link extensions, with coverage under additional active-dimension and moment conditions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript establishes an oracle Bernstein-von Mises theorem for the fractional posterior under spike-and-slab priors in generalized linear models with grouped sparsity. The argument proceeds by establishing sparse local asymptotic normality and Laplace approximation around support-specific pseudo-true centers, then invoking fixed prior mass, support penalization, recovery geometry, and beta-min separation to obtain contraction, exact support recovery, Gaussian mixture approximation, and collapse to the oracle Gaussian law. Model-specific verifications are supplied for Gaussian regression and for logistic, Poisson, probit, Gamma log-link, and negative-binomial log-link regression under stated sufficient conditions; the ordinary posterior receives only restricted treatment.

Significance. If the supportwise likelihood assumptions hold and close the LAN remainder uniformly, the result supplies a rigorous justification for asymptotic normality of fractional posteriors in sparse high-dimensional GLMs. This would strengthen the theoretical foundation for Bayesian inference procedures that rely on oracle properties and mixture collapse, particularly for the listed GLM families where explicit verifications are given.

major comments (1)
  1. The supportwise likelihood assumptions (invoked throughout the proof sketch in the abstract to control the sparse LAN expansion and enable collapse of the Gaussian mixture to the oracle law) are load-bearing yet non-standard. The manuscript must supply their precise statement and demonstrate that the model-specific sufficient conditions for the five GLM families uniformly bound the LAN remainder over the recovered supports; without this, the subsequent contraction and oracle BvM steps do not follow.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments on our manuscript. We address the single major comment below and agree that greater explicitness on the supportwise likelihood assumptions will improve clarity. The revised manuscript will incorporate the requested clarifications and verifications.

read point-by-point responses
  1. Referee: The supportwise likelihood assumptions (invoked throughout the proof sketch in the abstract to control the sparse LAN expansion and enable collapse of the Gaussian mixture to the oracle law) are load-bearing yet non-standard. The manuscript must supply their precise statement and demonstrate that the model-specific sufficient conditions for the five GLM families uniformly bound the LAN remainder over the recovered supports; without this, the subsequent contraction and oracle BvM steps do not follow.

    Authors: We agree that the supportwise likelihood assumptions are central and that their uniform control of the LAN remainder must be verified explicitly for the stated GLM families. These assumptions appear in the manuscript as Assumption 2.1 (Supportwise LAN), which requires that the log-likelihood ratio expansion holds with remainder o_p(1) uniformly over all supports of cardinality at most s_n. Sections 4.1–4.5 supply model-specific sufficient conditions (bounded design, beta-min separation, moment bounds) under which the LAN remainder is controlled for each of the five families. To address the referee’s concern directly, the revision will add a new Lemma 4.6 that aggregates these verifications and proves the uniform o_p(1) bound over the recovered supports, thereby closing the gap between the sufficient conditions and Assumption 2.1. This addition will be placed immediately before the main contraction and BvM arguments. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on external assumptions and standard expansions

full rationale

The claimed oracle BvM result is obtained by first establishing sparse LAN and Laplace approximation around support-specific centers, then applying fixed-prior mass, support penalization, recovery geometry and beta-min conditions to reach contraction, exact support recovery and collapse to the oracle Gaussian law. These steps invoke stated supportwise likelihood assumptions whose content is external to the derivation and are verified model-by-model under sufficient conditions; no equation or step is shown to reduce by construction to a fitted parameter, self-citation chain, or renamed input. The ordinary-posterior extensions are likewise handled under additional explicit conditions. The derivation chain is therefore self-contained against the listed assumptions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed on abstract only; full assumptions and proof details unavailable.

axioms (1)
  • domain assumption supportwise likelihood assumptions
    Invoked as the main condition enabling the oracle BvM result.

pith-pipeline@v0.9.1-grok · 5651 in / 1018 out tokens · 24479 ms · 2026-07-01T15:51:02.853956+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 39 canonical work pages

  1. [1]

    Grinshtein , Vadim V

    barticle [author] Abramovich , Felix F. Grinshtein , Vadim V. ( 2016 ). Model selection and minimax estimation in generalized linear models . IEEE Transactions on Information Theory 62 3721--3730 . 10.1109/TIT.2016.2555812 barticle

  2. [2]

    Ridgway , James J

    barticle [author] Alquier , Pierre P. Ridgway , James J. ( 2020 ). Concentration of tempered posteriors and of their variational approximations . The Annals of Statistics 48 1475--1497 . 10.1214/19-AOS1855 barticle

  3. [3]

    ( 2025 )

    barticle [author] Bai , Ray R. ( 2025 ). Bayesian group regularization in generalized linear models with a continuous spike-and-slab prior . Annals of the Institute of Statistical Mathematics . Published online 18 October 2025 . 10.1007/s10463-025-00964-7 barticle

  4. [4]

    , Moran , Gemma E G

    barticle [author] Bai , Ray R. , Moran , Gemma E G. E. , Antonelli , Joseph L J. L. , Chen , Yong Y. Boland , Mary R M. R. ( 2022 ). Spike-and-slab group lassos for grouped regression and sparse generalized additive models . Journal of the American Statistical Association 117 184--197 . 10.1080/01621459.2020.1765784 barticle

  5. [5]

    Ghosal , Subhashis S

    barticle [author] Belitser , Eduard E. Ghosal , Subhashis S. ( 2020 ). Empirical Bayes oracle uncertainty quantification for regression . The Annals of Statistics 48 3113--3137 . 10.1214/19-AOS1845 barticle

  6. [6]

    , Pati , Debdeep D

    barticle [author] Bhattacharya , Anirban A. , Pati , Debdeep D. Yang , Yun Y. ( 2019 ). Bayesian fractional posteriors . The Annals of Statistics 47 39--66 . 10.1214/18-AOS1712 barticle

  7. [7]

    barticle [author] Carvalho , Carlos M C. M. , Polson , Nicholas G N. G. Scott , James G J. G. ( 2010 ). The horseshoe estimator for sparse signals . Biometrika 97 465--480 . 10.1093/biomet/asq017 barticle

  8. [8]

    , Schmidt-Hieber , Johannes J

    barticle [author] Castillo , Isma \"e l I. , Schmidt-Hieber , Johannes J. van der Vaart , Aad A. ( 2015 ). Bayesian linear regression with sparse priors . The Annals of Statistics 43 1986--2018 . 10.1214/15-AOS1334 barticle

  9. [9]

    , Lin , Buyu B

    barticle [author] Ge , Changhao C. , Lin , Buyu B. Liu , Jun S J. S. ( 2025 ). A variational spike-and-slab approach for group variable selection . Bayesian Analysis . Advance publication . 10.1214/25-BA1537 barticle

  10. [10]

    barticle [author] George , Edward I E. I. McCulloch , Robert E R. E. ( 1993 ). Variable selection via G ibbs sampling . Journal of the American Statistical Association 88 881--889 . 10.1080/01621459.1993.10476353 barticle

  11. [11]

    ( 1997 )

    barticle [author] Ghosal , Subhashis S. ( 1997 ). Normal approximation to the posterior distribution for generalized linear models with many covariates . Mathematical Methods of Statistics 6 332--348 . 1475901 barticle

  12. [12]

    , Ghosh , Jayanta K

    barticle [author] Ghosal , Subhashis S. , Ghosh , Jayanta K. J. K. van der Vaart , Aad W. A. W. ( 2000 ). Convergence rates of posterior distributions . The Annals of Statistics 28 500--531 . 10.1214/aos/1016218228 barticle

  13. [13]

    van der Vaart , Aad W

    barticle [author] Ghosal , Subhashis S. van der Vaart , Aad W. A. W. ( 2007 ). Convergence rates of posterior distributions for non-i.i.d. observations . The Annals of Statistics 35 192--223 . 10.1214/009053606000001172 barticle

  14. [14]

    Zhang , Tong T

    barticle [author] Huang , Junzhou J. Zhang , Tong T. ( 2010 ). The benefit of group sparsity . The Annals of Statistics 38 1978--2004 . 10.1214/09-AOS778 barticle

  15. [15]

    Ghosal , Subhashis S

    barticle [author] Jeong , Seonghyun S. Ghosal , Subhashis S. ( 2021 ). Posterior contraction in sparse generalized linear models . Biometrika 108 367--379 . 10.1093/biomet/asaa074 barticle

  16. [16]

    ( 2007 )

    barticle [author] Jiang , Wenxin W. ( 2007 ). Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities . The Annals of Statistics 35 1487--1511 . 10.1214/009053607000000019 barticle

  17. [17]

    barticle [author] Johnson , Valen E V. E. Rossell , David D. ( 2012 ). Bayesian model selection in high-dimensional settings . Journal of the American Statistical Association 107 649--660 . 10.1080/01621459.2012.682536 barticle

  18. [18]

    , Evangelou , Marina M

    barticle [author] Komodromos , Michael M. , Evangelou , Marina M. , Filippi , Sarah S. Ray , Kolyan K. ( 2025 ). Group spike-and-slab variational Bayes . Bayesian Analysis . Advance publication . 10.1214/25-BA1542 barticle

  19. [19]

    ( 1986 )

    bbook [author] Le Cam , Lucien L. ( 1986 ). Asymptotic Methods in Statistical Decision Theory . Springer , New York . 10.1007/978-1-4612-4946-7 bbook

  20. [20]

    , Chae , Minwoo M

    barticle [author] Lee , Jeyong J. , Chae , Minwoo M. Martin , Ryan R. ( 2025 ). Advances in B ayesian model selection consistency for high-dimensional generalized linear models . The Annals of Statistics 53 2012--2039 . 10.1214/25-AOS2528 barticle

  21. [21]

    , Song , Qifan Q

    barticle [author] Liang , Faming F. , Song , Qifan Q. Yu , Kai K. ( 2013 ). Bayesian subset modeling for high-dimensional generalized linear models . Journal of the American Statistical Association 108 589--606 . 10.1080/01621459.2012.761942 barticle

  22. [22]

    , Pontil , Massimiliano M

    barticle [author] Lounici , Karim K. , Pontil , Massimiliano M. , van de Geer , Sara S. Tsybakov , Alexandre B A. B. ( 2011 ). Oracle inequalities and optimal inference under group sparsity . The Annals of Statistics 39 2164--2204 . 10.1214/11-AOS896 barticle

  23. [23]

    , Mess , Raymond R

    barticle [author] Martin , Ryan R. , Mess , Raymond R. Walker , Stephen G S. G. ( 2017 ). Empirical Bayes posterior concentration in sparse high-dimensional linear models . Bernoulli 23 1822--1847 . 10.3150/15-BEJ797 barticle

  24. [24]

    Syring , Nicholas N

    bincollection [author] Martin , Ryan R. Syring , Nicholas N. ( 2022 ). Direct Gibbs posterior inference on risk minimizers: Construction, concentration, and calibration . In Handbook of Statistics , 47 1--41 . Elsevier . 10.1016/bs.host.2022.06.004 bincollection

  25. [25]

    barticle [author] Mitchell , Toby J T. J. Beauchamp , John J J. J. ( 1988 ). Bayesian variable selection in linear regression . Journal of the American Statistical Association 83 1023--1032 . 10.1080/01621459.1988.10478694 barticle

  26. [26]

    barticle [author] Narisetty , Naveen N N. N. He , Xuming X. ( 2014 ). Bayesian variable selection with shrinking and diffusing priors . The Annals of Statistics 42 789--817 . 10.1214/14-AOS1207 barticle

  27. [27]

    barticle [author] Negahban , Sahand N S. N. , Ravikumar , Pradeep P. , Wainwright , Martin J M. J. Yu , Bin B. ( 2012 ). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers . Statistical Science 27 538--557 . 10.1214/12-STS400 barticle

  28. [28]

    Ro c kov \'a , Veronika V

    barticle [author] Nie , Lizhen L. Ro c kov \'a , Veronika V. ( 2023 ). Bayesian bootstrap spike-and-slab lasso . Journal of the American Statistical Association 118 2013--2028 . 10.1080/01621459.2022.2025815 barticle

  29. [29]

    ( 2012 )

    barticle [author] Rigollet , Philippe P. ( 2012 ). Kullback- L eibler aggregation and misspecified generalized linear models . The Annals of Statistics 40 639--665 . 10.1214/11-AOS961 barticle

  30. [30]

    ( 2018 )

    barticle [author] Ro c kov \'a , Veronika V. ( 2018 ). Bayesian estimation of sparse signals with a continuous spike-and-slab prior . The Annals of Statistics 46 401--437 . 10.1214/17-AOS1554 barticle

  31. [31]

    George , Edward I E

    barticle [author] Ro c kov \'a , Veronika V. George , Edward I E. I. ( 2018 ). The spike-and-slab LASSO . Journal of the American Statistical Association 113 431--444 . 10.1080/01621459.2016.1260469 barticle

  32. [32]

    ( 2012 )

    barticle [author] Spokoiny , Vladimir V. ( 2012 ). Parametric estimation. Finite sample theory . The Annals of Statistics 40 2877--2909 . 10.1214/12-AOS1054 barticle

  33. [33]

    ( 2017 )

    barticle [author] Spokoiny , Vladimir V. ( 2017 ). Penalized maximum likelihood estimation and effective dimension . Annales de l'Institut Henri Poincar \'e , Probabilit \'e s et Statistiques 53 389--429 . 10.1214/15-AIHP720 barticle

  34. [34]

    Martin , Ryan R

    barticle [author] Syring , Nicholas N. Martin , Ryan R. ( 2019 ). Calibrating general posterior credible regions . Biometrika 106 479--486 . 10.1093/biomet/asy054 barticle

  35. [35]

    bbook [author] Tadesse , Mahlet G M. G. Vannucci , Marina M. , eds. ( 2021 ). Handbook of Bayesian Variable Selection . Chapman and Hall/CRC , New York . 10.1201/9781003089018 bbook

  36. [36]

    Martin , Ryan R

    barticle [author] Tang , Yiqi Y. Martin , Ryan R. ( 2024 ). Empirical B ayes inference in sparse high-dimensional generalized linear models . Electronic Journal of Statistics 18 3212--3246 . 10.1214/24-EJS2274 barticle

  37. [37]

    barticle [author] van de Geer , Sara A S. A. ( 2008 ). High-dimensional generalized linear models and the lasso . The Annals of Statistics 36 614--645 . 10.1214/009053607000000929 barticle

  38. [38]

    bbook [author] van der Vaart , Aad W A. W. ( 1998 ). Asymptotic Statistics . Cambridge University Press , Cambridge . 10.1017/CBO9780511802256 bbook

  39. [39]

    , Wainwright , Martin J M

    barticle [author] Yang , Yun Y. , Wainwright , Martin J M. J. Jordan , Michael I M. I. ( 2016 ). On the computational complexity of high-dimensional B ayesian variable selection . The Annals of Statistics 44 2497--2532 . 10.1214/15-AOS1417 barticle

  40. [40]

    Lin , Yi Y

    barticle [author] Yuan , Ming M. Lin , Yi Y. ( 2006 ). Model selection and estimation in regression with grouped variables . Journal of the Royal Statistical Society Series B: Statistical Methodology 68 49--67 . 10.1111/j.1467-9868.2005.00532.x barticle