arxiv: 2605.11677 · v1 · submitted 2026-05-12 · 🧮 math.ST · stat.TH

Recognition: no theorem link

Bayesian and Empirical Bayesian Bootstrapping

Nils Lid Hjort

Authors on Pith no claims yet

Pith reviewed 2026-05-13 00:50 UTC · model grok-4.3

classification 🧮 math.ST stat.TH

keywords Bayesian bootstrapDirichlet process priorEfron bootstrapnonparametric Bayesian inferenceasymptotic equivalenceempirical Bayesresampling methodsposterior approximation

0 comments

The pith

Bayesian bootstrap with Dirichlet process prior equates to Efron's bootstrap and supports reading bootstrap histograms as approximate posteriors

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a nonparametric Bayesian bootstrap that places a Dirichlet process prior on the unknown distribution and uses simulation to approximate the posterior for any parameter defined as a functional of that distribution. It shows that this procedure, its approximations, Rubin's bootstrap, Efron's bootstrap, and delta-method intervals all agree to first order as the sample size grows, provided the functional is smooth. The same equivalence holds when prior parameters are estimated from the data in an empirical Bayes step, and the method carries over to regression and censored-data models. A reader would conclude that the classic bootstrap therefore admits a direct Bayesian interpretation.

Core claim

The author establishes that the Bayesian bootstrap is the nonparametric Bayes solution obtained from a Dirichlet process prior on the distribution, that Efron's bootstrap is recovered exactly when that prior is taken to be noninformative, that the bootstrap histogram approximates the posterior distribution of the parameter, and that the same first-order results follow from natural approximations to the Bayes solution or from empirical Bayes estimation of the prior.

What carries the argument

Simulation from the posterior distribution of the functional θ(P) induced by a Dirichlet process prior on the unknown distribution P.

If this is right

For any smooth functional, the exact Bayes solution, Bayesian bootstrap, Rubin's bootstrap, Efron's bootstrap, and delta-method intervals become first-order equivalent as sample size increases.
The procedure extends directly to semiparametric regression models.
Analogous results connect the method to bootstrapping for censored observations and general hazard-rate models.
Empirical Bayes versions of the procedure estimate the parameters of the Dirichlet process prior from the observed sample.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Bootstrap resampling outputs could be used directly as approximate posterior samples in settings where full nonparametric Bayesian computation is expensive.
The equivalence supplies a Bayesian rationale for using bootstrap standard errors or intervals when only frequentist resampling is feasible.
Similar Bayesian justifications might be developed for other resampling schemes such as permutation tests.

Load-bearing premise

The functional of interest must be smooth enough for the asymptotic equivalences between the different bootstrap procedures to hold.

What would settle it

A large-sample simulation in which Bayesian bootstrap credible intervals and Efron bootstrap confidence intervals for a smooth functional such as the mean exhibit materially different coverage rates would refute the claimed first-order equivalence.

read the original abstract

Let $X_1,\ldots,X_n$ be a random sample from an unknown probability distribution $P$ on the sample space ${\cal X}$, and let $\theta=\theta(P)$ be a parameter of interest. The present paper proposes a nonparametric `Bayesian bootstrap' method of obtaining Bayes estimates and Bayesian confidence limits for $\theta$. It uses a simple simulation technique to numerically approximate the exact posterior distribution of $\theta$ using a (non-degenerate) Dirichlet process prior for $P$. Asymptotic arguments are given which justify the use of the Bayesian bootstrap for any smooth functional $\theta(P)$. When the prior is fixed and the sample size grows five approaches become first-order equivalent: the exact Bayesian, the Bayesian bootstrap, Rubin's degenerate-prior bootstrap, Efron's bootstrap, and the classical one using delta methods. The Bayesian bootstrap method is also extended to the semiparametric regression case. A separate section treats similar ideas for censored data and for more general hazard rate models, where a connection is made to a `weird bootstrap' proposed by Gill. Finally empirical Bayesian versions of the procedure are discussed, where suitable parameters of the Dirichlet process prior are inferred from data. Our results lend Bayesian support to the classic Efron bootstrap. It is the Bayesian bootstrap under a noninformative reference prior; it is a limit of natural approximations to good Bayes solutions; it is an approximation to a natural empirical Bayesian strategy; and the formally incorrect reading of a bootstrap histogram as a posterior distribution for the parameter isn't so incorrect after all.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Hjort shows the Bayesian bootstrap with Dirichlet process prior is first-order equivalent to Efron's bootstrap for smooth functionals, giving a direct Bayesian justification without circularity.

read the letter

The core contribution is a simulation-based nonparametric Bayesian bootstrap that approximates the posterior for θ(P) under a Dirichlet process prior on P. For smooth functionals, this matches the usual bootstrap histogram asymptotically, along with Rubin's degenerate version and the delta-method normal limit. The paper also sketches extensions to semiparametric regression and to censored data via a connection to Gill's weird bootstrap, plus an empirical Bayes version that tunes the Dirichlet hyperparameters from the data. These links are clean and the argument that the bootstrap histogram can be read as an approximate posterior is presented without overclaiming. The asymptotic equivalences rest on the functional being smooth and the prior staying fixed while n grows, which is stated clearly. The main soft spot is that the differentiability conditions and remainder terms are not spelled out in full detail in the abstract-level description; if the influence function is only once differentiable or the convergence is not uniform, the first-order equivalence could leave higher-order discrepancies unaddressed. That said, the central claim does not appear to rely on circular fitting, and the simulation step is justified separately. The work is aimed at statisticians who already use the bootstrap and want a Bayesian reading of it, or who are building nonparametric Bayesian procedures. It is worth sending to referees because the unification is useful and the technical steps look reproducible once the full derivations are checked.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a nonparametric Bayesian bootstrap that simulates from the posterior of a smooth functional θ(P) under a Dirichlet process prior on the unknown distribution P. It presents asymptotic arguments establishing first-order equivalence, as n→∞ with fixed prior, among the exact Bayesian posterior, the Bayesian bootstrap approximation, Rubin's degenerate-prior bootstrap, Efron's multinomial bootstrap, and the delta-method normal approximation. The work extends the approach to semiparametric regression, censored data (linking to Gill's weird bootstrap), and empirical Bayes procedures that estimate Dirichlet hyperparameters from data, concluding that these connections supply Bayesian support for the classical Efron bootstrap.

Significance. If the equivalences are placed on a rigorous footing, the paper would usefully connect Bayesian nonparametric inference with frequentist resampling, showing how bootstrap histograms can serve as approximate posteriors and how empirical Bayes versions arise naturally. The extensions to regression and hazard models add practical value, and the explicit framing of Efron's bootstrap as a limit of Bayesian procedures offers a coherent justification that remains relevant for modern nonparametric statistics.

major comments (2)

[asymptotic arguments section] The asymptotic equivalence claims (abstract and the section presenting the limiting-distribution arguments) rest on the assumption that θ(P) is smooth, yet no explicit differentiability class is stated (e.g., Hadamard or Fréchet differentiability of the functional, or square-integrability of the influence function) and no remainder bounds or uniformity statements are supplied. This is load-bearing for the asserted first-order equivalence of the five procedures when the prior is fixed and n grows.
[Bayesian bootstrap simulation section] The simulation procedure that approximates the exact Dirichlet-process posterior is central to the Bayesian bootstrap, but the manuscript supplies no convergence rate for the Monte Carlo approximation or error bounds relative to the true posterior; without these, the justification that the simulated histogram can be read as an approximate posterior (and hence that Efron's bootstrap is 'not so incorrect') remains incomplete.

minor comments (2)

[Abstract] The abstract lists five approaches but the wording 'the exact Bayesian, the Bayesian bootstrap, Rubin's degenerate-prior bootstrap, Efron's bootstrap, and the classical one using delta methods' could be clarified to distinguish the exact posterior from its simulation-based approximation.
[empirical Bayes section] Notation for the Dirichlet process hyperparameters and their empirical Bayes estimation could be made more uniform across the empirical Bayes section to aid readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and detailed report. The comments highlight important points regarding the rigor of the asymptotic arguments and the simulation procedure. We respond to each major comment below and indicate the revisions we plan to make.

read point-by-point responses

Referee: [asymptotic arguments section] The asymptotic equivalence claims (abstract and the section presenting the limiting-distribution arguments) rest on the assumption that θ(P) is smooth, yet no explicit differentiability class is stated (e.g., Hadamard or Fréchet differentiability of the functional, or square-integrability of the influence function) and no remainder bounds or uniformity statements are supplied. This is load-bearing for the asserted first-order equivalence of the five procedures when the prior is fixed and n grows.

Authors: We agree that greater precision on the smoothness assumption would strengthen the paper. The term 'smooth functional' in the manuscript is intended to refer to functionals that are differentiable in the sense required for the delta method, specifically those admitting a first-order expansion with an influence function that is square-integrable under P. In the revised manuscript, we will explicitly state the assumption of Hadamard differentiability at the true P with square-integrable influence function, citing the relevant literature on asymptotic equivalence of bootstrap and delta-method approximations. We note that since the prior is fixed and we consider pointwise asymptotics as n → ∞, uniformity over neighborhoods of P is not required for the first-order equivalence claims. Remainder bounds follow from the standard theory under these differentiability conditions. revision: yes
Referee: [Bayesian bootstrap simulation section] The simulation procedure that approximates the exact Dirichlet-process posterior is central to the Bayesian bootstrap, but the manuscript supplies no convergence rate for the Monte Carlo approximation or error bounds relative to the true posterior; without these, the justification that the simulated histogram can be read as an approximate posterior (and hence that Efron's bootstrap is 'not so incorrect') remains incomplete.

Authors: The simulation in the Bayesian bootstrap is a Monte Carlo approximation to the posterior distribution induced by the Dirichlet process prior. While the manuscript does not provide explicit convergence rates, the number of Monte Carlo draws B can be chosen independently of n to make the approximation error arbitrarily small. We will revise the manuscript to include a discussion of the Monte Carlo error, noting that it is typically of order O(1/√B) in probability for the empirical distribution of the simulated values, and can be made negligible relative to the O(1/√n) statistical error by taking B large (e.g., B = 1000 or more). However, deriving non-asymptotic error bounds between the simulated histogram and the exact posterior would require additional assumptions on the functional θ and is beyond the scope of the current work, which focuses on the large-sample equivalence. revision: partial

Circularity Check

0 steps flagged

No circularity: asymptotic equivalences derived from independent limiting arguments

full rationale

The paper establishes first-order asymptotic equivalence (as n→∞ with fixed prior) among the exact posterior under Dirichlet process, its Bayesian bootstrap simulation, Rubin's version, Efron's multinomial bootstrap, and the delta-method normal limit, all for smooth functionals θ(P). These limits are obtained via standard weak-convergence arguments rather than parameter fitting, self-definition, or load-bearing self-citation. The claim that Efron's bootstrap thereby receives Bayesian support is a direct consequence of the derived limits, not a renaming or reduction of the result to its own inputs. The smoothness assumption is stated explicitly as a prerequisite; no ansatz is smuggled via citation and no prediction is forced by construction from a fitted subset.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The approach relies on standard properties of the Dirichlet process prior in Bayesian nonparametrics and asymptotic statistical theory for smooth functionals.

free parameters (1)

Dirichlet process hyperparameters
Fixed in the main method or inferred from data in the empirical Bayesian version.

axioms (1)

domain assumption θ(P) is a smooth functional of the distribution P
This is required for the asymptotic first-order equivalence to hold with other methods.

pith-pipeline@v0.9.0 · 5568 in / 1183 out tokens · 50270 ms · 2026-05-13T00:50:42.661908+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

[1]

and Freedman, D.A

Bickel, P.J. and Freedman, D.A. (1981). Some asymptotic theory for the bootstrap.Annals of Statistics9. 1196–1217. Billingsley, P. (1968).Convergence of Probability Measures.Wiley, Singapore. Boos, D.D. and Serﬂing, R.J. (1980). A note on diﬀerentials and the CLT and LIL for statistical functions with application to M-estimates.Annals of Statistics8, 618–...

work page 1981
[2]

Lo, A.Y. (1987). A large-sample study of the Bayesian bootstrap.Annals of Statistics15, 360–375. Lo, A.Y. (1991). A Bayesian bootstrap for censored data. Technical report, Department of Statistics, SUNY at Buﬀalo. Newton, M.A. and Raftery, A.E. (1991). Approximate Bayesian inference by the weighted likelihood bootstrap. Technical report, Department of Sta...

work page 1987
[3]

Academic Press, New York. Shao, J. (1989). Functional calculus and asymptotic theory for statistical analysis.Statis- tics and Probability Letters8, 397–405. Singh, K. (1981). On the asymptotic accuracy of Efron’s bootstrap.Annals of Statistics9, 1187–1195. Yamato, H. (1984). Characteristic functions of means of distributions chosen from a Dirich- let pro...

work page 1989