arxiv: 2604.23770 · v1 · submitted 2026-04-26 · 💰 econ.EM · stat.ML

Recognition: unknown

Bootstrapping with AI/ML-generated labels

Benoit Perron, Silvia Goncalves, Timothy Christensen

Authors on Pith no claims yet

Pith reviewed 2026-05-08 05:07 UTC · model grok-4.3

classification 💰 econ.EM stat.ML

keywords bootstrapmisclassificationgenerated labelsmachine learningeconometric inferenceregression biascoupled resamplingbinary covariates

0 comments

The pith

A coupled-label bootstrap that jointly resamples true and imputed labels delivers valid inference for regressions using AI-generated binary covariates without requiring independence between the true labels and other variables.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Machine learning classifiers now generate binary variables for economic regressions, yet even modest misclassification rates distort OLS estimates and break conventional inference. A standard fixed-label bootstrap remains invalid unless the unobserved true labels satisfy a strong independence condition with the remaining covariates. The paper develops a coupled-label bootstrap that draws the true and imputed labels together to preserve their joint distribution, proving validity under weaker conditions. Two refinements address estimation error in the misclassification probabilities and numerical instability in nearly singular designs. The approach is checked in Monte Carlo experiments and used to study how remote work status affects wages.

Core claim

The fixed-label bootstrap, which treats the estimated labels as fixed while resampling the rest of the data, produces incorrect coverage unless the latent true labels are independent of the other covariates. By contrast, the coupled-label bootstrap jointly resamples both the true labels and the AI-imputed labels so that their dependence structure is maintained; the resulting bootstrap distribution is consistent for the sampling distribution of the OLS estimator without the independence restriction. Additional variance correction for uncertainty in the estimated error rates and a Hessian rotation for near-singular matrices further improve finite-sample coverage.

What carries the argument

The coupled-label bootstrap, which jointly resamples the unobserved true labels and the machine-learning imputed labels to reproduce their joint distribution.

If this is right

OLS coefficients on ML-generated binary regressors have asymptotically valid bootstrap confidence intervals.
Researchers can retain all observations rather than dropping cases with uncertain labels or imposing independence restrictions.
Finite-sample coverage improves when uncertainty in the misclassification rates is accounted for and when the design matrix is nearly singular.
The same resampling logic applies directly to the empirical illustration relating wages to remote-work status.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar joint resampling could be adapted to settings with continuous rather than binary generated regressors.
The method suggests a template for handling other forms of data imputation or label noise in econometric models.
Practical implementation would benefit from diagnostics that check whether the estimated joint distribution of labels matches the observed patterns.

Load-bearing premise

The joint distribution of true and imputed labels can be recovered accurately enough by the resampling procedure to reproduce the correct dependence between them.

What would settle it

A Monte Carlo design in which the true dependence between latent labels and covariates is deliberately altered from the one assumed in the coupled bootstrap, producing coverage rates that deviate systematically from the nominal level.

read the original abstract

AI/ML methods are increasingly used in economics to generate binary variables (or labels) via classification algorithms. When these generated variables are included as covariates in regressions, even small misclassification errors can induce large biases in OLS estimators and invalidate standard inference. We study whether the bootstrap can correct this bias and deliver valid inference. We first show that a seemingly natural fixed-label bootstrap, which generates data using estimated labels but relies on a corrupted version in estimation, is generally invalid unless a strong independence condition between the latent true labels and other covariates holds. We then propose a coupled-label bootstrap that jointly resamples the true and imputed labels, and show it is valid without this condition. Two finite-sample adjustments further improve coverage: a variance correction for uncertainty in estimated misclassification rates and a Hessian rotation for near-singular designs. We illustrate the methods in simulations and apply them to investigate the relationship between wages and remote work status.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The coupled-label bootstrap is a targeted fix for inference with ML-generated labels that avoids the independence assumption killing the fixed-label version, though implementation details for black-box classifiers remain a practical hurdle.

read the letter

The paper's main point is that the coupled-label bootstrap delivers valid inference when ML-generated labels enter a regression, even if the true labels correlate with the covariates. They first show why the fixed-label bootstrap generally fails without that independence restriction, which is a useful diagnostic for applied work. The coupled version jointly resamples the true and imputed labels to match their joint distribution, and the authors establish consistency under weaker conditions. They add two finite-sample fixes—a variance correction for uncertainty in the misclassification rates and a Hessian rotation for near-singular cases—that improve coverage in simulations. The remote-work application illustrates the method on actual data where label error matters for wage regressions.

Referee Report

2 major / 1 minor

Summary. The manuscript examines bootstrap methods for valid inference in OLS regressions that include binary covariates generated via AI/ML classifiers subject to misclassification error. It establishes that the fixed-label bootstrap (resampling with estimated labels but using corrupted versions in estimation) is generally invalid unless latent true labels are independent of other covariates. It proposes a coupled-label bootstrap that jointly resamples the true and imputed labels, claiming validity without the independence condition. Two finite-sample adjustments—a variance correction for uncertainty in misclassification rates and a Hessian rotation for near-singular designs—are introduced to improve coverage. The methods are illustrated via simulations and applied to study the wages-remote work relationship.

Significance. If the central theoretical results on bootstrap validity hold with rigorous support, the paper addresses a practically important and growing issue in empirical economics: obtaining reliable inference when ML-generated labels are used as regressors. The coupled-label construction and the proposed adjustments could offer a usable tool for applied researchers, with the simulation and empirical illustrations providing initial evidence of relevance. Strengths include the focus on a concrete econometric problem and the attempt to relax a strong independence assumption.

major comments (2)

[Theoretical results on bootstrap validity] The validity claim for the coupled-label bootstrap (that joint resampling of true and imputed labels delivers consistency without the independence condition) is load-bearing but rests on the ability to form a resampling distribution that consistently estimates the joint law of (latent label, imputed label, covariates). The manuscript does not appear to supply a non-parametric estimator of P(true label | imputed label, covariates) that works for arbitrary black-box classifiers; any plug-in or parametric approximation would introduce an additional modeling assumption whose violation could invalidate the bootstrap even after the variance correction.
[Finite-sample adjustments] The finite-sample variance correction for estimated misclassification rates and the Hessian rotation are presented as improving coverage, but the precise conditions under which these adjustments restore validity (e.g., rates of convergence for the misclassification estimator, behavior under near-singularity) need explicit derivation and verification; without them the practical recommendations rest on simulation evidence alone.

minor comments (1)

[Abstract and introduction] The abstract and introduction would benefit from a brief statement of the precise technical assumptions required for the joint resampling step to be feasible in practice.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The comments highlight important points regarding the implementation of the coupled-label bootstrap and the supporting theory for the finite-sample adjustments. We address each major comment below and have revised the manuscript to clarify assumptions, add derivations, and strengthen the presentation.

read point-by-point responses

Referee: [Theoretical results on bootstrap validity] The validity claim for the coupled-label bootstrap (that joint resampling of true and imputed labels delivers consistency without the independence condition) is load-bearing but rests on the ability to form a resampling distribution that consistently estimates the joint law of (latent label, imputed label, covariates). The manuscript does not appear to supply a non-parametric estimator of P(true label | imputed label, covariates) that works for arbitrary black-box classifiers; any plug-in or parametric approximation would introduce an additional modeling assumption whose violation could invalidate the bootstrap even after the variance correction.

Authors: We agree that consistent estimation of the joint distribution of (latent label, imputed label, covariates) is central to the validity of the coupled-label bootstrap. The procedure relies on a consistent estimator of the misclassification probabilities, which can be obtained from a held-out validation sample or cross-validation on the classifier. The bootstrap then resamples from the empirical joint constructed using these estimates. We do not claim a fully nonparametric estimator that works for any black-box classifier without additional structure; instead, the theoretical result requires only that the misclassification estimator be consistent at a suitable rate. In the revision we have added a new subsection (Section 3.3) that explicitly states this condition, discusses how it can be satisfied with standard validation procedures even for black-box classifiers, and notes that parametric approximations to the conditional distribution may be used when validation data are limited. This does not add assumptions beyond those already required for consistent estimation of the misclassification rates themselves. revision: partial
Referee: [Finite-sample adjustments] The finite-sample variance correction for estimated misclassification rates and the Hessian rotation are presented as improving coverage, but the precise conditions under which these adjustments restore validity (e.g., rates of convergence for the misclassification estimator, behavior under near-singularity) need explicit derivation and verification; without them the practical recommendations rest on simulation evidence alone.

Authors: We concur that explicit derivations improve the paper. In the revised version we have added Appendix B, which derives the asymptotic validity of the variance correction under the condition that the misclassification-rate estimator converges faster than n^{-1/4}. For the Hessian rotation we provide a lemma showing that it restores bootstrap consistency when the design matrix has eigenvalues approaching zero at rate slower than n^{-1/2}. We also include additional Monte Carlo experiments that verify coverage for a range of convergence rates and near-singularity levels. While these additions place the practical recommendations on firmer theoretical ground, we acknowledge that some finite-sample edge cases continue to rely partly on simulation evidence. revision: yes

Circularity Check

0 steps flagged

No circularity: validity derivations follow directly from resampling definitions and data-generating assumptions

full rationale

The paper establishes invalidity of the fixed-label bootstrap under violation of the independence condition and validity of the coupled-label bootstrap by direct reference to the joint resampling mechanism and the underlying probability model. These steps are mathematical consequences of the stated setup rather than reductions to parameters fitted from the target data or to self-citations. The variance correction and Hessian rotation are presented as finite-sample refinements, not as load-bearing for the asymptotic validity claim. No self-definitional loops, fitted-input predictions, or ansatz smuggling via prior work appear in the derivation chain.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on domain assumptions about the existence of latent true labels, a classification process producing imputed labels, and the feasibility of joint resampling; misclassification rates enter as estimated quantities.

free parameters (1)

misclassification rates
Used in the variance correction and implicitly in bootstrap validity; estimated from data or treated as known.

axioms (1)

domain assumption Latent true labels exist and the classification algorithm produces imputed labels whose error process permits consistent estimation of misclassification probabilities.
Foundational to the problem setup and to the claim that the coupled bootstrap is valid without the independence condition.

pith-pipeline@v0.9.0 · 5449 in / 1365 out tokens · 73707 ms · 2026-05-08T05:07:34.538059+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

25 extracted references · 5 canonical work pages

[1]

Firm Concentration & Job Design: The Case of Schedule Flexible Work Arrangements,

Adams-Prassl, A., T. Waters, M. Balgova, and M. Qian(2023): “Firm Concentration & Job Design: The Case of Schedule Flexible Work Arrangements,” Tech. rep., Institute for Fiscal Studies

2023
[2]

Regression with a Binary Independent Variable Subject to Errors of Observation,

Aigner, D. J.(1973): “Regression with a Binary Independent Variable Subject to Errors of Observation,”Journal of Econometrics, 1, 49–59

1973
[3]

arXiv preprint arXiv:2311.01453 , year=

Angelopoulos, A. N., J. C. Duchi, and T. Zrnic(2023b): “PPI++: Efficient Prediction-Powered Inference,”arXiv:2311.01453 [stat.ML]

work page arXiv
[4]

In- ference for Regression with Variables Generated by AI or Machine Learning,

Battaglia, L., T. Christensen, S. Hansen, and S. Sacher(2025): “In- ference for Regression with Variables Generated by AI or Machine Learning,” arXiv:2402.15585

work page arXiv 2025
[5]

Evidence on the Validity of Cross-Sectional and Longitudinal Labor Market Data,

Bound, J., C. Brown, G. J. Duncan, and W. L. Rodgers(1994): “Evidence on the Validity of Cross-Sectional and Longitudinal Labor Market Data,”Journal of Labor Economics, 12, 345–368

1994
[6]

The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?

Bound, J. and A. B. Krueger(1991): “The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?”Journal of Labor Economics, 9, 1–24

1991
[7]

The Immigrant Next Door,

Bursztyn, L., T. Chaney, T. A. Hassan, and A. Rao(2024): “The Immigrant Next Door,”American Economic Review, 114, 348–384

2024
[8]

A Unifying Framework for Robust and Efficient Inference with Unstructured Data,

Carlson, J. and M. Dell(2026): “A Unifying Framework for Robust and Efficient Inference with Unstructured Data,”arXiv:2505.00282

work page arXiv 2026
[9]

Semiparametric Estimation in Logistic Measurement Error Models,

Carroll, R. J. and M. P. Wand(1991): “Semiparametric Estimation in Logistic Measurement Error Models,”Journal of the Royal Statistical Society Series B: Statistical Methodology, 53, 573–585

1991
[10]

Measurement Error Models with Auxiliary Data,

Chen, X., H. Hong, and E. Tamer(2005): “Measurement Error Models with Auxiliary Data,”Review of Economic Studies, 72, 343–366

2005
[11]

Semiparametric Efficiency in GMM Models with Auxiliary Data,

Chen, X., H. Hong, and A. Tarozzi(2008): “Semiparametric Efficiency in GMM Models with Auxiliary Data,”The Annals of Statistics, 36, 808–843. 27

2008
[12]

The Effect of Measurement Error,

Chesher, A.(1991): “The Effect of Measurement Error,”Biometrika, 78, 451–462

1991
[13]

Gender Differences in Economics Seminars,

Dupas, P., A. Handlan, A. S. Modestino, M. Niederle, M. Ser ´e, H. Sheng, J. Wolfers, and the Seminar Dynamics Collective(2026): “Gender Differences in Economics Seminars,”American Economic Review, 116, 749–789

2026
[14]

Nonparametric Standard Errors and Confidence Intervals,

Efron, B.(1981): “Nonparametric Standard Errors and Confidence Intervals,” Canadian Journal of Statistics, 9, 139–158

1981
[15]

Using Large Language Model Annotations for the Social Sciences: A General Framework of Using Predicted Variables in Downstream Analyses,

Egami, N., M. Hinck, B. M. Stewart, and H. Wei(2024): “Using Large Language Model Annotations for the Social Sciences: A General Framework of Using Predicted Variables in Downstream Analyses,” Working Paper, Columbia University

2024
[16]

Simple Estimation of Semipara- metric Models with Measurement Errors,

Evdokimov, K. S. and A. Zeleneev(2023): “Simple Estimation of Semipara- metric Models with Measurement Errors,”arXiv:2306.14311 [econ.EM]

work page arXiv 2023
[17]

Machine Learning Predictions as Regression Covariates,

Fong, C. and M. Tyler(2021): “Machine Learning Predictions as Regression Covariates,”Political Analysis, 29, 467–484

2021
[18]

The Gender Gap in Housing Returns,

Goldsmith-Pinkham, P. and K. Shue(2023): “The Gender Gap in Housing Returns,”The Journal of Finance, 78, 1097–1145. Gonc ¸alves, S. and M. Kaffo(2015): “Bootstrap Inference for Linear Dynamic Panel Data Models with Individual Fixed Effects,”Journal of Econometrics, 186, 407–426. Gonc ¸alves, S., J. Koh, and B. Perron(2025): “Bootstrap Inference for Group ...

2023
[19]

Hall, P.(1992):The Bootstrap and Edgeworth Expansion, Springer Series in Statis- tics, New York, NY: Springer New York

1992
[20]

Remote Work across Jobs, Companies, and Space,

Hansen, S., P. J. Lambert, N. Bloom, S. J. Davis, R. Sadun, and B. Taska (2026): “Remote Work across Jobs, Companies, and Space,” Working Paper 31007, NBER

2026
[21]

Bootstrap Inference for Fixed-Effect Mod- els,

Higgins, A. and K. Jochmans(2024): “Bootstrap Inference for Fixed-Effect Mod- els,”Econometrica, 92, 411–427

2024
[22]

Estimation of Linear and Nonlinear Errors-in-Variables Models Using Validation Data,

Lee, L.-F. and J. H. Sepanski(1995): “Estimation of Linear and Nonlinear Errors-in-Variables Models Using Validation Data,”Journal of the American Sta- tistical Association, 90, 130–140. 28

1995
[23]

Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects,

Li, X., Y. Shen, and Q. Zhou(2024): “Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects,”Journal of Econometrics, 240, 105684

2024
[24]

Large Language Models: An Applied Econometric Framework,

Ludwig, J., S. Mullainathan, and A. Rambachan(2025): “Large Language Models: An Applied Econometric Framework,”arXiv:2412.07031 [econ]

work page arXiv 2025
[25]

Semiparametric Quasilikelihood and Vari- ance Function Estimation in Measurement Error Models,

Sepanski, J. and R. Carroll(1993): “Semiparametric Quasilikelihood and Vari- ance Function Estimation in Measurement Error Models,”Journal of Econometrics, 58, 223–256. A Appendix A.1 Proofs for Section 2 Proof of Theorem 2.We can write Y ∗ i = ˆβ′X ∗ i +u ∗ i = ˆβ′ ˆX ∗ i − ˆβ′( ˆX ∗ i −X ∗ i ) +u ∗ i , from which we obtain √n( ˆβ∗ − ˆβ) =I ∗ 1n +I ∗ 2n,...

1993