Is Inference Conditional on Not Rejecting a Pre-test Less Reliable than Unconditional Inference?

Cl\'ement de Chaisemartin; Xavier D'Haultf{\oe}uille

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

Conditional inference after not rejecting a pre-test for asymptotic normality conditions remains valid, though typically conservative.

2026-05-23 23:17 UTC

load-bearing objection The paper shows conditional inference after a non-rejected pre-test stays valid under the maintained conditions, and the validity does not depend on asymptotic dependence between the estimator and the pre-test.

arxiv 2407.03725 v3 submitted 2024-07-04 econ.EM stat.ME

Is Inference Conditional on Not Rejecting a Pre-test Less Reliable than Unconditional Inference?

Cl\'ement de Chaisemartin , Xavier D'Haultf{\oe}uille This is my paper

classification econ.EM stat.ME

keywords pre-testingconditional inferenceasymptotic normalitycoverage probabilityeconometricshypothesis testing

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether restricting inference to cases where a pre-test for the conditions needed for asymptotic normality does not reject undermines the reliability of that inference. It establishes that when those conditions and mild regularity restrictions hold, the resulting conditional confidence intervals still achieve at least the nominal coverage level. This validity result holds even when the estimator and the pre-test statistic are asymptotically dependent. Readers who routinely run specification tests before reporting standard errors or intervals can therefore continue to do so without introducing invalidity, although the intervals will often be wider than they would be without the pre-test.

Core claim

Assume that an estimator is asymptotically normal for a target parameter under some conditions. Suppose also that one can test these conditions, and one conducts inference for the target only if the pre-test is not rejected. We show that if the tested conditions and mild regularity restrictions hold, conditional inference is still valid, albeit typically conservative. Validity holds regardless of the asymptotic dependence between the estimator and the pre-test. If the tested conditions do not hold, we exhibit conditions under which confidence intervals have larger conditional than unconditional coverage.

What carries the argument

The conditional coverage probability of the confidence interval given that the pre-test is not rejected; the proof shows this probability is bounded below by the nominal level whenever the maintained conditions hold.

Load-bearing premise

The estimator is asymptotically normal under the conditions that the pre-test is checking, and the pre-test is consistent for those conditions.

What would settle it

A concrete counter-example in which the tested conditions hold, the pre-test does not reject with positive probability, yet the conditional coverage probability falls strictly below the nominal level.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

When the tested conditions hold, conditional inference achieves correct or higher coverage.
The validity result does not require the estimator and pre-test to be asymptotically independent.
When the tested conditions fail, there exist cases in which conditional coverage exceeds unconditional coverage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The result may reduce reluctance to report diagnostic tests before main results in applied work.
Similar arguments could be examined for other common pre-tests such as those for instrument strength.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The paper shows conditional inference after a non-rejected pre-test stays valid under the maintained conditions, and the validity does not depend on asymptotic dependence between the estimator and the pre-test.

read the letter

The main takeaway is that pre-testing does not invalidate inference in the way many people worry. Under the paper's assumptions plus some mild regularity conditions, the conditional coverage probability after failing to reject the pre-test is still at least the nominal level, though usually conservative. This holds even when the estimator and the pre-test statistic are asymptotically dependent. When the tested conditions fail, the paper gives examples where the conditional intervals actually have better coverage than the unconditional ones. That pair of results is the concrete contribution. It sits on top of standard asymptotic normality arguments rather than introducing new machinery, which keeps the claim straightforward. The result directly addresses a recurring practical question in applied work where researchers test for things like instrument validity or heteroskedasticity before proceeding. The soft spot is that the abstract leaves the exact regularity restrictions and the definition of the pre-test somewhat implicit, so a reader has to wait for the full proof to see how restrictive they turn out to be in common settings. No circularity or hidden fitting appears in the stated claim. The paper is aimed at econometricians who routinely condition on pre-test outcomes and want a clear asymptotic statement rather than simulation evidence. It is worth sending to referees because the question is live in the literature and the answer is stated cleanly enough to be checked.

Referee Report

0 major / 1 minor

Summary. The manuscript examines whether inference for a target parameter, conducted only after not rejecting a pre-test for the conditions supporting asymptotic normality of an estimator, is less reliable than unconditional inference. The central claim is that if the tested conditions and mild regularity restrictions hold, then conditional coverage remains valid (typically conservative), and this holds irrespective of asymptotic dependence between the estimator and pre-test statistic. When the conditions fail, the paper exhibits cases where conditional coverage exceeds unconditional coverage.

Significance. If the result holds under the stated assumptions, it would be a useful clarification for econometric practice regarding pre-testing for model conditions, indicating that conditional inference does not undermine validity in the manner sometimes feared. The result addresses the dependence between estimator and pre-test, which is a relevant but often secondary concern in the literature on pre-test bias.

minor comments (1)

The abstract states the main result but does not outline the key steps in the derivation or provide explicit statements of the mild regularity restrictions; the full manuscript should include these to allow readers to verify the conditions under which the claim applies.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their careful reading and positive assessment of the manuscript. The referee correctly summarizes the central claim and notes its potential usefulness as a clarification for practice. No specific major comments are listed in the report, so we have no points requiring response or revision. We agree with the recommendation for minor revision only in the sense that we remain open to any editorial suggestions, but none are indicated here.

Circularity Check

0 steps flagged

Derivation self-contained under standard asymptotics

full rationale

The paper establishes validity of conditional inference after a non-rejected pre-test directly from the maintained assumption of asymptotic normality of the estimator plus mild regularity restrictions. The result is shown to hold irrespective of dependence between estimator and pre-test, and counter-examples are exhibited when assumptions fail. No equation reduces a claimed prediction to a fitted input by construction, no load-bearing premise rests on self-citation chains, and no ansatz is smuggled via prior work. The derivation is therefore independent of the target claim and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Based on abstract only; the paper starts from standard asymptotic normality and invokes unspecified mild regularity restrictions.

axioms (2)

domain assumption Estimator is asymptotically normal for target parameter under tested conditions
Stated as the foundational setup in the abstract.
ad hoc to paper Mild regularity restrictions hold
Invoked to support the validity result but not defined or justified in the abstract.

pith-pipeline@v0.9.0 · 5622 in / 1077 out tokens · 24724 ms · 2026-05-23T23:17:34.615372+00:00 · methodology

0 comments

read the original abstract

Assume that an estimator is asymptotically normal for a target parameter under some conditions. Suppose also that one can test these conditions, and one conducts inference for the target only if the pre-test is not rejected. Does such pre-testing undermine inference? We show that if the tested conditions and mild regularity restrictions hold, conditional inference is still valid, albeit typically conservative. Validity holds regardless of the asymptotic dependence between the estimator and the pre-test. If the tested conditions do not hold, we exhibit conditions under which confidence intervals have larger conditional than unconditional coverage.

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We show that if the tested conditions and mild regularity restrictions hold, conditional inference is still valid, albeit typically conservative. Validity holds regardless of the asymptotic dependence between the estimator and the pre-test.
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

The first point follows in particular from properties of convex functionals of Gaussian measures (Davydov et al., 1998), and the Gaussian correlation inequality (Royen, 2014).

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.