Anytime-valid FDR control with the stopped e-BH procedure

Aaditya Ramdas; Hongjian Wang; Sanjit Dandapanthula

Anytime-valid FDR control with the stopped e-BH procedure

T0 review · 0 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Stopped e-BH controls FDR whenever per-stream e-processes become global under a Markovian condition.

desk verdict A genuinely useful paper: it pinpoints a real filtration subtlety in stopped e-BH, proves a clean sufficient condition, and backs it with a concrete counterexample. read the letter →

arxiv 2502.08539 v3 pith:ZVEQZWJK submitted 2025-02-12 stat.ME math.STstat.TH

Hongjian Wang , Sanjit Dandapanthula , Aaditya Ramdas This is my paper · ORCID

classification stat.MEmath.STstat.TH MSC 62F0362L10

keywords e-processese-valuesFDRcontrole-BHproceduresequentialmultipletestingoptionalstoppingfiltrationMarkovassumption

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper closes a gap that matters for any sequential experiment with many hypotheses: it pins down when you can watch e-processes continuously, stop at any time, and still trust e-BH's false discovery rate guarantee. The subtlety is that an e-process built inside one hypothesis stream is only guaranteed to be an e-value when stopped at a stopping time of that stream's own filtration, while stopping all streams at a shared global stopping time can let one stream's past act as information about another stream's future. The paper proposes a Markovian causal condition---$Y_n$ is independent of everything observed before $n$ once the current covariate $X_n$ is known---under which locally constructed multiplicative e-processes become global e-processes, making the stopped e-BH procedure valid at any global stopping time. It further proves that stopped e-BH with global compound e-processes is not just sufficient but necessary for stopped FDR control, and it gives an explicit two-stream counterexample where the condition fails and the guarantee breaks.

What carries the argument

The carrying object is the multiplicative local e-process $M^g_n(\theta) = \prod_{i=1}^n E^g_i(\theta, X_i, Y^g_i)$ built from stepwise e-value functions, each $G_{n-1}$-measurable and satisfying the conditional expectation bound (5). The mechanism is a single conditional-expectation calculation: under Assumption 3.1, $Y_n$ is independent of the entire past given $X_n$, so $\mathbb{E}[E^g_n(\theta^*, X_n, Y^g_n)|\mathcal{F}_{n-1}] \le 1$ for every null $\theta^*$; hence the product is a supermartingale on the global look-ahead filtration, and taking the infimum over $\Theta^g_0$ preserves the e-process property. A second piece of machinery, global compound e-processes, makes the statement tight: stopped e-BH is necessary as well as sufficient, because any set process that controls stopped FDR can be represented as stopped e-BH applied to global compound e-processes.

What would settle it

Take two streams with $Y^2_n = Y^1_{n+1}$, $\theta=1/2$, and the global stopping time $\tau = 1 + \mathbf{1}\{Y^2_1=1\}$; the paper's own calculation gives $\mathbb{E}_{\theta=1/2} M^1_\tau = 1.25 > 1$, so the stopped local e-process is not an e-value, and running stopped e-BH with $\alpha$ near 0.1 at this stopping time should push the empirical FDR above $\alpha$.

Watch

Extended reading notes

Core claim

Under Assumption 3.1, the paper establishes that local e-processes are global e-processes. For a fixed hypothesis $g$, take any sequence of stepwise e-value functions $E^g_n(\theta, X_n, Y^g_n)$ satisfying $\sup_{\theta\in\Theta^g_0, x}\int E^g_n(\theta,x,y)\,p^g(dy|x,\theta) \le 1$. Then for every null parameter $\theta^*\in\Theta^g_0$, the product $M^g_n(\theta^*) = \prod_{i=1}^n E^g_i(\theta^*, X_i, Y^g_i)$ is a nonnegative supermartingale on the look-ahead global filtration $\mathcal{F}_n = \sigma(Y_i, X_j : i\le n,\,j\le n+1)$, and $U^g_n = \inf_{\theta\in\Theta^g_0} M^g_n(\theta)$ is an e-process on that filtration. Applying e-BH to $U^g_n$ across $g\in[G]$ at any $\mathcal{F}_n$-stopping time therefore controls the false discovery rate at level $\alpha$. The proof works because Assumption 3.1 lets the conditional law of $Y^g_n$ given the global past be replaced by the model's marginal conditional law $p^g(\cdot|X_n,\theta^*)$. When the assumption fails, a two-stream example with $Y^2_n = Y^1_{n+1}$ has $\mathbb{E}_{\theta=1/2} M^1_\tau = 1.25 > 1$ at a valid global stopping time, so the local e-process is not a global e-value and the guarantee is lost.

Load-bearing premise

The guarantee rests on $Y_n$ being independent of everything observed before time $n$ once $X_n$ is known; if past observations carry information about $Y_n$ beyond the current covariate, local e-processes can stop being global e-values and FDR control can fail.

Editorial extensions

If this is right

With the Markovian condition in force, a practitioner may monitor the e-BH rejection set at every time and stop at any global stopping time--for instance, only after a desired set of hypotheses is rejected--and still keep FDR at or below $\alpha$.
Arbitrary same-time dependence across hypotheses is allowed: the response variables $Y^1_n,\dots,Y^G_n$ may be correlated given $X_n$; only the temporal Markov structure is constrained.
If the condition fails, stopped e-BH on local e-processes can inflate FDR, and the inflation mirrors the $1+2^{-1}+\cdots+G^{-1}$ factor that BH faces with arbitrarily dependent p-values.
Stopped e-BH is representationally complete: any procedure meeting the stopped FDR bound can be written as stopped e-BH applied to global compound e-processes, so the method is not one option among many but the canonical one.
When the assumption is not credible, applying compound adjusters to the running maxima of local e-processes produces global compound e-processes and restores stopped FDR control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One consequence beyond the paper: Assumption 3.1 is a checkable no-unobserved-confounder condition, so in observational or longitudinal designs a natural diagnostic is to test whether past outcomes predict $Y_n$ after conditioning on $X_n$; if they do, practitioners should use an adjuster or a different design before trusting stopped e-BH.
The look-ahead filtration suggests $X_{n+1}$ can enter stopping decisions before responses are seen, which could support adaptive allocation of future samples after covariates are measured but before outcomes arrive; the paper notes the flexibility but does not pursue the design implications.
Because the counterexample's stopped local e-process is still a valid p-value, there may be a middle regime where BH applied to $1/M^g_\tau$ controls FDR under weaker dependence assumptions than full globality; quantifying that regime would be a natural extension.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this one is real. The paper identifies a gap that most of us in sequential testing have been quietly stepping over: e-processes are usually built for local filtrations, but stopped e-BH at a global stopping time needs global e-processes. Without that, FDR can inflate by up to about log G. The authors give a simple Markovian condition, Assumption 3.1, under which locally constructed multiplicative e-processes become global, and they demonstrate with a two-stream example that when the condition fails, the stopped e-process can have expectation 1.25 under the null, breaking e-BH.

The main theorem is proved in a few lines in Supp.6.1, and the proof is correct. The key step is replacing the conditional law of Y_n given F_{n-1} with the model's marginal law p^g(·|X_n, θ*); that is exactly where Assumption 3.1 is needed and the paper is explicit about it. The counterexample in Supp.5.5 is nicely concrete, with a table that verifies E(M_tau) = 1.25. I also appreciate the universality result in Supp.3: stopped e-BH with global compound e-processes is necessary and sufficient for stopped FDR control. That puts the contribution on firmer ground than just a single fix.

Soft spots are minor. The infimum in (7) is taken over an indexed family of supermartingales, and the paper doesn't state the measurability or regularity conditions needed to make that a clean e-process. All the worked examples are well-behaved, so this is a presentation issue, not a flaw. The claim that Assumption 3.1 holds in “most reasonable scenarios including genomics” is a bit too glib; it is a substantive causal assumption that should be argued case by case, not waved through. The paper does at least flag it honestly. The heavy reliance on prior work by the same group (e-BH, compound e-values, adjusters) is fine because those are established external results being used as lemmas, not the target of the proof.

Who is this for? Anyone doing sequential multiple testing, especially with bandit-style or streaming data where stopping decisions depend on the whole collection. It is a short, focused paper, appropriate for the special issue. I would send it to review; the central observation and the proof are solid, and the counterexample is instructive. I'd also cite it if I work on anytime-valid inference.

Referee Report

0 major / 4 minor

Summary. The paper studies sequential multiple testing with e-processes and the e-BH procedure. It points out that applying e-BH to stopped local e-processes does not automatically control FDR at a global stopping time, because local e-processes need not be martingales with respect to the global filtration. The paper introduces a look-ahead global filtration, imposes a Markovian conditional independence assumption (Assumption 3.1), and proves in Theorem 3.3 that products of stepwise e-value functions, and their infima over null parameter sets, are global e-processes. Corollary 3.4 then yields stopped FDR control for e-BH. The supplementary material provides examples (multivariate normal, SPRT, negative binomial regression, nonparametric heavy-tailed testing), a counterexample when Assumption 3.1 fails, a universality/converse result with compound e-processes, and adjuster-based remedies.

Significance. If correct, the paper settles a subtle but important gap between local and global e-processes in sequential multiple testing. The central proof in Supp.6.1 is short and correct, and the counterexample in Supp.5.5 verifies by direct computation that the failure is real rather than hypothetical. The paper also gives a converse in Supp.3.5 showing that global compound e-processes are exactly the objects needed for stopped FDR control, and it identifies a structural condition under which common local constructions (likelihood ratios, universal inference, Catoni-style e-values) become global. The exposition is concise, the examples are relevant, and the reliance on prior work is transparent: established results such as e-BH FDR control and compound e-value converses are used as lemmas, not proved circularly.

minor comments (4)

[§3, Eq. (7)] The proof of Theorem 3.3 in Supp.6.1 does not address measurability of the infimum U^g_n = inf_{θ ∈ Θ_0^g} M^g_n(θ). Since an uncountable infimum of F_n-measurable variables need not be F_n-measurable, the adaptedness required for an e-process is not automatic. Please add a standard regularity/measurability condition (for instance, separability of Θ_0^g and continuity or Borel measurability in θ, or restriction to a countable dense subset) and note that all examples in Supp.5 satisfy it.
[Supp.5.5] The table accompanying the counterexample is typeset ambiguously: the four possible values of (Y^1_1, Y^1_2) are not clearly separated into rows. Labeling the rows explicitly with the pair (Y^1_1, Y^1_2) would make the direct computation E_{θ=1/2} M^1_τ = 1.25 easier to verify.
[§2.2, Supp.1.1, Supp.3.5] Please correct the typographical errors: 'ganruantee' in Section 2.2, 'guanratee' in Example 2.2 and Supp.1.1, 'stardard' in Supp.1, and 'satisfiying' in Supp.3.5; in Supp.1.1, the global filtration should read σ(Y^g_1, ..., Y^g_n : g ∈ [G]).
[§2.2] The two bullet-point definitions of an e-process are stated as equivalent without proof or reference; the equivalence (or at least the direction used, that an e-process is dominated by a nonnegative supermartingale) is invoked in Supp.6.2. A brief justification or citation would improve rigor.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central Markovian lifting theorem is proved directly from stated assumptions and does not reduce to its inputs.

full rationale

The central derivation is a conditional mathematical proof, not a fit or definitional equivalence. Theorem 3.3 is proved in Supp.6.1 by directly computing E_theta*[E^g_n(theta*, X_n, Y^g_n) | F_{n-1}] and invoking Assumption 3.1 to replace the conditional law of Y^g_n given the past by p^g(·|X_n, theta*); the displayed inequality is exactly the stepwise e-value condition (5). U^g_n is then dominated by the supermartingale M^g_n(theta*) for each theta*, so it is an e-process on the look-ahead global filtration. This does not presuppose the stopped-FDR conclusion. The stopped-FDR step (Theorem 2.4 and Corollary 3.4) is a direct application of the e-BH FDR lemma (Lemma 2.1), an external result; no constant is fitted and no subset of the data is reused to define the target quantity. The paper's citations to prior work by the same group (e-BH, compound e-values, adjusters, universal inference) are used as lemmas or construction ingredients, not as substitutes for the proof of the Markovian lifting theorem; the adjuster discussion in Supp.4 is explicitly a remedy and is not needed for Corollary 3.4. The counterexample in Supp.5.5 verifies the necessity of Assumption 3.1 by direct calculation rather than by assumption. The necessity/universality result in Supp.3.5 is an application of an external converse theorem for compound e-values; even though the cited theorem shares an author, its assumptions do not include the stopped-e-BH claim, so it is independent support rather than a circular premise. Therefore no load-bearing step reduces by construction to its own input; any self-citations are routine and non-circular.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central proof relies on no fitted parameters and introduces no new physical or mathematical entities. The paper introduces a conceptual distinction between local and global e-processes and the Assumption 3.1 condition, but these are definitions and assumptions, not invented objects.

assumptions (5)

domain assumption Assumption 3.1: Y_n is conditionally independent of all past covariates and responses given X_n.
This is the load-bearing causal condition. Supp.6.1 invokes it in the second equality of the proof of Theorem 3.3, and Supp.5.5 shows failure of the assumption breaks globality.
domain assumption Marginal conditional model (2): each Y^g_n given X_n has distribution p^g(.,X_n,theta*) with a common parameter theta*, allowing arbitrary cross-stream dependence.
Defines the model class M and the null sets P^g that the theorem operates on.
domain assumption Stepwise e-value functions E^n_g satisfy the conditional bound (5) and are G_{n-1}-measurable.
Needed to construct the multiplicative processes M^n_g(theta) in (6); without (5) the supermartingale property fails.
standard math The e-BH FDR control lemma from Wang and Ramdas (2022) for e-values under arbitrary dependence.
Used as the starting point in Lemma 2.1 and Theorem 2.4.
standard math The compound e-value characterization of FDR control from Ignatiadis et al. (2024).
Used in the universality converse, Theorem Supp.3.5.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Anytime-valid FDR control with the stopped e-BH procedure." pith.science (2026). https://pith.science/paper/ZVEQZWJK

@misc{pith2026250208539,
  author       = {Pith},
  title        = {Pith review of: Anytime-valid FDR control with the stopped e-BH procedure},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZVEQZWJK}},
  note         = {Machine review of arXiv:2502.08539}
}

read the original abstract

The recent e-Benjamini-Hochberg (e-BH) procedure for multiple hypothesis testing is known to control the false discovery rate (FDR) under arbitrary dependence between the input e-values. This paper points out an important subtlety when applying the e-BH procedure with e-processes, which are sequential generalizations of e-values (where the data are observed sequentially). Since adaptively stopped e-processes are e-values, the e-BH procedure can be repeatedly applied at every time step, and one can continuously monitor the e-processes and the rejection sets obtained. One would hope that the "stopped e-BH procedure" (se-BH) has an FDR guarantee for the rejection set obtained at any stopping time. However, while this is true if the data in different streams are independent, it is not true in full generality, because each stopped e-process is an e-value only for stopping times in its own local filtration, but the se-BH procedure employs a stopping time with respect to a global filtration. This can cause information to leak across time, allowing one stream to know its future by knowing past data of another stream. This paper formulates a simple causal condition under which local e-processes are also global e-processes and thus the se-BH procedure does indeed control the FDR. The condition excludes unobserved confounding from the past and is met under most reasonable scenarios including genomics.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Self-Poisoning in Adaptive Out-of-Distribution Detection: A Sharp-Threshold Theory and Certified Label-Free Calibration
cs.LG 2026-07 conditional novelty 6.0 of 10

In adaptive OOD detection, bank impurity follows a mean-field urn law whose kernel slope acts as a reproduction number; a frozen-reserve gate removes the supercritical collapse, and a two-world theorem caps label-free...
Carefree multiple testing with e-processes
math.ST 2025-01 conditional novelty 6.0 of 10

Running-maximum e-BH can exceed the nominal FDR under arbitrary dependence, while adjusted running maxima restore FDR-sup control.

Reference graph

Works this paper leans on

13 extracted references · 8 canonical work pages · cited by 2 Pith papers

[1]

Benjamini and Y

Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57 0 (1): 0 289--300, 1995

work page 1995
[2]

Y. J. Choe and A. Ramdas. Combining evidence across filtrations. arXiv preprint arXiv:2402.09698, 2024

arXiv 2024
[3]

Dandapanthula and A

S. Dandapanthula and A. Ramdas. Multiple testing in multi-stream sequential change detection. arXiv e-prints, pages arXiv--2501, 2025

work page 2025
[4]

Finner, T

H. Finner, T. Dickhaus, and M. Roters. On the false discovery rate and an asymptotically optimal rejection curve . The Annals of Statistics, 37 0 (2): 0 596 -- 618, 2009. doi:10.1214/07-AOS569

work page doi:10.1214/07-aos569 2009
[5]

S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49 0 (2): 0 pp. 1055--1080, 2021. ISSN 00905364, 21688966

work page 2021
[6]

Ignatiadis, R

N. Ignatiadis, R. Wang, and A. Ramdas. Asymptotic and compound e-values: multiple testing and empirical B ayes. arXiv preprint arXiv:2409.19812, 2024

arXiv 2024
[7]

Shafer, A

G. Shafer, A. Shen, N. Vereshchagin, and V. Vovk. Test martingales, Bayes factors and p-values. Statistical Science, 26 0 (1): 0 84 -- 101, 2011. doi:10.1214/10-STS347

doi:10.1214/10-sts347 2011
[8]

Carefree multiple testing with e-processes

Y. Tavyrikov, J. J. Goeman, and R. de Heide. Carefree multiple testing with e-processes. arXiv preprint arXiv:2501.19360, 2025

work page Pith review arXiv 2025

Show all 13 references

[9]

Wang and A

H. Wang and A. Ramdas. Catoni-style confidence sequences for heavy-tailed mean estimation. Stochastic Processes and their Applications, 163: 0 168--202, 2023. ISSN 0304-4149. doi:10.1016/j.spa.2023.05.007

2023 doi
[10]

Wang and A

R. Wang and A. Ramdas. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84 0 (3): 0 822--852, 2022

2022
[11]

Wasserman, A

L. Wasserman, A. Ramdas, and S. Balakrishnan. Universal inference. Proceedings of the National Academy of Sciences, 117 0 (29): 0 16880--16890, 2020

2020
[12]

Z. Xu, R. Wang, and A. Ramdas. A unified framework for bandit multiple testing. Advances in Neural Information Processing Systems, 34: 0 16833--16845, 2021

2021
[13]

Z. Xu, L. Fischer, and A. Ramdas. Bringing closure to FDR control: beating the e-benjamini-hochberg procedure. arXiv preprint arXiv:2504.11759, 2025

2025 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Benjamini and Y

Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57 0 (1): 0 289--300, 1995

work page 1995

[2] [2]

Y. J. Choe and A. Ramdas. Combining evidence across filtrations. arXiv preprint arXiv:2402.09698, 2024

arXiv 2024

[3] [3]

Dandapanthula and A

S. Dandapanthula and A. Ramdas. Multiple testing in multi-stream sequential change detection. arXiv e-prints, pages arXiv--2501, 2025

work page 2025

[4] [4]

Finner, T

H. Finner, T. Dickhaus, and M. Roters. On the false discovery rate and an asymptotically optimal rejection curve . The Annals of Statistics, 37 0 (2): 0 596 -- 618, 2009. doi:10.1214/07-AOS569

work page doi:10.1214/07-aos569 2009

[5] [5]

S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49 0 (2): 0 pp. 1055--1080, 2021. ISSN 00905364, 21688966

work page 2021

[6] [6]

Ignatiadis, R

N. Ignatiadis, R. Wang, and A. Ramdas. Asymptotic and compound e-values: multiple testing and empirical B ayes. arXiv preprint arXiv:2409.19812, 2024

arXiv 2024

[7] [7]

Shafer, A

G. Shafer, A. Shen, N. Vereshchagin, and V. Vovk. Test martingales, Bayes factors and p-values. Statistical Science, 26 0 (1): 0 84 -- 101, 2011. doi:10.1214/10-STS347

doi:10.1214/10-sts347 2011

[8] [8]

Carefree multiple testing with e-processes

Y. Tavyrikov, J. J. Goeman, and R. de Heide. Carefree multiple testing with e-processes. arXiv preprint arXiv:2501.19360, 2025

work page Pith review arXiv 2025