pith. machine review for the scientific record. sign in

arxiv: 2605.09525 · v1 · submitted 2026-05-10 · 📊 stat.ME

Recognition: no theorem link

Simultaneous false discovery rate control in location families

Qingyuan Zhao, Wenjie Hu, Zijun Gao

Pith reviewed 2026-05-12 04:25 UTC · model grok-4.3

classification 📊 stat.ME
keywords false discovery rateBenjamini-Hochberg procedurelocation familiesmultiple testingsimultaneous controlp-valueshypothesis testing
0
0 comments X

The pith

In location families, the standard Benjamini-Hochberg procedure simultaneously controls the false discovery rate curve over all location parameters.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

When testing many hypotheses with data from location families, it is often desirable to bound the false discovery rate not only at the exact null but also across other parameter values that represent practically insignificant effects. The paper defines the FDR as a curve indexed by the location parameter and introduces a simple generalization of the Benjamini-Hochberg procedure that keeps this entire curve below any chosen level. A direct corollary is that the unmodified standard Benjamini-Hochberg procedure already delivers this simultaneous control at no extra cost. This matters in applications where analysts want uniform error guarantees rather than control only at zero effect.

Core claim

When observations follow location family distributions, a simple generalization of the Benjamini-Hochberg procedure controls the FDR curve, indexed by the location parameter, below any user-specified level. As a corollary of this result, the standard Benjamini-Hochberg procedure provides simultaneous control of the whole FDR curve for free.

What carries the argument

The FDR curve indexed by the location parameter, controlled by a threshold on p-values that exploits translation properties of location families to achieve uniform bounds.

If this is right

  • The FDR is bounded uniformly for every location parameter value, not only the null.
  • No separate procedure is required to handle near-null hypotheses in location-family settings.
  • The standard Benjamini-Hochberg threshold already delivers the full curve control without modification.
  • The guarantee applies directly to common location families such as the normal distribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The result may extend to other parametric families that share similar monotonicity or invariance properties in their likelihood ratios.
  • In large-scale testing domains such as genomics, existing applications of Benjamini-Hochberg already supply error control across a range of effect sizes.
  • Small deviations from exact location families could be tested numerically to check how quickly the simultaneous guarantee degrades.

Load-bearing premise

The observations come from location family distributions so the false discovery proportion can be tracked continuously as a curve over the location shift.

What would settle it

A location-family dataset and p-value configuration in which the FDR curve exceeds the target level for some parameter value under the generalized procedure would falsify the claim.

Figures

Figures reproduced from arXiv: 2605.09525 by Qingyuan Zhao, Wenjie Hu, Zijun Gao.

Figure 1
Figure 1. Figure 1: FDR curve control for Gaussian location model. The curve [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FDR curve control for breast cancer dataset testing the effect size with hypotheses [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FDR curve control for breast cancer dataset testing the signal-to-noise ratio with [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
read the original abstract

When testing a number of statistical hypotheses using data from location families, it is often useful to control the false discovery rate (FDR) not just for hypotheses of the null values but also of other parameter values that are deemed practically insignificant. Here we consider FDR as a curve indexed by the location parameter and suggest a simple generalization of the Benjamini-Hochberg procedure that controls the FDR curve below any user-specified level. As a corollary of our main result, we show that the standard Benjamini-Hochberg procedure -- designed to control the FDR at the null -- also provides simultaneous control of the whole FDR curve for free. We further demonstrate the implications of our results and some practical considerations with a numerical example.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that for data from location families, the false discovery rate (FDR) can be controlled simultaneously as a curve indexed by the location parameter θ. It proposes a simple generalization of the Benjamini-Hochberg (BH) procedure that keeps the entire FDR curve below any user-specified level α. As a corollary, the standard BH procedure (which controls FDR at the null θ=0) automatically provides simultaneous control of the full FDR curve for all θ, due to monotonicity properties specific to location families. Implications are illustrated via a numerical example.

Significance. If the central claims hold, the result is significant because it shows that existing FDR-controlling procedures deliver simultaneous control over a continuum of parameter values at no extra cost, which is valuable in multiple-testing settings where one wishes to bound FDR not only at the exact null but also for nearby alternatives deemed practically insignificant. The location-family structure is leveraged to obtain the monotonicity that makes the corollary possible, and the numerical example provides concrete illustration of the practical considerations.

major comments (2)
  1. [Corollary (following the main theorem)] The corollary that standard BH controls the entire FDR curve indexed by θ rests on the property that FDR(θ) ≤ FDR(0) for all θ (or that the curve is maximized at the null). While the abstract states this follows from the main result via location-family properties, the manuscript must explicitly derive or state the inequality FDR(θ) ≤ FDR(0) (including the precise definition of the indexed FDR functional) to confirm it holds uniformly over the class of location families, including asymmetric or heavy-tailed members. This inequality is load-bearing for the corollary.
  2. [Main theorem and proof] § on the main result (generalized BH): the proof that the proposed procedure controls the full FDR curve should be checked for any hidden dependence on the specific form of the location family or on the choice of critical values; if the control holds only under additional regularity conditions not stated in the abstract, these must be made explicit.
minor comments (2)
  1. [Numerical example] In the numerical example, state the exact location family (normal, Laplace, etc.), the sample sizes, and the grid of θ values used so that the demonstration is fully reproducible.
  2. [Setup and definitions] Clarify the notation for the FDR curve (e.g., whether it is E[V(θ)/R] or a different functional) at its first appearance and ensure it is used consistently.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive report. The two major comments highlight opportunities to strengthen the presentation of the corollary and the generality of the main theorem. We address each point below and will incorporate the suggested clarifications in a revised manuscript.

read point-by-point responses

  1. Referee: [Corollary (following the main theorem)] The corollary that standard BH controls the entire FDR curve indexed by θ rests on the property that FDR(θ) ≤ FDR(0) for all θ (or that the curve is maximized at the null). While the abstract states this follows from the main result via location-family properties, the manuscript must explicitly derive or state the inequality FDR(θ) ≤ FDR(0) (including the precise definition of the indexed FDR functional) to confirm it holds uniformly over the class of location families, including asymmetric or heavy-tailed members. This inequality is load-bearing for the corollary.

    Authors: We agree that an explicit derivation of FDR(θ) ≤ FDR(0) is necessary for a self-contained argument. The current proof invokes monotonicity properties of location families but does not spell out the steps from the definition of the indexed FDR functional. In the revision we will insert a dedicated paragraph immediately preceding the corollary that (i) recalls the precise definition of FDR(θ), (ii) derives the inequality using only the translation-equivariance of the test statistics and the ordering induced by the location parameter, and (iii) notes that the argument requires neither symmetry nor moment conditions, thereby covering asymmetric and heavy-tailed members of the class. revision: yes

  2. Referee: [Main theorem and proof] § on the main result (generalized BH): the proof that the proposed procedure controls the full FDR curve should be checked for any hidden dependence on the specific form of the location family or on the choice of critical values; if the control holds only under additional regularity conditions not stated in the abstract, these must be made explicit.

    Authors: The proof of the main theorem uses only the definition of a location family (i.e., the distribution of the test statistic is a translate of a fixed distribution) together with the standard BH critical-value construction; no further regularity on the density or tail behavior is invoked. We have re-examined the argument and confirm that it contains no hidden dependence on a particular parametric form or on the specific numerical values of the critical thresholds beyond the usual BH ordering. To address the referee’s concern we will add a short remark at the end of the proof section that lists the minimal assumptions employed and states that the result holds for any location family. revision: yes

Circularity Check

0 steps flagged

No circularity: theoretical derivation of FDR curve control is self-contained

full rationale

The paper derives a main result generalizing the Benjamini-Hochberg procedure to control the FDR curve indexed by the location parameter in location families, then obtains the corollary that standard BH controls the full curve as a direct consequence. No steps reduce by construction to fitted inputs, self-definitions, or self-citations; the argument relies on explicit distributional properties of the location family class without re-labeling assumptions as predictions. The derivation is independent and self-contained against the stated model assumptions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review prevents full audit; the result appears to rest on standard properties of location families and the definition of the FDR curve, with no free parameters or invented entities explicitly mentioned.

axioms (1)
  • domain assumption Data are i.i.d. from location families
    Invoked to define the FDR curve indexed by the location parameter

pith-pipeline@v0.9.0 · 5409 in / 1123 out tokens · 39763 ms · 2026-05-12T04:25:32.443786+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 58 canonical work pages

  1. [1]

    Statistical Science , volume=

    Two modeling strategies for empirical Bayes estimation , author=. Statistical Science , volume=

    work page

  2. [2]

    arXiv preprint arXiv:2201.09350 , year=

    Elementary proofs of several results on false discovery rate , author=. arXiv preprint arXiv:2201.09350 , year=

    work page arXiv

  3. [3]

    arXiv preprint arXiv:2401.16651 , year=

    A constructive approach to selective risk control , author=. arXiv preprint arXiv:2401.16651 , year=

    work page arXiv

  4. [4]

    Wasserstein, Ronald L and Lazar, Nicole A , journal=. The. 2016 , publisher=

    work page 2016

  5. [5]

    and others , title =

    Krammer, Julia and Pinker-Domenig, Katja and Robson, Mark E. and others , title =. Breast Cancer Research and Treatment , year =

    work page

  6. [6]

    New England Journal of Medicine , year =

    Hedenfalk, Ingrid and Duggan, David and Chen, Yidong and Radmacher, Michael and Bittner, Michael and Simon, Richard and Meltzer, Paul and Gusterson, Barry and Esteller, Manel and Raffeld, Mark and Yakhini, Zohar and Ben-Dor, Amir and Trent, Jeffrey , title =. New England Journal of Medicine , year =

    work page

  7. [7]

    and Tibshirani, Robert , title =

    Storey, John D. and Tibshirani, Robert , title =. Proceedings of the National Academy of Sciences , year =

    work page

  8. [8]

    Biometrika , pages=

    An adaptive null proportion estimator for false discovery rate control , author=. Biometrika , pages=. 2024 , publisher=

    work page 2024

  9. [9]

    Proceedings International Conference on Internet and Web Applications and Services , pages=

    Application of machine learning algorithms to an online recruitment system , author=. Proceedings International Conference on Internet and Web Applications and Services , pages=

    work page

  10. [10]

    Philosophical Transactions of the Royal Society of London

    Outline of a theory of statistical estimation based on the classical theory of probability , author=. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences , volume=. 1937 , publisher=

    work page 1937

  11. [11]

    Proceedings of the second Berkeley symposium on mathematical statistics and probability , volume=

    Asymptotically subminimax solutions of compound statistical decision problems , author=. Proceedings of the second Berkeley symposium on mathematical statistics and probability , volume=. 1951 , organization=

    work page 1951

  12. [12]

    Journal of Theoretical and Applied Information Technology , volume=

    An adaptive personnel selection model for recruitment using domain-driven data mining , author=. Journal of Theoretical and Applied Information Technology , volume=. 2016 , publisher=

    work page 2016

  13. [13]

    arXiv preprint arXiv:2303.01552 , year=

    Simultaneous Hypothesis Testing Using Internal Negative Controls with An Application to Proteomics , author=. arXiv preprint arXiv:2303.01552 , year=

    work page arXiv

  14. [14]

    Neuron , volume=

    In situ cell-type-specific cell-surface proteomic profiling in mice , author=. Neuron , volume=. 2022 , publisher=

    work page 2022

  15. [15]

    Test , volume=

    Exact testing with random permutations , author=. Test , volume=. 2018 , publisher=

    work page 2018

  16. [16]

    Statistica Sinica , volume=

    Selective sign-determining multiple confidence intervals with FCR control , author=. Statistica Sinica , volume=. 2020 , publisher=

    work page 2020

  17. [17]

    The Annals of Statistics , volume=

    Optimal rates for independence testing via U-statistic permutation tests , author=. The Annals of Statistics , volume=. 2021 , publisher=

    work page 2021

  18. [18]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    The conditional permutation test for independence while controlling for confounders , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2020 , publisher=

    work page 2020

  19. [19]

    2021 , publisher=

    Computer age statistical inference, student edition: algorithms, evidence, and data science , author=. 2021 , publisher=

    work page 2021

  20. [20]

    International Conference on Machine Learning , pages=

    Adaptive monte carlo multiple testing via multi-armed bandits , author=. International Conference on Machine Learning , pages=. 2019 , organization=

    work page 2019

  21. [21]

    arXiv preprint arXiv:1710.02776 , year=

    STAR: A general interactive framework for FDR control under structural constraints , author=. arXiv preprint arXiv:1710.02776 , year=

    work page arXiv

  22. [22]

    Bioinformatics (Oxford, England) , author =

    A flexible two-stage procedure for identifying gene sets that are differentially expressed , volume =. Bioinformatics (Oxford, England) , author =. 2009 , pmid =. doi:10.1093/bioinformatics/btp076 , abstract =

    work page doi:10.1093/bioinformatics/btp076 2009

  23. [23]

    Journal of the American Statistical Association , volume=

    Oracle and adaptive compound decision rules for false discovery rate control , author=. Journal of the American Statistical Association , volume=. 2007 , publisher=

    work page 2007

  24. [24]

    Biometrics , author =

    Comparing. Biometrics , author =. 1949 , note =. doi:10.2307/3001913 , abstract =

    work page doi:10.2307/3001913 1949

  25. [25]

    1999 , publisher=

    The analysis of variance , author=. 1999 , publisher=

    work page 1999

  26. [26]

    , month = jul, year =

    Johari, Ramesh and Pekelis, Leo and Walsh, David J. , month = jul, year =. Always. doi:10.48550/arXiv.1512.04922 , abstract =

    work page doi:10.48550/arxiv.1512.04922

  27. [27]

    Sequential Tests of Statistical Hypotheses

    Sequential. The Annals of Mathematical Statistics , author =. 1945 , note =. doi:10.1214/aoms/1177731118 , abstract =

    work page doi:10.1214/aoms/1177731118 1945

  28. [28]

    Annual Review of Statistics and Its Application , volume=

    Post-selection inference , author=. Annual Review of Statistics and Its Application , volume=. 2022 , publisher=

    work page 2022

  29. [29]

    , month = jun, year =

    Benjamini, Yoav and Hechtlinger, Yotam and Stark, Philip B. , month = jun, year =. Confidence Intervals for Selected Parameters , url =

    work page

  30. [30]

    J. A. Ferreira and A. H. Zwinderman , title =. The Annals of Statistics , number =. 2006 , doi =

    work page 2006

  31. [31]

    The Annals of Statistics , volume=

    The control of the false discovery rate in multiple testing under dependency , author=. The Annals of Statistics , volume=. 2001 , publisher=

    work page 2001

  32. [32]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    A direct approach to false discovery rates , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2002 , publisher=

    work page 2002

  33. [33]

    Journal of the Royal Statistical Society: Series B (Methodological) , volume=

    Controlling the false discovery rate: a practical and powerful approach to multiple testing , author=. Journal of the Royal Statistical Society: Series B (Methodological) , volume=. 1995 , publisher=

    work page 1995

  34. [34]

    Large-scale inference: empirical

    Efron, Bradley , year=. Large-scale inference: empirical

    work page

  35. [35]

    Journal of the American Statistical Association , volume=

    Empirical Bayes estimates for large-scale prediction problems , author=. Journal of the American Statistical Association , volume=. 2009 , publisher=

    work page 2009

  36. [36]

    Cancer cell , volume=

    Gene expression correlates of clinical prostate cancer behavior , author=. Cancer cell , volume=. 2002 , publisher=

    work page 2002

  37. [37]

    1979 , journal =

    Conditional Independence in Statistical Theory , author =. 1979 , journal =. doi:10.1111/j.2517-6161.1979.tb01052.x , date_added =

    work page doi:10.1111/j.2517-6161.1979.tb01052.x 1979

  38. [38]

    Publisher: Institute of Mathematical Statistics

    Extended Conditional Independence and Applications in Causal Inference , author =. 2017 , month = dec, journal =. doi:10.1214/16-AOS1537 , urldate =

    work page doi:10.1214/16-aos1537 2017

  39. [39]

    Electronic Journal of Statistics , volume=

    Two simple sufficient conditions for FDR control , author=. Electronic Journal of Statistics , volume=

    work page

  40. [40]

    The annals of applied statistics , volume=

    Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses , author=. The annals of applied statistics , volume=. 2006 , publisher=

    work page 2006

  41. [41]

    Journal of the American Statistical Association , volume=

    Multiple testing when many p-values are uniformly conservative, with application to testing qualitative interaction in educational interventions , author=. Journal of the American Statistical Association , volume=. 2019 , publisher=

    work page 2019

  42. [42]

    Journal of the American Statistical Association , volume=

    False discovery rate-adjusted multiple confidence intervals for selected parameters , author=. Journal of the American Statistical Association , volume=. 2005 , publisher=

    work page 2005

  43. [43]

    Biometrika , volume=

    Assessing replicability of findings across two studies of multiple features , author=. Biometrika , volume=. 2018 , publisher=

    work page 2018

  44. [44]

    Journal of the American Statistical Association , volume=

    Discovering findings that replicate from a primary study of high dimension to a follow-up study , author=. Journal of the American Statistical Association , volume=. 2013 , publisher=

    work page 2013

  45. [45]

    Journal of the Royal Statistical Society: Series B: Statistical Methodology , pages=

    Selective inference on multiple families of hypotheses , author=. Journal of the Royal Statistical Society: Series B: Statistical Methodology , pages=. 2014 , publisher=

    work page 2014

  46. [46]

    Biometrics , volume=

    Screening for partial conjunction hypotheses , author=. Biometrics , volume=. 2008 , publisher=

    work page 2008

  47. [47]

    The annals of applied statistics , volume=

    Multilayer knockoff filter: Controlled variable selection at multiple resolutions , author=. The annals of applied statistics , volume=. 2019 , publisher=

    work page 2019

  48. [48]

    Transactions of the American Mathematical Society , volume=

    Polynomials that are positive on an interval , author=. Transactions of the American Mathematical Society , volume=

    work page

  49. [49]

    arXiv preprint arXiv:2101.09195 , year=

    Randomization inference beyond the sharp null: Bounded null hypotheses and quantiles of individual treatment effects , author=. arXiv preprint arXiv:2101.09195 , year=

    work page arXiv

  50. [50]

    Statistical Science , volume=

    Multiple testing for exploratory research , author=. Statistical Science , volume=. 2011 , publisher=

    work page 2011

  51. [51]

    Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=

    Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2004 , publisher=

    work page 2004

  52. [52]

    Kuchibhotla and John E

    Arun K. Kuchibhotla and John E. Kolassa and Todd A. Kuffner , title =. Annual Review of Statistics and Its Application , volume = 9, number = 1, pages =. doi:10.1146/annurev-statistics-100421-044639 , url =

    work page doi:10.1146/annurev-statistics-100421-044639

  53. [53]

    The Annals of Statistics , volume = 41, number = 2, pages =

    Richard Berk and Lawrence Brown and Andreas Buja and Kai Zhang and Linda Zhao , title =. The Annals of Statistics , volume = 41, number = 2, pages =. doi:10.1214/12-aos1077 , url =

    work page doi:10.1214/12-aos1077

  54. [54]

    CoRR , year = 2014, url =

    Fithian, William and Sun, Dennis and Taylor, Jonathan , title =. CoRR , year = 2014, url =

    work page 2014

  55. [55]

    D., Sun, D

    Jason D. Lee and Dennis L. Sun and Yuekai Sun and Jonathan E. Taylor , title =. The Annals of Statistics , volume = 44, number = 3, pages =. doi:10.1214/15-aos1371 , url =

    work page doi:10.1214/15-aos1371

  56. [56]

    Annals of Statistics , pages =

    Panigrahi, Snigdha , title =. Annals of Statistics , pages =

    work page

  57. [57]

    The P-Filter:

    Barber, Rina Foygel and Ramdas, Aaditya , year =. The P-Filter:. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume =. doi:10.1111/rssb.12218 , urldate =

    work page doi:10.1111/rssb.12218

  58. [58]

    Bioinformatics , volume = 32, number = 11, pages =

    Vered Madar and Sandra Batista , title =. Bioinformatics , volume = 32, number = 11, pages =. doi:10.1093/bioinformatics/btw029 , url =

    work page doi:10.1093/bioinformatics/btw029