Pith. sign in

REVIEW 5 major objections 6 minor 45 references

Strategic A/B testing via Maximum Probability-driven Two-armed Bandit

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A weighted two-armed bandit statistic that concentrates the null and spreads the alternative could sharpen A/B tests for tiny effects.

desk verdict A real oracle-side theorem buried under an unproven bridge to the actual estimator, plus a few internal inconsistencies; worth a referee but not acceptance as is. read the letter →

arxiv 2506.22536 v1 pith:HUUAKII5 submitted 2025-06-27 stat.ML cs.LGmath.PR

Yu Zhang , Shanshan Zhao , Bokui Wan , Jinjuan Wang , Xiaodong Yan This is my paper
classification stat.MLcs.LGmath.PR MSC 62F0362P10
keywords A/Btestingtwo-armedbanditstrategiccentrallimittheoremaveragetreatmenteffectcausalinferencedoublemachinelearningpermutationtestspikedistribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a new test statistic for A/B testing that treats the comparison of treatment and control as a two-armed bandit problem. Its core claim is that by weighting the mean and volatility terms in a bandit-style statistic and using a sign-based policy, the statistic converges to a two-peaked "spike" distribution instead of a normal. Under the null hypothesis this distribution is more concentrated near zero, giving better type I error control; under the alternative it spreads out more, sharply increasing power to detect small average treatment effects. The authors support this with a strategic central limit theorem and with simulations and real ride-sharing data showing lower p-values than standard methods.

What carries the argument

The central object is the weighted mean-volatility statistic $T_{n,\lambda}(\theta_n) = \frac{1}{n}\sum_{i=1}^n \frac{\lambda}{1-\lambda}\bar{R}^{(\vartheta_i)}_n + \frac{1}{\sqrt{n}}\sum_{i=1}^n \frac{R^{(\vartheta_i)}_i}{\hat\sigma}$, where $R^{(1)}_i = Y_i(1)-Y_i(0)$ and $R^{(0)}_i = -R^{(1)}_i$, with $\hat\sigma$ the sample standard deviation and $\lambda\in(0,1)$ a weight. The sign-based policy $\vartheta^*_i$ chooses arm 1 when the current statistic $T_{i-1,\lambda}(\theta^*_{i-1})$ is nonnegative and arm 0 otherwise. This policy breaks exchangeability and produces the spike distribution $B(\omega_n,\sigma_0)$ via the strategic central limit theorem, which is the key technical result (Theorem 4.1). The parameter $\lambda$ controls the rate of convergence, and the paper recommends selecting it so that $\lambda\sigma/((1-\lambda)\sqrt{n}) \le 0.03$ to avoid inflating the type I error.

What would settle it

Simulate the actual PWTAB procedure with cross-fitted ML nuisance estimates under the null hypothesis with $n=20000$ and many replications, and examine whether the empirical distribution of $T_{n,\lambda}(\theta^*_n)$ is indeed more concentrated near zero than a standard normal and whether the empirical type I error stays at or below the nominal level for the recommended $\lambda$; if the spike shape or the type I error guarantee fails, the practical claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that the weighted two-armed bandit statistic $T_{n,\lambda}(\theta_n^*)$, under a sign-based optimal policy, converges to a spike distribution $B(\omega_n, \sigma_0)$ rather than a normal. The spike distribution has a density with two peaks located at $\pm \omega_n$, where $\omega_n = \frac{\lambda}{1-\lambda}\mu + \frac{\sqrt{n}}{\sigma}\mu$, and scale $\sigma_0 = \sqrt{1+\mu^2/\sigma^2}$. This shape yields two desired properties: under the null ($\mu=0$) the spike distribution degenerates to a standard normal, but with a tighter concentration than a plain CLT; under a positive alternative, the peaks move outward, so the rejection probability at a fixed threshold is larger than that of a normal-based test. Theorem 4.2 states that the limiting rejection probability is exactly the tail probability of this spike distribution, which is bounded by $\alpha$ under the null and approaches 1 under any fixed positive $\mu$, and that the sign-based policy is optimal among all policies in the sense of attaining the supremum rejection probability.

Load-bearing premise

The theorem is proved for an oracle statistic that observes both counterfactual outcomes, but the practical method substitutes estimated counterfactuals from double machine learning; the proof only shows that the mean of the estimates matches the true mean, not that the entire distributional spike result survives the estimation noise.

Editorial extensions

If this is right

  • If the claim holds, the same A/B test would detect smaller average treatment effects than a standard z-test or DIM at the same sample size, which is economically valuable in large-scale platforms.
  • Higher sensitivity could let experimenters run shorter or smaller experiments while maintaining power, directly reducing experimental cost.
  • The spike-distribution limit gives a closed-form expression for power, so practitioners could plan sample sizes by solving the tail-probability formula rather than relying on normal approximations.
  • The permutation (Cauchy-combination) layer would make the test robust to arbitrary ordering of the data, fixing the "p-value lottery" problem of order-dependent statistics.
  • If the oracle result extends to estimated counterfactuals, the method would provide a practical, ML-based A/B testing tool that combines double machine learning with the bandit statistic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's proof of Theorem 4.1 requires the estimated statistic $T_{n,\lambda}(\theta^*_n)$ to inherit the oracle distribution, but the argument only notes that $E(\hat\mu)=E(\Delta)$ under double robustness; it does not account for the variance of the estimation noise. A direct testable extension would be to simulate the estimated statistic under the null and check whether its empirical distrib
  • The spike distribution is essentially a folded or reflected normal with a shift; this suggests that similar power gains might be obtained by simpler sign-based modifications of standard z-statistics, without the full bandit machinery.
  • The paper's experiments compare power at a fixed sample size of 20,000; a useful extension would be to measure the required sample size reduction to achieve the same power, which would quantify the cost benefit more directly.
  • Since the policy depends on the running statistic, the method is inherently sequential; a natural connection is to sequential testing or early stopping, where the spike distribution could be used to derive stopping boundaries.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a permutation-based weighted two-armed bandit test (PWTAB) for average treatment effects in A/B testing. The oracle statistic T_{n,λ}(θ_n) combines a weighted mean term involving full-sample average rewards with a volatility-normalized sum of rewards selected by a sign-based adaptive policy. The authors prove a "strategic central limit theorem" (Theorems 4.1 and 4.2) asserting that the oracle statistic converges to a spike distribution B(ω_n, σ_0), which they claim controls type I error under H0 and maximizes rejection probability under H1. For the practical implementation, counterfactual effects are estimated by doubly robust machine learning with cross-fitting, λ is set from the estimated σ̂, and B permutations are aggregated by a Cauchy combination. Simulations and three ride-sharing datasets are reported.

Significance. If the oracle SCLT were fully established and the extension to estimated counterfactuals were proven, the spike-distribution result would be a novel and potentially useful device for improving power in large-scale A/B tests. The paper is also commendable for reporting experiments with several ML learners and for attempting a permutation-based robustness correction. However, the practical claim is not backed by the theorems as written: the bridge from oracle to estimated effects is a one-sentence assertion, the oracle proof has a gap in the telescoping argument, and the testing problem is one-sided while the rejection region is two-sided. These issues are load-bearing for the paper's central claims, so the current version requires substantial revision.

major comments (5)
  1. [§3.1, Eq. (5); Lemma B.2, Eq. (17)] The proof of Theorem 4.1 uses a telescoping expansion in which the increment T_{m,λ}(θ*_n) - T_{m-1,λ}(θ*_n) is represented as the single local term λ \bar R_m^{(ϑ_m)}/((1-λ)n) + R_m^{(ϑ_m)}/(√n σ̂). Under the definition in Eq. (5), the mean term of the statistic is (1/n)∑_{i=1}^n λ/(1-λ) \bar R_n^{(ϑ_i)}, where \bar R_n^{(ϑ_i)} is a full-sample average; when m increases, that global average (or its partial analog \bar R_m, if partial sums are intended) changes for all previous indices, so the difference is not the displayed local term. The neglected contribution from the change in the running average is O(1/n) per step and can accumulate to order O(1/√n), which is the same order as the claimed bound in Eq. (18). The proof does not control this term, so the oracle SCLT is not established as written.
  2. [Remark 4.4; Eq. (7); Algorithm 1] Theorems 4.1 and 4.2 are proved for the oracle statistic built from the true individual effects Δ_i. The practical statistic in Eq. (7) uses cross-fitted estimates bμ_i. Remark 4.4 bridges the two by asserting that E[bμ_i]=E[Δ_i] under double robustness, but the theorem's proof depends on the sequential conditional law of the selected reward, the sign-based policy in Eq. (10), and the volatility estimator σ̂. Replacing Δ_i by bμ_i changes the sign of individual increments, alters the policy trajectory, changes σ̂, and introduces cross-fitting dependence across all units. First-moment equality does not preserve the martingale and telescoping identities used in Lemmas B.2–B.4, so type I error control and the power optimality of the actual PWTAB procedure are not established.
  3. [§3.1 (after Eq. (5)); Theorem 4.2] The testing problem is one-sided (H0: μ≤0, H1: μ>0 in Eq. (1)), but the rejection region used throughout is two-sided: |T_{n,λ}|>z_{1-α/2}. The spike distribution B(ω_n, σ_0) in Eq. (12) has mass at both ±ω_n, so under the alternative the two-sided rule counts very negative values of T as evidence for μ>0, which is not a rejection region for the stated one-sided alternative. The sentence in Section 5 that the method achieves "largest statistical power in one-sided hypothesis testing with a two-tailed rejection region" is internally inconsistent, and the apparent power gain over the z-test is partly an artifact of this mismatch.
  4. [§3.1; Algorithm 1, line 8] The asymptotic theorems treat λ as a fixed constant, but in the algorithm λ is chosen as a data-dependent function of σ̂ (λ = τ√n/(σ̂ + τ√n)), and Section 3.1 also suggests bootstrap tuning of λ to control the type I error. The SCLT and the bound O(σ/((1-λ)√n)) do not automatically hold for a random λ correlated with the data. The type I error rates reported in Tables 2 and 4 are therefore an empirical calibration property rather than a consequence of Theorem 4.1, and the power comparison at the tuned λ is partly circular.
  5. [Abstract; Eq. (12), Theorem 4.1] The abstract claims a "more concentrated distribution under the null hypothesis." However, for μ=0, ω_n=0 and the density in Eq. (12) reduces exactly to the standard normal density. Thus the SCLT does not concentrate the null distribution relative to the CLT; the only distributional change is under the alternative, where the spike distribution spreads mass to ±ω_n. The claimed null concentration is not supported by Theorem 4.1.
minor comments (6)
  1. [§3.1] The sentence stating that under the null "P(|T_{n,λ}|<z_{1-α/2})<α" has the inequality direction reversed; it should presumably be P(|T_{n,λ}|>z_{1-α/2})≤α.
  2. [Theorem 4.1 and Appendix B] Theorem 4.1 assumes φ in a nonstandard space of continuous functions with finite limits at ±∞, while Lemmas B.2–B.4 assume φ∈C_b^3(R); the approximation argument should be stated before the lemmas, and the boundedness of rewards used in Lemma B.2 is not listed among the assumptions of Theorem 4.1.
  3. [Remark 4.5] Remark 4.5 says simulations show K=2 performs well, but no experiment varying K is reported; either add such an experiment or remove the claim.
  4. [Algorithm 1] The choice B=25 is justified only by an informal observation that larger B does not improve power; a sensitivity analysis or a more precise reference would be helpful.
  5. [Table 3(b)] In the XGBoost row of Table 3(b), PWTAB gives p=0.909 while WTAB gives p=0.491, which contradicts the claimed uniform superiority of PWTAB and is not discussed in the text.
  6. [Notation, Eq. (7)] The notation \bar μ_n^{(1)} = -\bar μ_n^{(0)} is used even though bμ_i are estimated and may not be exactly skew-symmetric; the definitions should be made explicit to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SCLT is derived by a self-contained SDE/martingale argument and benchmarked against standard normal quantiles; the main gap (Remark 4.4) is an unproven oracle-to-estimated transfer, not a circular reduction.

full rationale

The paper's central claim, the strategic central limit theorem for the weighted two-armed bandit statistic T_{n,lambda}(theta*_n), is not circular by construction. The limiting spike distribution B(omega_n, sigma_0) is derived in Appendix B from an explicit SDE (Equation (15)) and its transition density; the proof uses Taylor expansions, martingale estimates (Lemmas B.2-B.4), and standard Brownian motion, rather than assuming the conclusion. The optimal-policy statement in Lemma 2.1 is also proved in the appendix 'independently of Theorem 3.3 in [6]' using the paper's own Theorem 4.1 and Lemma B.4, so the optimality claim is not merely imported from the authors' prior work. The lambda selection in Algorithm 1 is a fixed formula lambda = tau*sqrt(n)/(sigma_hat + tau*sqrt(n)) that enforces the convergence-rate bound in Theorem 4.1; it is not fitted to the data's p-values, and the bootstrap-tuning paragraph is a suggestion, not part of the theoretical claims. The empirical benchmarks against standard normal critical values and against CUPED, DIM, and z-DML provide external checks. The notable weakness is Remark 4.4, which asserts that E(mu_hat) = E(Y(1)-Y(0)) implies the oracle Theorems 4.1 and 4.2 carry over to the estimated statistic. This is a missing proof and a real correctness risk for the practical PWTAB procedure, because the sign-dependent policy and the estimated volatility depend on the entire sequence and its variance, not just the first moment. However, this is an unproven transfer, not a circular reduction: no equation in the paper is equivalent to its inputs by construction, and no fitted parameter is renamed as a prediction. The paper therefore receives a circularity score of 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central theorem is a pure probability result about the oracle statistic. The free parameters λ, τ, B, K are tuning choices; λ in particular is set using the data (σ̂) and affects both power and type I error. The axioms are standard causal assumptions plus a stochastic-analysis existence result. No new physical or causal entities are introduced; the 'spike distribution' is derived from the SDE, not postulated ad hoc.

free parameters (4)
  • λ = data-dependent: λ = τ√n/(σ̂+τ√n) with τ typically 0.03; alternatively selected by bootstrap to control type I error
    Weighting parameter balancing the mean and volatility terms in the statistic; directly affects power and convergence rate.
  • τ = 0.03 (typical)
    Threshold used to choose λ so that the convergence rate term λσ/((1-λ)√n) is kept at most τ.
  • B = 25
    Number of permutations in PWTAB; chosen because larger B did not improve power.
  • K = 2
    Number of cross-fitting folds; stated as sufficient in Remark 4.5.
assumptions (5)
  • domain assumption Rubin Causal Model assumptions: consistency, no unmeasured confounding, positivity.
    Invoked in Section 2.1 and Assumption 1 to justify the counterfactual framework and DR estimation.
  • domain assumption Assumption 1: boundedness and positivity (e(X) ≥ ε, outcomes bounded).
    Required for the doubly robust estimator and DML convergence.
  • domain assumption Assumption 2: at least one of outcome regression or propensity score models is correctly specified.
    Used in Remark 4.4 to assert E(bμ) = E(Δ), the only bridge from oracle to estimated statistic.
  • standard math Existence and uniqueness of strong solutions to SDEs with discontinuous drift (Mel'nikov [30]).
    Used in the proof of Lemma B.1 to define the SDE whose transition density yields the spike distribution.
  • domain assumption Oracle access to both potential outcomes Y(1) and Y(0) in the theoretical analysis.
    Theorems 4.1 and 4.2 are proved for the oracle statistic; this assumption is later relaxed only by assertion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Strategic A/B testing via Maximum Probability-driven Two-armed Bandit." pith.science (2026). https://pith.science/paper/HUUAKII5

@misc{pith2026250622536,
  author       = {Pith},
  title        = {Pith review of: Strategic A/B testing via Maximum Probability-driven Two-armed Bandit},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HUUAKII5}},
  note         = {Machine review of arXiv:2506.22536}
}
read the original abstract

Detecting a minor average treatment effect is a major challenge in large-scale applications, where even minimal improvements can have a significant economic impact. Traditional methods, reliant on normal distribution-based or expanded statistics, often fail to identify such minor effects because of their inability to handle small discrepancies with sufficient sensitivity. This work leverages a counterfactual outcome framework and proposes a maximum probability-driven two-armed bandit (TAB) process by weighting the mean volatility statistic, which controls Type I error. The implementation of permutation methods further enhances the robustness and efficacy. The established strategic central limit theorem (SCLT) demonstrates that our approach yields a more concentrated distribution under the null hypothesis and a less concentrated one under the alternative hypothesis, greatly improving statistical power. The experimental results indicate a significant improvement in the A/B testing, highlighting the potential to reduce experimental costs while maintaining high statistical power.

Figures

Figures reproduced from arXiv: 2506.22536 by the authors.

Figure 1
Figure 1. The power under the optimal strategy θ ∗ n (brown shadow) (top) and the probability density plot of the test statistic Tn(θn) under different strategies (bottom). can be derived under the policy θn = (ϑ1, . . . , ϑn): Tn(θn) = 1 n Xn i=1 R¯(ϑi) n | {z } Mean + 1 √ n Xn i=1 R (ϑi) i σb | {z } Volatility , (2) where σb 2 = Pn i=1(R (1) i − R¯ (1) n ) 2/(n − 1) and R¯ (1) n = Pn i=1 R (1) i /n = −R¯ (0) n . This statis… view at source ↗
Figure 2
Figure 2. Plot of statistical power and empirical type I error rate as [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Density plots of asymptotic distributions under two hypotheses, compared with a standard [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Power comparisons of various methods across different settings, presented in descending [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Variance reduction compared to DIM (left [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: More power comparisons of various methods across different settings, presented in [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 39 canonical work pages

  1. [1]

    Estimating the effect of treatment on binary outcomes using full matching on the propensity score.Statistical methods in medical research, 26(6):2505– 2525, 2017

    Peter C Austin and Elizabeth A Stuart. Estimating the effect of treatment on binary outcomes using full matching on the propensity score.Statistical methods in medical research, 26(6):2505– 2525, 2017

    work page 2017

  2. [2]

    Doubly robust estimation in missing data and causal inference models

    Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962–973, 2005. 12

    work page 2005

  3. [3]

    Bagging predictors

    Leo Breiman. Bagging predictors. Machine learning, 24:123–140, 1996

    work page 1996

  4. [4]

    Variable selection for propensity score models

    M Alan Brookhart, Sebastian Schneeweiss, Kenneth J Rothman, Robert J Glynn, Jerry Avorn, and Til Stürmer. Variable selection for propensity score models. American journal of epidemi- ology, 163(12):1149–1156, 2006

    work page 2006

  5. [5]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016

    2016

  6. [6]

    Strategy-driven limit theorems associated bandit problems

    Zengjing Chen, Shui Feng, and Guodong Zhang. Strategy-driven limit theorems associated bandit problems. arXiv preprint arXiv:2204.04442, 2022

    arXiv 2022

  7. [7]

    Strategic two-sample test via the two- armed bandit process

    Zengjing Chen, Xiaodong Yan, and Guodong Zhang. Strategic two-sample test via the two- armed bandit process. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(4):1271–1298, 2023

    work page 2023

  8. [8]

    Double/debiased machine learning for treatment and structural parameters, 2018

    Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters, 2018

    2018

Show all 45 references
  1. [9]

    Applied causal inference powered by ml and ai

    Victor Chernozhukov, Christian Hansen, Nathan Kallus, Martin Spindler, and Vasilis Syrgkanis. Applied causal inference powered by ml and ai. arXiv preprint arXiv:2403.02467, 2024

    2024 arXiv

  2. [10]

    Improving the sensitivity of online controlled experiments by utilizing pre-experiment data

    Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In Proceedings of the sixth ACM international conference on Web search and data mining, pages 123–132, 2013

    2013

  3. [11]

    Statistical methods for research workers

    Ronald Aylmer Fisher. Statistical methods for research workers. In Breakthroughs in statistics: Methodology and distribution, pages 66–70. Springer, 1970

    1970

  4. [12]

    Doubly robust estimation of causal effects

    Michele Jonsson Funk, Daniel Westreich, Chris Wiesen, Til Stürmer, M Alan Brookhart, and Marie Davidian. Doubly robust estimation of causal effects. American journal of epidemiology, 173(7):761–767, 2011

    2011

  5. [13]

    Propensity scores in intensive care and anaesthesiology literature: a systematic review

    Etienne Gayat, Romain Pirracchio, Matthieu Resche-Rigon, Alexandre Mebazaa, Jean-Yves Mary, and Raphaël Porcher. Propensity scores in intensive care and anaesthesiology literature: a systematic review. Intensive care medicine, 36:1993–2003, 2010

    1993

  6. [14]

    Machine learning for variance reduction in online experiments

    Yongyi Guo, Dominic Coey, Mikael Konutgan, Wenting Li, Chris Schoener, and Matt Goldman. Machine learning for variance reduction in online experiments. Advances in Neural Information Processing Systems, 34:8637–8648, 2021

    2021

  7. [15]

    Bootstrap

    Tim Hesterberg. Bootstrap. Wiley Interdisciplinary Reviews: Computational Statistics, 3(6):497– 526, 2011

    2011

  8. [16]

    Efficient estimation of average treatment effects using the estimated propensity score

    Keisuke Hirano, Guido W Imbens, and Geert Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4):1161–1189, 2003

    2003

  9. [17]

    Focusing on the long-term: It’s good for users and business

    Henning Hohnhold, Deirdre O’Brien, and Diane Tang. Focusing on the long-term: It’s good for users and business. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1849–1858, 2015

    2015

  10. [18]

    Statistics and causal inference

    Paul W Holland. Statistics and causal inference. Journal of the American statistical Association, 81(396):945–960, 1986

    1986

  11. [19]

    Rubin causal model

    Guido W Imbens and Donald B Rubin. Rubin causal model. In Microeconometrics, pages 229–241. Springer, 2010

    2010

  12. [20]

    Toward optimal variance reduction in online controlled experiments

    Ying Jin and Shan Ba. Toward optimal variance reduction in online controlled experiments. Technometrics, 65(2):231–242, 2023

    2023

  13. [21]

    Lightgbm: A highly efficient gradient boosting decision tree

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017. 13

    2017

  14. [22]

    The parametric g-formula for time-to-event data: intuition and a worked example.Epidemiology, 25(6):889–897, 2014

    Alexander P Keil, Jessie K Edwards, David B Richardson, Ashley I Naimi, and Stephen R Cole. The parametric g-formula for time-to-event data: intuition and a worked example.Epidemiology, 25(6):889–897, 2014

    2014

  15. [23]

    Online controlled experiments at large scale

    Ron Kohavi, Alex Deng, Brian Frasca, Toby Walker, Ya Xu, and Nils Pohlmann. Online controlled experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1168–1176, 2013

    2013

  16. [24]

    Controlled ex- periments on the web: survey and practical guide

    Ron Kohavi, Roger Longbotham, Dan Sommerfield, and Randal M Henne. Controlled ex- periments on the web: survey and practical guide. Data mining and knowledge discovery , 18:140–181, 2009

    2009

  17. [25]

    Cambridge University Press, 2020

    Ron Kohavi, Diane Tang, and Ya Xu.Trustworthy online controlled experiments: A practical guide to a/b testing. Cambridge University Press, 2020

    2020

  18. [26]

    Identifiability of causal effects with multiple causes and a binary outcome

    Dehan Kong, Shu Yang, and Linbo Wang. Identifiability of causal effects with multiple causes and a binary outcome. Biometrika, 109(1):265–272, 2022

    2022

  19. [27]

    Strengths and limitations of meta-analysis

    Young Ho Lee. Strengths and limitations of meta-analysis. The Korean Journal of Medicine, 94(5):391–395, 2019

    2019

  20. [28]

    Cauchy combination test: a powerful test with analytic p-value calcu- lation under arbitrary dependency structures

    Yaowu Liu and Jun Xie. Cauchy combination test: a powerful test with analytic p-value calcu- lation under arbitrary dependency structures. Journal of the American Statistical Association, 115(529):393–402, 2020

    2020

  21. [29]

    P-values for high-dimensional regression

    Nicolai Meinshausen, Lukas Meier, and Peter Bühlmann. P-values for high-dimensional regression. Journal of the American Statistical Association, 104(488):1671–1681, 2009

    2009

  22. [30]

    On strong solutions of stochastic differential equations with nonsmooth coefficients

    Aleksandr Viktorovich Mel’nikov. On strong solutions of stochastic differential equations with nonsmooth coefficients. Theory of Probability & Its Applications, 24(1):147–150, 1979

    1979

  23. [31]

    Causal inference in statistics: An overview

    Judea Pearl. Causal inference in statistics: An overview. 2009

    2009

  24. [32]

    A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect

    James Robins. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical modelling, 7(9-12):1393–1512, 1986

    1986

  25. [33]

    Implementation of g-computation on a simulated data set: demonstration of a causal inference technique

    Jonathan M Snowden, Sherri Rose, and Kathleen M Mortimer. Implementation of g-computation on a simulated data set: demonstration of a causal inference technique. American journal of epidemiology, 173(7):731–738, 2011

    2011

  26. [34]

    Bounded, efficient and doubly robust estimation with inverse weighting

    Zhiqiang Tan. Bounded, efficient and doubly robust estimation with inverse weighting. Biometrika, 97(3):661–682, 2010

    2010

  27. [35]

    Control using predictions as covariates in switchback experiments

    Yixin Tang, Caixia Huang, David Kastelman, and Jared Bauman. Control using predictions as covariates in switchback experiments. 2020

    2020

  28. [36]

    Issues in stacked generalization

    Kai Ming Ting and Ian H Witten. Issues in stacked generalization. Journal of artificial intelligence research, 10:271–289, 1999

    1999

  29. [37]

    Invited commentary: G-computation–lost in translation? American journal of epidemiology, 173(7):739–742, 2011

    Stijn Vansteelandt and Niels Keiding. Invited commentary: G-computation–lost in translation? American journal of epidemiology, 173(7):739–742, 2011

    2011

  30. [38]

    Meta-analysis: Its strengths and limitations

    Esteban Walker, Adrian V Hernandez, and Michael W Kattan. Meta-analysis: Its strengths and limitations. Cleveland Clinic journal of medicine, 75(6):431, 2008

    2008

  31. [39]

    G-computation of average treatment effects on the treated and the untreated

    Aolin Wang, Roch A Nianogo, and Onyebuchi A Arah. G-computation of average treatment effects on the treated and the untreated. BMC medical research methodology, 17:1–5, 2017

    2017

  32. [40]

    Variance reduction in randomised trials by inverse probability weighting using the propensity score

    Elizabeth J Williamson, Andrew Forbes, and Ian R White. Variance reduction in randomised trials by inverse probability weighting using the propensity score. Statistics in medicine , 33(5):721–737, 2014

    2014

  33. [41]

    Stacked generalization

    David H Wolpert. Stacked generalization. Neural networks, 5(2):241–259, 1992. 14

    1992

  34. [42]

    The loop estimator: Adjusting for covariates in randomized experiments

    Edward Wu and Johann A Gagnon-Bartsch. The loop estimator: Adjusting for covariates in randomized experiments. Evaluation review, 42(4):458–488, 2018

    2018

  35. [43]

    Evaluating mobile apps with a/b and quasi a/b tests

    Ya Xu and Nanyu Chen. Evaluating mobile apps with a/b and quasi a/b tests. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 313–322, 2016

    2016

  36. [44]

    A survey on causal inference

    Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. A survey on causal inference. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(5):1–46, 2021

    2021

  37. [45]

    F m n (x) + Z m n m−1 n ˙F m n βY m−1 n , x β s βdY m−1 n , x β s + β2 2 Z m n m−1 n ¨F m n βY m−1 n , x β s ds # =E

    Baqun Zhang, Anastasios A Tsiatis, Eric B Laber, and Marie Davidian. A robust method for estimating optimal treatment regimes. Biometrics, 68(4):1010–1018, 2012. 15 A Additional experiments Table 5 presents the empirical type I error rates for all methods in the synthetic data...

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.