Pith. sign in

REVIEW 2 major objections 2 minor 31 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Ratio CATE for binary outcomes decomposes exactly into a product of two odds ratios.

2026-06-29 20:09 UTC pith:MP3KLLDK

load-bearing objection The paper's clean decomposition of ratio CATE into two odds ratios plus tailored DR augmentations is the real contribution, and the benchmarks back practical gains in low-conversion and confounded settings. the 2 major comments →

arxiv 2605.26288 v1 pith:MP3KLLDK submitted 2026-05-25 stat.ML cs.LGstat.ME

Beyond Differences: Doubly Robust Meta-Learners for Ratio-Based Treatment Effects

Michael Fuchs , Dominik Kreiss This is my paper
classification stat.ML cs.LGstat.ME
keywords ratio CATEdoubly robust meta-learnersQ-Learnerpropensity classificationobservational dataRCT benchmarksbinary outcomestreatment effect estimation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper focuses on estimating ratio-based conditional average treatment effects instead of differences when outcomes are binary. It constructs the Q-Learner by rewriting the ratio as the product of two odds ratios. This converts the estimation problem into two separate propensity classification tasks. Benchmarks across RCT and observational datasets show advantages in low-conversion settings and under confounding.

Core claim

The Q-Learner decomposes the ratio CATE τ(x) = E[Y|W=1,X=x] / E[Y|W=0,X=x] into a product of two odds ratios, reducing ratio-CATE estimation for binary outcomes to two propensity classification tasks. Doubly robust augmentations are derived for both S/T- and Q-style ratio learners with distinct robustness properties.

What carries the argument

The Q-Learner decomposition of the ratio CATE into a product of two odds ratios that reduces the problem to propensity classification tasks.

Load-bearing premise

The ratio CATE can be exactly decomposed into a product of two odds ratios without additional assumptions or loss of information.

What would settle it

A low-conversion RCT dataset where outcome-regression methods outperform the Q-Learner after standard hyperparameter tuning and propensity estimation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On seven RCT datasets the Q-Learner is the most consistently competitive method in low-conversion regimes.
  • On four observational datasets the doubly robust learners introduced here outperform other methods.
  • The doubly robust ratio learners serve as the natural default choice for confounded observational data.
  • The approach avoids both log-linear parametric restrictions and generic regression without robustness guarantees.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Marketers and clinicians working with relative effects such as lift or risk ratios could adopt these estimators in place of difference-based ones.
  • The classification reduction may extend to other ratio functionals if analogous exact decompositions can be found.
  • Systematic comparisons on additional datasets with controlled conversion rates would isolate when the low-conversion advantage appears.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces the Q-Learner for estimating ratio-based conditional average treatment effects (CATE) τ(x) = E[Y|W=1,X=x] / E[Y|W=0,X=x] for binary outcomes by decomposing τ(x) into a product of two odds ratios, reducing the task to two propensity classification problems. It further derives doubly robust augmentations for both S/T- and Q-style ratio learners, characterizes their robustness properties, and reports benchmark results showing the Q-Learner as most consistently competitive on seven RCT datasets in low-conversion regimes and the DR learners as superior on four observational datasets.

Significance. If the results hold, the work supplies a theoretically clean and practically advantageous approach to ratio CATE estimation in domains where ratios are the natural functional (medicine, pricing, marketing). The exact decomposition via Bayes' rule with no extra assumptions or information loss, the parameter-free reduction to classification tasks, and the explicit characterization of distinct robustness properties for the DR learners are genuine strengths. The empirical findings on RCT versus observational regimes provide actionable guidance for practitioners facing low-conversion or confounded data.

major comments (2)
  1. Abstract and experimental section: the central performance claims—that the Q-Learner is 'the most consistently competitive method in low-conversion regimes' on seven RCT datasets and that 'the DR learners introduced here decisively come out on top' on four observational datasets—require explicit reporting of model specifications, hyperparameter selection, data exclusion rules, evaluation metrics, and whether error bars or statistical tests accompany the rankings; without these the benchmark assertions cannot be verified.
  2. Methods section on the Q-Learner construction: while the decomposition τ(x) = [P(W=1|Y=1,X)/P(W=0|Y=1,X)] × [(1-e(x))/e(x)] is an exact identity for binary Y, the manuscript should state the precise conditions (binary outcome, no additional modeling assumptions) in a dedicated proposition or theorem so that readers can immediately see the scope of the reduction to propensity tasks.
minor comments (2)
  1. Notation: the symbol τ(x) is used for the ratio CATE; a brief sentence distinguishing it from the conventional difference-based CATE would prevent confusion for readers accustomed to the latter.
  2. The abstract states that the DR learners have 'distinct robustness properties'; a short table or enumerated list in the main text summarizing which nuisance functions must be correctly specified for each variant would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and positive assessment of the work. We address each major comment below and will revise the manuscript accordingly to improve clarity and reproducibility.

read point-by-point responses

  1. Referee: Abstract and experimental section: the central performance claims—that the Q-Learner is 'the most consistently competitive method in low-conversion regimes' on seven RCT datasets and that 'the DR learners introduced here decisively come out on top' on four observational datasets—require explicit reporting of model specifications, hyperparameter selection, data exclusion rules, evaluation metrics, and whether error bars or statistical tests accompany the rankings; without these the benchmark assertions cannot be verified.

    Authors: We agree that the experimental details must be fully specified to allow verification of the reported performance rankings. In the revised manuscript we will expand the experimental section (and update the abstract where space permits) to document: the precise base learners and model specifications, the hyperparameter selection protocol (including cross-validation folds and search ranges), data exclusion or preprocessing rules, the exact evaluation metrics employed, and the presence of error bars or formal statistical comparisons supporting the rankings. These additions will be placed in a dedicated reproducibility subsection. revision: yes

  2. Referee: Methods section on the Q-Learner construction: while the decomposition τ(x) = [P(W=1|Y=1,X)/P(W=0|Y=1,X)] × [(1-e(x))/e(x)] is an exact identity for binary Y, the manuscript should state the precise conditions (binary outcome, no additional modeling assumptions) in a dedicated proposition or theorem so that readers can immediately see the scope of the reduction to propensity tasks.

    Authors: We accept the suggestion. The revised manuscript will contain a new, self-contained proposition (placed early in the Methods section) that formally states the decomposition as an exact identity holding for any binary outcome Y ∈ {0,1} under the standard causal assumptions and with no further modeling restrictions. The proposition will explicitly note that the reduction to two propensity-classification problems follows directly from this identity. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper's central derivation for the Q-Learner is an exact identity obtained directly from Bayes' rule: the ratio CATE decomposes as τ(x) = [P(W=1|Y=1,X)/P(W=0|Y=1,X)] × [(1-e(x))/e(x)] with e(x)=P(W=1|X), reducing estimation to two propensity tasks without added assumptions or loss of information. This is a standard probabilistic identity, not a self-definition, fitted input renamed as prediction, or ansatz smuggled via citation. Doubly robust extensions are derived properties of this identity, and all performance claims rest on external benchmarks across seven RCT and four observational datasets rather than internal self-referential quantities. No load-bearing self-citation chains or uniqueness theorems from the authors appear in the provided material, rendering the derivation self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The paper introduces the Q-Learner as a new method. It relies on standard domain assumptions from causal inference for the doubly robust properties to hold. No explicit free parameters are identified in the abstract.

axioms (1)
  • domain assumption Standard causal assumptions including consistency, positivity, and no unmeasured confounding for the DR properties to hold
    Invoked implicitly for the doubly robust augmentations described in the abstract.
invented entities (1)
  • Q-Learner no independent evidence
    purpose: Decomposes the ratio-based CATE into a product of two odds ratios to enable estimation via two propensity classification tasks
    Newly introduced estimator in the paper; no independent evidence outside this work is provided in the abstract.

pith-pipeline@v0.9.1-grok · 5720 in / 1303 out tokens · 33747 ms · 2026-06-29T20:09:00.005274+00:00 · methodology

0 comments
read the original abstract

When treatment effects are naturally expressed as ratios -- as in medicine, pricing, and marketing -- the ratio-based CATE $\tau(x) = E[Y|W=1,X=x] / E[Y|W=0,X=x]$ is the appropriate estimand. Yet existing estimators either impose a log-linear parametric structure or apply generic regression without robustness guarantees for this functional. We introduce the Q-Learner, which decomposes $\tau(x)$ into a product of two odds ratios, reducing ratio-CATE estimation for binary outcomes to two propensity classification tasks. We further derive doubly robust augmentations for both S/T- and Q-style ratio learners and characterize their distinct robustness properties. In benchmarks on seven RCT datasets, the Q-Learner is the most consistently competitive method in low-conversion regimes, where its propensity-only construction sidesteps the imbalanced regression that hurts outcome-based estimators. On four observational datasets, where propensity must be estimated and confounding cannot be ruled out, the DR learners introduced here decisively come out on top, making them practitioners' natural default for confounded observational data.

Figures

Figures reproduced from arXiv: 2605.26288 by Dominik Kreiss, Michael Fuchs.

Figure 1
Figure 1. Figure 1: Qini ratio to the S-Learner across the seven RCT datasets, sorted by conversion rate. Each dot is one [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Qini ratio to the S-Learner on the four low-conversion RCT datasets. Values above the red baseline ( [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qini ratio to the S-Learner on the four observational datasets, sorted by conversion rate. The two solid lines [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-(learner, dataset) heatmap of Qinil/QiniS on the seven RCT datasets, sorted by ascending conversion rate. Green cells indicate that the learner beats the S-Learner; red cells indicate the opposite; neutral grey cells are not statistically distinguishable from S at z = 1.96 (paired test). Cells where the learner’s Qini is negative are flagged in deep red (“Qini < 0”), since the ratio breaks down. Colour… view at source ↗
Figure 5
Figure 5. Figure 5: Per-(learner, dataset) heatmap of Qinil/QiniS on the four observational datasets, sorted by ascending conversion rate. Green cells indicate that the learner beats the S-Learner; red cells indicate the opposite; neutral grey cells are not statistically distinguishable from S at z = 1.96 (paired test). Cells where the learner’s Qini is negative are flagged in deep red (“Qini < 0”), since the ratio breaks dow… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 5 canonical work pages

  1. [1]

    Abadie, A., Angrist, J., and Imbens, G. (2002). Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica, 70(1):91--117

    2002

  2. [2]

    Andersen, L. W. (2021). Absolute vs.\ relative effects---implications for subgroup analyses. Trials, 22(1):50

    2021

  3. [3]

    Athey, S., Tibshirani, J., and Wager, S. (2019). Generalized random forests. The Annals of Statistics, 47(2):1148--1178

    2019

  4. [4]

    Boughdiri, A., Josse, J., and Scornet, E. (2025). Quantifying treatment effects: Estimating risk ratios via observational studies. Proceedings of the 42nd International Conference on Machine Learning (ICML)

    2025

  5. [5]

    Cattaneo, M. D. (2010). Efficient semiparametric estimation of multi-valued treatment effects under ignorability. Journal of Econometrics, 155(2):138--154

    2010

  6. [6]

    Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1--C68

    2018

  7. [7]

    F., Speroff, T., Dawson, N

    Connors, A. F., Speroff, T., Dawson, N. V., et al. (1996). The effectiveness of right heart catheterization in the initial care of critically ill patients. JAMA, 276(11):889--897

    1996

  8. [8]

    and van der Schaar, M

    Curth, A. and van der Schaar, M. (2021). Doing great at estimating CATE? On the neglected assumptions in benchmark comparisons of treatment effect estimators. arXiv preprint arXiv:2107.13346

    work page arXiv 2021

  9. [9]

    Diemert, E., Betlei, A., Renaudin, C., and Amini, M. R. (2018). A large scale benchmark for uplift modeling. Proceedings of AdKDD & TargetAd, KDD

    2018

  10. [10]

    Foster, D. J. and Syrgkanis, V. (2023). Orthogonal statistical learning. Annals of Statistics, 51(3). arXiv:1901.09036

    work page arXiv 2023

  11. [11]

    and Hastie, T

    Gao, Z. and Hastie, T. (2025). Estimating heterogeneous treatment effects for general responses. Biometrics, 81(4):ujaf162

    2025

  12. [12]

    Hern \'a n, M. A. and Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC

    2020

  13. [13]

    Hillstrom, K. (2008). The MineThatData E-Mail Analytics And Data Mining Challenge. MineThatData Blog

    2008

  14. [14]

    Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T. Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. NeurIPS, 30

    2017

  15. [15]

    Kennedy, E. H. (2022). Semiparametric doubly robust targeted double machine learning: a review. arXiv:2203.06469

    work page arXiv 2022

  16. [16]

    Kennedy, E. H. (2023). Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, 17(2). arXiv:2004.14497

    work page arXiv 2023

  17. [17]

    R., Sekhon, J

    K \"u nzel, S. R., Sekhon, J. S., Bickel, P. J., and Yu, B. (2019). Metalearners for estimating heterogeneous treatment effects using machine learning. PNAS, 116(10):4156--4165

    2019

  18. [18]

    and Cand \`e s, E

    Lei, L. and Cand \`e s, E. J. (2021). Conformal inference of conditional average treatment effects. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 83(5):811--845

    2021

  19. [19]

    https://sklift.readthedocs.io/en/latest/datasets.html

    Lenta Uplift Modeling Dataset (2021). https://sklift.readthedocs.io/en/latest/datasets.html

    2021

  20. [20]

    M., Sontag, D., Zemel, R., and Welling, M

    Louizos, C., Shalit, U., Mooij, J. M., Sontag, D., Zemel, R., and Welling, M. (2017). Causal effect inference with deep latent-variable models. NeurIPS, 30

    2017

  21. [21]

    Open Data Science Community

    MegaFon Uplift Competition (2019). Open Data Science Community

    2019

  22. [22]

    and Wager, S

    Nie, X. and Wager, S. (2021). Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108(2):299--319

    2021

  23. [23]

    Radcliffe, N. J. and Surry, P. D. (2011). Real-world uplift modelling with significance-based uplift trees. White Paper, Stochastic Solutions

    2011

  24. [24]

    S., Robins, J

    Richardson, T. S., Robins, J. M., and Wang, L. (2017). On modeling and estimation for the relative risk and risk difference. Journal of the American Statistical Association, 112(519):1121--1130

    2017

  25. [25]

    M., Rotnitzky, A., and Zhao, L

    Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427):846--866

    1994

  26. [26]

    Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5):688--701

    1974

  27. [27]

    H., Lau, J., McIntosh, M

    Schmid, C. H., Lau, J., McIntosh, M. W., and Cappelleri, J. C. (1998). An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Statistics in Medicine, 17(17):1923--1942

    1998

  28. [28]

    Shirvaikar, V., Stor s, A., Lin, X., and Holmes, C. (2023). Targeting relative risk heterogeneity with causal forests. arXiv preprint arXiv:2309.15793

    work page arXiv 2023

  29. [29]

    https://retailhero.ai/

    X5 Retail Hero Uplift Competition (2020). https://retailhero.ai/

    2020

  30. [30]

    Yadlowsky, S., Pellegrini, F., Lionetto, F., Braune, S., and Tian, L. (2021). Estimation and validation of ratio-based conditional average treatment effects using observational data. Journal of the American Statistical Association, 116(536):1946--1960

    2021

  31. [31]

    Zou, G. (2004). A modified Poisson regression approach to prospective studies with binary data. American Journal of Epidemiology, 159(7):702--706

    2004