REVIEW 4 major objections 4 minor 36 references
Conditional Average Treatment Effect Estimation Under Hidden Confounders
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Outcome-only RCT data can correct hidden-confounder bias in CATE estimation, and the paper shows how with a combined marginal-plus-projection balancing regularizer.
desk verdict The outcome-only RCT setting is genuinely new and the experiments are solid, but the projection balancing component as implemented estimates the wrong quantity, so the central deconfounding claim is unsupported as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a pseudo-confounder generator $\psi$ that maps Gaussian noise to a variable $\tilde{U}$ emulating the unobserved confounder, feeding it into a CATE model $f_t(X,\tilde{U})$ whose conditional expectation $Z_t=\mathbb{E}[f_t(X,\tilde{U})|X]$ is the estimator. MB adds an adversarial critic $\tilde{g}$ enforcing $f_t(X,\tilde{U}) \stackrel{d}{=} Y_t$ for each $t$; PB adds a critic $g$ with values in $[-1,1]$ enforcing $\mathbb{E}[Z_t g(X)]\approx \mathbb{E}[Y'_t g(X)]$ against RCT outcomes $Y'_t$. The combined objective is $\min \mathbb{E}[(Z_T-Y)^2] + \alpha \sum_t ( \sup_{\tilde{g}}|\mathbb{E}[\tilde{g}(f_t)]-\mathbb{E}[\tilde{g}(Y'_t)]| + \sup_g |\mathbb{E}[Z_t g(X)]-\mathbb{E}[Y'_t g(X)]| )$, optimized adversarially with a schedule that raises $\alpha$ during training.
What would settle it
Simulate a hidden-confounder model where the CATE depends strongly on $X$ and the RCT outcomes are drawn from the true conditional distribution of $Y_t$ given $X$. Compute the projection-balancing loss twice: once with the paper's random outcome-feature pairing and once with true paired $X$ values from the RCT. If the random-pairing version stays near zero while the paired version does not, and adding PB does not reduce PEHE relative to the factual learner, the proposed mechanism is not aligning the projections the paper claims to align.
Extended reading notes
Core claim
Under hidden confounders the factual learner converges to $\mathbb{E}[Y|X,T=t]$, not the target $\mathbb{E}[Y_t|X]$, so observational data alone is biased. The paper's central claim is that this bias can be corrected by regularizing the factual objective with RCT outcomes. It proves an idealized projection constraint whose unique solution is the true conditional potential outcome, and a practical bound: for any estimator $Z_t$, $\mathbb{E}[|Z_t - \mathbb{E}[Y_t|X]|] \le L_p(Z_t) + \sqrt{\mathrm{Var}(Y_t)}$, where $L_p(Z_t)$ measures the worst-case mismatch between projections of $Z_t$ and projections of the RCT outcomes. The paper concludes that an estimator with small projection mismatch is close to the true conditional potential outcome, up to an irreducible variance floor, and implements this with a pseudo-confounder generator inside a neural CATE model.
Load-bearing premise
The projection-balancing penalty in the implemented algorithm pairs each RCT outcome with a randomly chosen observational feature vector, so the term it actually computes is an estimate of $\mathbb{E}[g(X)]\,\mathbb{E}[Y_t]$ rather than $\mathbb{E}[g(X)Y_t]$; this equivalence is valid only if RCT outcomes are independent of the features, an assumption the paper never states and one that fails whenever treatment effects truly vary with $X$.
Editorial extensions
If this is right
- Observational-data CATE models can be deconfounded using RCT outcome summaries alone, with no RCT covariates ever disclosed.
- The combined MB+PB penalty should keep $\sqrt{\epsilon_{\mathrm{PEHE}}}$ low even as the confounding degree $\Gamma$ grows, since the projection constraint shrinks the hypothesis class of the factual learner.
- A small RCT batch, on the order of 25 to 50 outcomes, is enough to stabilize the improvement, so the method is viable where experiments are costly.
- Because RCT features are not needed, the method applies when privacy rules, mismatched feature sets, or lost historical records prevent direct pooling of RCT and observational data.
Reading between the lines
- If the random pairing in the implemented PB loss is taken literally, the method's success on heterogeneous-effect data likely depends on the RCT outcomes being effectively independent of $X$ in the sample; that is a testable premise and not one the paper checks.
- The variance floor in Proposition 3.4 suggests the guarantee is weakest precisely when potential outcomes have large spread; reducing that floor, for example by modeling effect modifiers, is a natural extension the paper leaves open.
- A concrete variant worth testing: replace the random pairing with importance weighting or quantile matching of RCT outcomes to observational covariates, which should recover the true projection mismatch when the independence premise fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method for estimating conditional average treatment effects (CATE) under hidden confounders by leveraging a small outcome-only randomized controlled trial (RCT) dataset in addition to a large observational dataset. Two regularizers are introduced: Marginals Balancing (MB), which matches the distribution of predicted potential outcomes to the RCT outcome marginals via a pseudo-confounder generator, and Projections Balancing (PB), which is intended to match projections of the learned potential outcomes onto functions of the covariates to those of the true potential outcomes. The combined method, MB+PB, is evaluated on synthetic data and three real-world datasets (STAR, ACTG, NSW), reporting competitive or better PEHE than baselines and prior methods. The theoretical component includes Proposition 3.2, showing uniqueness of the ideal PB solution, and Proposition 3.4, an error bound for the practical PB learner.
Significance. If the central claim were valid, the paper would make a valuable contribution: using only RCT outcomes, without covariates, to deconfound CATE estimation is practically important for privacy and data-sharing settings. The paper also clearly identifies the non-uniqueness of MB and provides an explicit example. The empirical evaluation follows established benchmarks and demonstrates that a heuristic combination can yield low PEHE on several datasets. However, the theoretical and algorithmic core of the PB component has a fundamental flaw: the implemented PB loss does not estimate the quantity defined in Definition 3.3, and Proposition 3.4's proof relies on an unstated independence assumption. Because PB is the component that is supposed to resolve the non-identifiability of MB, the paper's central claim is not supported by its own derivations or algorithm.
major comments (4)
- [Algorithm 1, line 10] The implemented projection-balancing loss does not estimate the quantity in Definition 3.3. The first term in L_p pairs each RCT outcome y_i^t with a randomly selected observational covariate x_{\lambda(i)}; because the RCT and observational samples are independent, this empirical average converges to E[g(X)]E[Y'_t], not E[g(X)Y'_t]. The equality of these two quantities requires Y'_t \perp X, an assumption that is never stated and that fails whenever potential outcomes depend on X, i.e., whenever CATE is heterogeneous. As a consequence, a constant estimator Z_t = E[Y_t] achieves zero PB penalty, and the PB regularizer does not constrain the CATE at all. Since PB is the component that is claimed to make the solution unique and is half of MB+PB, the central claim that outcome-only RCT outcomes regularize CATE through projection balancing is unsupported.
- [Appendix A, proof of Proposition 3.4] The proof replaces E[Y'_t g(X)] by E[Y'_t]E[g(X)] in the third displayed inequality. This step is valid only under Y'_t \perp X, which is not part of the assumptions of Proposition 3.4 or Definition 3.3. The paper's own generative models (e.g., Section 3.1) specify Y_t as a function of X and U, so Y_t depends on X. Thus the proof of the central bound relies on an implicit independence assumption that contradicts the heterogeneous-CATE setting.
- [Proposition 3.4 and Remark 3.5] Even if the independence issue were resolved, the bound E[|Z_t - E[Y_t|X]|] \le L_p(Z_t) + \sqrt{\mathrm{Var}(Y_t)} contains an estimator-independent \sqrt{\mathrm{Var}(Y_t)} term. With L_p = 0 the bound reduces to \sqrt{\mathrm{Var}(Y_t)}, which is generally large for heterogeneous outcomes and does not control the estimation error of E[Y_t|X]. Therefore Remark 3.5's statement that a solution to the PB optimization has 'guaranteed performance' is not justified.
- [Definition 3.3] The expression E[Y'_t g(X)] is not well-defined by the problem setup. RCT data D_r^t contains outcomes only, with no covariates, so there is no joint sample of (Y'_t, X). Unless an explicit coupling between Y'_t and X is specified, the constraint in Eq. (7) cannot be estimated from the available data. The random pairing in Algorithm 1 implicitly selects an independent coupling, which reduces the constraint to matching E[g(X)] E[Y'_t]. This is a fundamental identifiability issue rather than a technical detail.
minor comments (4)
- [Algorithm 1, line 7] The factual loss term t_i(y_i - \hat{y}_i)^2 + (1-t_i)(y_i - \hat{y}_i)^2 simplifies to (y_i - \hat{y}_i)^2; the redundant notation appears to be a typo.
- [Figure 9] The synthetic results in Figure 9 are reported without error bars or standard deviations; adding them would help assess the claimed robustness of MB+PB across confounding levels.
- [Table 1] On STAR and ACTG, the reported differences between MB+PB and CorNet are within one standard deviation, and no significance tests are provided; the claim of 'outperforming CorNet in two of the three tasks' should be qualified accordingly.
- [Appendix B.2] The description of generating the small RCT dataset uses covariate-based selection ('birthday' for STAR, 'gender' for ACTG, 'age' for NSW); this should be clarified as part of the evaluation protocol for creating distribution shift, not as input to the method, to avoid an apparent contradiction with the outcome-only assumption.
Circularity Check
No significant circularity: RCT outcomes serve as external anchors, and the PB implementation gap is a validity concern, not a circular derivation.
full rationale
The paper's derivation chain is not circular. The proposed MB+PB method uses RCT outcomes as external targets for distributional and projection matching; these outcomes are not fitted parameters and are not derived from the CATE estimates themselves. The RCT data are sampled independently from the observational data, and the method optimizes a factual loss plus regularization terms anchored to those external outcomes. No claim reduces to its own inputs by construction: the pseudo-confounder generator is learned, but the balancing losses are evaluated against observed RCT outcomes, and the final CATE predictions are compared to independently constructed pseudo-ground truth in the real-data experiments. The theoretical guarantee in Proposition 3.4 is admittedly loose due to the additive sqrt(Var(Y_t)) term, and the implementation in Algorithm 1's projection-balancing loss pairs each RCT outcome with a randomly selected observational covariate, so the empirical PB term estimates E[g(X)]E[Y_t] rather than E[Y_t g(X)]. This is a substantive correctness and identifiability gap—it means the PB constraint may be weakly informative for heterogeneous CATE—but it is not circularity: the method still anchors to external RCT outcomes, and a constant predictor with zero PB loss would still be judged by PEHE against the true CATE. The paper also explicitly acknowledges the non-uniqueness of the MB solution in Appendix A, which is an honest limitation rather than a hidden reduction. No self-citations are load-bearing, no fitted parameter is renamed as a prediction, and no known result is merely relabeled. The central empirical claim is self-contained against the reported benchmarks and does not depend on the paper's own prior work for its validity.
Assumptions & free parameters
free parameters (4)
- alpha regularization schedule =
0.01 for epochs 1-1230, linear to 100 by epoch 1430, then 100
- balancing iterations =
5 when alpha is small, 50 when alpha is large
- tolerance C in Definition 3.3 =
not set
- outcome selection threshold c =
1 for STAR, 0 for ACTG, 0.25 for NSW
assumptions (6)
- domain assumption X is independent of U
- domain assumption Y_t is conditionally independent of T given X and U
- domain assumption RCT potential outcomes and observational potential outcomes have the same marginal distribution
- ad hoc to paper Y'_t is independent of X for the implemented PB loss
- domain assumption Positivity P(T=1|X) > 0
- ad hoc to paper The pseudo-confounder family can represent the confounding effect
invented entities (1)
-
pseudo-confounder U~ = psi(eta)
Cite this review
Pith. "Pith review of Conditional Average Treatment Effect Estimation Under Hidden Confounders." pith.science (2026). https://pith.science/paper/UJYYQOO2
@misc{pith2026250612304,
author = {Pith},
title = {Pith review of: Conditional Average Treatment Effect Estimation Under Hidden Confounders},
year = {2026},
howpublished = {\url{https://pith.science/paper/UJYYQOO2}},
note = {Machine review of arXiv:2506.12304}
}
read the original abstract
One of the major challenges in estimating conditional potential outcomes and conditional average treatment effects (CATE) is the presence of hidden confounders. Since testing for hidden confounders cannot be accomplished only with observational data, conditional unconfoundedness is commonly assumed in the literature of CATE estimation. Nevertheless, under this assumption, CATE estimation can be significantly biased due to the effects of unobserved confounders. In this work, we consider the case where in addition to a potentially large observational dataset, a small dataset from a randomized controlled trial (RCT) is available. Notably, we make no assumptions on the existence of any covariate information for the RCT dataset, we only require the outcomes to be observed. We propose a CATE estimation method based on a pseudo-confounder generator and a CATE model that aligns the learned potential outcomes from the observational data with those observed from the RCT. Our method is applicable to many practical scenarios of interest, particularly those where privacy is a concern (e.g., medical applications). Extensive numerical experiments are provided demonstrating the effectiveness of our approach for both synthetic and real-world datasets.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Bayesian inference of individualized treatment effects using multi-task gaussian processes
Ahmed M Alaa and Mihaela Van Der Schaar. Bayesian inference of individualized treatment effects using multi-task gaussian processes. Advances in neural information processing systems, 30, 2017
work page 2017
-
[2]
P Billingsley. Probability and measure. 3rd wiley. New York, 1995
work page 1995
-
[3]
Elise Chor, P. Lindsay Chase-Lansdale, Teresa Eckrich Sommer, Terri Sabol, Lauren Tighe, Jeanne Brooks-Gunn, Hirokazu Yoshikawa, Amanda Morris, and Christopher King. Three-year outcomes for low-income parents of young children in a two-generation education program. Journal of Research on Educational Effectiveness, 0 0 (0): 0 1--42, 2024
work page 2024
-
[4]
Causal inference methods for combining randomized trials and observational studies: a review
B \'e n \'e dicte Colnet, Imke Mayer, Guanhua Chen, Awa Dieng, Ruohong Li, Ga \"e l Varoquaux, Jean-Philippe Vert, Julie Josse, and Shu Yang. Causal inference methods for combining randomized trials and observational studies: a review. Statistical science, 39 0 (1): 0 165--191, 2024
work page 2024
-
[5]
Probability: theory and examples, volume 49
Rick Durrett. Probability: theory and examples, volume 49. Cambridge university press, 2019
2019
-
[6]
Causal-stonet: Causal inference for high-dimensional complex data
Yaxin Fang and Faming Liang. Causal-stonet: Causal inference for high-dimensional complex data. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=BtZ7vCt5QY
work page 2024
-
[7]
Causal machine learning for predicting treatment outcomes
Stefan Feuerriegel, Dennis Frauen, Valentyn Melnychuk, Jonas Schweisthal, Konstantin Hess, Alicia Curth, Stefan Bauer, Niki Kilbertus, Isaac S Kohane, and Mihaela van der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30 0 (4): 0 958--968, 2024
work page 2024
-
[8]
Monica Gandhi, Niloufar Ameli, Peter Bacchetti, Gerald B Sharp, Audrey L French, Mary Young, Stephen J Gange, Kathryn Anastos, Susan Holman, Alexandra Levine, et al. Eligibility criteria for hiv clinical trials and generalizability of results: the gap between published reports and study protocols. Aids, 19 0 (16): 0 1885--1896, 2005
work page 2005
Show all 36 references
-
[9]
Causal inference in public health
Thomas A Glass, Steven N Goodman, Miguel A Hernán, and Jonathan M Samet. Causal inference in public health. Annual Review of Public Health, 34: 0 61--75, 2013. doi:10.1146/annurev-publhealth-031811-124606
2013 doi
-
[10]
Priority issues concerning hiv infection among women
Ruth M Greenblatt. Priority issues concerning hiv infection among women. Women's Health Issues, 21 0 (6): 0 S266--S271, 2011
2011
-
[11]
Estimating heterogeneous treatment effects: Mutual information bounds and learning algorithms
Xingzhuo Guo, Yuchen Zhang, Jianmin Wang, and Mingsheng Long. Estimating heterogeneous treatment effects: Mutual information bounds and learning algorithms. In International Conference on Machine Learning, pages 12108--12121. PMLR, 2023
2023
-
[12]
A trial comparing nucleoside monotherapy with combination therapy in hiv-infected adults with cd4 cell counts from 200 to 500 per cubic millimeter
Scott M Hammer, David A Katzenstein, Michael D Hughes, Holly Gundacker, Robert T Schooley, Richard H Haubrich, W Keith Henry, Michael M Lederman, John P Phair, Manette Niu, et al. A trial comparing nucleoside monotherapy with combination therapy in hiv-infected adults with cd4...
1996
-
[13]
Sequential deconfounding for causal inference with unobserved confounders
Tobias Hatt and Stefan Feuerriegel. Sequential deconfounding for causal inference with unobserved confounders. In Causal Learning and Reasoning, pages 934--956. PMLR, 2024
2024
-
[14]
Combining observational and randomized data for estimating heterogeneous treatment effects
Tobias Hatt, Jeroen Berrevoets, Alicia Curth, Stefan Feuerriegel, and Mihaela van der Schaar. Combining observational and randomized data for estimating heterogeneous treatment effects. arXiv preprint arXiv:2202.12891, 2022 a
2022 arXiv
-
[15]
Generalizing off-policy learning under sample selection bias
Tobias Hatt, Daniel Tschernutter, and Stefan Feuerriegel. Generalizing off-policy learning under sample selection bias. In Uncertainty in Artificial Intelligence, pages 769--779. PMLR, 2022 b
2022
-
[16]
Causal Inference: What If
Miguel A Hern \'a n and James M Robins. Causal Inference: What If. Chapman & Hall/CRC, Boca Raton, 2020
2020
-
[17]
Bayesian nonparametric modeling for causal inference
Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20 0 (1): 0 217--240, 2011
2011
-
[18]
Causal inference in the social sciences
Guido W Imbens. Causal inference in the social sciences. Annual Review of Statistics and Its Application, 11, 2024
2024
-
[19]
Causal inference in statistics, social, and biomedical sciences
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015
2015
-
[20]
Confounding-robust policy improvement
Nathan Kallus and Angela Zhou. Confounding-robust policy improvement. Advances in neural information processing systems, 31, 2018
2018
-
[21]
Removing hidden confounding by experimental grounding
Nathan Kallus, Aahlad Manas Puli, and Uri Shalit. Removing hidden confounding by experimental grounding. Advances in neural information processing systems, 31, 2018
2018
-
[22]
Interval estimation of individual-level causal effects under unobserved confounding
Nathan Kallus, Xiaojie Mao, and Angela Zhou. Interval estimation of individual-level causal effects under unobserved confounding. In The 22nd international conference on artificial intelligence and statistics, pages 2281--2290. PMLR, 2019
2019
-
[23]
Evaluating the econometric evaluations of training programs with experimental data
Robert J LaLonde. Evaluating the econometric evaluations of training programs with experimental data. The American economic review, pages 604--620, 1986
1986
-
[24]
Removing hidden confounding in recommendation: a unified multi-task learning approach
Haoxuan Li, Kunhan Wu, Chunyuan Zheng, Yanghao Xiao, Hao Wang, Zhi Geng, Fuli Feng, Xiangnan He, and Peng Wu. Removing hidden confounding in recommendation: a unified multi-task learning approach. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[25]
Bounds on representation-induced confounding bias for treatment effect estimation
Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. Bounds on representation-induced confounding bias for treatment effect estimation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=d3xKPQVjSc
2024
-
[26]
B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding
Miruna Oprescu, Jacob Dorn, Marah Ghoummaid, Andrew Jesson, Nathan Kallus, and Uri Shalit. B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding. In International Conference on Machine Learning, pages 26599--26618. PMLR, 2023
2023
-
[27]
The logic of scientific discovery
Karl Popper. The logic of scientific discovery. Routledge, 2005
2005
-
[28]
Robust agents learn causal world models
Jonathan Richens and Tom Everitt. Robust agents learn causal world models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=pOoKI3ouv1
2024
-
[29]
Rosenbaum
Paul R. Rosenbaum. Observational Studies. Springer, New York, 2nd edition, 2002
2002
-
[30]
The central role of the propensity score in observational studies for causal effects
Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70 0 (1): 0 41--55, 1983
1983
-
[31]
Meta-learners for partially-identified treatment effects across multiple environments
Jonas Schweisthal, Dennis Frauen, Mihaela Van Der Schaar, and Stefan Feuerriegel. Meta-learners for partially-identified treatment effects across multiple environments. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machin...
2024
-
[32]
Estimating individual treatment effect: generalization bounds and algorithms
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning, pages 3076--3085. PMLR, 2017
2017
-
[33]
Adapting neural networks for the estimation of treatment effects
Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. Advances in neural information processing systems, 32, 2019
2019
-
[34]
Does matching overcome lalonde's critique of nonexperimental estimators? Journal of econometrics, 125 0 (1-2): 0 305--353, 2005
Jeffrey A Smith and Petra E Todd. Does matching overcome lalonde's critique of nonexperimental estimators? Journal of econometrics, 125 0 (1-2): 0 305--353, 2005
2005
-
[35]
Sense and sensitivity analysis: Simple post-hoc analysis of bias due to unobserved confounding
Victor Veitch and Anisha Zaveri. Sense and sensitivity analysis: Simple post-hoc analysis of bias due to unobserved confounding. Advances in neural information processing systems, 33: 0 10999--11009, 2020
2020
-
[36]
Estimation and inference of heterogeneous treatment effects using random forests
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113 0 (523): 0 1228--1242, 2018
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.