Constructing external comparator groups via transportability in mean or in effect measure
Pith reviewed 2026-05-10 01:40 UTC · model grok-4.3
The pith
Causal effects in target populations can be identified by combining index trial data with external comparators under either transportability of potential outcome means or transportability of effect measures.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We delineate external comparator analyses under two distinct but related identification strategies. The first relies on exchangeability of potential outcome means, using information only on the treatments to be compared. The second relies on transportability in effect measure, requiring additional use of information on a third treatment common to the populations. We propose estimators for identifying observed data functionals, with a particular focus on semiparametric efficient augmented weighting estimators that incorporate models for the probability of trial participation, the probability of treatment, and conditional outcome means. We derive the asymptotic properties of these estimators,,
What carries the argument
Semiparametric efficient augmented weighting estimators incorporating models for the probability of trial participation, the probability of treatment, and conditional outcome means.
Load-bearing premise
Exchangeability of potential outcome means or transportability in effect measure must hold for the combined populations.
What would settle it
A data-generating process in which transportability fails and the augmented weighting estimators exhibit bias relative to the known true causal effect.
read the original abstract
Learning about causal effects in target populations and their subsets may be facilitated by combining information from multiple sources. One major class of study designs that combine information involves appending an index study with data from an external comparator, which may facilitate head-to-head comparisons of treatments initially studied in different populations. We delineate external comparator analyses under two distinct, but related, identification strategies. The first strategy relies on exchangeability (transportability) of potential outcome means, which uses information only on the treatments that are to be compared. The second strategy relies on transportability in effect measure, requiring additional use of information on a third treatment common to the populations that have been combined. In a time-fixed setting with a point treatment and non-failure time outcome, we examine identification and estimation under a basic setup where information from an index trial is combined with a second, and external to the index trial, data source. We propose estimators for identifying observed data functionals, with a particular focus on semiparametric efficient augmented weighting estimators that incorporate models for the probability of trial participation, the probability of treatment, and conditional outcome means. We derive the asymptotic properties of these augmented weighting estimators -- including robustness to model misspecification and slower rates of convergence for some nuisance function models -- and use simulation to compare their finite sample performance to estimators based only on outcome modeling or weighting. Last, we provide a practical demonstration of the proposed methods by combining the ACCEPT and PHOENIX 1 randomized trials to evaluate the effect of various biologic agents on plaque psoriasis, a chronic inflammatory disorder.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper delineates two identification strategies for causal effects when appending an index trial with external comparator data: transportability of potential outcome means (using only the treatments of interest) and transportability in effect measure (requiring a third common treatment). It proposes semiparametric efficient augmented weighting estimators that incorporate models for trial participation probability, treatment probability, and conditional outcome means; derives their asymptotic properties including robustness to misspecification and slower nuisance convergence rates; compares finite-sample performance to pure outcome modeling and weighting estimators via simulation; and applies the methods to combine ACCEPT and PHOENIX 1 trials for evaluating biologic agents in plaque psoriasis.
Significance. If the asymptotic robustness properties hold for both strategies, the work provides a principled and efficient framework for external comparator analyses that improves upon standard approaches by allowing partial nuisance misspecification and slower convergence rates. The simulation comparisons and real-data demonstration add practical value for clinical research settings where direct randomization in the target population is infeasible.
major comments (2)
- [§4] §4 (Estimation under transportability in effect measure) and the associated influence function derivation: the claimed robustness to partial nuisance misspecification (double or triple robustness) is not fully verified for the effect-measure strategy. The influence function must be shown to cancel bias terms when only subsets of the three nuisance models (trial participation, treatment, outcome) are correct; otherwise the practical advantage over pure outcome modeling is reduced, as noted in the stress-test concern. Please provide the explicit expansion or theorem establishing the conditions.
- [§5] Theorem on asymptotic normality (likely §5): the slower rates of convergence permitted for nuisance estimators (e.g., n^{-1/4}) must be explicitly tied to both identification strategies and verified to ensure the central limit theorem and efficiency claims hold uniformly; the current outline leaves open whether the effect-measure case requires stricter rates.
minor comments (2)
- [Abstract] The abstract states the focus on augmented weighting estimators but could more clearly contrast the differing assumption sets and robustness properties of the two transportability strategies.
- [Simulation section] In the simulation section, include the exact data-generating processes and parameter values to facilitate reproducibility of the finite-sample comparisons.
Simulated Author's Rebuttal
We thank the referee for their careful reading and valuable comments on our manuscript. We have carefully considered the major comments and provide point-by-point responses below. We will make revisions to address the concerns raised.
read point-by-point responses
-
Referee: [§4] §4 (Estimation under transportability in effect measure) and the associated influence function derivation: the claimed robustness to partial nuisance misspecification (double or triple robustness) is not fully verified for the effect-measure strategy. The influence function must be shown to cancel bias terms when only subsets of the three nuisance models (trial participation, treatment, outcome) are correct; otherwise the practical advantage over pure outcome modeling is reduced, as noted in the stress-test concern. Please provide the explicit expansion or theorem establishing the conditions.
Authors: We appreciate this observation. The manuscript claims triple robustness for the augmented weighting estimator under transportability in effect measure, but we acknowledge that the explicit bias cancellation expansion for cases where only subsets of the nuisance models are correct was not detailed in the main text or appendix. We will add a new subsection or appendix entry providing the full influence function expansion, demonstrating that the estimator is consistent if any two of the three nuisance functions (trial participation probability, treatment probability, and conditional outcome mean) are correctly specified. This will strengthen the presentation of the robustness properties and directly address the concern about practical advantages. revision: yes
-
Referee: [§5] Theorem on asymptotic normality (likely §5): the slower rates of convergence permitted for nuisance estimators (e.g., n^{-1/4}) must be explicitly tied to both identification strategies and verified to ensure the central limit theorem and efficiency claims hold uniformly; the current outline leaves open whether the effect-measure case requires stricter rates.
Authors: We thank the referee for highlighting this point. The asymptotic normality result in Theorem 5.1 is derived under conditions that apply to both identification strategies, allowing nuisance estimators to converge at rates slower than n^{-1/2} (specifically n^{-1/4} under standard regularity conditions for the influence function to be asymptotically linear). However, to ensure clarity, we will revise the theorem statement and its proof to explicitly state the conditions for both the mean transportability and effect measure transportability cases, confirming that the same rate requirements suffice for the central limit theorem to hold and that the efficiency claims are uniform across strategies. No stricter rates are needed for the effect-measure case. revision: yes
Circularity Check
No significant circularity; derivations rely on standard identification assumptions and semiparametric efficiency theory
full rationale
The paper delineates two identification strategies (transportability of potential outcome means or in effect measure) and proposes augmented weighting estimators incorporating models for trial participation, treatment probability, and conditional outcome means. Asymptotic properties, including robustness to misspecification, are derived from standard semiparametric efficiency theory rather than reducing to fitted quantities or self-citations by construction. No load-bearing steps equate predictions to inputs via definition, renaming, or author-specific uniqueness theorems. The central claims remain independent of the paper's own fitted values.
Axiom & Free-Parameter Ledger
free parameters (1)
- models for trial participation probability, treatment probability, and conditional outcome means
axioms (3)
- domain assumption Exchangeability (transportability) of potential outcome means between index and external populations conditional on covariates
- domain assumption Transportability in effect measure, requiring a common third treatment
- standard math Standard positivity and consistency assumptions for causal inference
Reference graph
Works this paper leans on
-
[1]
Ung, L., Wang, G., Haneuse, S., Hern´ an, M. A. & Dahabreh, I. J.Identification strate- gies for combining an experimental study with external data2026. arXiv: 2406.03302 [stat.ME]
work page internal anchor Pith review Pith/arXiv arXiv
-
[2]
Griffiths, C. E.et al.Comparison of ustekinumab and etanercept for moderate-to-severe psoriasis.New England Journal of Medicine362,118–128 (2010)
work page 2010
-
[3]
Leonardi, C. L.et al.Efficacy and safety of ustekinumab, a human interleukin-12/23 monoclonal antibody, in patients with psoriasis: 76-week results from a randomised, double-blind, placebo-controlled trial (PHOENIX 1).The Lancet371,1665–1674 (2008)
work page 2008
-
[4]
Guyatt, G. H. & Rennie, D. Users’ guides to the medical literature.Journal of the American Medical Association270,2096–2097 (1993)
work page 2096
-
[5]
Dans, A. L., Dans, L. F., Guyatt, G. H., Richardson, S., Group, E.-B. M. W.,et al. Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient.Journal of the American Medical Association279, 545–549 (1998)
work page 1998
-
[6]
Gehan, E. A. & Freireich, E. J. Non-randomized controls in cancer clinical trials.New England Journal of Medicine290,198–203 (1974)
work page 1974
-
[7]
Pocock, S. J. The combination of randomized and historical controls in clinical trials. Journal of Chronic Diseases29,175–188 (1976)
work page 1976
-
[8]
Bucher, H. C., Guyatt, G. H., Griffith, L. E. & Walter, S. D. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology50,683–691 (1997)
work page 1997
-
[9]
Network meta-analysis for indirect treatment comparisons.Statistics in Medicine21,2313–2324 (2002)
Lumley, T. Network meta-analysis for indirect treatment comparisons.Statistics in Medicine21,2313–2324 (2002). 32
work page 2002
-
[10]
Caldwell, D. M., Ades, A. & Higgins, J. Simultaneous comparison of multiple treat- ments: combining direct and indirect evidence.British Medical Journal331,897–900 (2005)
work page 2005
-
[11]
Ioannidis, J. P. Integration of evidence from multiple meta-analyses: a primer on um- brella reviews, treatment networks and multiple treatments meta-analyses.Canadian Medical Association Journal181,488–493 (2009)
work page 2009
-
[12]
Rubin, D. B. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of Educational Psychology66,688 (1974)
work page 1974
-
[13]
Robins, J. M. A new approach to causal inference in mortality studies with a sus- tained exposure period – application to control of the healthy worker survivor effect. Mathematical Modelling7,1393–1512 (1986)
work page 1986
-
[14]
On the application of probability theory to agricultural experi- ments
Splawa-Neyman, J. On the application of probability theory to agricultural experi- ments. Essay on principles. Section 9. [Translated from Splawa-Neyman, J (1923) in Roczniki Nauk Rolniczych Tom X, 1–51]. Trans. by Dabrowska, D. M. & Speed, T. P. Statistical Science5,465–472 (1990)
work page 1923
-
[15]
Shook-Sa, B. E.et al.Fusing trial data for treatment comparisons: Single vs multi-span bridging.Statistics in Medicine43,793–815 (2024)
work page 2024
-
[16]
Zivich, P. N.et al.HIV prevention among men who have sex with men: tenofovir alafe- namide combination preexposure prophylaxis versus placebo.The Journal of Infectious Diseases229,1123–1130 (2024)
work page 2024
-
[17]
Robins, J. M., Rotnitzky, A. & Zhao, L. P. Estimation of regression coefficients when some regressors are not always observed.Journal of the American Statistical Association 89,846–866 (1994)
work page 1994
-
[18]
Rotnitzky, A., Robins, J. M. & Scharfstein, D. O. Semiparametric regression for re- peated outcomes with nonignorable nonresponse.Journal of the American Statistical Association93,1321–1339 (1998). 33
work page 1998
-
[19]
Bang, H. & Robins, J. M. Doubly robust estimation in missing data and causal inference models.Biometrics61,962–973 (2005)
work page 2005
-
[20]
Robins, J. M. & Morgenstern, H. The foundations of confounding in epidemiology. Computers & Mathematics with Applications14,869–916 (1987)
work page 1987
-
[21]
Robins, J. M. Confidence intervals for causal parameters.Statistics in Medicine7,773– 785 (1988)
work page 1988
-
[22]
Robins, J. M. Covariance Adjustment in Randomized Experiments and Observational Studies: Comment.Statistical Science17,309–321 (2002)
work page 2002
-
[23]
Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A. & Hern´ an, M. A. Extending inferences from a randomized trial to a new target population.Statistics in Medicine39,1999–2014 (2020)
work page 1999
-
[24]
Dahabreh, I. J.et al.Study designs for extending causal inferences from a randomized trial to a target population.American Journal of Epidemiology190,1632–1642 (2021)
work page 2021
-
[25]
Robins, J. M. & Greenland, S. Causal inference without counterfactuals: comment. Journal of the American Statistical Association95,431–435 (2000)
work page 2000
-
[26]
VanderWeele, T. J. Concerning the consistency assumption in causal inference.Epi- demiology20,880–883 (2009)
work page 2009
-
[27]
Dahabreh, I. J., Robins, J. M., Haneuse, S. J.-P. A. & Hern´ an, M. A.Generalizing causal inferences from randomized trials: counterfactual and graphical identification
-
[28]
arXiv: 1906.10792[stat.ME]
work page Pith review arXiv 1906
-
[29]
Ung, L., VanderWeele, T. J. & Dahabreh, I. J. Generalizing and transporting causal inferences from randomized trials in the presence of trial engagement effects.Epidemi- ology36,500–510 (2025)
work page 2025
-
[30]
Landsberger, H. A.Hawthorne revisited: management and the worker, its critics, and developments in human relations in industry(Cornell University, Ithaca, NY, 1958). 34
work page 1958
-
[31]
Greenland, S., Pearl, J. & Robins, J. M. Causal diagrams for epidemiologic research. Epidemiology,37–48 (1999)
work page 1999
-
[32]
Pearl, J.Causality2nd (Cambridge University Press, Cambridge, UK, 2009)
work page 2009
-
[33]
Richardson, T. S. & Robins, J. M.Single world intervention graphs: a primerinSecond UAI workshop on causal structure learning, Bellevue, Washington(2013)
work page 2013
-
[34]
Bareinboim, E. & Pearl, J. Causal inference and the data-fusion problem.Proceedings of the National Academy of Sciences113,7345–7352 (2016)
work page 2016
-
[35]
Dahabreh, I. J., Robertson, S. E. & Hern´ an, M. A.Generalizing and transporting infer- ences about the effects of treatment assignment subject to non-adherence2022. arXiv: 2211.04876[stat.ME]
-
[36]
Dahabreh, I. J., Robertson, S. E., Petito, L. C., Hern´ an, M. A. & Steingrimsson, J. A. Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple randomized trials to a target population.Biometrics79,1057– 1072 (2023)
work page 2023
-
[37]
& Pearl, J.Transportability of Causal Effects: Completeness Resultsin AAAI(2012), 698–704
Bareinboim, E. & Pearl, J.Transportability of Causal Effects: Completeness Resultsin AAAI(2012), 698–704
work page 2012
-
[38]
VanderWeele, T. J. On the distinction between interaction and effect modification. Epidemiology,863–871 (2009)
work page 2009
-
[39]
Glasziou, P. P. & Irwig, L. M. An evidence based approach to individualising treatment. British Medical Journal311,1356–1359 (1995)
work page 1995
-
[40]
Dahabreh, I. J., Robertson, S. E. & Steingrimsson, J. A. Learning about treatment effects in a new target population under transportability assumptions for relative effect measures.European Journal of Epidemiology,1–9 (2024). 35
work page 2024
-
[41]
Wang, G., Levis, A., Steingrimsson, J. & Dahabreh, I.Causal inference under trans- portability assumptions for conditional relative effect measures2024. arXiv: 2402.02702 [stat.ME]
-
[42]
Athey, S. & Imbens, G. W. Identification and inference in nonlinear difference-in- differences models.Econometrica74,431–497 (2006)
work page 2006
-
[43]
Goodman-Bacon, A. Difference-in-differences with variation in treatment timing.Jour- nal of econometrics225,254–277 (2021)
work page 2021
-
[44]
Hampel, F. R. The influence curve and its role in robust estimation.Journal of the American Statistical Association69,383–393 (1974)
work page 1974
-
[45]
Bickel, P. J., Klaassen, C. A., Wellner, J. A. & Ritov, Y.Efficient and adaptive esti- mation for semiparametric models(Johns Hopkins University Press Baltimore, 1993)
work page 1993
-
[46]
Tsiatis, A.Semiparametric theory and missing data(Springer Science & Business Me- dia, 2007)
work page 2007
-
[47]
Kennedy, E. H. Semiparametric doubly robust targeted double machine learning: a review.Handbook of Statistical Methods for Precision Medicine,207–236 (2024)
work page 2024
-
[48]
Kennedy, E. H. Semiparametric theory.arXiv:1709.06418(2017)
work page Pith review arXiv 2017
-
[49]
Chernozhukov, V.et al.Double/debiased/neyman machine learning of treatment ef- fects.American Economic Review107,261–65 (2017)
work page 2017
-
[50]
Chernozhukov, V.et al.Double/debiased machine learning for treatment and structural parameters.The Econometrics Journal21,C1–C68 (2018)
work page 2018
-
[51]
Bancroft, T. A. On biases in estimation due to the use of preliminary tests of signifi- cance.The Annals of Mathematical Statistics15,190–204 (1944)
work page 1944
-
[52]
Giles, J. A. & Giles, D. E. Pre-test estimation and testing in econometrics: recent developments.Journal of Economic Surveys7,145–197 (1993). 36
work page 1993
-
[53]
J., Matthews, A., Steingrimsson, J
Dahabreh, I. J., Matthews, A., Steingrimsson, J. A., Scharfstein, D. O. & Stuart, E. A. Using trial and observational data to assess effectiveness: Trial emulation, transporta- bility, benchmarking, and joint analysis.Epidemiologic Reviews,mxac011 (2023)
work page 2023
-
[54]
Stefanski, L. A. & Boos, D. D. The calculus of M-estimation.The American Statistician 56,29–38 (2002)
work page 2002
-
[55]
Efron, B. & Tibshirani, R. J.An Introduction to the Bootstrap(Chapman & Hall/CRC, 1994)
work page 1994
-
[56]
E.et al.Double robust variance estimation with parametric working models.Biometrics81,ujaf054 (2025)
Shook-Sa, B. E.et al.Double robust variance estimation with parametric working models.Biometrics81,ujaf054 (2025)
work page 2025
-
[57]
Wu, H.et al. Why Is the Double-Robust Estimator for Causal Inference Not Doubly Robust for Variance Estimation?2025. arXiv: 2511.17907[stat.ME]
-
[58]
Robertson, S. E., Steingrimsson, J. A. & Dahabreh, I. J. Using numerical methods to design simulations: revisiting the balancing intercept.American Journal of Epidemiol- ogy191,1283–1289 (2022)
work page 2022
-
[59]
Gelfand, J. M.et al.Risk of myocardial infarction in patients with psoriasis.Journal of the American Medical Association296,1735–1741 (2006)
work page 2006
-
[60]
Griffiths, C. E., Armstrong, A. W., Gudjonsson, J. E. & Barker, J. N. Psoriasis.The Lancet397,1301–1315 (2021)
work page 2021
-
[61]
Krumholz, H. M. & Waldstreicher, J. The Yale Open Data Access (YODA) project– a mechanism for data sharing.The New England Journal of Medicine375,403–405 (2016)
work page 2016
-
[62]
Hern´ an, M. A. & Robins, J. M. Using big data to emulate a target trial when a ran- domized trial is not available.American Journal of Epidemiology183,758–764 (2016). 37
work page 2016
-
[63]
Hern´ an, M. A., Dahabreh, I. J., Dickerman, B. A. & Swanson, S. A. The target trial framework for causal inference from observational data: why and when is it helpful? Annals of Internal Medicine178,402–407 (2025)
work page 2025
-
[64]
Ung, L. & Dahabreh, I. J. Keep asking: What do I want? What do I have? What do I do? An approach for combining information to learn about a target population. European Journal of Epidemiology,1–10 (2025)
work page 2025
-
[65]
Robins, J. Marginal Structural Models, 1997 Proceedings of the American Statistical Association, section on Bayesian statistical science, pp. 1–10 (1998)
work page 1997
-
[66]
Leonardi, C. L.et al.Etanercept as monotherapy in patients with psoriasis.New Eng- land Journal of Medicine349,2014–2022 (2003)
work page 2014
-
[67]
Dahabreh, I. J., Ung, L., Hern´ an, M. A. & Chiu, Y.-H.The role of assignment in defining and identifying causal effects in randomized trials2025. arXiv: 2408.14710 [stat.ME]. 38
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.