Identification strategies for combining an experimental study with external data

Guanbo Wang; Issa J. Dahabreh; Lawson Ung; Miguel A. Hern\'an; Sebastien Haneuse

arxiv: 2406.03302 · v4 · submitted 2024-06-05 · 📊 stat.ME · math.ST· stat.TH

Identification strategies for combining an experimental study with external data

Lawson Ung , Guanbo Wang , Sebastien Haneuse , Miguel A. Hern\'an , Issa J. Dahabreh This is my paper

Pith reviewed 2026-05-24 00:29 UTC · model grok-4.3

classification 📊 stat.ME math.STstat.TH

keywords causal inferenceexternal dataidentification strategiespotential outcomesexperimental studiesobservational datageneralizabilitytransportability

0 comments

The pith

Identification strategies for combining experimental studies with external data form a separate class of causal problems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper describes basic study templates that combine information from experimental studies, including randomized and single-group trials, with external experimental or observational data. It uses the potential outcomes framework to elaborate identification strategies for potential outcome means and average treatment effects. The paper argues these strategies inherit ideas from single-source causal studies and combining-information methods but merit consideration as a separate class because they differ in scientific motivations, target population definitions, sampling, data structures, and identifiability conditions. This formalization is motivated by increasing use in clinical practice and regulatory evaluations of drugs and devices.

Core claim

The central claim is that identification strategies for analyses that combine information from experimental studies with external data inherit ideas relevant to the study of causation in single-source studies and the related literature on combining information, but merit consideration as a separate class of causal problems because they differ in terms of their scientific motivations, definitions of the target population, sampling, data structures, and identifiability conditions. In formalizing identification strategies for the analyses described, the paper provides a conceptual foundation to support the systematic use and evaluation of such efforts.

What carries the argument

Study templates that combine experimental studies with external data, with identification strategies for potential outcome means and average treatment effects elaborated via the potential outcomes framework.

Load-bearing premise

The potential outcomes framework can be directly extended to elaborate identifiability conditions for combined experimental and external data sources without additional unstated restrictions on data structures or sampling.

What would settle it

A concrete combined-data scenario in which the identifiability conditions for potential outcome means or average treatment effects cannot be stated using only the potential outcomes framework and the listed differences in motivations, populations, sampling, and structures would falsify the claim that these form a distinct class.

read the original abstract

There is increasing interest in combining information from experimental studies, including randomized and single-group trials, with information from external experimental or observational data sources. Such efforts are usually motivated by the desire to compare treatments evaluated in different studies -- for instance, by constructing external comparator groups for some index study -- or to estimate treatment effects with greater precision. Proposals to combine experimental studies with external data were made at least as early as the 1970s, but in recent years have come under increasing consideration within clinical practice and by regulatory agencies involved in drug and device evaluation, particularly with the increasing availability of trial and observational data. In this paper, we describe basic study templates that combine information from experimental studies with external data, and use the potential (counterfactual) outcomes framework to elaborate identification strategies for potential outcome means and average treatment effects. We argue that these identification strategies inherit ideas relevant to the study of causation in single-source studies and the related literature on combining information (e.g., generalizability and transportability methods), but merit consideration as a separate class of causal problems because they differ in terms of their scientific motivations, definitions of the target population, sampling, data structures, and identifiability conditions. In formalizing identification strategies for the analyses described herein, we hope to provide a conceptual foundation to support the systematic use and evaluation of such efforts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This paper organizes existing causal ideas on mixing trials with external data into templates but adds no new identification results or derivations.

read the letter

The main thing to know is that this paper describes basic study templates for combining experimental studies with external data and uses the potential outcomes framework to outline identification strategies for means and average treatment effects. It argues these merit separate treatment because of differences in motivations, target populations, sampling, data structures, and conditions, but it builds directly on prior generalizability and transportability work without producing anything absent from that literature. The motivation around regulatory interest in external comparators is clear and timely. What it does reasonably well is pull those threads together into a structured overview for readers who need a conceptual map of the issues. The soft spot is the claim of distinctness. It lists descriptive differences but does not demonstrate non-reducibility to existing methods, and there are no derivations, proofs, or examples to check how the identifiability conditions actually play out in practice. The argument stays at the level of standard causal assumptions without new technical content. This is for causal methodologists and epidemiologists working on hybrid designs in clinical research who want a reference for thinking through these problems. A reader seeking new tools or formal advances will not find them here, but someone wanting a clear framing of the practical challenges might get modest value. I would send it for peer review to see whether the community thinks the proposed classification is useful enough to stand on its own.

Referee Report

1 major / 1 minor

Summary. The manuscript describes basic study templates that combine experimental studies (randomized or single-group trials) with external experimental or observational data, motivated by constructing comparators or improving precision. Using the potential outcomes framework, it elaborates identification strategies for potential outcome means and average treatment effects. The central argument is that these strategies inherit from single-source causal inference and transportability literature but form a distinct class due to differences in scientific motivations, target population definitions, sampling, data structures, and identifiability conditions, providing a conceptual foundation for their systematic use.

Significance. If the identification strategies are correctly specified and the distinct-class claim is substantiated, the work offers a useful organizing framework for researchers and regulators combining trial and external data in drug/device evaluation. The paper's explicit grounding in the potential outcomes framework and its focus on formalizing identifiability conditions are strengths that could support clearer evaluation of such analyses.

major comments (1)

[Abstract] Abstract: The claim that the described identification strategies 'merit consideration as a separate class of causal problems' is supported only by a descriptive enumeration of differences in motivations, target populations, sampling, data structures, and identifiability conditions. The manuscript does not provide a formal argument or concrete counter-example demonstrating that these strategies are not reducible to existing generalizability or transportability methods; this weakens the load-bearing assertion of distinctness.

minor comments (1)

[Abstract] The abstract references proposals 'at least as early as the 1970s' without specific citations; adding one or two foundational references would improve historical context without altering the argument.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive feedback. We respond to the major comment below.

read point-by-point responses

Referee: [Abstract] Abstract: The claim that the described identification strategies 'merit consideration as a separate class of causal problems' is supported only by a descriptive enumeration of differences in motivations, target populations, sampling, data structures, and identifiability conditions. The manuscript does not provide a formal argument or concrete counter-example demonstrating that these strategies are not reducible to existing generalizability or transportability methods; this weakens the load-bearing assertion of distinctness.

Authors: We agree that the manuscript supports the claim of distinctness through an enumeration of differences in motivations, target populations, sampling, data structures, and identifiability conditions rather than a formal proof of non-reducibility to generalizability or transportability methods. This enumeration is the core of our argument, as these differences produce identifiability conditions and practical considerations (e.g., blending RCT internal validity with external comparator data for a shared target) that are not the primary focus of existing transportability frameworks. We do not claim or demonstrate mathematical irreducibility in all cases, as the paper's aim is to provide a conceptual organizing framework rather than a formal uniqueness proof. To strengthen the presentation, we will revise the abstract and add a brief illustrative example in the introduction clarifying one such distinction. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper is a conceptual discussion that elaborates identification strategies for combining experimental and external data using the standard potential outcomes framework. It draws on existing literature for generalizability and transportability but makes no derivations, equations, or parameter fits that reduce to self-referential inputs. The claim that these strategies form a distinct class rests on enumerated differences in motivations, populations, sampling, and conditions rather than any self-definition, fitted prediction, or load-bearing self-citation chain. No steps meet the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the potential outcomes framework and standard identifiability conditions from causal inference; no new free parameters, invented entities, or ad-hoc axioms are introduced in the abstract.

axioms (2)

domain assumption Potential outcomes framework applies to combined experimental and external data sources
Invoked throughout the abstract as the basis for elaborating identification strategies.
domain assumption Differences in motivations, target populations, sampling, data structures, and identifiability conditions justify treating combined studies as a separate class
Stated as the reason these strategies merit separate consideration.

pith-pipeline@v0.9.0 · 5785 in / 1263 out tokens · 17661 ms · 2026-05-24T00:29:14.733853+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Constructing external comparator groups via transportability in mean or in effect measure
stat.ME 2026-04 unverdicted novelty 6.0

Proposes semiparametric efficient augmented weighting estimators for causal effects under transportability of means or effect measures when appending external comparators to an index trial.

Reference graph

Works this paper leans on

86 extracted references · 86 canonical work pages · cited by 1 Pith paper · 1 internal anchor

[1]

Gehan, E. A. & Freireich, E. J. Non-randomized controls in cancer clinical trials. New England Journal of Medicine 290, 198–203 (1974)

work page 1974
[2]

Pocock, S. J. The combination of randomized and historica l controls in clinical trials. Journal of Chronic Diseases 29, 175–88 (1976)

work page 1976
[3]

Dahabreh, I. J. Combining information to answer epidemio logical questions about a target population. American Journal of Epidemiology, kwad014 (2024)

work page 2024
[4]

Pocock, S. J. Allocation of patients to treatment in clini cal trials. Biometrics 35, 183–97 (1979)

work page 1979
[5]

Fleming, T. R. Historical Controls, Data Banks, and Rando mized Trials in Clinical. Cancer Treatment Reports 66, 1101–1105 (1982)

work page 1982
[6]

Sacks, H., Chalmers, T. C. & Smith, H. Randomized versus hi storical controls for clinical trials. The American Journal of Medicine 72, 233–240 (1982)

work page 1982
[7]

P., Selwyn, M

Dempster, A. P., Selwyn, M. R. & Weeks, B. J. Combining hist orical and randomized controls for assessing trends in proportions. Journal of the American Statistical Association 78, 221– 227 (1983)

work page 1983
[8]

Krisam, J., Weber, D., Schlenk, R. F. & Kieser, M. Enhancing single-arm phase II trials by inclusion of matched control patients 2020. arXiv: 2007.15935 [stat.ME]

work page arXiv 2020
[9]

Single-arm Trials with Historical Controls: S tudy Designs to Avoid Time-related Biases

Suissa, S. Single-arm Trials with Historical Controls: S tudy Designs to Avoid Time-related Biases. Epidemiology 32, 94–100 (2021)

work page 2021
[10]

Ghadessi, M. et al. A roadmap to using historical controls in clinical trials–b y Drug Informa- tion Association Adaptive Design Scientiﬁc Working Group (DIA-ADSWG). Orphanet Journal of Rare Diseases 15, 1–19 (2020)

work page 2020
[11]

Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products Web Page

US Food and Drug Administration. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products Web Page. 2023. https://www.fda.gov/regulatory-information/searc 24

work page 2023
[12]

Burcu, M. et al. Real-world evidence to support regulatory decision-makin g for medicines: Considerations for external control arms. Pharmacoepidemiology and Drug Safety 29, 1228– 1235 (2020)

work page 2020
[13]

Thorlund, K., Dron, L., Park, J. J. H. & Mills, E. J. Synthe tic and External Controls in Clinical Trials - A Primer for Researchers. Clinical Epidemiology 12, 457–467 (2020)

work page 2020
[14]

Wang, G. et al. Evaluating hybrid controls methodology in early-phase onc ology trials: A simulation study based on the MORPHEUS-UC trial. Pharmaceutical Statistics 23, 31–45 (2024)

work page 2024
[15]

Segal, B. D. & Tan, W. K. A note on the amount of information borrowed from external da ta in hybrid controlled trials with time-to-event outcomes 2020. arXiv: 2010.00433 [stat.ME]

work page arXiv 2020
[16]

Ventz, S. et al. The design and evaluation of hybrid controlled trials that l everage external data and randomization. Nature Communications 13, 5783 (2022)

work page 2022
[17]

Rippin, G. et al. A Review of Causal Inference for External Comparator Arm Stu dies. Drug Safety, 1–23 (2022)

work page 2022
[18]

Tan, K. et al. Emulating Control Arms for Cancer Clinical Trials Using Ext ernal Cohorts Created From Electronic Health Record-Derived Real-World Data. Clinical Pharmacology & Therapeutics 111, 168–178 (2022)

work page 2022
[19]

& Pearl, J

Bareinboim, E. & Pearl, J. Causal inference and the data- fusion problem. Proceedings of the National Academy of Sciences 113, 7345–7352 (2016)

work page 2016
[20]

Shook-Sa, B. E. et al. Fusing Trial Data for Treatment Comparisons: Single ve rsus Multi-Span Bridging 2023. arXiv: 2305.00845 [stat.AP]

work page arXiv 2023
[21]

Breskin, A. et al. Fusion designs and estimators for treatment eﬀects. Statistics in Medicine 40, 3124–3137 (2021)

work page 2021
[22]

Carrigan, G. et al. Using Electronic Health Records to Derive Control Arms for E arly Phase Single-Arm Lung Cancer Trials: Proof-of-Concept in Random ized Controlled Trials. Clinical Pharmacology & Therapeutics 107, 369–377 (2020). 25

work page 2020
[23]

Lim, J. et al. Minimizing Patient Burden Through the Use of Historical Sub ject-Level Data in Innovative Conﬁrmatory Clinical Trials: Review of Methods and Opportunities. Therapeutic Innovation & Regulatory Science 52, 546–559 (2018)

work page 2018
[24]

Hall, K. T. et al. Historical Controls in Randomized Clinical Trials: Opport unities and Chal- lenges. Clinical Pharmacology & Therapeutics 109, 343–351 (2021)

work page 2021
[25]

J., Baio, G., Berlin, J

Hatswell, A. J., Baio, G., Berlin, J. A., Irs, A. & Freeman tle, N. Regulatory approval of pharmaceuticals without a randomised controlled study: an alysis of EMA and FDA approvals 1999–2014. BMJ Open 6, e011666 (2016)

work page 1999
[26]

Curtis, L. H. et al. Regulatory and HTA Considerations for Development of Real- World Data Derived External Controls. Clinical Pharmacology & Therapeutics 114, 303–315 (2023)

work page 2023
[27]

Guideline on clinical trials in small populations Report

The European Medicines Agency. Guideline on clinical trials in small populations Report. (2006). https://www.ema.europa.eu/en/documents/scientiﬁc-guideline/guideline-clinical-trials-small-populations

work page 2006
[28]

M., Grimson, F., Layton, D., Pocock, S

Gray, C. M., Grimson, F., Layton, D., Pocock, S. & Kim, J. A Framework for Methodological Choice and Evidence Assessment for Studies Using External C omparators from Real-World Data. Drug Safety 43, 623–633 (2020)

work page 2020
[29]

Rahman, R. et al. Leveraging external data in the design and analysis of clini cal trials in neuro-oncology. Lancet Oncology 22, e456–e465 (2021)

work page 2021
[30]

& Zhou, X.-H

Li, X., Miao, W., Lu, F. & Zhou, X.-H. Improving eﬃciency o f inference in clinical trials with external control data. Biometrics 79, 394–403 (2021)

work page 2021
[31]

Valancius, M. et al. A Causal Inference Framework for Leveraging External Con trols in Hybrid Trials 2023. arXiv: 2305.08969 [stat.ME]

work page arXiv 2023
[32]

Dahabreh, I. J. et al. Study designs for extending causal inferences from a random ized trial to a target population. American Journal of Epidemiology 190, 1632–1642 (2021)

work page 2021
[33]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A. & Hern´ an, M. A. Ex- tending inferences from a randomized trial to a new target po pulation. Statistics in Medicine 39, 1999–2014 (2020)

work page 1999
[34]

& Dahabreh, I

Chiu, Y.-H. & Dahabreh, I. J. Selection on treatment in the target population of generali zabil- lity and transportability analyses 2022. arXiv: 2209.08758 [stat.ME]. 26

work page arXiv 2022
[35]

Robins, J. M. Conﬁdence intervals for causal parameters . Statistics in Medicine 7, 773–785 (1988)

work page 1988
[36]

J., Klaassen, C

Bickel, P. J., Klaassen, C. A., Wellner, J. A. & Ritov, Y. Eﬃcient and adaptive estimation for semiparametric models (Johns Hopkins University Press Baltimore, 1993)

work page 1993
[37]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E. & Hern´ an, M. A. Generalizing and transporting inferences about the eﬀects of treatment assignment subject to non-adh erence 2022. arXiv: 2211.04876 [stat.ME]

work page arXiv 2022
[38]

A., Robins, J

Hern´ an, M. A., Robins, J. M., et al. Per-protocol analyses of pragmatic trials. N Engl J Med 377, 1391–1398 (2017)

work page 2017
[39]

Rubin, D. B. Estimating causal eﬀects of treatments in ran domized and nonrandomized stud- ies. Journal of Educational Psychology 66, 688–701 (1974)

work page 1974
[40]

Robins, J. M. A new approach to causal inference in mortal ity studies with a sustained ex- posure period – application to control of the healthy worker survivor eﬀect. Mathematical Modelling 7, 1393–1512 (1986)

work page 1986
[41]

On the application of probability the ory to agricultural experiments

Splawa-Neyman, J. On the application of probability the ory to agricultural experiments. Es- say on principles. Section 9. [Translated from Splawa-Neym an, J (1923) in Roczniki Nauk Rolniczych Tom X, 1–51]. Trans. by Dabrowska, D. M. & Speed, T . P. Statistical Science 5, 465–472 (1990)

work page 1923
[42]

Robins, J. M. & Greenland, S. Causal inference without co unterfactuals: comment. Journal of the American Statistical Association 95, 431–435 (2000)

work page 2000
[43]

K., Lesko, C

Westreich, D., Edwards, J. K., Lesko, C. R., Stuart, E. & C ole, S. R. Transportability of trial results using inverse odds of sampling weights. American Journal of Epidemiology 186, 1010–1014 (2017)

work page 2017
[44]

Rudolph, K. E. & van der Laan, M. J. Robust estimation of en couragement design intervention eﬀects transported across sites. Journal of the Royal Statistical Society. Series B (Statist ical Methodology) 79, 1509–1525 (2017)

work page 2017
[45]

Dahabreh, I. J. & Hern´ an, M. A. Extending inferences fro m a randomized trial to a target population. European Journal of Epidemiology 34, 719–722 (2019). 27

work page 2019
[46]

Landsberger, H. A. Hawthorne Revisited: Management and the Worker, Its Critics, and De- velopments in Human Relations in Industry. (Cornell University, Ithaca, NY, 1958)

work page 1958
[47]

J., Robins, J

Dahabreh, I. J., Robins, J. M. & Hern´ an, M. A. Benchmarki ng Observational Methods by Comparing Randomized Trials and Their Emulations. Epidemiology (Cambridge, Mass.) 31, 614–619 (2020)

work page 2020
[48]

J., Petito, L

Dahabreh, I. J., Petito, L. C., Robertson, S. E., Hern´ an , M. A. & Steingrimsson, J. A. Toward causally interpretable meta-analysis: Transporting infe rences from multiple randomized trials to a new target population. Epidemiology (Cambridge, Mass.) 31, 334–344 (2020)

work page 2020
[49]

Generalizing causal inferences from randomized trials: counterfactual and graphical identification

Dahabreh, I. J., Robins, J. M., Haneuse, S. J.-P. & Hern´ a n, M. A. Generalizing causal in- ferences from randomized trials: counterfactual and graph ical identiﬁcation. arXiv preprint arXiv:1906.10792 (2019)

work page internal anchor Pith review Pith/arXiv arXiv 1906
[50]

Causality 2nd Edition (Cambridge University Press, Cambridge, UK, 20 09)

Pearl, J. Causality 2nd Edition (Cambridge University Press, Cambridge, UK, 20 09)

work page
[51]

Richardson, T. S. & Robins, J. M. Single world intervention graphs (SWIGs): A uniﬁcation of the counterfactual and graphical approaches to causality tech. rep. 128. https://www.csss.washington.edu/researc (Center for Statistics and the Social Sciences, University of Washington, 2013)

work page 2013
[52]

Randomization analysis of ex perimental data: the Fisher random- ization test

Rubin, D. B. Discussion of “Randomization analysis of ex perimental data: the Fisher random- ization test”. Journal of the American Statistical Association 75, 591–593 (1980)

work page 1980
[53]

Rubin, D. B. Statistics and causal inference: Comment: W hich ifs have causal answers. Journal of the American Statistical Association 81, 961–962 (1986)

work page 1986
[54]

Hern´ an, M. A. & VanderWeele, T. J. Compound treatments a nd transportability of causal inference. Epidemiology (Cambridge, Mass.) 22, 368 (2011)

work page 2011
[55]

Greenland, S., Robins, J. M. & Pearl, J. Confounding and c ollapsibility in causal inference. Statistical Science, 29–46 (1999)

work page 1999
[56]

VanderWeele, T. J. Concerning the consistency assumpti on in causal inference. Epidemiology (Cambridge, Mass.) 20, 880–883 (2009)

work page 2009
[57]

Halloran, M. E. & Struchiner, C. J. Causal inference in in fectious diseases. Epidemiology (Cambridge, Mass.), 142–151 (1995). 28

work page 1995
[58]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E., Tchetgen Tchetgen, E. J., Stuart, E. A. & Hern´ an, M. A. Generalizing causal inferences from individuals in random ized trials to all trial-eligible indi- viduals. Biometrics 75, 685–694 (2018)

work page 2018
[59]

Verma, T. S. & Pearl, J. in Probabilistic and Causal Inference: The Works of Judea Pearl 221–236 (2022)

work page 2022
[60]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E. & Steingrimsson, J. A. L earning about treatment eﬀects in a new target population under transportability assumptions for relative eﬀect measures. arXiv preprint arXiv:2202.11622 (2022)

work page arXiv 2022
[61]

& Dahabreh, I

Wang, G., Levis, A., Steingrimsson, J. & Dahabreh, I. Cau sal inference under transportability assumptions for conditional relative eﬀect measures. arXiv preprint arXiv:2402.02702 (2024)

work page arXiv 2024
[62]

& Imbens, G

Athey, S. & Imbens, G. W. Identiﬁcation and inference in n onlinear diﬀerence-in-diﬀerences models. Econometrica 74, 431–497 (2006)

work page 2006
[63]

B., Colicino, E., Schwartz, J

Sofer, T., Richardson, D. B., Colicino, E., Schwartz, J. & Tchetgen, E. J. T. On negative outcome control of unobserved confounding as a generalizat ion of diﬀerence-in-diﬀerences. Statistical science: a review journal of the Institute of Mat hematical Statistics 31, 348 (2016)

work page 2016
[64]

Hern´ an, M. A. & Robins, J. M. Causal Inference: What If chap. 10 (Chapman & Hall/CRC, Boca Raton, FL, 2024)

work page 2024
[65]

Stefanski, L. A. & Boos, D. D. The calculus of M-estimatio n. The American Statistician 56, 29–38 (2002)

work page 2002
[66]

& Tibshirani, R

Efron, B. & Tibshirani, R. J. An introduction to the bootstrap Monographs on Statistics a nd Applied Probability 57 (Chapman & Hall/CRC, Boca Raton, Florida, USA, 1993)

work page 1993
[67]

Interval estimation by simulation as an al ternative to and extension of conﬁdence intervals

Greenland, S. Interval estimation by simulation as an al ternative to and extension of conﬁdence intervals. International Journal of Epidemiology 33, 1389–1397 (2004)

work page 2004
[68]

& Geng, Z

Wu, P., Luo, S. & Geng, Z. On the Comparative Analysis of Average Treatment Eﬀects Esti - mation via Data Combination 2023. arXiv: 2311.00528 [stat.ME]

work page arXiv 2023
[69]

& van der Laan, L

Van der Laan, M., Qiu, S. & van der Laan, L. Adaptive-TMLE f or the Average Treatment Ef- fect based on Randomized Controlled Trial Augmented with Re al-World Data. arXiv preprint arXiv:2405.07186 (2024). 29

work page arXiv 2024
[70]

Amiri-Kordestani, L. et al. A Food and Drug Administration analysis of survival outcome s comparing the Adjuvant Paclitaxel and Trastuzumab trial wi th an external control from his- torical clinical trials. Annals of Oncology 31, 1704–1708 (2020)

work page 2020
[71]

& Korn, E

Freidlin, B. & Korn, E. L. Augmenting randomized clinica l trial data with historical control data: Precision medicine applications. JNCI: Journal of the National Cancer Institute (2022)

work page 2022
[72]

Signorovitch, J. E. et al. Comparative eﬀectiveness without head-to-head trials. Pharmacoeco- nomics 28, 935–945 (2010)

work page 2010
[73]

Seeger, J. D. et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiology and Drug Safety 29, 1382–1392 (2020)

work page 2020
[74]

Mishra-Kalyani, P. et al. External control arms in oncology: current use and future di rections. Annals of Oncology 33, 376–383 (2022)

work page 2022
[75]

Signorovitch, J. et al. Matching-adjusted indirect comparison (MAIC) results con ﬁrmed by head-to-head trials: a case study in psoriasis. Journal of Dermatological Treatment 34, 2169574 (2023)

work page 2023
[76]

& Peto, R

Doll, R. & Peto, R. Randomised controlled trials and retr ospective controls. British Medical Journal 280, 44 (1980)

work page 1980
[77]

Diehl, L. F. & Perry, D. A comparison of randomized concur rent control groups with matched historical control groups: are historical controls valid? Journal of Clinical Oncology 4, 1114– 1120 (1986)

work page 1986
[78]

J., Proskorovsky, I

Ishak, K. J., Proskorovsky, I. & Benedict, A. Simulation and matching-based approaches for indirect comparison of treatments. Pharmacoeconomics 33, 537–549 (2015)

work page 2015
[79]

& Sekhon, J

Hartman, E., Grieve, R., Ramsahai, R. & Sekhon, J. S. From SATE to PATT: combining experimental with observational studies to estimate popul ation treatment eﬀects. Journal of the Royal Statistical Society Series A (Statistics in Socie ty) 10, 1111 (2013)

work page 2013
[80]

O., Brooks, M

Lu, Y., Scharfstein, D. O., Brooks, M. M., Quach, K. & Kenn edy, E. H. Causal Inference for Comprehensive Cohort Studies. arXiv preprint arXiv:1910.03531 (2019). 30

work page arXiv 1910

Showing first 80 references.

[1] [1]

Gehan, E. A. & Freireich, E. J. Non-randomized controls in cancer clinical trials. New England Journal of Medicine 290, 198–203 (1974)

work page 1974

[2] [2]

Pocock, S. J. The combination of randomized and historica l controls in clinical trials. Journal of Chronic Diseases 29, 175–88 (1976)

work page 1976

[3] [3]

Dahabreh, I. J. Combining information to answer epidemio logical questions about a target population. American Journal of Epidemiology, kwad014 (2024)

work page 2024

[4] [4]

Pocock, S. J. Allocation of patients to treatment in clini cal trials. Biometrics 35, 183–97 (1979)

work page 1979

[5] [5]

Fleming, T. R. Historical Controls, Data Banks, and Rando mized Trials in Clinical. Cancer Treatment Reports 66, 1101–1105 (1982)

work page 1982

[6] [6]

Sacks, H., Chalmers, T. C. & Smith, H. Randomized versus hi storical controls for clinical trials. The American Journal of Medicine 72, 233–240 (1982)

work page 1982

[7] [7]

P., Selwyn, M

Dempster, A. P., Selwyn, M. R. & Weeks, B. J. Combining hist orical and randomized controls for assessing trends in proportions. Journal of the American Statistical Association 78, 221– 227 (1983)

work page 1983

[8] [8]

Krisam, J., Weber, D., Schlenk, R. F. & Kieser, M. Enhancing single-arm phase II trials by inclusion of matched control patients 2020. arXiv: 2007.15935 [stat.ME]

work page arXiv 2020

[9] [9]

Single-arm Trials with Historical Controls: S tudy Designs to Avoid Time-related Biases

Suissa, S. Single-arm Trials with Historical Controls: S tudy Designs to Avoid Time-related Biases. Epidemiology 32, 94–100 (2021)

work page 2021

[10] [10]

Ghadessi, M. et al. A roadmap to using historical controls in clinical trials–b y Drug Informa- tion Association Adaptive Design Scientiﬁc Working Group (DIA-ADSWG). Orphanet Journal of Rare Diseases 15, 1–19 (2020)

work page 2020

[11] [11]

Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products Web Page

US Food and Drug Administration. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products Web Page. 2023. https://www.fda.gov/regulatory-information/searc 24

work page 2023

[12] [12]

Burcu, M. et al. Real-world evidence to support regulatory decision-makin g for medicines: Considerations for external control arms. Pharmacoepidemiology and Drug Safety 29, 1228– 1235 (2020)

work page 2020

[13] [13]

Thorlund, K., Dron, L., Park, J. J. H. & Mills, E. J. Synthe tic and External Controls in Clinical Trials - A Primer for Researchers. Clinical Epidemiology 12, 457–467 (2020)

work page 2020

[14] [14]

Wang, G. et al. Evaluating hybrid controls methodology in early-phase onc ology trials: A simulation study based on the MORPHEUS-UC trial. Pharmaceutical Statistics 23, 31–45 (2024)

work page 2024

[15] [15]

Segal, B. D. & Tan, W. K. A note on the amount of information borrowed from external da ta in hybrid controlled trials with time-to-event outcomes 2020. arXiv: 2010.00433 [stat.ME]

work page arXiv 2020

[16] [16]

Ventz, S. et al. The design and evaluation of hybrid controlled trials that l everage external data and randomization. Nature Communications 13, 5783 (2022)

work page 2022

[17] [17]

Rippin, G. et al. A Review of Causal Inference for External Comparator Arm Stu dies. Drug Safety, 1–23 (2022)

work page 2022

[18] [18]

Tan, K. et al. Emulating Control Arms for Cancer Clinical Trials Using Ext ernal Cohorts Created From Electronic Health Record-Derived Real-World Data. Clinical Pharmacology & Therapeutics 111, 168–178 (2022)

work page 2022

[19] [19]

& Pearl, J

Bareinboim, E. & Pearl, J. Causal inference and the data- fusion problem. Proceedings of the National Academy of Sciences 113, 7345–7352 (2016)

work page 2016

[20] [20]

Shook-Sa, B. E. et al. Fusing Trial Data for Treatment Comparisons: Single ve rsus Multi-Span Bridging 2023. arXiv: 2305.00845 [stat.AP]

work page arXiv 2023

[21] [21]

Breskin, A. et al. Fusion designs and estimators for treatment eﬀects. Statistics in Medicine 40, 3124–3137 (2021)

work page 2021

[22] [22]

Carrigan, G. et al. Using Electronic Health Records to Derive Control Arms for E arly Phase Single-Arm Lung Cancer Trials: Proof-of-Concept in Random ized Controlled Trials. Clinical Pharmacology & Therapeutics 107, 369–377 (2020). 25

work page 2020

[23] [23]

Lim, J. et al. Minimizing Patient Burden Through the Use of Historical Sub ject-Level Data in Innovative Conﬁrmatory Clinical Trials: Review of Methods and Opportunities. Therapeutic Innovation & Regulatory Science 52, 546–559 (2018)

work page 2018

[24] [24]

Hall, K. T. et al. Historical Controls in Randomized Clinical Trials: Opport unities and Chal- lenges. Clinical Pharmacology & Therapeutics 109, 343–351 (2021)

work page 2021

[25] [25]

J., Baio, G., Berlin, J

Hatswell, A. J., Baio, G., Berlin, J. A., Irs, A. & Freeman tle, N. Regulatory approval of pharmaceuticals without a randomised controlled study: an alysis of EMA and FDA approvals 1999–2014. BMJ Open 6, e011666 (2016)

work page 1999

[26] [26]

Curtis, L. H. et al. Regulatory and HTA Considerations for Development of Real- World Data Derived External Controls. Clinical Pharmacology & Therapeutics 114, 303–315 (2023)

work page 2023

[27] [27]

Guideline on clinical trials in small populations Report

The European Medicines Agency. Guideline on clinical trials in small populations Report. (2006). https://www.ema.europa.eu/en/documents/scientiﬁc-guideline/guideline-clinical-trials-small-populations

work page 2006

[28] [28]

M., Grimson, F., Layton, D., Pocock, S

Gray, C. M., Grimson, F., Layton, D., Pocock, S. & Kim, J. A Framework for Methodological Choice and Evidence Assessment for Studies Using External C omparators from Real-World Data. Drug Safety 43, 623–633 (2020)

work page 2020

[29] [29]

Rahman, R. et al. Leveraging external data in the design and analysis of clini cal trials in neuro-oncology. Lancet Oncology 22, e456–e465 (2021)

work page 2021

[30] [30]

& Zhou, X.-H

Li, X., Miao, W., Lu, F. & Zhou, X.-H. Improving eﬃciency o f inference in clinical trials with external control data. Biometrics 79, 394–403 (2021)

work page 2021

[31] [31]

Valancius, M. et al. A Causal Inference Framework for Leveraging External Con trols in Hybrid Trials 2023. arXiv: 2305.08969 [stat.ME]

work page arXiv 2023

[32] [32]

Dahabreh, I. J. et al. Study designs for extending causal inferences from a random ized trial to a target population. American Journal of Epidemiology 190, 1632–1642 (2021)

work page 2021

[33] [33]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A. & Hern´ an, M. A. Ex- tending inferences from a randomized trial to a new target po pulation. Statistics in Medicine 39, 1999–2014 (2020)

work page 1999

[34] [34]

& Dahabreh, I

Chiu, Y.-H. & Dahabreh, I. J. Selection on treatment in the target population of generali zabil- lity and transportability analyses 2022. arXiv: 2209.08758 [stat.ME]. 26

work page arXiv 2022

[35] [35]

Robins, J. M. Conﬁdence intervals for causal parameters . Statistics in Medicine 7, 773–785 (1988)

work page 1988

[36] [36]

J., Klaassen, C

Bickel, P. J., Klaassen, C. A., Wellner, J. A. & Ritov, Y. Eﬃcient and adaptive estimation for semiparametric models (Johns Hopkins University Press Baltimore, 1993)

work page 1993

[37] [37]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E. & Hern´ an, M. A. Generalizing and transporting inferences about the eﬀects of treatment assignment subject to non-adh erence 2022. arXiv: 2211.04876 [stat.ME]

work page arXiv 2022

[38] [38]

A., Robins, J

Hern´ an, M. A., Robins, J. M., et al. Per-protocol analyses of pragmatic trials. N Engl J Med 377, 1391–1398 (2017)

work page 2017

[39] [39]

Rubin, D. B. Estimating causal eﬀects of treatments in ran domized and nonrandomized stud- ies. Journal of Educational Psychology 66, 688–701 (1974)

work page 1974

[40] [40]

Robins, J. M. A new approach to causal inference in mortal ity studies with a sustained ex- posure period – application to control of the healthy worker survivor eﬀect. Mathematical Modelling 7, 1393–1512 (1986)

work page 1986

[41] [41]

On the application of probability the ory to agricultural experiments

Splawa-Neyman, J. On the application of probability the ory to agricultural experiments. Es- say on principles. Section 9. [Translated from Splawa-Neym an, J (1923) in Roczniki Nauk Rolniczych Tom X, 1–51]. Trans. by Dabrowska, D. M. & Speed, T . P. Statistical Science 5, 465–472 (1990)

work page 1923

[42] [42]

Robins, J. M. & Greenland, S. Causal inference without co unterfactuals: comment. Journal of the American Statistical Association 95, 431–435 (2000)

work page 2000

[43] [43]

K., Lesko, C

Westreich, D., Edwards, J. K., Lesko, C. R., Stuart, E. & C ole, S. R. Transportability of trial results using inverse odds of sampling weights. American Journal of Epidemiology 186, 1010–1014 (2017)

work page 2017

[44] [44]

Rudolph, K. E. & van der Laan, M. J. Robust estimation of en couragement design intervention eﬀects transported across sites. Journal of the Royal Statistical Society. Series B (Statist ical Methodology) 79, 1509–1525 (2017)

work page 2017

[45] [45]

Dahabreh, I. J. & Hern´ an, M. A. Extending inferences fro m a randomized trial to a target population. European Journal of Epidemiology 34, 719–722 (2019). 27

work page 2019

[46] [46]

Landsberger, H. A. Hawthorne Revisited: Management and the Worker, Its Critics, and De- velopments in Human Relations in Industry. (Cornell University, Ithaca, NY, 1958)

work page 1958

[47] [47]

J., Robins, J

Dahabreh, I. J., Robins, J. M. & Hern´ an, M. A. Benchmarki ng Observational Methods by Comparing Randomized Trials and Their Emulations. Epidemiology (Cambridge, Mass.) 31, 614–619 (2020)

work page 2020

[48] [48]

J., Petito, L

Dahabreh, I. J., Petito, L. C., Robertson, S. E., Hern´ an , M. A. & Steingrimsson, J. A. Toward causally interpretable meta-analysis: Transporting infe rences from multiple randomized trials to a new target population. Epidemiology (Cambridge, Mass.) 31, 334–344 (2020)

work page 2020

[49] [49]

Generalizing causal inferences from randomized trials: counterfactual and graphical identification

Dahabreh, I. J., Robins, J. M., Haneuse, S. J.-P. & Hern´ a n, M. A. Generalizing causal in- ferences from randomized trials: counterfactual and graph ical identiﬁcation. arXiv preprint arXiv:1906.10792 (2019)

work page internal anchor Pith review Pith/arXiv arXiv 1906

[50] [50]

Causality 2nd Edition (Cambridge University Press, Cambridge, UK, 20 09)

Pearl, J. Causality 2nd Edition (Cambridge University Press, Cambridge, UK, 20 09)

work page

[51] [51]

Richardson, T. S. & Robins, J. M. Single world intervention graphs (SWIGs): A uniﬁcation of the counterfactual and graphical approaches to causality tech. rep. 128. https://www.csss.washington.edu/researc (Center for Statistics and the Social Sciences, University of Washington, 2013)

work page 2013

[52] [52]

Randomization analysis of ex perimental data: the Fisher random- ization test

Rubin, D. B. Discussion of “Randomization analysis of ex perimental data: the Fisher random- ization test”. Journal of the American Statistical Association 75, 591–593 (1980)

work page 1980

[53] [53]

Rubin, D. B. Statistics and causal inference: Comment: W hich ifs have causal answers. Journal of the American Statistical Association 81, 961–962 (1986)

work page 1986

[54] [54]

Hern´ an, M. A. & VanderWeele, T. J. Compound treatments a nd transportability of causal inference. Epidemiology (Cambridge, Mass.) 22, 368 (2011)

work page 2011

[55] [55]

Greenland, S., Robins, J. M. & Pearl, J. Confounding and c ollapsibility in causal inference. Statistical Science, 29–46 (1999)

work page 1999

[56] [56]

VanderWeele, T. J. Concerning the consistency assumpti on in causal inference. Epidemiology (Cambridge, Mass.) 20, 880–883 (2009)

work page 2009

[57] [57]

Halloran, M. E. & Struchiner, C. J. Causal inference in in fectious diseases. Epidemiology (Cambridge, Mass.), 142–151 (1995). 28

work page 1995

[58] [58]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E., Tchetgen Tchetgen, E. J., Stuart, E. A. & Hern´ an, M. A. Generalizing causal inferences from individuals in random ized trials to all trial-eligible indi- viduals. Biometrics 75, 685–694 (2018)

work page 2018

[59] [59]

Verma, T. S. & Pearl, J. in Probabilistic and Causal Inference: The Works of Judea Pearl 221–236 (2022)

work page 2022

[60] [60]

J., Robertson, S

Dahabreh, I. J., Robertson, S. E. & Steingrimsson, J. A. L earning about treatment eﬀects in a new target population under transportability assumptions for relative eﬀect measures. arXiv preprint arXiv:2202.11622 (2022)

work page arXiv 2022

[61] [61]

& Dahabreh, I

Wang, G., Levis, A., Steingrimsson, J. & Dahabreh, I. Cau sal inference under transportability assumptions for conditional relative eﬀect measures. arXiv preprint arXiv:2402.02702 (2024)

work page arXiv 2024

[62] [62]

& Imbens, G

Athey, S. & Imbens, G. W. Identiﬁcation and inference in n onlinear diﬀerence-in-diﬀerences models. Econometrica 74, 431–497 (2006)

work page 2006

[63] [63]

B., Colicino, E., Schwartz, J

Sofer, T., Richardson, D. B., Colicino, E., Schwartz, J. & Tchetgen, E. J. T. On negative outcome control of unobserved confounding as a generalizat ion of diﬀerence-in-diﬀerences. Statistical science: a review journal of the Institute of Mat hematical Statistics 31, 348 (2016)

work page 2016

[64] [64]

Hern´ an, M. A. & Robins, J. M. Causal Inference: What If chap. 10 (Chapman & Hall/CRC, Boca Raton, FL, 2024)

work page 2024

[65] [65]

Stefanski, L. A. & Boos, D. D. The calculus of M-estimatio n. The American Statistician 56, 29–38 (2002)

work page 2002

[66] [66]

& Tibshirani, R

Efron, B. & Tibshirani, R. J. An introduction to the bootstrap Monographs on Statistics a nd Applied Probability 57 (Chapman & Hall/CRC, Boca Raton, Florida, USA, 1993)

work page 1993

[67] [67]

Interval estimation by simulation as an al ternative to and extension of conﬁdence intervals

Greenland, S. Interval estimation by simulation as an al ternative to and extension of conﬁdence intervals. International Journal of Epidemiology 33, 1389–1397 (2004)

work page 2004

[68] [68]

& Geng, Z

Wu, P., Luo, S. & Geng, Z. On the Comparative Analysis of Average Treatment Eﬀects Esti - mation via Data Combination 2023. arXiv: 2311.00528 [stat.ME]

work page arXiv 2023

[69] [69]

& van der Laan, L

Van der Laan, M., Qiu, S. & van der Laan, L. Adaptive-TMLE f or the Average Treatment Ef- fect based on Randomized Controlled Trial Augmented with Re al-World Data. arXiv preprint arXiv:2405.07186 (2024). 29

work page arXiv 2024

[70] [70]

Amiri-Kordestani, L. et al. A Food and Drug Administration analysis of survival outcome s comparing the Adjuvant Paclitaxel and Trastuzumab trial wi th an external control from his- torical clinical trials. Annals of Oncology 31, 1704–1708 (2020)

work page 2020

[71] [71]

& Korn, E

Freidlin, B. & Korn, E. L. Augmenting randomized clinica l trial data with historical control data: Precision medicine applications. JNCI: Journal of the National Cancer Institute (2022)

work page 2022

[72] [72]

Signorovitch, J. E. et al. Comparative eﬀectiveness without head-to-head trials. Pharmacoeco- nomics 28, 935–945 (2010)

work page 2010

[73] [73]

Seeger, J. D. et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiology and Drug Safety 29, 1382–1392 (2020)

work page 2020

[74] [74]

Mishra-Kalyani, P. et al. External control arms in oncology: current use and future di rections. Annals of Oncology 33, 376–383 (2022)

work page 2022

[75] [75]

Signorovitch, J. et al. Matching-adjusted indirect comparison (MAIC) results con ﬁrmed by head-to-head trials: a case study in psoriasis. Journal of Dermatological Treatment 34, 2169574 (2023)

work page 2023

[76] [76]

& Peto, R

Doll, R. & Peto, R. Randomised controlled trials and retr ospective controls. British Medical Journal 280, 44 (1980)

work page 1980

[77] [77]

Diehl, L. F. & Perry, D. A comparison of randomized concur rent control groups with matched historical control groups: are historical controls valid? Journal of Clinical Oncology 4, 1114– 1120 (1986)

work page 1986

[78] [78]

J., Proskorovsky, I

Ishak, K. J., Proskorovsky, I. & Benedict, A. Simulation and matching-based approaches for indirect comparison of treatments. Pharmacoeconomics 33, 537–549 (2015)

work page 2015

[79] [79]

& Sekhon, J

Hartman, E., Grieve, R., Ramsahai, R. & Sekhon, J. S. From SATE to PATT: combining experimental with observational studies to estimate popul ation treatment eﬀects. Journal of the Royal Statistical Society Series A (Statistics in Socie ty) 10, 1111 (2013)

work page 2013

[80] [80]

O., Brooks, M

Lu, Y., Scharfstein, D. O., Brooks, M. M., Quach, K. & Kenn edy, E. H. Causal Inference for Comprehensive Cohort Studies. arXiv preprint arXiv:1910.03531 (2019). 30

work page arXiv 1910