Pith. sign in

REVIEW 2 major objections 7 minor 158 references

Uncertainty Quantification and Causal Considerations for Off-Policy Decision Making

T0 review · 2 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis claims that reweighting marginal outcome densities, conformal predictive intervals, and confounding-robust causal bounds can make off-policy evaluation reliable for decision-making.

desk verdict A thesis bundling three solid, already-published OPE papers; the finite-sample guarantees depend on an unquantified ratio-estimation error that the authors themselves flag, which is a real limitation but not a fatal one. read the letter →

arxiv 2502.06011 v1 pith:7NX64PPT submitted 2025-02-09 stat.ML cs.LG

Muhammad Faaiz Taufiq This is my paper · ORCID
classification stat.MLcs.LG
keywords off-policyevaluationcontextualbanditsmarginaldensityratioconformalpredictionfinite-samplecoverageunmeasuredconfoundingcausalboundsdigitaltwinfalsification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis aims to make off-policy evaluation usable when logged data were produced under a different policy and when standard assumptions fail. It proposes three results: a Marginal Ratio estimator that reweights outcomes by the marginal density ratio $w(y)=p_{\pi^*}(y)/p_{\pi_b}(y)$ and is unbiased with variance no larger than IPW or the MIPS family, and with lower variance than DR under stated conditions; Conformal Off-Policy Prediction, which turns logged bandit data into predictive intervals for outcomes under a new policy with finite-sample coverage even when the ratio weights must be estimated; and longitudinal causal bounds that remain valid under arbitrary unmeasured confounding, used to falsify digital twin models rather than certify them. If correct, these results give practitioners a lower-variance estimate of a policy's expected outcome, a distribution-free interval for individual outcomes, and a sound way to detect when a simulator is wrong in specific scenarios.

What carries the argument

The load-bearing object is the marginal ratio between target and behaviour distributions: for MR it is $w(y)=p_{\pi^*}(y)/p_{\pi_b}(y)$, realised as the conditional expectation $\mathbb{E}_{\pi_b}[\rho(A,X)\mid Y=y]$ and solved by regression; for COPP it is the conditional ratio $w(x,y)=dP_{\pi^*}(y\mid x)/dP_{\pi_b}(y\mid x)$, used to reweight nonconformity scores under weighted exchangeability. For the digital twin result, the machinery is the longitudinal potential-outcome bound of Theorem 4.4.1: the observed data identify $Q_{\mathrm{lo}}$ and $Q_{\mathrm{up}}$ by replacing unobserved potential outcomes with pre-specified worst-case values $y_{\mathrm{lo}},y_{\mathrm{up}}$ whenever the recorded action sequence differs from the target sequence, which yields a sound falsification test.

What would settle it

Run the paper's own synthetic bandit setup with the behaviour policy, target policy, and outcome model known, so the marginal ratios $w(y)$ and $w(x,y)$ can be computed exactly rather than estimated. If, at any finite calibration size, the empirical variance of MR exceeds that of IPW, or the empirical marginal coverage of COPP falls below $1-\alpha$, the corresponding theorem fails; with exact weights either outcome would be a direct counterexample.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the shift between the behaviour and target policies can be summarised by the marginal ratio of outcome densities rather than the full policy ratio, and that this single change yields three advances. In contextual bandits, the MR estimator $\hat{\theta}_{\mathrm{MR}} = \frac{1}{n}\sum_i w(y_i)y_i$ with $w(y)=\mathbb{E}_{\pi_b}[\rho(A,X)\mid Y=y]$ is unbiased when weights are known, has variance no larger than IPW's, and no larger than the variance of a generalised family of MIPS estimators. In the same setting, COPP reweights conformal nonconformity scores by $w(x,y)=dP_{\pi^*}(y\mid x)/dP_{\pi_b}(y\mid x)$ and yields predictive intervals satisfying $\mathbb{P}_{(X,Y)\sim \pi^*}(Y\in \hat{C}_n(X))\ge 1-\alpha-\Delta_w$ with estimated weights. For sequential decisions, Theorem 4.4.1 bounds the unidentifiable interventional mean $Q=\mathbb{E}[Y(a_{1:t})\mid X_{0:t}(a_{1:t})\in B_{0:t}]$ between two identifiable observed-data expectations $Q_{\mathrm{lo}}$ and $Q_{\mathrm{up}}$ under only worst-case outcome limits, which is then used to test a digital twin's predictions and falsify it when the twin's output falls outside these bounds.

Load-bearing premise

The methods' practical value depends on accurate estimation of the marginal ratios and on pre-specified worst-case outcome limits, neither of which is guaranteed by the data.

Editorial extensions

If this is right

  • In contextual bandits, OPE can be unbiased with lower variance than IPW and MIPS when the marginal ratios are known, so value estimates no longer degrade as quickly with large action or context spaces or large policy shifts.
  • Because MR uses every datapoint for each arm's expected outcome, average treatment effect estimation becomes more sample-efficient, especially when treatment groups are imbalanced.
  • COPP gives context-dependent predictive intervals for target-policy outcomes with finite-sample coverage, so risk-sensitive decisions can be based on quantiles or worst-case outcomes rather than only point estimates.
  • The longitudinal causal bounds allow digital twins to be falsified from i.i.d. observational trajectories even under arbitrary unmeasured confounding, so a twin can be shown wrong in specific patient or action scenarios without unconfoundedness assumptions.
  • Since the bounds are optimal without further assumptions, certification of a twin from observational data alone is impossible in general; the honest output of such data is a list of failures, not a certificate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If MR's variance advantage is stable under weight estimation, the same marginal-ratio trick could be used inside policy-gradient objectives to reduce gradient variance in off-policy optimization, not just evaluation.
  • COPP's intervals could be turned into robust policy-learning objectives that optimise worst-case outcomes with finite-sample coverage guarantees, a step the thesis suggests but does not develop.
  • The causal falsification bounds are stated for digital twins but apply to any simulator of a stochastic dynamical system; in principle they could audit reinforcement-learning environments or economic models from purely observational logs.
  • A natural extension would be to combine the falsification bounds with sensitivity-analysis assumptions on the strength of confounding, which would make 'not falsified' closer to a certification while openly paying the cost of an untestable assumption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. This thesis presents three self-contained contributions to off-policy evaluation (OPE). Chapter 2 introduces the Marginal Ratio (MR) estimator, which weights observations by the marginal density ratio w(y) = p_{π*}(y)/p_{πb}(y). It proves exact variance comparisons against IPW (Prop. 2.3.2), a conditional comparison against DR (Prop. 2.3.3), and dominance over the MIPS family under the MIPS action-embedding assumption (Theorem 2.3.2), and it analyzes the impact of estimating w(y) by regression (Prop. 2.3.4). Chapter 3 proposes Conformal Off-Policy Prediction (COPP), which adapts weighted conformal prediction to policy shifts and produces predictive intervals for target-policy outcomes with finite-sample marginal coverage 1−α under exact weights (Prop. 3.4.1), coverage 1−α−Δw under approximate weights (Prop. 3.4.2), and asymptotic conditional coverage (Prop. 3.4.3). Chapter 4 formulates digital-twin assessment as causal inference, proves that certification from observational data is unsound in general (Theorem 4.3.1), derives longitudinal bounds on interventional means that are valid under arbitrary unmeasured confounding (Theorem 4.4.1) with sharpness and impossibility results (Props. 4.4.1–4.4.2, Theorem 4.4.2), and builds an exact hypothesis-testing falsification procedure applied to the Pulse Physiology Engine using the MIMIC-III sepsis cohort. A short concluding chapter discusses limitations and future work.

Significance. Each chapter contains correct and useful theoretical results. The variance decomposition in Prop. 2.3.2 is exact and parameter-free; the formal link between MR and a generalized MIPS family (Theorem 2.3.2, App. A.4) is a clean structural insight; COPP's analysis honestly exposes the Δw degradation rather than hiding it; and Chapter 4's bounds generalize Manski to longitudinal settings, with the sharpness construction (Prop. 4.4.2) and the non-identifiability results (Theorems 4.3.1 and 4.4.2) delimiting the contribution precisely. The Chapter 4 case study is a major strength: 11,677 MIMIC-III trajectories, 1,442 hypotheses, exact Hoeffding-based confidence intervals, Holm correction, and an honest D0/D sample split that preserves conditional type-I error. The MR-OPE code is publicly released.

major comments (2)
  1. [§3.4.1 (Prop. 3.4.2), §3.3.1 (Eq. 3.7)] The central finite-sample guarantee of Chapter 3 is coverage ≥ 1−α−Δw with Δw = ½E_{Pπb}|ŵ−w| (Prop. 3.4.2), but the thesis provides no finite-sample bound on Δw and no procedure for estimating it from data. Eq. (3.7) compounds the estimation error of π̂b and P̂(y|x,a) with the Monte Carlo error of the h-sample average, and the proposition treats ŵ as fixed given Dtr; consequently the theorem neither controls the gap between the empirical coverage reported in Tables 3.1–3.2 and the nominal level, nor tells the user how to check it. If Δw is comparable to α, the stated lower bound is vacuous while the empirical coverage may still fall below 1−α. Since the abstract and introduction claim 'finite-sample guarantees' without this proviso, the headline claim is conditional on a nuisance quantity the user cannot verify. I recommend either proving a finite-sample bound on Δw under explicit assumptions on the estimators (for example, a bounded density ratio together with a ratio estimator whose L1 error is controlled), or reframing the claims across the chapter and reporting empirical Δw alongside coverage in the experiments.
  2. [§2.3.1 (Prop. 2.3.4, Eq. 2.4), App. A.3] The claimed variance advantage of MR over IPW with estimated weights is established only as E[ε²]→0, and Eq. (2.4) shows the variance difference is (1/n)(E[Var(ρ̂Y|Y)] − Var(εY) − 2Cov(w̃Y,εY)), so for any finite training size m the advantage can be negative. The rate E[ε²]=O(m^{−2/3}) quoted in App. A.3 is imported from a neural-network generalization bound (Lai et al., 2023) whose regularity conditions are not verified for the two-step estimator actually used in §2.5, and no finite-sample bound on Bias(θ̃MR)−Bias(θ̃IPW)=E[εY] is given. Since all experiments use estimated weights, the reported MSE gains are not covered by the theorems in the finite-sample regime. The limitation is openly acknowledged in §2.6 and Chapter 5, but the abstract's statement that MR 'achieves significant variance reduction... while maintaining unbiasedness' is stronger than what Prop. 2.3.4 delivers. I recommend adding a finite-sample analysis of the weight-estimation step under stated assumptions, or qualifying the abstract and Section 2.1 claims and reporting the regression error ε in the experiments.
minor comments (7)
  1. [§2.1 (abstract), §2.3.1 (Prop. 2.3.3)] The variance comparison with DR is conditional: Prop. 2.3.3 requires Eπb[Varπb(ρY|Y)] ≥ Eπb[Varπb(ρμ|X)]. The abstract and Section 2.1 say MR shows 'benefits' and 'better variance properties' compared with IPW and DR without this qualifier; I recommend stating explicitly that unconditional dominance is proven only against IPW and, under Assumption 2.3.1, against the MIPS family, with the DR comparison holding under the stated condition.
  2. [§4.6.1, §4.5.1] The data-driven choice of ylo and yup from D0 is sound because f is defined as a clipped extraction function, so condition (4.5) holds by construction and the D0/D split preserves conditional type-I error. The main text should nevertheless emphasize that the falsification concerns the mean of the clipped functional, not the raw physiological quantity, and that the ad hoc selection of B0:t (acknowledged in §4.7) shapes which failure modes are discovered.
  3. [Theorem 4.4.1] The three quantities in (4.6) are conditioned on two different events: {X0:N(A1:N)∈B0:N} for the bounds and {X0:t(a1:t)∈B0:t} for the target Q. The proof in App. C.7 is correct, but a remark explaining that the events coincide on {A1:t=a1:t} and that the worst-case substitution handles the complement would make the statement substantially easier to parse.
  4. [§2.3.1, Eq. (2.4)] The display of Eq. (2.4) is garbled by the underbrace markup, and the notation Varπb[ρ̂(A,X)Y|Y] uses Y both as a random variable and as the conditioning value; please standardize the notation and re-typeset the equation.
  5. [Chapter 5] The concluding chapter summarizes the three papers but does not discuss the relationship between the no-unmeasured-confounding assumption that Chapters 2 and 3 require and the arbitrary-confounding setting of Chapter 4; a short paragraph on this connection, including when each set of tools is applicable, would strengthen the integration the thesis claims.
  6. [§3.1] The claim that COPP is 'the first such method based on CP that can be applied to stochastic policies and continuous action spaces' is asserted rather than argued; given the closely related work of Lei and Candès (2021) and the concurrent ITE-sensitivity papers discussed in §3.5, the novelty claim should be either supported by a more systematic comparison or softened.
  7. [General] There are several typos and inconsistencies, e.g., 'throught' in the Acknowledgements, 'CIF AR-100' in the header of Table 2.1, and 'Mnist' vs 'MNIST' in §2.5.2; a careful proofread is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the three chapters derive their guarantees from stated assumptions and explicitly track approximation errors, with self-citations only as chapter provenance.

full rationale

Each chapter proves its central claims from stated assumptions rather than assuming the conclusion. Chapter 2's MR estimator is unbiased by a change of measure when w(y)=p_{pi*}/p_{pi_b} is known, but the paper's actual theoretical content is the variance comparison (Prop. 2.3.2), which follows from the law of total variance, and the finite-m analysis (Prop. 2.3.4), which explicitly tracks the approximation error epsilon. Chapter 3's coverage guarantee (Props. 3.4.1-3.4.2) is an application of weighted exchangeability; Delta_w is an explicit error term, not a hidden fitted parameter, and the authors state the limitation that weights must be estimated. Chapter 4's Theorem 4.4.1 is proved from consistency and bounded-support assumptions, generalizing Manski's bounds; the falsification hypotheses Hlo/Hup are constructed so that rejection implies non-correctness, and the proof does not assume the twin is incorrect. The data-chosen ylo/yup are acknowledged and sample-split; they affect the tested target but do not make the test a tautology. Self-citations appear only as thesis chapter provenance, not as load-bearing justifications. The unquantified Delta_w and epsilon are genuine limitations but are not circular reductions, so the honest finding is no circularity.

Assumptions & free parameters 1 free parameters · 7 assumptions · 0 invented entities

The thesis introduces no new postulated entities, particles, or mediators. The assumptions are standard OPE and causal inference conditions. The only data-chosen scalar parameters are the worst-case outcome bounds ylo,yup in the Chapter 4 case study. The central claims depend on these assumptions but not on any invented object.

free parameters (1)
  • ylo, yup worst-case outcome bounds = 0.2 and 0.8 quantiles of each physiological quantity in D0
    In the Chapter 4 case study (Section 4.6.1), the bounds ylo and yup are chosen as the 20th and 80th percentiles of the corresponding physiological quantity in the exploratory dataset D0. These values directly determine the width of the causal bounds and the power of the falsification test. Sensitivity analysis is reported in Appendix C.9.9.
assumptions (7)
  • domain assumption No unmeasured confounding (strong ignorability) for contextual bandit OPE
    Assumed in Chapters 2 and 3 (Assumption 1.1.1): the agent's action depends only on the observed context plus independent randomness. This is needed for standard IPW/DR and for the COPP weight calculation.
  • domain assumption Support overlap: π*(a|x)>0 implies πb(a|x)>0
    Assumption 2.2.1 ensures importance sampling weights are finite and unbiased. It is standard in OPE and is used throughout Chapter 2.
  • standard math Weighted exchangeability of calibration and test data under policy shift
    COPP builds on Tibshirani et al. 2019. Proposition 3.4.1 uses weighted exchangeability with weights w(x,y)=dPπ*_{Y|X}/dPπb_{Y|X} to obtain finite-sample marginal coverage.
  • domain assumption Potential outcomes framework with consistency
    Chapter 4, Section 4.3.1 assumes the observed trajectory follows (4.1) where X_t(A1:t) equals the potential outcome when actions A1:t are chosen. This is the standard consistency assumption in causal inference.
  • domain assumption Twin outputs are i.i.d. across runs and independent of the real-world process
    Section 4.2.2 assumes the twin's internal randomness U1:T is independent of the real-world potential outcomes and that repeated executions produce i.i.d. copies. This makes the twin dataset D_hat(a1:t) i.i.d.
  • domain assumption Observational trajectories are i.i.d. copies of (4.1)
    Section 4.3.1 assumes the observed dataset is i.i.d. copies of the trajectory (X0, A1, X1(A1), ..., AT, XT(A1:T)). This is the only distributional assumption needed for the falsification test.
  • domain assumption Outcome boundedness ylo ≤ Y(a1:t) ≤ yup almost surely
    Condition (4.5) in Theorem 4.4.1 requires the outcome to lie in a known bounded range. In the case study, this is enforced by clipping f to [ylo, yup] using data-chosen quantiles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncertainty Quantification and Causal Considerations for Off-Policy Decision Making." pith.science (2026). https://pith.science/paper/7NX64PPT

@misc{pith2026250206011,
  author       = {Pith},
  title        = {Pith review of: Uncertainty Quantification and Causal Considerations for Off-Policy Decision Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NX64PPT}},
  note         = {Machine review of arXiv:2502.06011}
}
read the original abstract

Off-policy evaluation (OPE) is a critical challenge in robust decision-making that seeks to assess the performance of a new policy using data collected under a different policy. However, the existing OPE methodologies suffer from several limitations arising from statistical uncertainty as well as causal considerations. In this thesis, we address these limitations by presenting three different works. Firstly, we consider the problem of high variance in the importance-sampling-based OPE estimators. We introduce the Marginal Ratio (MR) estimator, a novel OPE method that reduces variance by focusing on the marginal distribution of outcomes rather than direct policy shifts, improving robustness in contextual bandits. Next, we propose Conformal Off-Policy Prediction (COPP), a principled approach for uncertainty quantification in OPE that provides finite-sample predictive intervals, ensuring robust decision-making in risk-sensitive applications. Finally, we address causal unidentifiability in off-policy decision-making by developing novel bounds for sequential decision settings, which remain valid under arbitrary unmeasured confounding. We apply these bounds to assess the reliability of digital twin models, introducing a falsification framework to identify scenarios where model predictions diverge from real-world behaviour. Our contributions provide new insights into robust decision-making under uncertainty and establish principled methods for evaluating policies in both static and dynamic settings.

Figures

Figures reproduced from arXiv: 2502.06011 by the authors.

Figure 1.1
Figure 1.1. The discrepancy between observational data and interventional behaviour in the presence of unmeasured confounding: the range of outcomes observed in the data for patients who were administered the drug (blue) differs from what would be observed if the drug were administered to the general population (red). will rarely hold, and so OPE procedures based on this assumption may yield unreliable results in practice [Murp… view at source ↗
Figure 2.1
Figure 2.1. Bayesian network corresponding to Assumption 2.3.1. Intuition The context-embedding pair (X, E) can be seen as a representation of the context-action pair (X, A) which contains less ‘redundant information’ regarding the outcome Y . Intuitively, the MIPS estimator, which only considers the shift in the distribution of (X, E) is therefore more efficient than the IPW estimator (which considers the shift in the distribu… view at source ↗
Figure 2.2
Figure 2.2. Results for synthetic data experiment. In 2.2a we have α ∗ = 0.8 and in 2.2b we have n = 800. Results We compute the target policy value using the n evaluation datapoints. Here, the MSE of the estimators is computed over 10 different sets of logged data replicated with different seeds. The results presented have context dimension d = 1000, number of actions na = 100 and training data size m = 5000. More experiments … view at source ↗
Figures from the paper (5 more)
Figure 2
Figure 2. Figure 2: b plots the results for increasing policy shift. [PITH_FULL_IMAGE:figures/full_fig_p038_2.png]
Figure 3.1
Figure 3.1. Figure 3.1: Left (a): Conformal Off-Policy Prediction against standard off-policy evaluation methods. Right (b): 90% predictive intervals for Y against X for COPP, competing methods and the oracle. 3.1 Introduction Before deploying a decision-making policy to production, it is u…
Figure 4.1
Figure 4.1. Figure 4.1: Distributions of − log10 plo and − log10 pup across hypotheses, grouped by physiolog￾ical quantity. Higher values indicate greater evidence in favour of rejection. fewer rejections than our more general result given in Theorem 4.4.1, even when considered as a proport…
Figure 4.2
Figure 4.2. Figure 4.2: Estimates and 95% confidence intervals for Qb t and Qobs t at each 1 ≤ t ≤ 4 for two choices of (B0:4, a1:4), where bY (a1:t) and Y (a1:t) correspond to HCO3 concentration. The dashed lines indicate lower and upper 95% confidence intervals for Qlo, Qup respectively. …
Figure 4.3
Figure 4.3. Figure 4.3: Raw glucose values from the observational data and twin for two choices of (B0:t , a1:t), with confidence intervals for Qb and Qup shown below. The horizontal axes are truncated to the .025 and .975 quantiles of the observational data for clarity. Untruncated plots a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

158 extracted references · 41 canonical work pages

  1. [1]

    Sim2real for autonomous vehicle control using executable digital twin

    Jean Pierre Allamaa, Panagiotis Patrinos, Herman Van der Auweraer , and Tong Duy Son. Sim2real for autonomous vehicle control using executable digital twin. IFAC-PapersOnLine, 55 0 (24): 0 385--391, 2022. ISSN 2405-8963. doi:https://doi.org/10.1016/j.ifacol.2022.10.314. URL https://www.sciencedirect.com/science/article/pii/S2405896322023461. 10th IFAC Sym...

    work page doi:10.1016/j.ifacol.2022.10.314 2022

  2. [2]

    Best arm identification for contaminated bandits

    Jason Altschuler, Victor-Emmanuel Brunel, and Alan Malek. Best arm identification for contaminated bandits. Journal of Machine Learning Research, 20 0 (91): 0 1--39, 2019

    2019

  3. [3]

    Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices

    AMSE. Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices. AMSE, 2018

    2018

  4. [4]

    Angelopoulos and Stephen Bates

    Anastasios N. Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021

    arXiv 2021

  5. [5]

    A survey on digital twin: Definitions, characteristics, applications, and design implications

    Barbara Rita Barricelli, Elena Casiraghi, and Daniela Fogli. A survey on digital twin: Definitions, characteristics, applications, and design implications. IEEE access, 7: 0 167653--167671, 2019

    2019

  6. [6]

    Online decision making with high-dimensional covariates

    Hamsa Bastani and Mohsen Bayati. Online decision making with high-dimensional covariates. Operations Research, 68, 11 2019. doi:10.1287/opre.2019.1902

    arXiv 2019

  7. [7]

    A distributional perspective on reinforcement learning

    Marc G Bellemare, Will Dabney, and R \'e mi Munos. A distributional perspective on reinforcement learning. In International Conference on Machine Learning, pages 449--458, 2017

    2017

  8. [8]

    Tuegel, Anthony R

    Nicholas Bellinger, Eric J. Tuegel, Anthony R. Ingraffea, Thomas G. Eason, and S. Michael Spottswood. Reengineering aircraft structural life prediction using a digital twin. International Journal of Aerospace Engineering, 2011: 0 154798, 2011. doi:10.1155/2011/154798. URL https://doi.org/10.1155/2011/154798

    work page doi:10.1155/2011/154798 2011

Show all 158 references
  1. [9]

    The control of the false discovery rate in multiple testing under dependency

    Yoav Benjamini and Daniel Yekutieli. The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics, 29 0 (4): 0 1165 -- 1188, 2001. doi:10.1214/aos/1013699998. URL https://doi.org/10.1214/aos/1013699998

    2001

  2. [10]

    The offset tree for learning with partial labels

    Alina Beygelzimer and John Langford. The offset tree for learning with partial labels. CoRR, abs/0812.4044, 2008. URL http://arxiv.org/abs/0812.4044

    2008 arXiv

  3. [11]

    A contextual bandit bake-off

    Alberto Bietti, Alekh Agarwal, and John Langford. A contextual bandit bake-off. arXiv preprint arXiv:1802.04064, 2018

    2018 arXiv

  4. [12]

    Webb, Andinet Enquobahrie, Jared Vicory, Jerry Heneghan, Robert Hubal, Stephanie TerMaath, Philip Asare, and Rachel B

    Aaron Bray, Jeffrey B. Webb, Andinet Enquobahrie, Jared Vicory, Jerry Heneghan, Robert Hubal, Stephanie TerMaath, Philip Asare, and Rachel B. Clipp. Pulse Physiology Engine: an Open-Source Software Platform for Computational Modeling of Human Medical Simulation . SN Comprehens...

    2019 doi

  5. [13]

    Mining gold from implicit models to improve likelihood-free inference

    Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. Mining gold from implicit models to improve likelihood-free inference. Proceedings of the National Academy of Sciences, 117 0 (10): 0 5242--5249, 2020

    2020

  6. [14]

    Random forests

    Leo Breiman. Random forests. Machine Learning, 45 0 (1): 0 5--32, 2001. doi:10.1023/A:1010933404324. URL https://doi.org/10.1023/A:1010933404324

    2001 doi

  7. [15]

    Universal off-policy evaluation

    Yash Chandak, Scott Niekum, Bruno Castro da Silva, Erik Learned-Miller, Emma Brunskill, and Philip S Thomas. Universal off-policy evaluation. arXiv preprint arXiv:2104.12820, 2021

    2021 arXiv

  8. [16]

    The health digital twin to tackle cardiovascular disease—a review of an emerging interdisciplinary field

    Genevieve Coorey, Gemma A Figtree, David F Fletcher, Victoria J Snelson, Stephen Thomas Vernon, David Winlaw, Stuart M Grieve, Alistair McEwan, Jean Yee Hwa Yang, Pierre Qian, et al. The health digital twin to tackle cardiovascular disease—a review of an emerging interdiscipli...

    2022

  9. [17]

    Causal falsification of digital twins, 2023

    Rob Cornish, Muhammad Faaiz Taufiq, Arnaud Doucet, and Chris Holmes. Causal falsification of digital twins, 2023. URL https://arxiv.org/abs/2301.07210

    2023 arXiv

  10. [18]

    The 'digital twin' to enable the vision of precision cardiology

    Jorge Corral-Acero, Francesca Margara, Maciej Marciniak, Cristobal Rodero, Filip Loncaric, Yingjing Feng, Andrew Gilbert, Joao F Fernandes, Hassaan A Bukhari, Ali Wajdan, et al. The 'digital twin' to enable the vision of precision cardiology. European Heart Journal, 41 0 (48):...

    2020

  11. [19]

    A note on data-splitting for the evaluation of significance levels

    David R Cox. A note on data-splitting for the evaluation of significance levels. Biometrika, 62 0 (2): 0 441--444, 1975

    1975

  12. [20]

    Verification and validation of digital twins and virtual testbeds

    Ulrich Richard Dahmen, Tobias Osterloh, and Heinz-Jürgen Roßmann. Verification and validation of digital twins and virtual testbeds. International journal of advances in engineering sciences and applied mathematics, 11 0 (1): 0 47--64, 2022. ISSN 0975-5616. doi:10.11591/ijaas....

    2022 doi

  13. [21]

    A. C. Davison and D. V. Hinkley. Bootstrap Methods and their Application. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1997. doi:10.1017/CBO9780511802843

    1997 doi

  14. [22]

    The mnist database of handwritten digit images for machine learning research

    Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29 0 (6): 0 141--142, 2012

    2012

  15. [23]

    UCI machine learning repository, 2017

    Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml

    2017

  16. [24]

    Doubly robust policy evaluation and optimization

    Miroslav Dudík, Dumitru Erhan, John Langford, and Lihong Li. Doubly robust policy evaluation and optimization. Statistical Science, 29 0 (4), 2014 a

    2014

  17. [25]

    Doubly robust policy evaluation and optimization

    Miroslav Dudík, Dumitru Erhan, John Langford, and Lihong Li. Doubly robust policy evaluation and optimization. Statistical Science, 29 0 (4): 0 485--511, 2014 b . ISSN 08834237, 21688745. URL http://www.jstor.org/stable/43288496

    2014

  18. [26]

    B. Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7 0 (1): 0 1 -- 26, 1979. doi:10.1214/aos/1176344552. URL https://doi.org/10.1214/aos/1176344552

    1979

  19. [27]

    More robust doubly robust off-policy evaluation

    Mehrdad Farajtabar, Yinlam Chow, and Mohammad Ghavamzadeh. More robust doubly robust off-policy evaluation. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, ...

    2018

  20. [28]

    More robust doubly robust off-policy evaluation

    Mehrdad Farajtabar, Mohammad Ghavamzadeh, and Yinlam Chow. More robust doubly robust off-policy evaluation. 2018 b

    2018

  21. [29]

    Conformal off-policy evaluation in markov decision processes

    Daniele Foffano, Alessio Russo, and Alexandre Proutiere. Conformal off-policy evaluation in markov decision processes. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 3087--3094. IEEE, 2023

    2023

  22. [30]

    The limits of distribution-free conditional predictive inference

    Rina Foygel Barber, Emmanuel J Cand\`es, Aaditya Ramdas, and Ryan J Tibshirani. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10 0 (2): 0 455--482, 2021

    2021

  23. [31]

    A deep reinforcement learning approach to marginalized importance sampling with the successor representation

    Scott Fujimoto, David Meger, and Doina Precup. A deep reinforcement learning approach to marginalized importance sampling with the successor representation. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 13...

    2021

  24. [32]

    Credibility assessment of patient-specific computational modeling using patient-specific cardiac modeling as an exemplar

    Suran Galappaththige, Richard A Gray, Caroline Mendonca Costa, Steven Niederer, and Pras Pathmanathan. Credibility assessment of patient-specific computational modeling using patient-specific cardiac modeling as an exemplar. PLoS computational biology, 18 0 (10): 0 e1010541, 2022

    2022

  25. [33]

    Digital twin: Mitigating unpredictable, undesirable emergent behavior in complex systems

    Michael Grieves and John Vickers. Digital twin: Mitigating unpredictable, undesirable emergent behavior in complex systems. In Transdisciplinary Perspectives on Complex Systems, pages 85--113. Springer, 2017

    2017

  26. [34]

    Theoretical comparison of bootstrap confidence intervals

    Peter Hall. Theoretical comparison of bootstrap confidence intervals. The Annals of Statistics, 16 0 (3): 0 927 -- 953, 1988. doi:10.1214/aos/1176350933. URL https://doi.org/10.1214/aos/1176350933

    1988

  27. [35]

    Patient-specific in silico endovascular repair of abdominal aortic aneurysms: application and validation

    Andr \'e Hemmler, Brigitta Lutz, G \"u nay Kalender, Christian Reeps, and Michael W Gee. Patient-specific in silico endovascular repair of abdominal aortic aneurysms: application and validation. Biomechanics and Modeling in Mechanobiology, 18 0 (4): 0 983--1004, 2019

    2019

  28. [36]

    Causal Inference: What If

    Miguel A Hern \'a n and James M Robins. Causal Inference: What If. Chapman and Hall/CRC, Boca Raton, 2020

    2020

  29. [37]

    Hesterberg

    Tim C. Hesterberg. What teachers should know about the bootstrap: Resampling in the undergraduate statistics curriculum. The American Statistician, 69 0 (4): 0 371--386, 2015. doi:10.1080/00031305.2015.1089789. URL https://doi.org/10.1080/00031305.2015.1089789. PMID: 27019512

    2015 arXiv

  30. [38]

    Paul W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 81 0 (396): 0 945--960, 1986. ISSN 01621459. URL http://www.jstor.org/stable/2289064

    1986

  31. [39]

    A simple sequentially rejective multiple test procedure

    Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6 0 (2): 0 65--70, 1979. ISSN 03036898, 14679469. URL http://www.jstor.org/stable/4615733

    1979

  32. [40]

    D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47 0 (260): 0 663--685, 1952. ISSN 01621459. URL http://www.jstor.org/stable/2280784

    1952

  33. [41]

    Lipton, and Kamyar Azizzadenesheli

    Audrey Huang, Liu Leqi, Zachary C. Lipton, and Kamyar Azizzadenesheli. Off-policy risk assessment in contextual bandits. arXiv preprint arXiv:2104.08977, 2021

    2021 arXiv

  34. [42]

    Correcting sample selection bias by unlabeled data

    Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Sch \"o lkopf, and Alex J Smola. Correcting sample selection bias by unlabeled data. In Advances in Neural Information Processing Systems 19, pages 601--608, 2007

    2007

  35. [43]

    Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics

    Guido W Imbens. Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics. Journal of Economic Literature, 58 0 (4): 0 1129--79, 2020

    2020

  36. [44]

    Confidence intervals for partially identified parameters

    Guido W Imbens and Charles F Manski. Confidence intervals for partially identified parameters. Econometrica, 72 0 (6): 0 1845--1857, 2004

    2004

  37. [45]

    Digital twin of an urban-integrated hydroponic farm

    Melanie Jans-Singh, Kathryn Leeming, Ruchi Choudhary, and Mark Girolami. Digital twin of an urban-integrated hydroponic farm. Data-Centric Engineering, 1: 0 e20, 2020. doi:10.1017/dce.2020.21

    2020 doi

  38. [46]

    Doubly robust off-policy value evaluation for reinforcement learning

    Nan Jiang and Lihong Li. Doubly robust off-policy value evaluation for reinforcement learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research...

    2016

  39. [47]

    Sensitivity analysis of individual treatment effects: A robust conformal inference approach

    Ying Jin, Zhimei Ren, and Emmanuel J Cand \`e s. Sensitivity analysis of individual treatment effects: A robust conformal inference approach. arXiv preprint arXiv:2111.12161, 2021

    2021 arXiv

  40. [48]

    Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. Mimic-iii, a freely accessible critical care database. Scientific Data, 3 0 (1): 0 160035, 2016. doi:10.10...

    2016 doi

  41. [49]

    Characterising the digital twin: A systematic literature review

    David Jones, Chris Snider, Aydin Nassehi, Jason Yon, and Ben Hicks. Characterising the digital twin: A systematic literature review. CIRP Journal of Manufacturing Science and Technology, 29: 0 36--52, 2020

    2020

  42. [50]

    Double reinforcement learning for efficient off-policy evaluation in markov decision processes

    Nathan Kallus and Masatoshi Uehara. Double reinforcement learning for efficient off-policy evaluation in markov decision processes. J. Mach. Learn. Res., 21 0 (1), jun 2022. ISSN 1532-4435

    2022

  43. [51]

    Confounding-robust policy improvement

    Nathan Kallus and Angela Zhou. Confounding-robust policy improvement. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceed...

    2018

  44. [52]

    Minimax-optimal policy learning under unobserved confounding

    Nathan Kallus and Angela Zhou. Minimax-optimal policy learning under unobserved confounding. Management Science, 67, 10 2020. doi:10.1287/mnsc.2020.3699

    2020

  45. [53]

    Optimal off-policy evaluation from multiple logging policies

    Nathan Kallus, Yuta Saito, and Masatoshi Uehara. Optimal off-policy evaluation from multiple logging policies. In International Conference on Machine Learning, pages 5247--5256. PMLR, 2021

    2021

  46. [54]

    A probabilistic graphical model foundation for enabling predictive digital twins at scale

    Michael G Kapteyn, Jacob VR Pretorius, and Karen E Willcox. A probabilistic graphical model foundation for enabling predictive digital twins at scale. Nature Computational Science, 1 0 (5): 0 337--347, 2021

    2021

  47. [55]

    Being optimistic to be conservative: Quickly learning a cvar policy

    Ramtin Keramati, Christoph Dann, Alex Tamkin, and Emma Brunskill. Being optimistic to be conservative: Quickly learning a cvar policy. In AAAI Conference on Artificial Intelligence, volume 34, pages 4436--4443, 2020

    2020

  48. [56]

    Digital twin for legacy systems: Simulation model testing and validation

    Adnan Khan, Martin Dahl, Petter Falkman, and Martin Fabian. Digital twin for legacy systems: Simulation model testing and validation. In 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE). IEEE, 2018

    2018

  49. [57]

    Digital twin concepts with uncertainty for nuclear power applications

    Brendan Kochunas and Xun Huan. Digital twin concepts with uncertainty for nuclear power applications. Energies, 14 0 (14): 0 4235, 2021

    2021

  50. [58]

    Celi, Omar Badawi, Anthony C

    Matthieu Komorowski, Leo A. Celi, Omar Badawi, Anthony C. Gordon, and A. Aldo Faisal. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine, 24 0 (11): 0 1716--1720, 2018. doi:10.1038/s41591-018-0213-5. URL http...

    2018 doi

  51. [59]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009

    2009

  52. [60]

    Conformal off-policy prediction for multi-agent systems

    Tom Kuipers, Renukanandan Tumu, Shuo Yang, Milad Kazemi, Rahul Mangharam, and Nicola Paoletti. Conformal off-policy prediction for multi-agent systems. arXiv preprint arXiv:2403.16871, 2024

    2024 arXiv

  53. [61]

    Measurement bias and effect restoration in causal inference

    Manabu Kuroki and Judea Pearl. Measurement bias and effect restoration in causal inference . Biometrika, 101 0 (2): 0 423--437, 03 2014. ISSN 0006-3444. doi:10.1093/biomet/ast066. URL https://doi.org/10.1093/biomet/ast066

    2014 doi

  54. [62]

    Confident off-policy evaluation and selection through self-normalized importance weighting

    Ilja Kuzborskij, Claire Vernade, Andr \' a s Gy \" o rgy, and Csaba Szepesv \'a ri. Confident off-policy evaluation and selection through self-normalized importance weighting. In International Conference on Artificial Intelligence and Statistics, pages 640--648, 2021

    2021

  55. [63]

    Generalization ability of wide neural networks on R , 2023

    Jianfa Lai, Manyun Xu, Rui Chen, and Qian Lin. Generalization ability of wide neural networks on R , 2023. URL https://arxiv.org/abs/2302.05933

    2023 arXiv

  56. [64]

    Development and verification of a digital twin patient model to predict treatment response in sepsis

    Amos Lal, Guangxi Li, Edin Cubro, Sarah Chalmers, Heyi Li, Vitaly Herasevich, Yue Dong, Brian Pickering, Kilickaya Oguz, and Ognjen Gajic. Development and verification of a digital twin patient model to predict treatment response in sepsis. Critical Care Medicine, 49: 0 611--6...

    2021

  57. [65]

    Illustrating reinforcement learning from human feedback (rlhf)

    Nathan Lambert, Louis Castricato, Leandro von Werra, and Alex Havrilla. Illustrating reinforcement learning from human feedback (rlhf). Hugging Face Blog, 2022. https://huggingface.co/blog/rlhf

    2022

  58. [66]

    Fast virtual deployment of self-expandable stents: method and in vitro evaluation for intracranial aneurysmal stenting

    Ignacio Larrabide, Minsuok Kim, Luca Augsburger, Maria Cruz Villa-Uriol, Daniel Rüfenacht, and Alejandro F Frangi. Fast virtual deployment of self-expandable stents: method and in vitro evaluation for intracranial aneurysmal stenting. Medical Image Analysis, 16 0 (3): 0 721—73...

    2012 doi

  59. [67]

    Bandit Algorithms

    Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge University Press, 2020

    2020

  60. [68]

    Dynamic treatment regimes: practical design considerations

    Philip W Lavori and Ree Dawson. Dynamic treatment regimes: practical design considerations. Clinical trials, 1 0 (1): 0 9--20, 2004

    2004

  61. [69]

    Distribution-free prediction bands for non-parametric regression

    Jing Lei and Larry Wasserman. Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society: Series B , pages 71--96, 2014

    2014

  62. [70]

    Conformal inference of counterfactuals and individual treatment effects

    Lihua Lei and Emmanuel J Cand \`e s. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society: Series B , pages 911--938, 2021

    2021

  63. [71]

    Addressing Extreme Propensity Scores via the Overlap Weights

    Fan Li, Laine E Thomas, and Fan Li. Addressing Extreme Propensity Scores via the Overlap Weights . American Journal of Epidemiology, 188 0 (1): 0 250--257, 09 2018. ISSN 0002-9262. doi:10.1093/aje/kwy201. URL https://doi.org/10.1093/aje/kwy201

    2018 doi

  64. [72]

    Schapire

    Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, page 661–670, New York, NY, USA, 2010. Association for Computing...

    2010

  65. [73]

    Optimal rates for spectral algorithms with least-squares regression over hilbert spaces

    Junhong Lin, Alessandro Rudi, Lorenzo Rosasco, and Volkan Cevher. Optimal rates for spectral algorithms with least-squares regression over hilbert spaces. Applied and Computational Harmonic Analysis, 48 0 (3): 0 868--890, 2020. ISSN 1063-5203. doi:https://doi.org/10.1016/j.ach...

    2020 doi

  66. [74]

    Triply robust off-policy evaluation, 2019

    Anqi Liu, Hao Liu, Anima Anandkumar, and Yisong Yue. Triply robust off-policy evaluation, 2019. URL https://arxiv.org/abs/1911.05811

    2019 arXiv

  67. [75]

    Breaking the curse of horizon: Infinite-horizon off-policy estimation

    Qiang Liu, Lihong Li, Ziyang Tang, and Dengyong Zhou. Breaking the curse of horizon: Infinite-horizon off-policy estimation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume...

    2018

  68. [76]

    Trustworthy llms: a survey and guideline for evaluating large language models' alignment, 2024

    Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and guideline for evaluating large language models' alignment, 2024. URL https://arxiv.org/abs/2308.05374

    2024 arXiv

  69. [77]

    Understanding the curse of horizon in off-policy evaluation via conditional importance sampling

    Yao Liu, Pierre-Luc Bacon, and Emma Brunskill. Understanding the curse of horizon in off-policy evaluation via conditional importance sampling. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  70. [78]

    B ayesian counterfactual risk minimization

    Ben London and Ted Sandler. B ayesian counterfactual risk minimization. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4125--4133. PMLR,...

    2019

  71. [79]

    Causal effect inference with deep latent-variable models

    Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 6449–6459, Red Hook, N...

    2017

  72. [80]

    Rethinking importance weighting for transfer learning

    Nan Lu, Tianyi Zhang, Tongtong Fang, Takeshi Teshima, and Masashi Sugiyama. Rethinking importance weighting for transfer learning. CoRR, abs/2112.10157, 2021. URL https://arxiv.org/abs/2112.10157

    2021 arXiv

  73. [81]

    Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues

    Yuqian Lu, Chao Liu, I Kevin, Kai Wang, Huiyue Huang, and Xun Xu. Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robotics and Computer-Integrated Manufacturing, 61: 0 101837, 2020

    2020

  74. [82]

    Charles F. Manski. Anatomy of the selection problem. The Journal of Human Resources, 24 0 (3): 0 343--360, 1989. ISSN 0022166X. URL http://www.jstor.org/stable/145818

    1989

  75. [83]

    Charles F. Manski. Nonparametric bounds on treatment effects. The American Economic Review, 80 0 (2): 0 319--323, 1990. ISSN 00028282. URL http://www.jstor.org/stable/2006592

    1990

  76. [84]

    Identification Problems in the Social Sciences

    Charles F Manski. Identification Problems in the Social Sciences . Harvard University Press, 1995

    1995

  77. [85]

    Partial Identification of Probability Distributions

    Charles F Manski. Partial Identification of Probability Distributions. Springer, 2003

    2003

  78. [86]

    A modular computational framework for medical digital twins

    Joseph Masison, Jonathan Beezley, Yu Mei, Henrique Assis Lopes Ribeiro, Adam C Knapp, L Sordo Vieira, Bandita Adhikari, Yogesh Scindia, Michael Grauer, Brian Helba, et al. A modular computational framework for medical digital twins. Proceedings of the National Academy of Scien...

    2021

  79. [87]

    A Full-Body Model of Burn Pathophysiology and Treatment Using the BioGears Engine

    Matthew McDaniel and Austin Baird. A Full-Body Model of Burn Pathophysiology and Treatment Using the BioGears Engine . In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 261--264, 2019. doi:10.1109/EMBC.2019.8857686

    2019

  80. [88]

    Keller, Steven White, and Austin Baird

    Matthew McDaniel, Jonathan M. Keller, Steven White, and Austin Baird. A Whole-Body Mathematical Model of Sepsis Progression and Treatment Designed in the BioGears Physiology Engine . Frontiers in Physiology, 10: 0 1321, 2019. ISSN 1664-042X. doi:10.3389/fphys.2019.01321. URL h...

    2019

  81. [89]

    Simulating ratios of normalizing constants via a simple identity: a theoretical exploration

    Xiao-Li Meng and Wing Hung Wong. Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statistica Sinica, pages 831--860, 1996

    1996

  82. [90]

    Subgaussian and differentiable importance sampling for off-policy evaluation and learning

    Alberto Maria Metelli, Alessio Russo, and Marcello Restelli. Subgaussian and differentiable importance sampling for off-policy evaluation and learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Proces...

    2021

  83. [91]

    S. A. Murphy. An experimental design for the development of adaptive treatment strategies. Statistics in Medicine, 24 0 (10): 0 1455--1481, 2005. doi:https://doi.org/10.1002/sim.2022. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.2022

    2005 doi

  84. [92]

    Optimal dynamic treatment regimes

    Susan A Murphy. Optimal dynamic treatment regimes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65 0 (2): 0 331--355, 2003

    2003

  85. [93]

    Off-policy policy evaluation for sequential decisions under unobserved confounding

    Hongseok Namkoong, Ramtin Keramati, Steve Yadlowsky, and Emma Brunskill. Off-policy policy evaluation for sequential decisions under unobserved confounding. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Sy...

    2020

  86. [94]

    Newey and James R

    Whitney K. Newey and James R. Robins. Cross-fitting and fast remainder rates for semiparametric estimation, 2018. URL https://arxiv.org/abs/1801.09138

    2018 arXiv

  87. [95]

    Scaling digital twins from the artisanal to the industrial

    Steven A Niederer, Michael S Sacks, Mark Girolami, and Karen Willcox. Scaling digital twins from the artisanal to the industrial. Nature Computational Science, 1 0 (5): 0 313--320, 2021

    2021

  88. [96]

    Learning robust decision policies from observational data

    Muhammad Osama, Dave Zachariah, and Peter Stoica. Learning robust decision policies from observational data. arXiv preprint arXiv:2006.02355, 2020

    2006 arXiv

  89. [97]

    Causality

    Judea Pearl. Causality. Cambridge University Press, 2 edition, 2009. doi:10.1017/CBO9780511803161

    2009 doi

  90. [98]

    The Logic of Scientific Discovery

    Karl Popper. The Logic of Scientific Discovery. Routledge, 2005

    2005

  91. [99]

    Introducing LETOR 4.0 datasets

    Tao Qin and Tie - Yan Liu. Introducing LETOR 4.0 datasets. arXiv preprint arXiv:1306.2597, 2013

    2013 arXiv

  92. [100]

    A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect

    James Robins. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling, 7 0 (9): 0 1393--1512, 1986. ISSN 0270-0255. doi:https://doi.org/10.1016/0270-0255(86)90...

    1986

  93. [101]

    Cand\`es

    Yaniv Romano, Evan Patterson, and Emmanuel J. Cand\`es. Conformalized quantile regression. In Advances in Neural Information Processing Systems, volume 32, pages 3543--3553, 2019

    2019

  94. [102]

    Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel J. Candès. With malice toward none: Assessing uncertainty via equalized coverage. Harvard Data Science Review, 2 0 (2), 4 2020

    2020

  95. [103]

    Rosenbaum

    Paul R. Rosenbaum. Observational Studies. Springer, New York, NY, 2002

    2002

  96. [104]

    Rosenbaum and Donald B

    Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70 0 (1): 0 41--55, 1983. ISSN 00063444. URL http://www.jstor.org/stable/2335942

    1983

  97. [105]

    Conditional importance sampling for off-policy learning

    Mark Rowland, Anna Harutyunyan, Hado Hasselt, Diana Borsa, Tom Schaul, R \'e mi Munos, and Will Dabney. Conditional importance sampling for off-policy learning. In International Conference on Artificial Intelligence and Statistics, pages 45--55. PMLR, 2020

    2020

  98. [106]

    Roy and William L

    Christopher J. Roy and William L. Oberkampf. A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Computer Methods in Applied Mechanics and Engineering, 200 0 (25): 0 2131--2144, 2011. ISSN 0045-7825. doi:https://doi.o...

    2011 doi

  99. [107]

    Donald B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66: 0 688–701, 1974. doi:https://doi.org/10.1037/h0037350

    1974 doi

  100. [108]

    Causal inference using potential outcomes

    Donald B Rubin. Causal inference using potential outcomes. Journal of the American Statistical Association, 100 0 (469): 0 322--331, 2005. doi:10.1198/016214504000001880. URL https://doi.org/10.1198/016214504000001880

    2005 doi

  101. [109]

    Off-policy bandits with deficient support

    Noveen Sachdeva, Yi Su, and Thorsten Joachims. Off-policy bandits with deficient support. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20, page 965–975, New York, NY, USA, 2020. Association for Computing Machinery. I...

    2020

  102. [110]

    Construction with digital twin information systems

    Rafael Sacks, Ioannis Brilakis, Ergo Pikas, Haiyan Xie, and Mark Girolami. Construction with digital twin information systems. Data-Centric Engineering, 1, 2020

    2020

  103. [111]

    Off-policy evaluation for large action spaces via embeddings

    Yuta Saito and Thorsten Joachims. Off-policy evaluation for large action spaces via embeddings. In Proceedings of the 39th International Conference on Machine Learning, pages 19089--19122. PMLR, 2022

    2022

  104. [112]

    Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation

    Yuta Saito, Aihara Shunsuke, Matsutani Megumi, and Narita Yusuke. Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation. arXiv preprint arXiv:2008.07146, 2020

    2008 arXiv

  105. [113]

    Evaluating the robustness of off-policy evaluation

    Yuta Saito, Takuma Udagawa, Haruka Kiyohara, Kazuki Mogi, Yusuke Narita, and Kei Tateno. Evaluating the robustness of off-policy evaluation. In Proceedings of the 15th ACM Conference on Recommender Systems, RecSys '21, page 114–123, New York, NY, USA, 2021. Association for Com...

    2021

  106. [114]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017

    2017

  107. [115]

    Seymour, Vincent X

    Christopher W. Seymour, Vincent X. Liu, Theodore J. Iwashyna, Frank M. Brunkhorst, Thomas D. Rea, André Scherag, Gordon Rubenfeld, Jeremy M. Kahn, Manu Shankar-Hari, Mervyn Singer, Clifford S. Deutschman, Gabriel J. Escobar, and Derek C. Angus. Assessment of Clinical Criteria ...

    2016

  108. [116]

    A tutorial on conformal prediction

    Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9 0 (3), 2008

    2008

  109. [117]

    Improving predictive inference under covariate shift by weighting the log-likelihood function

    Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90 0 (2): 0 227--244, 2000. ISSN 0378-3758. doi:https://doi.org/10.1016/S0378-3758(00)00115-4. URL https://www.sc...

    doi

  110. [118]

    Deutschman, Christopher Warren Seymour, Manu Shankar-Hari, Djillali Annane, Michael Bauer, Rinaldo Bellomo, Gordon R

    Mervyn Singer, Clifford S. Deutschman, Christopher Warren Seymour, Manu Shankar-Hari, Djillali Annane, Michael Bauer, Rinaldo Bellomo, Gordon R. Bernard, Jean-Daniel Chiche, Craig M. Coopersmith, Richard S. Hotchkiss, Mitchell M. Levy, John C. Marshall, Greg S. Martin, Steven ...

    2016

  111. [119]

    Balanced off-policy evaluation in general action spaces

    Arjun Sondhi, David Arbour, and Drew Dimmery. Balanced off-policy evaluation in general action spaces. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings...

    2020

  112. [120]

    Learning optimal conformal classifiers

    David Stutz, Krishnamurthy Dvijotham, Ali Taylan Cemgil, and Arnaud Doucet. Learning optimal conformal classifiers. International Conference on Representation Learning, 2022

    2022

  113. [121]

    Doubly robust off-policy evaluation with shrinkage

    Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, and Miroslav Dud \' k. Doubly robust off-policy evaluation with shrinkage. CoRR, abs/1907.09623, 2019 a . URL http://arxiv.org/abs/1907.09623

    1907 arXiv

  114. [122]

    CAB : Continuous adaptive blending for policy evaluation and learning

    Yi Su, Lequn Wang, Michele Santacatterina, and Thorsten Joachims. CAB : Continuous adaptive blending for policy evaluation and learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of ...

    2019

  115. [123]

    Doubly robust off-policy evaluation with shrinkage

    Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, and Miroslav Dud\' k. Doubly robust off-policy evaluation with shrinkage. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  116. [124]

    Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation

    Masashi Sugiyama and Motoaki Kawanabe. Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation. The MIT Press, 2012. ISBN 9780262017091. URL http://www.jstor.org/stable/j.ctt5hhbtm

    2012

  117. [125]

    Covariate shift adaptation by importance weighted cross validation

    Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert M\"uller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8 0 (5): 0 985--1005, 2007

    2007

  118. [126]

    Direct importance estimation with model selection and its application to covariate shift adaptation

    Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul von Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in Neural Information Processing Systems 20, pages 1433--1440, 2008

    2008

  119. [127]

    Counterfactual risk minimization: Learning from logged bandit feedback

    Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15, page 814–823. JMLR.org, 2015 a

    2015

  120. [128]

    The self-normalized estimator for counterfactual learning

    Adith Swaminathan and Thorsten Joachims. The self-normalized estimator for counterfactual learning. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015 b . URL http...

    2015

  121. [129]

    Batch learning from logged bandit feedback through counterfactual risk minimization

    Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization. Journal of Machine Learning Research, 16 0 (52): 0 1731--1755, 2015 c

    2015

  122. [130]

    The self-normalized estimator for counterfactual learning

    Adith Swaminathan and Thorsten Joachims. The self-normalized estimator for counterfactual learning. In Advances in Neural Information Processing Systems, volume 28, 2015 d

    2015

  123. [131]

    Off-policy evaluation for slate recommendation

    Adith Swaminathan, Akshay Krishnamurthy, Alekh Agarwal, Miroslav Dud \' k, John Langford, Damien Jose, and Imed Zitouni. Off-policy evaluation for slate recommendation. In Advances in Neural Information Processing Systems, 2017 a

    2017

  124. [132]

    Off-policy evaluation for slate recommendation

    Adith Swaminathan, Akshay Krishnamurthy, Alekh Agarwal, Miroslav Dud\' k, John Langford, Damien Jose, and Imed Zitouni. Off-policy evaluation for slate recommendation. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 3...

    2017

  125. [133]

    A distributional approach for causal inference using propensity scores

    Zhiqiang Tan. A distributional approach for causal inference using propensity scores. Journal of the American Statistical Association, 101 0 (476): 0 1619--1637, 2006

    2006

  126. [134]

    Conformal off-policy prediction in contextual bandits

    Muhammad Faaiz Taufiq, Jean-Francois Ton, Rob Cornish, Yee Whye Teh, and Arnaud Doucet. Conformal off-policy prediction in contextual bandits. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022....

    2022

  127. [135]

    Manifold restricted interventional shapley values

    Muhammad Faaiz Taufiq, Patrick Bl\"obaum, and Lenon Minorics. Manifold restricted interventional shapley values. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, vo...

    2023

  128. [136]

    Marginal density ratio for off-policy evaluation in contextual bandits

    Muhammad Faaiz Taufiq, Arnaud Doucet, Rob Cornish, and Jean-Francois Ton. Marginal density ratio for off-policy evaluation in contextual bandits. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 b . URL https://openreview.net/forum?id=noyleECBam

    2023

  129. [137]

    Achievable fairness on your data with utility guarantees, 2024

    Muhammad Faaiz Taufiq, Jean-Francois Ton, and Yang Liu. Achievable fairness on your data with utility guarantees, 2024. URL https://arxiv.org/abs/2402.17106

    2024 arXiv

  130. [138]

    An introduction to proximal causal learning, 2020

    Eric J Tchetgen Tchetgen, Andrew Ying, Yifan Cui, Xu Shi, and Wang Miao. An introduction to proximal causal learning, 2020

    2020

  131. [139]

    Thomas and Emma Brunskill

    Philip S. Thomas and Emma Brunskill. Data-efficient off-policy policy evaluation for reinforcement learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML'16, page 2139–2148. JMLR.org, 2016

    2016

  132. [140]

    Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh

    Philip S. Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High-confidence off-policy evaluation. In AAAI Conference on Artificial Intelligence, 2015

    2015

  133. [141]

    An introduction to the bootstrap

    Robert J Tibshirani and Bradley Efron. An introduction to the bootstrap. Monographs on statistics and applied probability, 57: 0 1--436, 1993

    1993

  134. [142]

    Tibshirani, Rina Foygel Barber, Emmanuel J

    Ryan J. Tibshirani, Rina Foygel Barber, Emmanuel J. Cand\`es, and Aaditya Ramdas. Conformal prediction under covariate shift. In Advances in Neural Information Processing Systems, 2019

    2019

  135. [143]

    Dynamic treatment regimes: Statistical methods for precision medicine

    Anastasios A Tsiatis, Marie Davidian, Shannon T Holloway, and Eric B Laber. Dynamic treatment regimes: Statistical methods for precision medicine. Chapman and Hall/CRC, 2019

    2019

  136. [144]

    A review of off-policy evaluation in reinforcement learning, 2022

    Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. A review of off-policy evaluation in reinforcement learning, 2022. URL https://arxiv.org/abs/2212.06355

    2022 arXiv

  137. [145]

    Empirical study of off-policy policy evaluation for reinforcement learning

    Cameron Voloshin, Hoang Minh Le, Nan Jiang, and Yisong Yue. Empirical study of off-policy policy evaluation for reinforcement learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL https://openreview.net...

    2021

  138. [146]

    Conditional validity of inductive conformal predictors

    Vladimir Vovk. Conditional validity of inductive conformal predictors. In Steven C. H. Hoi and Wray Buntine, editors, Proceedings of the Asian Conference on Machine Learning, volume 25 of Proceedings of Machine Learning Research, pages 475--490, Singapore Management University...

    2012

  139. [147]

    Algorithmic Learning in a Random World

    Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World. Springer Science & Business Media, 2005

    2005

  140. [148]

    Optimal and adaptive off-policy evaluation in contextual bandits

    Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dud\' k. Optimal and adaptive off-policy evaluation in contextual bandits. In International Conference on Machine Learning, page 3589–3597, 2017 a

    2017

  141. [149]

    Optimal and adaptive off-policy evaluation in contextual bandits

    Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dud\' k. Optimal and adaptive off-policy evaluation in contextual bandits. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 3589–3597. JMLR.org, 2017 b

    2017

  142. [150]

    Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling

    Tengyang Xie, Yifei Ma, and Yu-Xiang Wang. Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information ...

    2019

  143. [151]

    Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling

    Tengyang Xie, Yifei Ma, and Yu-Xiang Wang. Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information ...

    2019

  144. [152]

    Kernel single proxy control for deterministic confounding, 2024

    Liyuan Xu and Arthur Gretton. Kernel single proxy control for deterministic confounding, 2024

    2024

  145. [153]

    Deep proxy causal learning and its application to confounded bandit policy evaluation

    Liyuan Xu, Heishiro Kanagawa, and Arthur Gretton. Deep proxy causal learning and its application to confounded bandit policy evaluation. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https...

    2021

  146. [154]

    Contextual-bandit based personalized recommendation with time-varying user interests

    Xiao Xu, Fang Dong, Yanghua Li, Shaojian He, and Xin Li. Contextual-bandit based personalized recommendation with time-varying user interests. Proceedings of the AAAI Conference on Artificial Intelligence, 34: 0 6518--6525, 04 2020. doi:10.1609/aaai.v34i04.6125

    2020 doi

  147. [155]

    Bounds on the conditional and average treatment effect with unobserved confounding factors

    Steve Yadlowsky, Hongseok Namkoong, Sanjay Basu, John Duchi, and Lu Tian. Bounds on the conditional and average treatment effect with unobserved confounding factors . The Annals of Statistics, 50 0 (5): 0 2587 -- 2615, 2022. doi:10.1214/22-AOS2195. URL https://doi.org/10.1214/...

    2022 doi

  148. [156]

    Conformal sensitivity analysis for individual treatment effects

    Mingzhang Yin, Claudia Shi, Yixin Wang, and David M Blei. Conformal sensitivity analysis for individual treatment effects. arXiv preprint arXiv:2112.03493, 2021

    2021 arXiv

  149. [157]

    Near-optimal reinforcement learning in dynamic treatment regimes

    Junzhe Zhang and Elias Bareinboim. Near-optimal reinforcement learning in dynamic treatment regimes. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associat...

    2019

  150. [158]

    Conformal off-policy prediction

    Yingying Zhang, Chengchun Shi, and Shikai Luo. Conformal off-policy prediction. In International Conference on Artificial Intelligence and Statistics, pages 2751--2768. PMLR, 2023

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.