Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework

Mohammad M. Ghassemi; Niloufar Eghbali; Tuka Alhanai

REVIEW 5 major objections 6 minor 33 references

Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that adding a conformal prediction layer to deep Q-learning yields safer, more conservative ventilator action selection and improves estimated 90-day survival over clinician and prior RL policies.

desk verdict Conformal integration is genuine, but the 90-day survival claim rests on an unvalidated FQE mapping, not measured outcomes. read the letter →

arxiv 2412.12597 v1 pith:KMAQKNVU submitted 2024-12-17 cs.LG

Niloufar Eghbali , Tuka Alhanai , Mohammad M. Ghassemi This is my paper · ORCID

classification cs.LG

keywords conformalpredictionofflinereinforcementlearningdeepQ-learningmechanicalventilationuncertaintyquantificationdistributionshiftintensivecaredecisionsupport90-daysurvival

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a distribution-free conformal prediction layer can solve a specific safety problem in offline reinforcement learning for intensive care: the tendency of a deep Q-agent to act on overestimated values in unfamiliar patient states. ConformalDQN learns action probabilities for the clinician behavior policy, calibrates a threshold on held-out data, and refuses to consider any action whose probability falls below that threshold unless no action qualifies. In the paper's evaluation on a large ICU dataset, this filtering raised the estimated 90-day survival rate to 83.9%, above the 81.65% of a conservative Q-learning baseline and the 74.9% of physician practice, while keeping recommended ventilator settings inside lung-protective ranges. The authors present the method as a step toward clinically usable decision support with an interpretable confidence measure for each recommendation.

What carries the argument

The load-bearing object is the conformal confidence threshold $\tau$ and the confident-action set it defines. Calibrated with $n$ held-out state-action pairs at confidence level $1-\alpha$, the threshold guarantees $P(a^* \in \{a : P_\omega(a|s) \geq 1-\tau\}) \geq 1-\alpha$ under exchangeability, where $a^*$ is the expert action. At decision time the agent takes $\arg\max_{a \in A_c} Q(s,a)$ when $A_c$ is nonempty and falls back to $\arg\max_a Q(s,a)$ otherwise; the composite training loss combines Double DQN loss, negative log-likelihood of expert actions, and L2 regularization on logits to keep the probability head well calibrated. The threshold thus acts as a tunable, post-training filter that blocks high-Q but low-confidence actions.

What would settle it

Using the paper's calibration set and threshold, compute how often the expert's actual action falls in the confident set on the deliberately unusual out-of-distribution patients; if the rate is well below the target confidence, the out-of-distribution safety mechanism is not doing what is claimed.

Watch

Extended reading notes

Core claim

The central claim is that the conformal threshold, applied to the action probabilities of the behavioral policy, is a sufficient safety mechanism for offline deep Q-learning in mechanical ventilation. ConformalDQN extends Double DQN with a probability head $P_\omega(a|s)$ trained to imitate the clinician behavior policy, computes the threshold $\tau$ as the $\lceil(n+1)(1-\alpha)\rceil/n$ quantile of the nonconformity scores $1-P_\omega(a_i|s_i)$ on a calibration set, and restricts action selection to the confident set $\{a : P_\omega(a|s) \geq 1-\tau\}$. The authors report that this mechanism yields the highest mean initial Q-value (0.639), the strongest negative correlation between predicted Q-values and mortality ($-0.563$), and stable Q-values under an out-of-distribution test, and they interpret these results as evidence that the method improves expected 90-day survival and avoids unsafe actions.

Load-bearing premise

The safety claim rests on the assumption that a threshold tuned on typical patients also protects deliberately unusual patients, even though the statistical guarantee only covers patients drawn from the same distribution as the tuning data.

Editorial extensions

If this is right

If the reported Q-value estimates are right, ConformalDQN recommendations correspond to an estimated 90-day survival of 83.9%, a gain over both the CQL baseline (81.65%) and observed physician practice (74.9%).
Clinicians could adjust the confidence threshold after training to shift the agent between conservative and exploratory behavior without retraining the network.
The method's output includes an interpretable confidence set of acceptable actions, which is the kind of information a human-in-the-loop decision support system needs.
In the paper's OOD evaluation, ConformalDQN keeps mean initial Q-values below the overestimation threshold while standard DDQN exceeds it, supporting the claim that uncertainty filtering mitigates distribution shift.
The same architecture can be transferred to other treatment decisions such as drug dosing, where offline data and safety constraints dominate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors list state-conditioned conformal prediction as future work; if realized, it would replace the current population-level coverage guarantee with per-patient confidence sets, which is what a clinician would need at the bedside.
The filter-plus-maximization pattern does not depend on Double DQN specifically; any offline RL method that can estimate behavior-policy action probabilities could wear the same conformal safety layer, including continuous-action variants the authors suggest.
Because the reported survival rates are estimated by Fitted Q-Evaluation rather than observed outcomes, an independent off-policy evaluator or a prospective pilot is the natural next check on the 83.9% figure.
The survival comparison would be sharper if future work reports observed 90-day mortality on the test cohort directly, rather than mapping Q-values through binned physician outcomes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Honest take: the conformal-DQN integration is real and the mechanics are sound, but the paper overclaims the survival benefit. The headline numbers are FQE estimates mapped through a physician-only calibration curve; that mapping is doing real work and is not validated. There is a legitimate kernel here: the composite loss tying the Q-network to a behavioral probability estimator, the post-training threshold filter, and the OOD stability check all make sense as a method contribution for trustworthy offline RL in critical care.

What is actually new: combining split conformal calibration with a Double DQN for ventilator settings, and using the calibrated threshold to restrict the action set at deployment. The method is an incremental variant, not a breakthrough, but it is clearly described and the data pipeline follows DeepVent closely. The paper gives credit to prior work and the code is available.

Where it is soft. First, the 90-day survival claim. Table 2 is not a measured outcome. It takes FQE initial-state Q-values, bins the physician policy's Q-values, computes observed survival in those bins, then maps each policy's mean Q through that function. That assumes a single Q-to-survival calibration transfers across policies whose action distributions and Q-scales differ. No confidence intervals, no independent estimator. The stress-test note is right that the mapping is numerically inconsistent with the reward structure. Second, the conformal guarantee is only marginal over the calibration distribution; it does not extend to the deliberately shifted OOD set, so the sentence claiming protection against out-of-distribution states is too strong. Third, several hyperparameters (α, λ, L2 weight) are tuned on validation and not fully reported; reproducibility is partial.

The method itself is sensible, and the OOD figure showing stable Q-values below the overestimation threshold is a fair check. The paper needs a revision that softens the claims, reports conditional or OOD-robust conformal methods, and validates the survival estimator. I would not cite the survival improvement as a result, but I would cite the method as an example of conformal action filtering.

Who should read it: people working on offline RL safety in healthcare and on conformal prediction for decision support. It deserves serious peer review; the flaws are fixable and the topic is important. I would accept it for review with major revision expectations.

Referee Report

5 major / 6 minor

Summary. The paper proposes ConformalDQN, a framework that augments Double DQN with an estimate of the behavioral policy's action probabilities P_ω(a|s), a conformal threshold τ computed on a calibration set, and an action-selection rule that restricts Q-value maximization to actions with probability at least 1−τ. The method is trained and evaluated on a MIMIC-IV cohort of mechanically ventilated patients, and the paper claims improved 90-day survival relative to physician practice, CQL, and DDQN, as well as robustness to out-of-distribution states. The evaluation uses Fitted Q-Evaluation (FQE), correlation analyses, action-distribution plots, and mean initial Q-values on in-distribution and deliberately constructed out-of-distribution subsets.

Significance. If the central claims were established, the framework would be a useful contribution to safe offline RL in clinical decision support: it provides a distribution-free calibration procedure for action filtering, releases code, and targets a clinically meaningful problem. However, the headline survival benefit is not supported by the evidence: Table 2 is an indirect FQE-based mapping, not a measurement of outcomes under the learned policies, and the conformal guarantee in Eq. (12) is a marginal coverage statement about behavioral actions, not a safety guarantee for the chosen Q-optimal action or for out-of-distribution states. The paper's contribution is best assessed as a modest action-filtering mechanism with a correctly stated but limited conformal guarantee; the clinical and OOD-safety claims require substantial additional work.

major comments (5)

[Evaluation, Table 2] The reported 90-day survival rates are not measured outcomes. They are obtained by fitting FQE on the same MIMIC-IV data and the same sparse terminal reward, binning physician Q-values, and mapping each policy's mean initial Q-value through that binning function. FQE is an approximate off-policy estimator with known bias in offline RL, and the paper provides no confidence intervals, no importance-sampling estimator, and no direct validation of the Q-to-survival calibration. Furthermore, with the terminal reward in Eq. (8) (+1 survive, −1 die), a mean Q of 0.639 implies expected survival probability (0.639+1)/2 = 0.8195, not the reported 83.89%, so the physician-Q binning mapping is doing unexplained numerical work. The abstract's claim that ConformalDQN 'outperforms other methods by increasing the 90-day survival rate' is therefore unsupported by the evidence presented.
[Conformal predictor calibration, Eq. (12)] Equation (12) states P(a* ∈ {a : P_ω(a|s) ≥ 1−τ}) ≥ 1−α for 'an optimal behavioral action', but the conformal guarantee is marginal over the calibration distribution and only covers inclusion of a behavioral action drawn exchangeably from that distribution. It does not cover inclusion of the argmax-Q action, nor does it guarantee that the selected action is safe or clinically appropriate, and the phrase 'optimal behavioral action' is never defined. In the OOD evaluation, the test distribution is deliberately shifted by selecting patients with top or bottom 1% initial features, so exchangeability between calibration and test fails and τ has no stated coverage guarantee. The claim that the filter avoids potentially harmful actions in unfamiliar states is not supported by Eq. (12).
[Uncertainty-Aware Action Selection, Eq. (14)] The fallback rule in Eq. (14), which selects arg max_a Q(s,a) when A_c is empty, removes the conformal filter precisely in unfamiliar states where no action meets the probability threshold. There is no coverage or safety guarantee for this fallback, and it can select actions with low behavioral probability and potentially high extrapolation error exactly when the model is least reliable. This undercuts the stated safety motivation and should either be removed, analyzed, or replaced with an explicit abstention mechanism.
[Out-of-Distribution Evaluation, Figure 4] The 'overestimation threshold' set at 1.0 is not a valid threshold for the objective actually trained. The maximum return is 1 only in the absence of intermediate rewards and with discount factor γ = 1, whereas the experiments use γ = 0.75 and the reward in Eq. (8) includes intermediate rewards weighted by λ. Comparing mean initial Q-values against this fixed line therefore conflates scale differences with overestimation and does not establish that standard DDQN overestimates or that CQL and ConformalDQN do not.
[RL Problem Formulation, Eq. (8)] The reward function depends on a 'modified APACHE II score' AP(s), but the paper never defines how this score is computed from the 44 state variables, which variables enter it, or how the modification differs from APACHE II. Without this definition, the reward, all learned Q-values, and Table 2 cannot be reproduced or audited, and the claim that the reward aligns with 90-day survival cannot be checked.

minor comments (6)

[Loss function, Eq. (9)] The text says 'L_θ is the standard Double DQN loss describes as eq. 9', but the Double DQN loss is defined in Eq. (3), not Eq. (9); this cross-reference should be corrected.
[Methods and Appendix] There are several typographical errors, including 'Correspondance', 'weight-asjusted', 'loctate', and 'V olume'; the paper should be carefully proofread.
[Experimental Setups] The composite loss in Eq. (9) contains an NLL term and an L2 regularization term, but the grid search lists learning rate, discount factor, layer count, and units only; the values of λ, the L2 regularization weight, and any weight on the NLL term are not reported, making the training configuration incompletely specified.
[Evaluation] The FQE estimator is described only by a reference; the paper should report the FQE architecture, training details, target update procedure, and whether FQE was trained on the same train/calibration splits as the policies being evaluated.
[Data split and OOD selection] The OOD dataset is described as consisting of patients with top or bottom 1% initial features, but it is not stated whether these patients are disjoint from the training, calibration, and test splits; if the OOD selection is performed on the full cohort before splitting, information leakage and overlapping evaluation sets are possible and should be clarified.
[Table 2] Table 2 reports mean initial Q-values with standard deviations for five runs, but the corresponding survival-rate columns are shown without any uncertainty intervals; given that the survival mapping is nonlinear, the standard deviation of the survival estimate should be reported or propagated.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 90-day survival comparison reduces to a fitted Q-to-survival calibration on the same FQE Q-values used to rank policies; the conformal contribution itself is not circular.

fitted input called prediction [Results section, paragraph preceding Table 2 and Table 2 itself]
"We binned the physician’s Q-values into intervals, calculated the observed 90-day survival rate for each bin, and mapped the mean Q-values of each policy to corresponding survival rates for interpretability."

The 'Survival Rate' column in Table 2 is not a measured outcome of ConformalDQN, DeepVent(CQL), or the physician policy; it is constructed by fitting a Q-to-survival binning function on physician FQE Q-values and then reading each policy's survival rate off its mean initial Q-value. Because the FQE Q-values are trained with the same sparse terminal reward (+1 survive, -1 die within 90 days), the reported survival percentages are a transformation of the very Q-values being compared. The abstract's claim that ConformalDQN 'outperforming other methods by increasing the 90-day survival rate' therefore reduces by construction to the claim that ConformalDQN has a higher mean FQE Q-value; no direct outcome evaluation or independent off-policy estimator is reported.

full rationale

The paper's conformal mechanism is not circular in the sense of this review: Eq. 12 is a direct consequence of the quantile definition in Eq. 11 under exchangeability, and the paper makes no load-bearing use of self-citations or imported uniqueness theorems. The central circularity is the survival-rate comparison. Table 2 derives the 90-day survival percentages by binning the physician policy's FQE Q-values, computing observed mortality per bin, and mapping each policy's mean Q-value through that fitted curve. Since the same FQE Q-values were trained with a terminal reward defined by 90-day survival, the 'predicted' survival rates are fitted values rather than independent measurements; the ranking of policies on 'Survival Rate' is forced by the ranking of their mean Q-values. This is the load-bearing evidence for the abstract's survival-improvement claim. A secondary concern, noted for completeness rather than counted as circularity, is that FQE is an approximate off-policy estimator and the paper reports no confidence intervals or validation of the Q-to-survival transfer; that is a correctness/validity risk, not a circularity. Accordingly, the score reflects partial circularity: one central prediction reduces by construction, while the conformal uncertainty-quantification contribution retains independent content.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The method imports the data pipeline, reward, and clinical proxy from Kondrup et al. and Peine et al., adds a behavioral policy network Pω trained with a cloning loss, and evaluates with an FQE model trained on the same data. The most consequential fitted quantities are α, γ, and the unspecified FQE configuration, which together determine the reported survival estimates. No new physical or clinical entities are introduced; the conformal filter is a decision rule, not a new entity.

free parameters (5)

α (conformal significance level) = 0.15
Selected by grid search on the validation set to maximize the reported performance; the paper presents it as a fixed confidence level, but tuning it on validation weakens the conformal guarantee's transfer to the test set.
discount factor γ = 0.75
Tuned by grid search; enters every Q-value and the FQE comparisons.
λ (intermediate reward weight) = not reported
Imported from Kondrup et al. into Eq. 8; controls the balance between immediate physiological change and terminal survival in the reward.
L2 regularization weight in Eq. 9 = not reported
The loss includes a 'λ||i||2' term but the paper never states this λ's value, making part of the training objective unspecified.
FQE configuration = not reported
The architecture, discount factor, and training details of the Fitted Q-Evaluation model that produces the survival estimates are not described, yet the central comparison depends on it.

assumptions (6)

domain assumption Exchangeability of calibration and test states
Split conformal prediction requires exchangeability; the OOD evaluation (top/bottom 1% features) deliberately violates this, so the coverage guarantee in Eq. 12 does not extend to those states.
domain assumption Patient state is Markovian for ventilator decisions
The MDP formulation assumes the 44 variables summarize all clinically relevant history for the Bellman updates.
domain assumption The reward function is a valid clinical proxy
The paper adopts the Kondrup et al. reward combining modified APACHE-II change and 90-day survival; the policy optimizes this proxy, not a validated outcome measure.
domain assumption FQE gives unbiased value estimates for the compared policies
The ranking of policies relies on Fitted Q-Evaluation, which is sensitive to distribution shift and model misspecification; the paper does not address these failure modes.
ad hoc to paper The modified APACHE II score is computable from extracted features
The paper defines 'AP' as a modified APACHE II score but does not specify the modification or its computation; every intermediate reward in Eq. 8 depends on it.
domain assumption Maximum discounted return equals 1 for the overestimation threshold
The paper declares Q-values above 1.0 to be overestimation, but with any discount factor below 1 the maximum discounted terminal reward is below 1; the FQE discount is not stated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework." pith.science (2026). https://pith.science/paper/KMAQKNVU

@misc{pith2026241212597,
  author       = {Pith},
  title        = {Pith review of: Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KMAQKNVU}},
  note         = {Machine review of arXiv:2412.12597}
}

read the original abstract

Mechanical Ventilation (MV) is a critical life-support intervention in intensive care units (ICUs). However, optimal ventilator settings are challenging to determine because of the complexity of balancing patient-specific physiological needs with the risks of adverse outcomes that impact morbidity, mortality, and healthcare costs. This study introduces ConformalDQN, a novel distribution-free conformal deep Q-learning approach for optimizing mechanical ventilation in intensive care units. By integrating conformal prediction with deep reinforcement learning, our method provides reliable uncertainty quantification, addressing the challenges of Q-value overestimation and out-of-distribution actions in offline settings. We trained and evaluated our model using ICU patient records from the MIMIC-IV database. ConformalDQN extends the Double DQN architecture with a conformal predictor and employs a composite loss function that balances Q-learning with well-calibrated probability estimation. This enables uncertainty-aware action selection, allowing the model to avoid potentially harmful actions in unfamiliar states and handle distribution shifts by being more conservative in out-of-distribution scenarios. Evaluation against baseline models, including physician policies, policy constraint methods, and behavior cloning, demonstrates that ConformalDQN consistently makes recommendations within clinically safe and relevant ranges, outperforming other methods by increasing the 90-day survival rate. Notably, our approach provides an interpretable measure of confidence in its decisions, which is crucial for clinical adoption and potential human-in-the-loop implementations.

Figures

Figures reproduced from arXiv: 2412.12597 by the authors.

**Figure 2.** Overview of the Method. (1)Preparing a structured dataset of past experiences.(2) Training a Double Deep Q-Network [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 4.** Comparison of mean initial Q-values for in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 3.** Figure 3: Distribution of ventilator settings chosen by dif [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 22 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

N.; and Bates, S

Angelopoulos, A. N.; and Bates, S. 2021. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511

arXiv 2021
[4]

Balasubramanian, V.; Ho, S.-S.; and Vovk, V. 2014. Conformal prediction for reliable machine learning: theory, adaptations and applications. Newnes

work page 2014
[5]

Coppola, S.; Froio, S.; and Chiumello, D. 2014. Protective lung ventilation during general anesthesia: is there any evidence? Critical Care, 18: 1--7

work page 2014
[6]

Coronato, A.; Naeem, M.; De Pietro, G.; and Paragliola, G. 2020. Reinforcement learning for intelligent healthcare applications: A survey. Artificial intelligence in medicine, 109: 101964

work page 2020
[7]

Fujimoto, S.; Meger, D.; and Precup, D. 2019. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, 2052--2062. PMLR

work page 2019
[8]

Gottesman, O.; Johansson, F.; Komorowski, M.; Faisal, A.; Sontag, D.; Doshi-Velez, F.; and Celi, L. A. 2019. Guidelines for reinforcement learning in healthcare. Nature medicine, 25(1): 16--18

work page 2019

Show all 33 references

[9]

Jingkun, M.; Fengxi, L.; Chunxin, L.; and Pixuan, Z. 2024. Ventilator Treatment Policy Control based on BCQ off-line Deep Reinforcement Learning

2024
[10]

A.; and Mark, R

Johnson, A.; Bulgarelli, L.; Pollard, T.; Horng, S.; Celi, L. A.; and Mark, R. 2020. Mimic-iv. PhysioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021), 49--55

2020
[11]

A.; Wagner, D

Knaus, W. A.; Wagner, D. P.; Draper, E. A.; Zimmerman, J. E.; Bergner, M.; Bastos, P. G.; Sirio, C. A.; Murphy, D. J.; Lotring, T.; Damiano, A.; et al. 1991. The APACHE III prognostic system: risk prediction of hospital mortality for critically III hospitalized adults. Chest, ...

1991
[12]

D.; Precup, D.; and Basu, S

Kondrup, F.; Jiralerspong, T.; Lau, E.; de Lara, N.; Shkrob, J.; Tran, M. D.; Precup, D.; and Basu, S. 2023. Towards safe mechanical ventilation treatment using deep offline reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 15...

2023
[13]

Kumar, A.; Fu, J.; Soh, M.; Tucker, G.; and Levine, S. 2019. Stabilizing off-policy q-learning via bootstrapping error reduction. Advances in neural information processing systems, 32

2019
[14]

Kumar, A.; Zhou, A.; Tucker, G.; and Levine, S. 2020. Conservative q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems, 33: 1179--1191

2020
[15]

Lange, S.; Gabel, T.; and Riedmiller, M. 2012. Batch reinforcement learning. In Reinforcement learning: State-of-the-art, 45--73. Springer

2012
[16]

Le, H.; Voloshin, C.; and Yue, Y. 2019. Batch policy learning under constraints. In International Conference on Machine Learning, 3703--3712. PMLR

2019
[17]

S.; Mahendra, M.; and Aswani, A

Lee, J. S.; Mahendra, M.; and Aswani, A. 2024. Methodology for Interpretable Reinforcement Learning for Optimizing Mechanical Ventilation. arXiv preprint arXiv:2404.03105

2024 arXiv
[18]

Levine, S.; Kumar, A.; Tucker, G.; and Fu, J. 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643

2020 arXiv
[19]

A.; Veness, J.; Bellemare, M

Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Human-level control through deep reinforcement learning. nature, 518(7540): 529--533

2015
[20]

M \"o hlenkamp, S.; and Thiele, H. 2020. Ventilation of COVID-19 patients in intensive care units. Herz, 45(4): 329--331

2020
[21]

B.; Schmeink, A.; Ascheid, G.; Thiemermann, C.; Schuppert, A.; Kindle, R.; et al

Peine, A.; Hallawa, A.; Bickenbach, J.; Dartmann, G.; Fazlic, L. B.; Schmeink, A.; Ascheid, G.; Thiemermann, C.; Schuppert, A.; Kindle, R.; et al. 2021. Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical ...

2021
[22]

J.; and Slutsky, A

Pham, T.; Brochard, L. J.; and Slutsky, A. S. 2017. Mechanical ventilation: state of the art. In Mayo Clinic Proceedings, volume 92, 1382--1400. Elsevier

2017
[23]

Shafer, G.; and Vovk, V. 2008. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3)

2008
[24]

Silva, P.; Rocco, P.; and Pelosi, P. 2022. Personalized mechanical ventilation settings: Slower is better! In Annual Update in Intensive Care and Emergency Medicine 2022, 113--127. Springer

2022
[25]

S.; and Ranieri, V

Slutsky, A. S.; and Ranieri, V. M. 2013. Ventilator-induced lung injury. New England Journal of Medicine, 369(22): 2126--2136

2013
[26]

Suo, D.; Agarwal, N.; Xia, W.; Chen, X.; Ghai, U.; Yu, A.; Gradu, P.; Singh, K.; Zhang, C.; Minasyan, E.; et al. 2021. Machine learning for mechanical ventilation control. arXiv preprint arXiv:2102.06779

2021 arXiv
[27]

S.; and Barto, A

Sutton, R. S.; and Barto, A. G. 1998. Reinforcement learning: an introduction MIT Press. Cambridge, MA, 22447: 10

1998
[28]

Van Hasselt, H.; Guez, A.; and Silver, D. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30

2016
[29]

Vazquez, J.; and Facelli, J. C. 2022. Conformal prediction in clinical medical sciences. Journal of Healthcare Informatics Research, 6(3): 241--252

2022
[30]

J.; and Dayan, P

Watkins, C. J.; and Dayan, P. 1992. Q-learning. Machine learning, 8: 279--292

1992
[31]

Yu, C.; Liu, J.; Nemati, S.; and Yin, G. 2021. Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR), 55(1): 1--36

2021
[32]

Yuan, Y.; Shi, J.; Yang, J.; Li, C.; Cai, Y.; and Tang, B. 2023. Conservative Q-Learning for Mechanical Ventilation Treatment Using Diagnose Transformer-Encoder. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2346--2351. IEEE

2023
[33]

Zein, H.; Baratloo, A.; Negida, A.; and Safari, S. 2016. Ventilator weaning and spontaneous breathing trials; an educational review. Emergency, 4(2): 65

2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

N.; and Bates, S

Angelopoulos, A. N.; and Bates, S. 2021. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511

arXiv 2021

[4] [4]

Balasubramanian, V.; Ho, S.-S.; and Vovk, V. 2014. Conformal prediction for reliable machine learning: theory, adaptations and applications. Newnes

work page 2014

[5] [5]

Coppola, S.; Froio, S.; and Chiumello, D. 2014. Protective lung ventilation during general anesthesia: is there any evidence? Critical Care, 18: 1--7

work page 2014

[6] [6]

Coronato, A.; Naeem, M.; De Pietro, G.; and Paragliola, G. 2020. Reinforcement learning for intelligent healthcare applications: A survey. Artificial intelligence in medicine, 109: 101964

work page 2020

[7] [7]

Fujimoto, S.; Meger, D.; and Precup, D. 2019. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, 2052--2062. PMLR

work page 2019

[8] [8]

Gottesman, O.; Johansson, F.; Komorowski, M.; Faisal, A.; Sontag, D.; Doshi-Velez, F.; and Celi, L. A. 2019. Guidelines for reinforcement learning in healthcare. Nature medicine, 25(1): 16--18

work page 2019