Pith. sign in

REVIEW 4 major objections 5 minor 68 references

MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representation

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that adding LLM-encoded clinical notes to the state representation of an offline reinforcement-learning agent yields better sepsis treatment policies than using structured data alone, with gains shown on three ICU datasets

desk verdict Plausible architecture for adding clinical notes to offline RL, but the evaluation does not support the headline claim and there is a possible temporal leakage of the terminal outcome into the note features. read the letter →

arxiv 2508.07681 v1 pith:E2PBSEOG submitted 2025-08-11 cs.LG cs.AI

Yooseok Lim , ByoungJun Jeon , Seong-A Park , Jisoo Lee , Sae Won Choi , Chang Wook Jeong , Ho-Geol Ryu , Hongyeol Lee
show 1 more author
Hyun-Lim Yang
This is my paper
classification cs.LGcs.AI
keywords sepsisofflinereinforcementlearningmultimodalstaterepresentationclinicalnoteslargelanguagemodelsconservativeQ-learningMIMIC-IIIMIMIC-IV
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an offline reinforcement-learning agent for sepsis treatment learns better policies when its state contains LLM-encoded clinical notes alongside lab values and vital signs. The proposed framework, MORE-CLEAR, summarizes notes with an LLM, treats the first note as a persistent context vector, and fuses text and structured data through gated fusion and bidirectional cross-modal attention before feeding the state into conservative Q-learning. On MIMIC-III, MIMIC-IV, and a private ICU dataset, multimodal policies score higher than single-modality baselines on off-policy evaluation metrics. The authors also introduce a Behavioral Discrepancy Estimated Survival Rate (BDESR) that reports higher survival in episodes where clinicians' actions closely matched the policy's recommendations. A sympathetic reader would take this as evidence that unstructured clinical text, which most sepsis RL systems discard, carries decision-relevant signal for learning treatment policies.

What carries the argument

The central object is the fused multimodal state representation. Clinical notes are first summarized by an LLM (Gemma-3-27B-it) and encoded into dense vectors; the initial note becomes a context vector. A gated fusion mechanism computes a learnable sigmoid gate between the context vector and the current observation note. A bidirectional cross-modal attention module lets the structured embedding and the note embedding attend to each other in both directions, producing a final state vector $s = [\tilde{\ell}; \tilde{n}] \in \mathbb{R}^{2d}$. This state feeds a Conservative Q-Learning policy, whose regularization suppresses out-of-distribution overestimation.

What would settle it

Conduct a matched-cohort or prospective evaluation in which patients whose clinicians followed MORE-CLEAR's recommendations are compared with patients receiving standard care, controlling for SOFA and other acuity measures; if the survival benefit disappears once acuity is controlled, the central claim fails. A cheaper check is to recompute BDESR on a held-out split with propensity-matching on severity and see whether the low-discrepancy survival advantage persists.

Watch

Extended reading notes

Core claim

MORE-CLEAR's central claim is that multimodal state representation materially improves offline RL policies for sepsis management. The framework encodes structured observations with an MLP-style encoder, encodes clinical notes with a frozen pretrained LLM after structured summarization, and combines the two streams using a gated fusion that injects the initial note as a persistent context vector plus a bidirectional cross-modal attention module. The resulting fused state is fed into Conservative Q-Learning. Across two public datasets and one private Korean ICU cohort, the multimodal configurations (Bert+CQL, CB+CQL, Llama+CQL) outperform unimodal baselines on OPERA, DR, FQE, and mostly WIS me

Load-bearing premise

The evaluation assumes that the off-policy estimators and the BDESR survival gap—comparing episodes with high versus low divergence from the policy's actions—accurately reflect true clinical benefit, even though that gap is confounded by patient acuity and by the policy having been trained on those same patient trajectories.

Editorial extensions

If this is right

  • LLM-encoded clinical notes can serve as a practical input modality for ICU offline RL, not just for prediction tasks.
  • Multimodal policies trained with MORE-CLEAR generalize across hospital systems and time periods; the MIMIC-IV-trained policy showed the strongest cross-dataset transfer.
  • Encoding the initial clinical note as a persistent context vector is a concrete way to inject episode-level history into every decision point.
  • Clinical notes help compensate for missing and irregularly sampled structured data.
  • The framework is designed to extend beyond sepsis to other sequential critical-care decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the BDESR gap is real and causal, it suggests that clinicians already acting in line with the learned policy are doing better, which would make deployment less risky than in settings where the policy recommends unfamiliar actions; the paper does not establish this causality.
  • Because text-only policies underperform structured-only policies on several metrics, the value of MORE-CLEAR may come less from standalone text than from cross-modal fusion stabilizing Q-value estimation, as the Bellman-residual analysis hints.
  • A testable extension would be to vary the quality of note summarization; if summaries lose clinically relevant nuance, raw notes with longer context or a different LLM could change the conclusions.
  • The high-BDESR survival gap may partly reflect that sicker patients receive more intensive treatment and therefore deviate more from any policy, so severity matching could separate policy signal from acuity confound.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MORE-CLEAR, a multimodal offline reinforcement learning framework for sepsis treatment in the ICU. The method augments structured EHR features (labs and vitals) with LLM-summarized clinical notes, using a context-aware gated fusion of the initial note and time-specific notes, followed by bidirectional cross-modal attention. The resulting state representation is fed into offline RL algorithms (CQL, BCQ). Evaluation is carried out on MIMIC-III, MIMIC-IV, and a private ICU dataset, using OPE metrics (OPERA, DR, FQE, WIS), a newly proposed BDESR metric, cross-dataset validation, and Bellman-residual analysis. The paper claims that adding LLM-encoded clinical notes significantly improves estimated survival and policy performance compared to unimodal baselines.

Significance. If the central claim were established, the paper would make a useful contribution to clinically oriented offline RL: it is among the first to use LLM-based note summarization for state representation in medical offline RL, it spans two public datasets and a private cohort, and the framework components (gated fusion, cross-modal attention) are clearly described and ablated. The authors also provide an anonymous code link, which is helpful. However, the evaluation as presented does not currently support the main claim. Two load-bearing issues stand out: (1) a plausible temporal label-leakage mechanism involving future notes (e.g., discharge summaries) that would directly encode the terminal reward, and (2) the BDESR metric, which is circular because the 'low discrepancy' cohort is defined by agreement with a policy trained on the same episodes and is confounded by patient acuity. In addition, the OPE results are internally weak (near-ties on MIMIC-IV and very high FQE variance). The paper is therefore promising but needs substantial revision before its claims can be accepted.

major comments (4)
  1. [§4.4 and §5.1.2] Temporal integrity of the clinical-note state is not established. Section 5.1.2 says clinical notes include 'nursing records, physician documentation, discharge summaries, and other information,' and each episode spans 24h before to 48h after sepsis onset. Section 4.4 constructs trajectories 'by concatenating all notes recorded within fixed time intervals,' but the paper does not state that each 4-hour state contains only notes authored at or before that state's timestamp. If a discharge summary (which summarizes the hospitalization and typically indicates survival or death) is recorded within the episode window and is fed into the state, the terminal reward (+1/−1 for 90-day survival) is directly embedded in the state. The RL agent could then achieve high OPE values by exploiting this shortcut rather than by learning a clinically meaningful treatment policy. This would also explain why
  2. [§5.2.1, Eqs. (15)–(18)] The BDESR metric cannot support the paper's clinical-effectiveness claims. The metric stratifies episodes by discrepancy between the learned policy and clinician actions, then compares survival in the low- and high-discrepancy cohorts. Because the policy was trained on these same episodes, the survival gap is self-referential: a policy that memorizes favorable trajectories will have low discrepancy in survivors precisely because those trajectories were used to train it, not because the policy causes better outcomes. The comparison is also confounded by patient acuity: sicker patients tend to receive more variable and more intensive treatments and have worse survival, so high discrepancy and high mortality may be jointly driven by severity. The manuscript's own interpretation in §6.1 ('likely attributable to including a greater number of trajectories that culminated in mortality') acknowl
  3. [Table 2 and §6.1] The claim that MORE-CLEAR 'significantly improves' OPE metrics is not supported by the reported numbers. On MIMIC-IV, OPERA for structured CQL alone is 3.862 ± 0.04, while multimodal configurations are 3.861, 3.877, and 3.810 — statistically indistinguishable. WIS gains are small (0.753 → 0.766) and overlap across seeds. FQE for CB+CQL on MIMIC-IV is 7.522 ± 3.33, an enormous standard deviation that makes the 'outstanding performance' claim untenable. No significance tests, confidence intervals, or seed-level distributions are provided despite the contribution statements claiming 'statistically significant improvement.' I request that the authors (i) report per-seed results or confidence intervals, (ii) perform paired significance tests against structured-data-only CQL, and (iii) include a sanity check of the OPE estimators (e.g., evaluating the behavior policy and comparing with the emp
  4. [Table 4 and §6.3] The cross-dataset validation in Table 4 is reported only for the multimodal CB+CQL model, with no unimodal structured-data baseline. As a result, the external-validation section cannot be used to support the contribution that multimodal state representation improves generalization. To substantiate the generalization claim, the same cross-dataset protocol should be applied to structured-data-only CQL (and ideally text-only) under identical tuning. The current table can only show that CB+CQL transfers at some absolute level, not that the multimodal representation is responsible for the transfer.
minor comments (5)
  1. [Table 7 (Appendix C)] The entry '6.495 ± 5,347' appears to be a typo for '5.347' (or '5.347' with a comma). Please correct the decimal separator.
  2. [§6.2] The text says 'window size(W )=3; see Appendix B', but the window-size ablation is in Appendix C, not Appendix B. Also, the relationship between 'W=3' and '12 hours' should be stated explicitly (4-hour intervals × 3).
  3. [Table 3 caption] In the table header, 'BCQ [4]' is used for the structured BCQ column, but reference [4] is the sepsis RL paper by Choi et al., not the original BCQ paper. Please use the correct citation for BCQ [12].
  4. [§5.1.2] The episode definition says '24 hours prior to 48 hours after the suspected onset of sepsis,' but it is not specified whether the episode terminal step T is at 48 hours after onset or at ICU/discharge time. This matters for interpreting the terminal reward in Eq. (14) and for the OPE evaluation horizon.
  5. [§4.4] The phrase 'the structured text is fed into the LLMs, which employ average pooling on the hidden state of the final layer' is ambiguous about which LLM is used for embedding versus summarization. Please clarify whether the embedding LLM is the same Gemma-3-27B-it used for summarization or a separate encoder (e.g., ClinicalBERT or Llama3.1-8B).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MORE-CLEAR's central claims rest on external OPE comparisons and ablations, not on a derivation whose inputs already contain the conclusion.

full rationale

MORE-CLEAR is an empirical systems paper rather than a formal derivation. The state representation is built from raw clinical notes via LLM summarization (Section 4.4), context-aware gated fusion (Section 4.5), and bidirectional cross-modal attention (Section 4.6); none of these components is defined in terms of the OPE metrics or the survival outcome. The main claim that adding clinical notes improves policy performance is supported by comparisons to unimodal baselines on MIMIC-III, MIMIC-IV, and a private dataset (Table 2), using standard off-policy estimators (WIS, DR, FQE, OPERA). These estimators are not constructed in a way that forces the multimodal advantage. Self-citations (e.g., [30,31]) appear only as background on RL in clinical settings and are not load-bearing for the multimodal contribution; no uniqueness theorem or ansatz is imported from the authors' prior work. The BDESR metric (Section 5.2.1, Eqs. 17-18) is self-referential as an evaluation tool—the survival labels in Eq. 18 are exactly the terminal rewards from Eq. 14, so comparing survival across policy-discrepancy cohorts partly re-reads the training signal rather than independently validating clinical benefit—and the inclusion of discharge summaries in the note trajectories (Section 5.1.2) raises a temporal leakage risk. However, these are threats to evaluation validity, not instances where a stated prediction is equivalent to its inputs by construction; the central multimodal result retains independent empirical content. Therefore no significant circularity is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the validity of OPE metrics and the new BDESR metric, neither of which is independently validated. The free parameters listed are hyperparameters that influence the reported results without sensitivity analysis. No new physical entities are introduced; the BDESR is a novel evaluation construct.

free parameters (4)
  • BDESR percentile p = 20
    The percentile cutoff for defining low/high discrepancy cohorts in Eq. 17 is a hyperparameter chosen without sensitivity analysis; it determines which episodes enter the survival comparison.
  • BDESR discrepancy weights alpha, beta = not reported (alpha + beta = 1)
    Weights for IV fluid vs vasopressor discrepancy are chosen with alpha+beta=1 but their values are not reported or optimized; they affect the discrepancy scores and thus the cohort definitions.
  • CQL regularization coefficient = 2.0
    This offline RL hyperparameter is fixed for all models and directly affects the conservative Q-value penalty; no sensitivity analysis is provided, and results may depend on it.
  • Context window size W = 3
    The stack-window ablation in Appendix C selects W=3 for the best OPERA/DR; this choice is made from the same test sets and is a free parameter tuned on evaluation metrics.
assumptions (4)
  • domain assumption OPE estimators (WIS, DR, FQE, OPERA) are treated as reliable proxies for true policy performance.
    Section 5.2 and 6.1 base the main claim on these estimates without comparison to a ground-truth policy value; OPE is known to be biased in high-dimensional state spaces.
  • domain assumption The initial clinical note in an episode provides a stable context vector that does not leak future outcomes.
    Section 4.5 uses oMn_0 as context, but the paper never defines which note is 'initial' nor checks missingness or content; if the first note contains admission information only, it's fine, but if it contains later documentation or if no note exists at t=0, the context vector construction is undefined.
  • domain assumption The 90-day survival reward at the terminal step is a valid and available outcome for all episodes.
    Section 5.1.2 defines reward; for patients still hospitalized at T, survival at 90 days after discharge is not yet known, so labels may be proxied or censored, which is not discussed.
  • domain assumption LLM summarization preserves the clinically relevant information from raw notes.
    Section 4.4 uses Gemma-3-27B-it to summarize notes, but no evaluation is done to verify that summaries retain the information needed for RL decisions; the LLM selection in Appendix B is based on QA and summarization benchmarks, not on downstream task performance.
invented entities (1)
  • BDESR metric (Behavioral Discrepancy Estimated Survival Rate)
    purpose: To estimate the clinical effectiveness of an RL policy by comparing survival between episodes with low and high policy-clinician action discrepancy.
    This is a new metric defined in Section 5.2.1. It has no external validation, and the causal interpretation (that high discrepancy indicates the policy is avoiding harmful actions) is unsupported. It is a correlational statistic, not an estimate of a policy's survival benefit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representation." pith.science (2026). https://pith.science/paper/E2PBSEOG

@misc{pith2026250807681,
  author       = {Pith},
  title        = {Pith review of: MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E2PBSEOG}},
  note         = {Machine review of arXiv:2508.07681}
}
read the original abstract

Sepsis, a life-threatening inflammatory response to infection, causes organ dysfunction, making early detection and optimal management critical. Previous reinforcement learning (RL) approaches to sepsis management rely primarily on structured data, such as lab results or vital signs, and on a dearth of a comprehensive understanding of the patient's condition. In this work, we propose a Multimodal Offline REinforcement learning for Clinical notes Leveraged Enhanced stAte Representation (MORE-CLEAR) framework for sepsis control in intensive care units. MORE-CLEAR employs pre-trained large-scale language models (LLMs) to facilitate the extraction of rich semantic representations from clinical notes, preserving clinical context and improving patient state representation. Gated fusion and cross-modal attention allow dynamic weight adjustment in the context of time and the effective integration of multimodal data. Extensive cross-validation using two public (MIMIC-III and MIMIC-IV) and one private dataset demonstrates that MORE-CLEAR significantly improves estimated survival rate and policy performance compared to single-modal RL approaches. To our knowledge, this is the first to leverage LLM capabilities within a multimodal offline RL for better state representation in medical applications. This approach can potentially expedite the treatment and management of sepsis by enabling reinforcement learning models to propose enhanced actions based on a more comprehensive understanding of patient conditions.

Figures

Figures reproduced from arXiv: 2508.07681 by the authors.

Figure 1
Figure 1. MORE-CLEAR framework 4.3 MORE-CLEAR framework MORE-CLEAR ( [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Performance of policies under different clinical note integration strategies [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. DR estimator performance across training iterations under cross-dataset validation [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Bellman residual distributions across modalities [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Radar plots comparing OPE metrics between raw and summarized clinical notes [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 49 canonical work pages

  1. [1]

    Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies

    Daniel A Adler, Fei Wang, David C Mohr, and Tanzeem Choudhury. Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies. Plos one, 17(4):e0266516, 2022

    work page 2022

  2. [2]

    Ta-rnn: An attention-based time-aware recurrent neural network architecture for electronic health records

    Mohammad Al Olaimat, Serdar Bozdag, and Alzheimer’s Disease Neuroimaging Initiative. Ta-rnn: An attention-based time-aware recurrent neural network architecture for electronic health records. Bioinformatics, 40(Supplement_1):i169–i179, 2024

    work page 2024

  3. [3]

    New mci detection method based on transformer and eeg data

    Siwar Chaabene, Brahim Haroun Hassan, Amal Boudaya, Lotfi Chaari, and Bassem Bouaziz. New mci detection method based on transformer and eeg data. In 2023 31st European Signal Processing Conference (EUSIPCO), pages 1200–1204. IEEE, 2023

    work page 2023

  4. [4]

    Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records

    Yunho Choi, Songmi Oh, Jin Won Huh, Ho-Taek Joo, Hosu Lee, Wonsang You, Cheng-mok Bae, Jae-Hun Choi, and Kyung-Joong Kim. Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records. Communications Medicine, 4(1):245, 2024

    work page 2024

  5. [5]

    Kormedconceptsqa

    ChuGyouk. Kormedconceptsqa. https://huggingface.co/datasets/ChuGyouk/ KorMedConceptsQA, 2024. 13

    work page 2024

  6. [6]

    A discourse-aware attention model for abstractive summarization of long documents

    Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Paper...

    doi:10.18653/v1/n18-2097 2018

  7. [7]

    Clinicians’ reasoning as reflected in electronic clinical note-entry and reading/retrieval: a systematic review and qualitative synthesis

    Tiago K Colicchio and James J Cimino. Clinicians’ reasoning as reflected in electronic clinical note-entry and reading/retrieval: a systematic review and qualitative synthesis. Journal of the American Medical Informatics Association, 26(2):172–184, 2019

    work page 2019

  8. [8]

    Bio-medical: A high-performance biomedical language model

    ContactDoctor. Bio-medical: A high-performance biomedical language model. https://huggingface.co/ContactDoctor/Bio-Medical-Llama-3-8B, 2024

    work page 2024

Show all 68 references
  1. [9]

    Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021

    Laura Evans, Andrew Rhodes, Waleed Alhazzani, Massimo Antonelli, Craig M Coopersmith, Craig French, Flávia R Machado, Lauralyn Mcintyre, Marlies Ostermann, Hallie C Prescott, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2...

    2021

  2. [10]

    Offline inverse constrained reinforcement learning for safe-critical decision making in healthcare

    Nan Fang, Guiliang Liu, and Wei Gong. Offline inverse constrained reinforcement learning for safe-critical decision making in healthcare. arXiv preprint arXiv:2410.07525, 2024

    2024 arXiv

  3. [11]

    Semi-markov offline reinforcement learning for healthcare

    Mehdi Fatemi, Mary Wu, Jeremy Petch, Walter Nelson, Stuart J Connolly, Alexander Benz, Anthony Carnicelli, and Marzyeh Ghassemi. Semi-markov offline reinforcement learning for healthcare. In Conference on Health, Inference, and Learning, pages 119–137. PMLR, 2022

    2022

  4. [12]

    Off-policy deep reinforcement learning without exploration

    Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International conference on machine learning , pages 2052–2062. PMLR, 2019

  5. [13]

    Approximate solutions to Markov decision processes

    Geoffrey J Gordon. Approximate solutions to Markov decision processes. Carnegie Mellon University, 1999

    1999

  6. [14]

    A primer on reinforcement learning in medicine for clinicians

    Pushkala Jayaraman, Jacob Desman, Moein Sabounchi, Girish N Nadkarni, and Ankit Sakhuja. A primer on reinforcement learning in medicine for clinicians. NPJ Digital Medicine, 7(1):337, 2024

    2024

  7. [15]

    Doubly robust off-policy value evaluation for reinforcement learning

    Nan Jiang and Lihong Li. Doubly robust off-policy value evaluation for reinforcement learning. In International conference on machine learning, pages 652–661. PMLR, 2016

    2016

  8. [16]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams, 2020

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams, 2020. URL https://arxiv.org/abs/2009.13081

    2020 arXiv

  9. [17]

    Cohen, and Xinghua Lu

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering, 2019. URL https://arxiv.org/abs/ 1909.06146

    2019 arXiv

  10. [18]

    Mimic-iii, a freely accessible critical care database

    Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9, 2016

    2016

  11. [19]

    Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317, 2019

    2019

  12. [20]

    Mimic-iv, a freely accessible electronic health record dataset

    Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1, 2023

    2023

  13. [21]

    A conservative q- learning approach for handling distribution shift in sepsis treatment strategies

    Pramod Kaushik, Sneha Kummetha, Perusha Moodley, and Raju S Bapi. A conservative q- learning approach for handling distribution shift in sepsis treatment strategies. arXiv preprint arXiv:2203.13884, 2022. 14

    2022 arXiv

  14. [22]

    An empirical study of representation learning for reinforcement learning in healthcare

    Taylor W Killian, Haoran Zhang, Jayakumar Subramanian, Mehdi Fatemi, and Marzyeh Ghassemi. An empirical study of representation learning for reinforcement learning in healthcare. arXiv preprint arXiv:2011.11235, 2020

    2011 arXiv

  15. [23]

    Deep reinforcement learning for autonomous driving: A survey

    B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE transactions on intelligent transportation systems, 23(6):4909–4926, 2021

    2021

  16. [24]

    Automated detection and classification of sleep apnea types using electrocardiogram (ecg) and electroencephalogram (eeg) features

    Onur Kocak, Tuncay Bayrak, Aykut Erdamar, Levent Ozparlak, Ziya Telatar, and Osman Erogul. Automated detection and classification of sleep apnea types using electrocardiogram (ecg) and electroencephalogram (eeg) features. Advances in Electrocardiograms-Clinical Applications, p...

    2012

  17. [25]

    The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care

    Matthieu Komorowski, Leo A Celi, Omar Badawi, Anthony C Gordon, and A Aldo Faisal. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nature medicine, 24(11):1716–1720, 2018

    2018

  18. [26]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021

    2021 arXiv

  19. [27]

    Stabilizing off- policy q-learning via bootstrapping error reduction

    Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off- policy q-learning via bootstrapping error reduction. Advances in neural information processing systems, 32, 2019

    2019

  20. [28]

    Conservative q-learning for offline reinforcement learning

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems , 33: 1179–1191, 2020

    2020

  21. [29]

    Kormedmcqa: Multi-choice question answering benchmark for korean healthcare professional licensing examinations, 2024

    Sunjun Kweon, Byungjin Choi, Gyouk Chu, Junyeong Song, Daeun Hyeon, Sujin Gan, Jueon Kim, Minkyu Kim, Rae Woong Park, and Edward Choi. Kormedmcqa: Multi-choice question answering benchmark for korean healthcare professional licensing examinations, 2024. URL https://arxiv.org/a...

    2024 arXiv

  22. [30]

    Reinforcement learning model for optimizing dexmedetomidine dosing to prevent delirium in critically ill patients

    Hong Yeul Lee, Soomin Chung, Dongwoo Hyeon, Hyun-Lim Yang, Hyung-Chul Lee, Ho Geol Ryu, and Hyeonhoon Lee. Reinforcement learning model for optimizing dexmedetomidine dosing to prevent delirium in critically ill patients. npj Digital Medicine, 7(1):325, 2024

    2024

  23. [31]

    Development and validation of a reinforcement learning model for ventilation control during emergence from general anesthesia

    Hyeonhoon Lee, Hyun-Kyu Yoon, Jaewon Kim, Ji Soo Park, Chang-Hoon Koo, Dongwook Won, and Hyung-Chul Lee. Development and validation of a reinforcement learning model for ventilation control during emergence from general anesthesia. npj Digital Medicine, 6(1):145, 2023

    2023

  24. [32]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  25. [33]

    Multimodal medical supervised image fusion method by cnn

    Yi Li, Junli Zhao, Zhihan Lv, and Zhenkuan Pan. Multimodal medical supervised image fusion method by cnn. Frontiers in neuroscience, 15:638976, 2021

    2021

  26. [34]

    Timing of norepinephrine initiation in patients with septic shock: a systematic review and meta-analysis

    Yuting Li, Hongxiang Li, and Dong Zhang. Timing of norepinephrine initiation in patients with septic shock: a systematic review and meta-analysis. Critical Care, 24:1–9, 2020

    2020

  27. [35]

    Reinforcement learning using neural networks in estimating an optimal dynamic treatment regime in patients with sepsis

    Weijie Liang and Jinzhu Jia. Reinforcement learning using neural networks in estimating an optimal dynamic treatment regime in patients with sepsis. Computer Methods and Programs in Biomedicine, page 108754, 2025

    2025

  28. [36]

    Weighted importance sampling for off-policy learning with linear function approximation

    A Rupam Mahmood, Hado P Van Hasselt, and Richard S Sutton. Weighted importance sampling for off-policy learning with linear function approximation. Advances in neural information processing systems, 27, 2014

    2014

  29. [37]

    Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database

    Paul E Marik, Walter T Linde-Zwirble, Edward A Bittner, Jennifer Sahatjian, and Douglas Hansell. Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database. Intensive care medicine, 43:625–632, 2017. 15

    2017

  30. [38]

    The epidemiology of sepsis in the united states from 1979 through 2000

    Greg S Martin, David M Mannino, Stephanie Eaton, and Marc Moss. The epidemiology of sepsis in the united states from 1979 through 2000. New England Journal of Medicine, 348(16): 1546–1554, 2003

    1979

  31. [39]

    How to keep good clinical records

    Alexander Mathioudakis, Ilona Rousalova, Ane Aamli Gagnat, Neil Saad, and Georgia Hardav- ella. How to keep good clinical records. Breathe, 12(4):369–373, 2016

    2016

  32. [40]

    Deep offline reinforcement learning for real-world treatment optimization applications

    Mila Nambiar, Supriyo Ghosh, Priscilla Ong, Yu En Chan, Yong Mong Bee, and Pavitra Krishnaswamy. Deep offline reinforcement learning for real-world treatment optimization applications. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pag...

    2023

  33. [41]

    An analysis of clinical queries in an electronic health record search utility.International journal of medical informatics, 79(7):515–522, 2010

    Karthik Natarajan, Daniel Stein, Samat Jain, and Noémie Elhadad. An analysis of clinical queries in an electronic health record search utility.International journal of medical informatics, 79(7):515–522, 2010

    2010

  34. [42]

    Challenges with reinforcement learning model transportability for sepsis treatment in emergency care

    Peter C Nauka, Jason N Kennedy, Emily B Brant, Matthieu Komorowski, Romain Pirracchio, Derek C Angus, and Christopher W Seymour. Challenges with reinforcement learning model transportability for sepsis treatment in emergency care. npj Digital Medicine, 8(1):1–5, 2025

    2025

  35. [43]

    Opera: Automatic offline policy evaluation with re-weighted aggregates of multiple estimators

    Allen Nie, Yash Chandak, Christina Yuan, Anirudhan Badrinath, Yannis Flet-Berliac, and Emma Brunskill. Opera: Automatic offline policy evaluation with re-weighted aggregates of multiple estimators. Advances in Neural Information Processing Systems, 37:103652–103680, 2024

    2024

  36. [44]

    Medmcqa : A large- scale multi-subject multi-choice dataset for medical domain question answering, 2022

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa : A large- scale multi-subject multi-choice dataset for medical domain question answering, 2022. URL https://arxiv.org/abs/2203.14371

    2022 arXiv

  37. [45]

    The role of proteomics and genomics in the development of colorectal cancer diagnostic tools and potential new treatments

    Gaurav Paraskar, Sankha Bhattacharya, and Anitha Kuttiappan. The role of proteomics and genomics in the development of colorectal cancer diagnostic tools and potential new treatments. ACS Pharmacology & Translational Science, 2025

    2025

  38. [46]

    How essential are unstructured clinical narratives and information fusion to clinical trial recruitment? AMIA Summits on Translational Science Proceedings, 2014:218, 2014

    Preethi Raghavan, James L Chen, Eric Fosler-Lussier, and Albert M Lai. How essential are unstructured clinical narratives and information fusion to clinical trial recruitment? AMIA Summits on Translational Science Proceedings, 2014:218, 2014

    2014

  39. [47]

    Evidence-based multimodal fusion on structured ehrs and free-text notes for icu outcome prediction

    Yucheng Ruan, Daniel J Tan, See Kiong Ng, Ling Huang, and Mengling Feng. Evidence-based multimodal fusion on structured ehrs and free-text notes for icu outcome prediction. arXiv preprint arXiv:2501.04389, 2025

    2025 arXiv

  40. [48]

    Management of sepsis

    James A Russell. Management of sepsis. New England Journal of Medicine, 355(16):1699– 1713, 2006

    2006

  41. [49]

    Integrating physiological time series and clinical notes with deep learning for improved icu mortality prediction.arXiv preprint arXiv:2003.11059, 2020

    Satya Narayan Shukla and Benjamin M Marlin. Integrating physiological time series and clinical notes with deep learning for improved icu mortality prediction.arXiv preprint arXiv:2003.11059, 2020

    2003 arXiv

  42. [50]

    The third international consensus definitions for sepsis and septic shock (sepsis-3)

    Mervyn Singer, Clifford S Deutschman, Christopher Warren Seymour, Manu Shankar-Hari, Djillali Annane, Michael Bauer, Rinaldo Bellomo, Gordon R Bernard, Jean-Daniel Chiche, Craig M Coopersmith, et al. The third international consensus definitions for sepsis and septic shock (se...

    2016

  43. [51]

    Reinforcement learning in robotic applications: a comprehensive survey

    Bharat Singh, Rajesh Kumar, and Vinay Pratap Singh. Reinforcement learning in robotic applications: a comprehensive survey. Artificial Intelligence Review, 55(2):945–990, 2022

    2022

  44. [52]

    An effective multimodal image fusion method using mri and pet for alzheimer’s disease diagnosis

    Juan Song, Jian Zheng, Ping Li, Xiaoyuan Lu, Guangming Zhu, and Peiyi Shen. An effective multimodal image fusion method using mri and pet for alzheimer’s disease diagnosis. Frontiers in digital health, 3:637386, 2021

    2021

  45. [53]

    Reinforcement learning: An introduction, volume 1

    Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  46. [54]

    Gemma Team. Gemma 3. 2025. URL https://goo.gle/Gemma3Report. 16

    2025

  47. [55]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025. URL https://arxiv.org/abs/2505.09388

    2025 arXiv

  48. [56]

    Ehr-based prediction modelling meets multimodal deep learning: A systematic review of structured and textual data fusion methods

    Ariel Soares Teles, Ivan Rodrigues de Moura, Francisco Silva, Angus Roberts, and Daniel Stahl. Ehr-based prediction modelling meets multimodal deep learning: A systematic review of structured and textual data fusion methods. Information Fusion, page 102981, 2025

    2025

  49. [57]

    Mlp-mixer: An all-mlp architecture for vision

    Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision. In M. Ranzato, A. Beygelzim...

    2021

  50. [58]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation langua...

    2023 arXiv

  51. [59]

    Offline safe reinforcement learning for sepsis treatment: Tackling variable-length episodes with sparse rewards

    Rui Tu, Zhipeng Luo, Chuanliang Pan, Zhong Wang, Jie Su, Yu Zhang, and Yifan Wang. Offline safe reinforcement learning for sepsis treatment: Tackling variable-length episodes with sparse rewards. Human-Centric Intelligent Systems, 5(1):63–76, 2025

    2025

  52. [60]

    Assessment of the worldwide burden of critical illness: the intensive care over nations (icon) audit

    Jean-Louis Vincent, John C Marshall, Silvio A Ñamendys-Silva, Bruno François, Ignacio Martin-Loeches, Jeffrey Lipman, Konrad Reinhart, Massimo Antonelli, Peter Pickkers, Hassane Njimi, et al. Assessment of the worldwide burden of critical illness: the intensive care over natio...

    2014

  53. [61]

    Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study

    Jason Waechter, Anand Kumar, Stephen E Lapinsky, John Marshall, Peter Dodek, Yaseen Arabi, Joseph E Parrillo, R Phillip Dellinger, Allan Garland, Cooperative Antimicrobial Therapy of Septic Shock Database Research Group, et al. Interaction between fluids and vasoactive agents ...

    2014

  54. [62]

    Clinical knowledge-guided deep reinforcement learning for sepsis antibiotic dosing recommen- dations

    Yuan Wang, Anqi Liu, Jucheng Yang, Lin Wang, Ning Xiong, Yisong Cheng, and Qin Wu. Clinical knowledge-guided deep reinforcement learning for sepsis antibiotic dosing recommen- dations. Artificial Intelligence in Medicine, 150:102811, 2024

    2024

  55. [63]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pages 1995–2003. PMLR, 2016

    1995

  56. [64]

    Q-learning

    Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8:279–292, 1992

    1992

  57. [65]

    Me llama: Foundation large language models for medical applications, 2024

    Qianqian Xie, Qingyu Chen, Aokun Chen, Cheng Peng, Yan Hu, Fongci Lin, Xueqing Peng, Jimin Huang, Jeffrey Zhang, Vipina Keloth, Huan He, Lucila Ohno-Machido, Yonghui Wu, Hua Xu, and Jiang Bian. Me llama: Foundation large language models for medical applications, 2024

    2024

  58. [66]

    Vasopressor initiation within 1 hour of fluid loading is associated with increased mortality in septic shock patients: analysis of national registry data

    Hye Ju Yeo, Young Seok Lee, Tae Hwa Kim, Jin Ho Jang, Heung Bum Lee, Dong Kyu Oh, Mi Hyeon Park, Chae-Man Lim, Woo Hyun Cho, et al. Vasopressor initiation within 1 hour of fluid loading is associated with increased mortality in septic shock patients: analysis of national regis...

    2022

  59. [67]

    Fusion deep learning approach combining diffuse optical tomography and ultrasound for improving breast cancer classification

    Menghao Zhang, Minghao Xue, Shuying Li, Yun Zou, and Quing Zhu. Fusion deep learning approach combining diffuse optical tomography and ultrasound for improving breast cancer classification. Biomedical Optics Express, 14(4):1636–1646, 2023

    2023

  60. [68]

    Prism: Mitigating ehr data sparsity via learning from missing feature calibrated prototype patient representations

    Yinghao Zhu, Zixiang Wang, Long He, Shiyun Xie, Xiaochen Zheng, Liantao Ma, and Cheng- wei Pan. Prism: Mitigating ehr data sparsity via learning from missing feature calibrated prototype patient representations. In Proceedings of the 33rd ACM International Conference on Inform...

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.