pith. sign in

arxiv: 2605.23320 · v1 · pith:UOPKMNJTnew · submitted 2026-05-22 · 💻 cs.AI

Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning

Sijia Li , Xiaoyu Tan , Qixing Wang , Weiyi Zhao , Chen Zhan , Teqi Hao , Xuemin Wang , Lei Gu
show 2 more authors
Roland Eils Xihe Qiu
This is my paper

Pith reviewed 2026-05-25 04:25 UTC · model grok-4.3

classification 💻 cs.AI
keywords ventilator decision supporthuman-in-the-loopmulti-agent systemscontextual banditspreference learningICU clinical decision supportsequential decision making
0
0 comments X

The pith

A multi-agent system uses a contextual bandit to adapt ventilator recommendations to individual clinician preferences from accepted decisions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents VDSS, a human-in-the-loop multi-agent framework for sequential ventilator decisions that must track evolving patient physiology while respecting safety limits and clinician-specific styles. It coordinates modular agents through structured contract interfaces and updates a contextual bandit from the final accepted plan at each cycle to personalize future suggestions. Structured rejection feedback triggers targeted replanning to cut unproductive loops. Retrospective replay of ICU trajectories with expert review shows higher recommendation acceptance and fewer rounds to an acceptable plan than alternatives. This setup aims to make AI assistance in critical care more controllable and clinically usable.

Core claim

VDSS coordinates modular decision components through contract-driven structured interfaces, performs online preference adaptation with a contextual bandit that updates clinician-specific preferences from the final accepted decision at each adjustment cycle, and uses structured rejection feedback to trigger targeted replanning; retrospective ICU trajectory replay with expert review indicates higher recommendation acceptability and fewer interaction rounds to reach an acceptable plan.

What carries the argument

Contextual bandit that learns clinician-specific preferences online from final accepted decisions to guide recommendations inside the contract-driven multi-agent coordination.

If this is right

  • Recommendations align more closely with individual clinician tuning styles without requiring explicit manual configuration.
  • Structured rejection feedback reduces the number of unproductive iterations needed to reach an acceptable ventilator plan.
  • Traceable evidence from the modular agents supports review and auditing of each decision step.
  • The framework maintains safety boundaries during sequential decisions while allowing personalization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same preference-learning loop could apply to other sequential clinical tasks such as fluid management or sedation adjustments where practitioner styles differ.
  • Real-time integration with continuous monitoring data would allow the bandit to adapt as patient trajectories change within a single stay.
  • Deployment would require safeguards to detect when learned preferences begin to conflict with updated safety protocols.

Load-bearing premise

That the final accepted decisions provide sufficient signal for a contextual bandit to learn stable, generalizable clinician preferences that respect safety boundaries across new patients and real-time trajectories.

What would settle it

A test on new ICU trajectories where acceptance rates do not rise and interaction rounds do not fall when the bandit-driven recommendations are used compared to non-adaptive baselines.

Figures

Figures reproduced from arXiv: 2605.23320 by Chen Zhan, Lei Gu, Qixing Wang, Roland Eils, Sijia Li, Teqi Hao, Weiyi Zhao, Xiaoyu Tan, Xihe Qiu, Xuemin Wang.

Figure 1
Figure 1. Figure 1: Overview of VDSS. Panel A shows the patient and clinical data hub with a context window and long term memory. Panel B depicts the VDSS multi agent rea￾soning engine from detection and phase goal inference to hold adjust gating, planning, safety checking, and feedback driven revision. Panel C summarizes the human in the loop interface and online clinician preference optimization with a contextual bandit. 1.… view at source ↗
Figure 2
Figure 2. Figure 2: Regret over 100 cycles from a single clinician, defined as Kt − 1 and capped by Kmax, shows an overall downward trend. served across other backbones, suggesting that the improvement is driven by the VDSS workflow rather than any single model. We do not emphasize conventional reinforcement learning because ventilation modes differ substantially in parameter sets and semantics, and per mode data are comparat… view at source ↗
Figure 3
Figure 3. Figure 3: Case study of one VDSS adjustment cycle. Kmax [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
read the original abstract

Ventilator decision support requires sequential decisions that track evolving physiology and disease trajectories while respecting safety boundaries and clinician specific tuning styles. Rule based approaches rarely generalize personalization, and end to end reinforcement learning or single large language model systems remain difficult to control and audit. We propose the Ventilator Decision Support System (VDSS), a human in the loop multi agent framework that coordinates modular decision components through contract driven structured interfaces and produces traceable evidence for review. VDSS performs online preference adaptation with a contextual bandit, updating clinician specific preferences from the final accepted decision at each adjustment cycle and using them to guide subsequent recommendations. Structured rejection feedback triggers targeted replanning to reduce unproductive iterations and improve interaction stability. Retrospective ICU trajectory replay with expert review indicates higher recommendation acceptability and fewer interaction rounds to reach an acceptable plan, supporting clinically deployable human AI collaboration.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes the Ventilator Decision Support System (VDSS), a human-in-the-loop multi-agent framework coordinating modular components via contract-driven interfaces for sequential ventilator decisions. It performs online preference adaptation via a contextual bandit updated solely from the final accepted decision per adjustment cycle, with structured rejection feedback triggering replanning. The central claim is that retrospective ICU trajectory replay with expert review demonstrates higher recommendation acceptability and fewer interaction rounds to reach an acceptable plan, supporting clinically deployable human-AI collaboration.

Significance. If the retrospective results hold under prospective validation, the work could advance safe, auditable personalization in high-stakes ICU settings by combining multi-agent modularity with preference learning, addressing generalization limits of rule-based systems and control issues in end-to-end RL. The use of traceable evidence and rejection-triggered replanning is a concrete strength for auditability.

major comments (2)
  1. [Abstract / Contextual Bandit Preference Learning] Abstract (and methods description of the contextual bandit): The update rule relies exclusively on the final accepted decision as a single positive label per cycle, with no mention of negative examples, rejection signals beyond replanning triggers, regularization, exploration mechanisms, or explicit safety constraint projection. This is load-bearing for the claim of 'stable, generalizable' clinician preferences, as it leaves open the risk of overfitting to replay-distribution choices and violating physiological limits on out-of-distribution trajectories.
  2. [Abstract / Retrospective Evaluation] Evaluation description (retrospective replay): The abstract asserts 'higher recommendation acceptability and fewer interaction rounds' but reports no cohort size, baseline comparisons, quantitative metrics (e.g., acceptability rates, round counts with error bars), statistical tests, or implementation details of the bandit or multi-agent coordination. This undermines the central empirical support for clinical deployability.
minor comments (1)
  1. [Abstract] Abstract contains minor phrasing issues (e.g., 'end to end' should be hyphenated; 'clinician specific' should be 'clinician-specific').

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the two major comments below and will revise the manuscript accordingly to improve clarity and empirical presentation.

read point-by-point responses

  1. Referee: [Abstract / Contextual Bandit Preference Learning] Abstract (and methods description of the contextual bandit): The update rule relies exclusively on the final accepted decision as a single positive label per cycle, with no mention of negative examples, rejection signals beyond replanning triggers, regularization, exploration mechanisms, or explicit safety constraint projection. This is load-bearing for the claim of 'stable, generalizable' clinician preferences, as it leaves open the risk of overfitting to replay-distribution choices and violating physiological limits on out-of-distribution trajectories.

    Authors: We agree the abstract and methods description are high-level and omit several implementation details. The design uses only the final accepted decision as a positive label to learn directly from clinician-accepted outcomes; rejections trigger replanning but are deliberately not treated as negative labels, since they may reflect transient issues rather than preference mismatch. Safety boundaries are enforced through the contract interfaces of the modular agents rather than the bandit. The current text does not describe regularization, exploration, or explicit projection steps. We will revise the methods section to fully specify the bandit update rule, any regularization or exploration used, and add an explicit limitations paragraph discussing overfitting risk to the replay distribution together with mitigation via expert review. This revision will be made without altering the core approach. revision: yes

  2. Referee: [Abstract / Retrospective Evaluation] Evaluation description (retrospective replay): The abstract asserts 'higher recommendation acceptability and fewer interaction rounds' but reports no cohort size, baseline comparisons, quantitative metrics (e.g., acceptability rates, round counts with error bars), statistical tests, or implementation details of the bandit or multi-agent coordination. This undermines the central empirical support for clinical deployability.

    Authors: The abstract is a concise summary; the full manuscript contains the requested details (cohort size from the ICU replay, baseline comparisons, acceptability rates and round counts with error bars, statistical tests, and implementation specifics) in the evaluation section. To strengthen the abstract's support for the claims, we will revise it to include the key quantitative results while remaining within length limits. This change directly addresses the concern without misrepresenting the existing results. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical system description with no load-bearing derivations or self-referential reductions.

full rationale

The paper presents a multi-agent ventilator decision support framework that incorporates a contextual bandit for online preference adaptation from final accepted decisions. No equations, first-principles derivations, or predictions appear in the provided text that reduce by construction to fitted inputs or self-citations. The evaluation on retrospective ICU replay is an empirical claim about acceptability and interaction rounds, not a mathematical result forced by the learning mechanism itself. Standard bandit updates from positive labels do not meet the criteria for self-definitional, fitted-input-called-prediction, or uniqueness-imported circularity. The system is self-contained against external benchmarks via expert review.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the central claim rests on unstated assumptions about bandit convergence and expert-review validity.

pith-pipeline@v0.9.0 · 5698 in / 1097 out tokens · 24379 ms · 2026-05-25T04:25:58.257221+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    British journal of anaesthesia105(1), 26–33 (2010)

    Bion, J.F., Abrusci, T., Hibbert, P.: Human factors in the management of the critically ill patient. British journal of anaesthesia105(1), 26–33 (2010)

    work page 2010

  2. [2]

    Annals of internal medicine170(5), 285–297 (2019)

    Cox,C.E.,White,D.B.,Hough,C.L.,Jones,D.M.,Kahn,J.M.,Olsen,M.K.,Lewis, C.L., Hanson, L.C., Carson, S.S.: Effects of a personalized web-based decision aid for surrogate decision makers of patients with prolonged mechanical ventilation: a randomized clinical trial. Annals of internal medicine170(5), 285–297 (2019)

    work page 2019

  3. [3]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Dewes, R., Dimitrova, R.: Contract-based design and verification of multi-agent systems with quantitative temporal requirements. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 23152–23159 (2025)

    work page 2025

  4. [4]

    American journal of respiratory and crit- ical care medicine195(9), 1253–1263 (2017)

    Fan, E., Del Sorbo, L., Goligher, E.C., Hodgson, C.L., Munshi, L., Walkey, A.J., Adhikari, N.K., Amato, M.B., Branson, R., Brower, R.G., et al.: An official ameri- can thoracic society/european society of intensive care medicine/society of critical care medicine clinical practice guideline: mechanical ventilation in adult patients with acute respiratory d...

    work page 2017

  5. [5]

    Respi- ratory Care69(8), 1042–1054 (2024)

    Goodfellow, L.T., Miller, A.G., Varekojis, S.M., LaVita, C.J., Glogowski, J.T., Hess, D.R.: Aarc clinical practice guideline: patient-ventilator assessment. Respi- ratory Care69(8), 1042–1054 (2024)

    work page 2024

  6. [6]

    Critical Care Medicine49(7), 1068–1082 (2021) 10 S

    Gray, B.M., Vandergrift, J.L., Barnhart, B.J., Reddy, S.G., Chesluk, B.J., Stevens, J.S., Lipner, R.S., Lynn, L.A., Barnett, M.L., Landon, B.E.: Changes in stress and workplace shortages reported by us critical care physicians treating coronavirus disease 2019 patients. Critical Care Medicine49(7), 1068–1082 (2021) 10 S. Li et al

    work page 2019

  7. [7]

    NPJ digital medicine5(1), 97 (2022)

    Henry, K.E., Kornfield, R., Sridharan, A., Linton, R.C., Groh, C., Wang, T., Wu, A., Mutlu, B., Saria, S.: Human–machine teaming is key to ai adoption: clinicians’ experiences with a deployed machine learning system. NPJ digital medicine5(1), 97 (2022)

    work page 2022

  8. [8]

    JAMA internal medicine183(8), 824–831 (2023)

    Jivraj, N.K., Hill, A.D., Shieh, M.S., Hua, M., Gershengorn, H.B., Ferrando-Vivas, P., Harrison, D., Rowan, K., Lindenauer, P.K., Wunsch, H.: Use of mechanical ventilation across 3 countries. JAMA internal medicine183(8), 824–831 (2023)

    work page 2023

  9. [9]

    Annals of intensive care1(1), 51 (2011)

    Jouvet, P., Hernert, P., Wysocki, M.: Development and implementation of explicit computerized protocols for mechanical ventilation in children. Annals of intensive care1(1), 51 (2011)

    work page 2011

  10. [10]

    Chest117(2), 467–475 (2000)

    Kollef, M.H., Shapiro, S.D., Clinkscale, D., Cracchiolo, L., Clayton, D., Wilner, R., Hossin, L.: The effect of respiratory therapist-initiated treatment protocols on patient outcomes and resource utilization. Chest117(2), 467–475 (2000)

    work page 2000

  11. [11]

    arXiv preprint arXiv:1706.09090 (2017)

    Lei, H., Lu, Y., Tewari, A., Murphy, S.A.: An actor-critic contextual ban- dit algorithm for personalized mobile health interventions. arXiv preprint arXiv:1706.09090 (2017)

    work page arXiv 2017

  12. [12]

    Journal of medical Internet research22(7), e18477 (2020)

    Liu, S., See, K.C., Ngiam, K.Y., Celi, L.A., Sun, X., Feng, M.: Reinforcement learning for clinical decision support in critical care: comprehensive review. Journal of medical Internet research22(7), e18477 (2020)

    work page 2020

  13. [13]

    The Lancet Digital Health2(10), e537–e548 (2020)

    Liu, X., Rivera, S.C., Moher, D., Calvert, M.J., Denniston, A.K., Ashrafian, H., Beam, A.L., Chan, A.W., Collins, G.S., Deeks, A.D.J., et al.: Reporting guide- lines for clinical trial reports for interventions involving artificial intelligence: the consort-ai extension. The Lancet Digital Health2(10), e537–e548 (2020)

    work page 2020

  14. [14]

    Heart & Lung73, 139–152 (2025)

    Muñoz, J., Ruíz-Cacho, R., Fernández-Araujo, N.J., Candela, A., Visedo, L.C., Muñoz-Visedo, J.: Artificial intelligence in the management of patient-ventilator asynchronies: A scoping review. Heart & Lung73, 139–152 (2025)

    work page 2025

  15. [15]

    British Journal of Anaesthesia133(1), 164–177 (2024)

    Murali, M., Ni, M., Karbing, D.S., Rees, S.E., Komorowski, M., Marshall, D., Ramnarayan, P., Patel, B.V.: Clinical practice, decision-making, and use of clinical decision support systems in invasive mechanical ventilation: a narrative review. British Journal of Anaesthesia133(1), 164–177 (2024)

    work page 2024

  16. [16]

    New England Journal of Medicine342(18), 1301–1308 (2000)

    Network, A.R.D.S.: Ventilation with lower tidal volumes as compared with tradi- tional tidal volumes for acute lung injury and the acute respiratory distress syn- drome. New England Journal of Medicine342(18), 1301–1308 (2000)

    work page 2000

  17. [17]

    Packer, C., Fang, V., Patil, S., Lin, K., Wooders, S., Gonzalez, J.: Memgpt: towards llms as operating systems. (2023)

    work page 2023

  18. [18]

    NPJ digital medicine4(1), 32 (2021)

    Peine, A., Hallawa, A., Bickenbach, J., Dartmann, G., Fazlic, L.B., Schmeink, A., Ascheid, G., Thiemermann, C., Schuppert, A., Kindle, R., et al.: Development and validation of a reinforcement learning algorithm to dynamically optimize mechan- ical ventilation in critical care. NPJ digital medicine4(1), 32 (2021)

    work page 2021

  19. [19]

    JAMA: Journal of the American Medical Association307(23) (2012)

    Ranieri, V.M., Rubenfeld, G.D., Taylor Thompson, B., Ferguson, N.D., Caldwell, E., Fan, E., Camporota, L., Slutsky, A.S.: Acute respiratory distress syndrome: the berlin definition. JAMA: Journal of the American Medical Association307(23) (2012)

    work page 2012

  20. [20]

    Jmir Ai3, e53207 (2024)

    Rosenbacke, R., Melhus, Å., McKee, M., Stuckler, D.: How explainable artificial intelligence can increase or decrease clinicians’ trust in ai applications in health care: systematic review. Jmir Ai3, e53207 (2024)

    work page 2024

  21. [21]

    Intensive and Critical Care Nursing75, 103367 (2023)

    Shi, F., Li, Y., Zhao, Y.: How do nurses manage their work under time pressure? occurrence of implicit rationing of nursing care in the intensive care unit: A quali- tative study. Intensive and Critical Care Nursing75, 103367 (2023)

    work page 2023

  22. [22]

    New England Journal of Medicine369(22), 2126–2136 (2013) Human-in-the-Loop VDSS with Bandit Learning 11

    Slutsky, A.S., Ranieri, V.M.: Ventilator-induced lung injury. New England Journal of Medicine369(22), 2126–2136 (2013) Human-in-the-Loop VDSS with Bandit Learning 11

    work page 2013

  23. [23]

    Intensive care medicine32(10), 1515–1522 (2006)

    Thille, A.W., Rodriguez, P., Cabello, B., Lellouche, F., Brochard, L.: Patient- ventilator asynchrony during assisted mechanical ventilation. Intensive care medicine32(10), 1515–1522 (2006)

    work page 2006

  24. [24]

    Nature medicine25(1), 44–56 (2019)

    Topol, E.J.: High-performance medicine: the convergence of human and artificial intelligence. Nature medicine25(1), 44–56 (2019)

    work page 2019

  25. [25]

    bmj377(2022)

    Vasey, B., Nagendran, M., Campbell, B., Clifton, D.A., Collins, G.S., Denaxas, S., Denniston, A.K., Faes, L., Geerts, B., Ibrahim, M., et al.: Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: Decide-ai. bmj377(2022)

    work page 2022

  26. [26]

    American Journal of Respiratory and Critical Care Medicine209(5), 553–562 (2024)

    von Wedel, D., Redaelli, S., Suleiman, A., Wachtendorf, L.J., Fosset, M., San- ter, P., Shay, D., Munoz-Acuna, R., Chen, G., Talmor, D., et al.: Adjustments of ventilator parameters during operating room–to–icu transition and 28-day mortal- ity. American Journal of Respiratory and Critical Care Medicine209(5), 553–562 (2024)

    work page 2024

  27. [27]

    In: First conference on language modeling (2024)

    Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al.: Autogen: Enabling next-gen llm applications via multi- agent conversations. In: First conference on language modeling (2024)

    work page 2024