Pith. sign in

REVIEW 4 major objections 3 minor 38 references

TumorBoard: Evidence-Grounded Multi-Agent Decision Support for Longitudinal Neuro-Oncology

T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TumorBoard shows that structured coordination, not extra tokens, drives multi-agent decision quality and safety in neuro-oncology.

desk verdict Solid architecture paper with a credible matched-budget main result, but the ablation token imbalance and the self-built benchmark keep the mechanism claims provisional. read the letter →

arxiv 2608.03190 v1 pith:TYMNATW4 submitted 2026-08-04 cs.AI

Yantong Liu , Zheyu Zhang , Runpeng Liu , Mu Xitang , Seong-Yoon Shin , Hyun-Ae Lee This is my paper · ORCID
classification cs.AI
keywords multi-agentsystemsclinicaldecisionsupportneuro-oncologyclaim-evidenceledgersafetygovernorevidencegroundinglongitudinalreasoningactiongraphs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that multi-agent medical decision support improves decisions only when coordination is structured, not free-form. TumorBoard builds a typed claim-evidence ledger, an adversarial critic, and a safety governor so that specialist agents exchange auditable claims instead of prose. On a 360-case hidden neuro-oncology benchmark with matched token budgets, it reaches an action F1 of 0.772, beating the strongest budget-matched council by 3.1 points, and cutting harmful recommendation release from 0.117 to 0.039 when the governor is active. Removing any one mechanism produces the predicted failure pattern, which the authors take as evidence that the protocol itself, not extra compute, accounts for the gain. A sympathetic reader would care because this offers a concrete recipe for making clinical multi-agent systems safer and more auditable.

What carries the argument

The load-bearing object is a typed claim-evidence ledger: a directed graph whose nodes are atomic claims carrying claim text, type, temporal scope, confidence, prerequisites, evidence pointers, and conflicts, and whose edges encode support, contradiction, supersession, and dependency. Evidence nodes are immutable, a claim cannot support its own ancestors, and a recommendation inherits the weakest confidence among its required upstream claims. The ledger makes coordination failures visible as graph defects; an adversarial critic then targets missing prerequisites, stale sources, temporal inconsistencies, and false consensus, while a safety governor releases, qualifies, or defers each recommen

What would settle it

Have an independent neuro-oncology team re-annotate the 360 hidden cases (action graphs and harm labels) without seeing the system outputs, then rerun the budget-matched comparison on the same frozen prompts. If TumorBoard's action-F1 advantage over the typed council shrinks or reverses under those external labels, the measured coordination gain is an artifact of benchmark self-confirmation rather than a property of the architecture.

Watch

Extended reading notes

Core claim

TumorBoard's central claim is that structured coordination—a typed claim-evidence ledger, adversarial critique, and a gated safety release—is the source of multi-agent decision quality in longitudinal neuro-oncology, and that this can be demonstrated by holding model, retrieval corpus, and total inference budget fixed while varying only the communication protocol. Under that constraint, full TumorBoard reaches an action F1 of 0.772 versus 0.741 for the strongest typed council, with evidence entailment of 0.914 and recommendation-to-evidence coverage of 0.927. Perturbation tests show the system defers 84.2% of cases when decisive evidence is deleted and caps harmful recommendations at 5.8%; e

Load-bearing premise

The load-bearing premise is that the benchmark's gold action graphs and harm labels, authored and adjudicated by the research group with no external validation, do not encode the same expectations the system is prompted to produce.

Editorial extensions

If this is right

  • Multi-agent medical systems should be evaluated with matched token budgets, because protocol structure, not compute, may explain reported gains.
  • The safety governor turns risk tolerance into an explicit release threshold, allowing deployments to choose how to trade harmful releases against false deferrals.
  • The claim-evidence ledger preserves the full reasoning dependency chain, making each recommendation traceable to specific evidence for human review and regulatory audit.
  • The same coordination protocol could transfer to other longitudinal, high-stakes decision domains where prerequisites and temporal validity matter, such as oncology treatment sequencing or chronic disease management.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because the gold action graphs and harm labels were authored and adjudicated by the research group with no external validation, the measured F1 gain over the typed council could partly reflect the designers' expectations rather than a generalizable property of the architecture.
  • Editorial inference: The 84.2% deferral under evidence deletion, while promising, may indicate an over-cautious bias; a testable extension would compare deferral decisions against human expert consensus on the same incomplete cases.
  • Editorial inference: The ledger records per-agent message cost, so the same data could be mined to identify which specialist role contributes unique evidence versus redundant prose, enabling a smaller council without loss of accuracy.
  • Editorial inference: The 4.3-point false-deferral cost is presented as an operational trade-off, but it lacks a clinical utility weighting; future work could link deferral decisions to downstream patient outcomes to set the threshold empirically.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper presents TumorBoard, a multi-agent decision-support system for longitudinal neuro-oncology. The architecture combines a timeline curator and shared case state, role-limited specialist agents (radiology, neuropathology, molecular, guidelines, therapy), a typed claim-evidence ledger, an adversarial critic, and a safety governor. The authors report a 360-case hidden benchmark with budget-matched comparisons against single-agent and multi-agent baselines. They claim action F1 of 0.772, evidence entailment of 0.914, a 3.1-point gain over the strongest baseline (adjusted p=0.0012), and a 7.8-point reduction in harmful release when the safety governor is enabled. They also present perturbation tests (evidence deletion, conflict, guideline shift, role corruption, prompt injection) and ablations of the ledger, critic, and governor.

Significance. If the empirical claims are sound, TumorBoard is a meaningful step toward auditable, evidence-grounded clinical decision support. The paper has real strengths: paired case-level statistics with bootstrap CIs and Holm correction, prespecified primary and safety endpoints, perturbation tests with parameterized severity, a hidden test set, and a commitment to retaining prompts, messages, ledger states, and outputs for audit. The typed claim-evidence ledger and explicit safety governor are sensible mechanisms for a high-stakes domain. The measured gains are modest but plausible. However, the benchmark is fully self-constructed with no external validation or public release, and the ablation token-budget mismatch documented below weakens the paper's central mechanistic attribution. The result is still worth serious consideration, but several load-bearing points need to be addressed before the claims can be accepted as stated.

major comments (4)
  1. [§5.5 / Table 5] The paper's own acceptance criterion for crediting coordination gains is control for generated tokens (§1, §5.5). Table 5 violates this criterion for the mechanism ablations: Full TumorBoard uses 14,220 tokens/case, No critic uses 11,380 (a 20% reduction), No governor uses 13,310, and No ledger uses 14,050. The reported No-critic degradation in Action F1 (0.749 vs 0.772) and contradiction resolution (-0.112) is therefore confounded by a 20% inference-compute reduction. The 'No critic' ablation cannot isolate the challenge mechanism from additional thinking time. The central attribution 'structured coordination as the source of the measured multi-agent advantage' (§7, Abstract) is not established by the reported comparisons. I ask for either token-equalized ablation runs (e.g., proportional critic iterations or filler reasoning) or an explicit covariate adjustment for generated tokens.
  2. [§5.1 / §7] The benchmark is self-constructed: gold action graphs and harm labels are authored and adjudicated by the research group, with no external validation or public release. The evaluation ontology (action graph nodes/edges, prerequisite and safety classes) closely mirrors the system's typed ledger schema and safety governor rules. This raises a circularity risk: the measured F1 and harmful-release reductions may partly reflect the benchmark encoding the same expectations the system is prompted to produce. Since the primary comparative claim rests on this benchmark, independent external adjudication (or at minimum a second institution's expert panel) and public release of case and label templates is needed. The paper should explicitly flag this limitation; it currently presents the benchmark as unproblematic.
  3. [Table 5 / §6.3] The No-governor row in Table 5 reports Action F1 0.776, numerically higher than Full TumorBoard's 0.772, despite using fewer tokens. The text in §6.3 says removing the governor 'increased harmful release by 0.078' but does not report that the decision-quality effect is negligible or slightly negative. The conclusion 'the ledger, critic, and safety governor jointly improve decision quality' (Conclusion) therefore overstates the evidence for the governor's effect on decision quality; at most the governor contributes to safety, not to F1. This should be stated precisely.
  4. [§5.4 / §6.2] The perturbation tests are a clear strength, but the safety outcomes are measured against the researchers' own harm labels, and the perturbation generator and the governor share the same prerequisite and risk rules. The 84.2% deferral under evidence deletion and the 7.8pp reduction in harmful release could thus be partly by construction: when the decisive fact is removed, a prerequisite check is expected to fail. This is not a trivial result, but it is less informative without an independent enumeration of harm types or a held-out clinician panel scoring potential harm with the governor's decision masked. Reporting such an external human harm review on a random subset would strengthen the safety claim considerably.
minor comments (3)
  1. [Table 5] The 'Tokens' column reports values such as '14220.000' and '11380.000'; these should be integers. Also, the column header in Table 3, 'Deferral F1', appears to denote a deferral-related rate rather than an F1 score; please clarify the metric name.
  2. [§5.5 / Appendix D] The appendix says every numeric statement maps to a locked CSV export, but the release 'will contain' prompts, retrieval snapshots, and logs rather than providing them now. For reproducibility, at least the prompts and retrieval collection procedure should be included as supplementary material.
  3. [§6.1 / Table 3] Table 3 reports many point estimates without confidence intervals for baseline methods. The paired CI is given for the primary TumorBoard-vs-typed-council comparison, but CIs for other baselines would help assess the significance of the ordering in Table 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main comparative and safety claims are evaluated against external expert labels; no self-citation chain or fitted-input-as-prediction defect is present.

full rationale

The paper's central claims are an empirical comparison against budget-matched baselines and ablations of its own components. The primary action-F1, entailment, coverage, and harmful-release numbers are measured against an adjudicated action graph and expert harm labels, not against the system's internal ledger. The ablations remove components and observe changes in these external metrics; while the 'No critic' ablation also reduces generated tokens (Table 5: 11,380 vs 14,220, a 20% drop), that is a potential experimental confound, not a circular reduction—the critic's contribution is not defined as the token count, and the headline gain over the typed council is reported at a matched budget. The process metrics (contradiction resolution, unsupported consensus) are internal to the ledger, but they are secondary and do not carry the main decision-quality claim. There are no self-citations, imported uniqueness theorems, or ansatz-smuggling citations; the references are external and none is load-bearing in a way that reduces a result to a prior assumption. The benchmark labels are author-adjudicated, which raises external-validity concerns, but the test set is hidden and no fitting to it is described, so this is not circularity. Overall, the derivation chain is self-contained with respect to the measured outcomes.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the self-built benchmark and labels, the assumption that guideline clauses as encoded are correct, and the assumption that the LLM backbone and token-budget control are adequate. No numeric free parameters are fitted to make the result; the safety threshold and critic iteration budget are tunable but not reported, so the headline operating point is not pinned down.

free parameters (2)
  • safety release threshold = not reported (only operating points on a threshold curve)
    Section 6.2 reports a specific operating point (harmful release 0.039, false deferral 0.043 above baseline) and Section 7 says threshold selection is tunable. The exact threshold values used to produce the reported results are not given, so the headline safety numbers are threshold-dependent.
  • critic iteration budget = not reported
    Section 4.4 says debate terminates when there is no unresolved high-risk contradiction or when the iteration budget is exhausted. The budget value is not specified, and it affects the balance between contradiction resolution and cost.
assumptions (4)
  • domain assumption The expert-authored action graphs and 'decisive evidence' markings in the benchmark are correct and unbiased.
    Section 5.1 describes a two-pass adjudication process but provides no external validation or inter-rater reliability for the gold labels.
  • domain assumption EANO/RANO/NCCN/WHO guideline clauses, encoded as retrievable evidence units, are treated as ground truth for treatment validity.
    Sections 2.3 and 4.2 use versioned guidelines as decision constraints; guideline correctness is assumed, not validated.
  • domain assumption The LLM backbone (not named in the paper) is instruction-following and deterministic enough that measured differences across variants are attributable to protocol, not haphazard model behavior.
    Section 5.2 says the same model version and temperature are used across variants, but the model is not identified.
  • domain assumption The 'matched token budget' acts as an effective control for compute across variants.
    Sections 5.2 and 6.3 claim a matched budget, but Table 5 shows token usage varies across variants from 11,380 to 14,560, so the budget cap may not bind equally.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TumorBoard: Evidence-Grounded Multi-Agent Decision Support for Longitudinal Neuro-Oncology." pith.science (2026). https://pith.science/paper/TYMNATW4

@misc{pith2026260803190,
  author       = {Pith},
  title        = {Pith review of: TumorBoard: Evidence-Grounded Multi-Agent Decision Support for Longitudinal Neuro-Oncology},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYMNATW4}},
  note         = {Machine review of arXiv:2608.03190}
}
read the original abstract

Neuro-oncology decisions require coordinated interpretation of serial MRI, pathology, molecular markers, treatment history, performance status, and evolving guidelines. We present TumorBoard, a multi-agent decision-support system built around a shared longitudinal case state and an auditable claim-evidence ledger. Specialist agents for radiology, neuropathology, molecular diagnosis, guidelines, and therapy planning produce atomic claims with provenance. An adversarial critic exposes contradictions, and a safety governor releases, qualifies, or defers recommendations according to evidence sufficiency and temporal validity. On a 360-case hidden benchmark at a matched token budget, TumorBoard achieved an action F1 of 0.772 and evidence entailment of 0.914. It exceeded the strongest typed-council baseline by 3.1 percentage points (95% CI: 1.6 to 4.7, adjusted p = 0.0012), while recommendation-to-evidence coverage reached 0.927. Under evidence deletion, the system deferred 84.2% of unsafe cases and limited harmful recommendations to 5.8%. The safety governor reduced harmful release by 7.8 percentage points at a false-deferral cost of 4.3 percentage points. Ablation studies of the ledger, critic, and governor produced the predicted failure patterns, establishing structured coordination as the source of the measured multi-agent advantage.

Figures

Figures reproduced from arXiv: 2608.03190 by the authors.

Figure 1
Figure 1. Longitudinal TumorBoard workflow. Heterogeneous patient records enter a timeline curator, shared case state, specialist council, safety governor, and provenance-preserving output. MetaGPT, and AgentBench provide orchestration patterns and evaluation lessons [4, 5, 6, 7]. TumorBoard replaces free-form conversation with a typed protocol. Messages contain claims, evidence pointers, confidence, prerequisites, contradict… view at source ↗
Figure 2
Figure 2. Specialist communication network. Every proposal enters a claim-evidence ledger, and contradictions remain explicit until resolved by the chair [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Claim-evidence ledger and safety gate. Atomic claims pass retrieval, entailment, conflict resolution, and risk checks before release. 5.5 5.5 Statistical analysis Primary comparison uses paired case-level bootstrap confi￾dence intervals for action-graph F1 and a mixed-effects ordinal model for expert ratings. Harmful recommendation rates use exact paired tests. Inter-rater reliability uses weighted kappa and intracl… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Evaluation matrix separating decision quality, evidence fidelity, process safety, and operational cost [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Paired action-F1 gain over the strongest budget-matched baseline across case strata. 6.2 6.2 Safety under evidence and guideline per￾turbation Under evidence deletion, TumorBoard deferred 84.2% of cases and limited harmful recommendations to 5.8%. Harmful rec￾ommendati…
Figure 6
Figure 6. Figure 6: Harmful-release and false-deferral rates across safety release thresholds [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 19 canonical work pages

  1. [1]

    ReAct: Synergizing Reasoning and Acting in Language Models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, et al.. ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations, 2023.https://arxiv.org/ abs/2210.03629

    arXiv 2023

  2. [2]

    Tool- former: Language Models Can Teach Themselves to Use Tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, et al.. Tool- former: Language Models Can Teach Themselves to Use Tools. Advances in Neural Information Processing Systems, 2023. https://arxiv.org/abs/2302.04761

    arXiv 2023

  3. [3]

    Reflexion: Language Agents with Verbal Reinforcement Learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. Advances in Neural Infor- mation Processing Systems, 2023. https://arxiv.org/ abs/2303.11366

    arXiv 2023

  4. [4]

    AutoGen: Enabling Next-Gen LLM Ap- plications via Multi-Agent Conversation

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erhan Yilmaz, et al.. AutoGen: Enabling Next-Gen LLM Ap- plications via Multi-Agent Conversation. arXiv, 2023. https: //arxiv.org/abs/2308.08155

    arXiv 2023

  5. [5]

    CAMEL: Communica- tive Agents for Mind Exploration of Large Scale Language Model Society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, Bernard Ghanem. CAMEL: Communica- tive Agents for Mind Exploration of Large Scale Language Model Society. Advances in Neural Information Process- ing Systems, 2023. https://arxiv.org/abs/2303. 17760

    work page 2023

  6. [6]

    MetaGPT: Meta Pro- gramming for Multi-Agent Collaborative Framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, et al.. MetaGPT: Meta Pro- gramming for Multi-Agent Collaborative Framework. Interna- tional Conference on Learning Representations, 2024. https: //arxiv.org/abs/2308.00352

    arXiv 2024

  7. [7]

    AgentBench: Evaluating LLMs as Agents

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, et al.. AgentBench: Evaluating LLMs as Agents. International Conference on Learning Representations, 2024. https://arxiv.org/abs/2308.03688

    arXiv 2024

  8. [8]

    Large Language Models as Agents in the Clinic

    Paul Hager, Friederike Jungmann, Roberto Holland, Keno Bhagat, Irina Hubrecht, Michael Knauer, et al.. Large Lan- guage Models as Agents in the Clinic. arXiv, 2023. https: //arxiv.org/abs/2309.10895

    work page Pith review arXiv 2023

Show all 38 references
  1. [9]

    Kung, Morgan Cheatham, Arielle Medenilla, Cza- rina Sillos, Lorie De Leon, Camille Elepaño, et al

    Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Cza- rina Sillos, Lorie De Leon, Camille Elepaño, et al.. Perfor- mance of ChatGPT on USMLE: Potential for AI-assisted med- ical education using large language models. PLOS Digital Health, 2023. https://doi.org/10.1371/journ...

    2023 doi

  2. [10]

    Sara Mahdavi, Jason Lee, Hyung Won Chung, et al

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Lee, Hyung Won Chung, et al.. Large language models encode clinical knowledge. Nature, 2023. https://doi. org/10.1038/s41586-023-06291-2

    2023 doi

  3. [11]

    Collins, Karel G.M

    Professor Gary S. Collins, Karel G.M. Moons, Paula Dhiman, Richard D Riley, Andrew L. Beam, Ben Van Calster, et al.. TRIPOD+AI statement: updated guidance for reporting clin- ical prediction models that use regression or machine learn- ing methods. BMJ, 2024. https://doi.org/1...

    2024

  4. [12]

    van den Bent, Matthias Preusser, Émilie Le Rhun, Jörg C

    Michael Weller, Martin J. van den Bent, Matthias Preusser, Émilie Le Rhun, Jörg C. Tonn, Giuseppe Minniti, et al.. EANO guidelines on the diagnosis and treatment of diffuse gliomas of adulthood. Nature Reviews Clinical Oncology, 2020. https: //doi.org/10.1038/s41571-020-00447-z

    2020 doi

  5. [13]

    Krumholz, Jure Leskovec, Eric J

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M. Krumholz, Jure Leskovec, Eric J. Topol, et al.. Foundation models for generalist medical artificial in- telligence. Nature, 2023. https://doi.org/10.1038/ s41586-023-05881-4

    2023

  6. [14]

    ChatGPT in medicine: an overview of its applications, advantages, lim- itations, future prospects, and ethical considerations

    Tirth Dave, Sai Anirudh Athaluri, Satyam Singh. ChatGPT in medicine: an overview of its applications, advantages, lim- itations, future prospects, and ethical considerations. Fron- tiers in Artificial Intelligence, 2023. https://doi.org/ 10.3389/frai.2023.1169595

    2023

  7. [15]

    Wen, Michael Weller, Eudocia Q

    Patrick Y . Wen, Michael Weller, Eudocia Q. Lee, Brian M. Alexander, Jill S. Barnholtz-Sloan, Floris P Barthel, et al.. Glioblastoma in adults: a Society for Neuro-Oncology (SNO) and European Society of Neuro-Oncology (EANO) consen- sus review on current management and future ...

    2020

  8. [16]

    Kolbinger, Hannah Sophie Muti, Zunamys I

    Jan Clusmann, Fiona R. Kolbinger, Hannah Sophie Muti, Zunamys I. Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, et al.. The future landscape of large language mod- els in medicine. Communications Medicine, 2023. https: //doi.org/10.1038/s43856-023-00370-1

    2023 doi

  9. [17]

    Bertalan Meskó, Eric J. Topol. The imperative for regulatory oversight of large language models (or generative AI) in health- care. npj Digital Medicine, 2023. https://doi.org/10. 1038/s41746-023-00873-0

    2023

  10. [18]

    K. K. Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, et al.. Toward expert- level medical question answering with large language models. Nature Medicine, 2025. https://doi.org/10.1038/ s41591-024-03423-7

    2025

  11. [19]

    Retrieval-Augmented Generation for Large Language Models: A Survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, et al.. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv (Cornell University), 2023. https://doi.org/10.48550/arxiv.2312.10997

    2023 doi arXiv

  12. [20]

    Adapted large language models can outperform medical experts in clinical text summarization

    Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean- Benoit Delbrouck, Asad Aali, Christian Bluethgen, et al.. Adapted large language models can outperform medical experts in clinical text summarization. Nature Medicine, 2024.https: //doi.org/10.1038/s41591-024-02855-5

    2024 doi

  13. [21]

    Structured infor- mation extraction from scientific text with large language mod- els

    John Dagdelen, Alexander Dunn, Sang-Hoon Lee, Nicholas Walker, Andrew Rosen, Gerbrand Ceder, et al.. Structured infor- mation extraction from scientific text with large language mod- els. Nature Communications, 2024. https://doi.org/ 10.1038/s41467-024-45563-x

    2024 doi

  14. [22]

    A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, et al.. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. conference-paper, 2024. https://doi.org/10.1145/ 3637528.3671470

    2024

  15. [23]

    Evaluation and mitigation of the limitations of large language models in clinical decision-making

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, et al.. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine, 2024. https: //doi.org/10.1038/s41591-024-03097-1

    2024 doi

  16. [24]

    Ahluwalia, Joachim M

    Burt Nabors, Jana Portnow, Manmeet S. Ahluwalia, Joachim M. Baehring, Henry Brem, Steven Brem, et al.. Central Ner- vous System Cancers, Version 3.2020, NCCN Clinical Prac- tice Guidelines in Oncology. Journal of the National Com- prehensive Cancer Network, 2020.https://doi.or...

    2020

  17. [25]

    Pappas, Carol A

    Peter G. Pappas, Carol A. Kauffman, David R. Andes, Daniel K. Benjamin, Thierry Calandra, John E. Edwards, et al.. Clinical Practice Guidelines for the Management Candidiasis: 2009 Update by the Infectious Diseases Society of America. Clinical Infectious Diseases, 2009. https:...

    2009

  18. [26]

    The TRIPOD-LLM reporting guideline for studies using large language models

    Jack Gallifant, Majid Afshar, Saleem Ameen, Yindalon Aphinyanaphongs, Shan Chen, Giovanni Cacciamani, et al.. The TRIPOD-LLM reporting guideline for studies using large language models. Nature Medicine, 2025. https://doi. org/10.1038/s41591-024-03425-5

    2025 doi

  19. [27]

    Guthier, Danielle S

    Elizabeth Huynh, Ahmed Hosny, Christian V . Guthier, Danielle S. Bitterman, Steven Petit, Daphne A. Haas-Kogan, et al.. Artificial intelligence in radiation oncology. Nature Reviews Clinical Oncology, 2020. https://doi.org/10.1038/ s41571-020-0417-8

    2020

  20. [28]

    Wright, Barron L

    Siru Liu, Aileen P. Wright, Barron L. Patterson, Jonathan P. Wanderer, Robert W. Turer, Scott D. Nelson, et al.. Using AI- generated suggestions from ChatGPT to optimize clinical deci- sion support. Journal of the American Medical Informatics As- sociation, 2023. https://doi.o...

    2023 doi

  21. [29]

    Nestor, Ali Soroush, Pierre Elias, et al

    Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G. Nestor, Ali Soroush, Pierre Elias, et al.. Evaluating large lan- guage models on medical evidence summarization. npj Digital Medicine, 2023. https://doi.org/10.1038/ s41746-023-00896-7

    2023

  22. [30]

    Dalal, Jennifer L

    Cyril Zakka, Rohan Shad, Akash Chaurasia, Alex R. Dalal, Jennifer L. Kim, Michael Moor, et al.. Almanac - Retrieval- Augmented Language Models for Clinical Medicine. NEJM AI, 2024.https://doi.org/10.1056/aioa2300068

    2024 doi

  23. [31]

    A framework for human evaluation of large language models in healthcare derived from literature review

    Thomas Yu Chow Tam, Sonish Sivarajkumar, Sumit Kapoor, Alisa V Stolyar, Katelyn Polanska, Karleigh R McCarthy, et al.. A framework for human evaluation of large language models in healthcare derived from literature review. npj Digital Medicine, 2024. https://doi.org/10.1038/ s...

    2024

  24. [32]

    Keloth, Xu Zuo, et al

    Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina K. Keloth, Xu Zuo, et al.. Improving large language models for clinical named entity recognition via prompt engineering. Jour- nal of the American Medical Informatics Association, 2024. https://doi.org/10.1093/jamia/ocad259

    2024 doi

  25. [33]

    Opportunities and challenges for ChatGPT and large language models in biomedicine and health

    Shubo Tian, Qiao Jin, Lana Yeganova, Po-Ting Lai, Qingqing Zhu, Xiuying Chen, et al.. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Briefings in Bioinformatics, 2023. https://doi.org/10. 1093/bib/bbad493

    2023

  26. [34]

    Large lan- guage models in health care: Development, applications, and challenges

    Rui Yang, Ting Fang Tan, Wei Lu, Arun James Thirunavukarasu, Daniel Shu Wei Ting, Nan Liu. Large lan- guage models in health care: Development, applications, and challenges. Health care science, 2023. https://doi.org/ 10.1002/hcs2.61

    2023 doi

  27. [35]

    Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine

    Stefan Harrer. Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine. EBioMedicine, 2023. https://doi.org/ 10.1016/j.ebiom.2023.104512

    2023

  28. [36]

    Kamel Boulos

    Peng Zhang, Maged N. Kamel Boulos. Generative AI in Medicine and Healthcare: Promises, Opportunities and Chal- lenges. Future Internet, 2023. https://doi.org/10. 3390/fi15090286

    2023

  29. [37]

    Large Language Models in Healthcare and Medical Domain: A Review

    Zabir Al Nazi, Wei Peng. Large Language Models in Healthcare and Medical Domain: A Review. In- formatics, 2024. https://doi.org/10.3390/ informatics11030057

    2024

  30. [38]

    Miao, Eduardo Rodriguez Al- maraz, Madhumita Sushil, Atul J

    Nikita Mehandru, Brenda Y . Miao, Eduardo Rodriguez Al- maraz, Madhumita Sushil, Atul J. Butte, Ahmed M. Alaa. Evaluating large language models as agents in the clinic. npj Digital Medicine, 2024. https://doi.org/10.1038/ s41746-024-01083-y Table A1:Appendix Table A1. Field co...

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.