pith. sign in

arxiv: 2606.07853 · v1 · pith:5YML43W4new · submitted 2026-06-05 · 💻 cs.CL · cs.AI

Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese

Giordano de Pinho Souza , Glaucia Melo , Josefino Cabral Melo Lima , Daniel Schneider This is my paper

Pith reviewed 2026-06-27 21:38 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords clinical LLM evaluationbilingual benchmarkPortuguese medical casescross-lingual performancediagnosis retrievalclinical decision support
0
0 comments X

The pith

The performance gap between English and Portuguese in clinical LLMs depends on the task, not a general language deficit.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds ClinicalBr, a set of 2,892 parallel Portuguese-English medical cases drawn from Brazilian journals, and tests four models on four clinical tasks. It finds that English holds a clear lead only in diagnosis retrieval, while the gap vanishes or reverses in differential diagnosis, exam recommendation, and treatment planning. Brazilian-endemic conditions prove no harder than the rest of the corpus. Exam recommendation emerges as the weakest task for every model and language.

Core claim

The Portuguese-English performance gap is task-dependent, not general. In diagnosis retrieval, English yields a consistent advantage across all models, with +7.5-12.1 accuracy points. This advantage disappears in differential diagnosis, exam recommendation, and treatment planning, where confidence intervals cross zero for most models and Portuguese completeness scores are marginally higher. Brazilian-endemic conditions proved easier than the full corpus, not harder.

What carries the argument

The ClinicalBr benchmark of parallel Portuguese-English case reports from 28 Brazilian journals, structured into four clinical decision tasks.

If this is right

  • English pre-training data already captures Brazilian tropical conditions adequately.
  • Model developers can focus language-specific effort on diagnosis retrieval rather than every clinical task.
  • Exam recommendation remains the hardest task and needs targeted improvement in both languages.
  • Bilingual evaluation reveals that overall English dominance claims do not hold for clinical decision support.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Task-specific multilingual evaluation may be more useful than single aggregated scores for clinical applications.
  • Future benchmarks could test whether adding Portuguese clinical text during fine-tuning closes the remaining diagnosis gap.

Load-bearing premise

The 2,892 cases from SciELO journals are representative of real clinical decisions and the four tasks measure decision quality without language-specific scoring biases.

What would settle it

A follow-up study on a larger set of real Brazilian hospital records that finds consistent English superiority across all four tasks, or that reveals systematic differences in how Portuguese and English answers are scored.

Figures

Figures reproduced from arXiv: 2606.07853 by Daniel Schneider, Giordano de Pinho Souza, Glaucia Melo, Josefino Cabral Melo Lima.

Figure 1
Figure 1. Figure 1: Accuracy in All cases 4 Results 4.1 The Benchmark ClinicalBr is a Brazilian Portuguese clinical decision benchmark built from the SciELO Online Library. This library covers open-access academic literature from countries across South America. In this work, we use the SciELO Brazil collection of Medicine Journals, which consists of several case-report-type papers containing real information about how real cl… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison 4.4 Exam Recommendation [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Precision, recall, and F1 in all exam recommendation cases [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Treatment subscores The treatment evaluation constitutes the final component of our benchmark assessment. In this task, LLM judges evaluate open-ended responses rather than applying a binary right-or-wrong check. Because relying on a single judge can introduce systematic bias and variations stemming from model training, we employed three distinct judges: Qwen-2.5-72B, Llama-3.3-70B, and gpt-oss-120B. These… view at source ↗
Figure 6
Figure 6. Figure 6: Treatment forest 4.6 Brazilian-endemic Cases Diagnosis (accuracy) DDx (CUI F1) Model Lang Total Tropical ∆ Total Tropical ∆ o3-mini PT 0.558 0.606 +0.048 0.236 0.257 +0.021 EN 0.639 0.648 +0.008 0.231 0.234 +0.002 DeepSeek-R1 PT 0.541 0.601 +0.061 0.263 0.275 +0.013 EN 0.615 0.642 +0.027 0.245 0.240 −0.004 Sabiá-4 PT 0.434 0.513 +0.079 0.214 0.250 +0.036 EN 0.541 0.580 +0.039 0.212 0.230 +0.018 MedGemma-27… view at source ↗
Figure 7
Figure 7. Figure 7: Tropical Forest 5 Discussion Cross-lingual gaps between language models can become a safety concern when they are utilized in a clinical decision support tool. Language models are used daily to provide accessible care information and support for users and healthcare workers, but when it comes to their implementation in critical scenarios, language performance needs to be evaluated. Our results showed that … view at source ↗
read the original abstract

Large Language Models are transforming the support for clinical decision and their application in real scenarios. Yet, most benchmarks are conducted in English, and cross-lingual evaluation is needed to tackle the language gaps in global access. We introduce ClinicalBr, the first bilingual benchmark for clinical decision built from real Brazilian case reports. The corpus contains 2,892 cases drawn from 28 SciELO medical journals, spanning 18 specialties, and is structured as parallel Portuguese-English pairs. Each case supports four evaluation tasks: diagnosis retrieval, differential diagnosis, exam recommendation, and treatment planning. We evaluate four models: MedGemma-27B, Sabi\'a-4, DeepSeek-R1, and o3-mini, across both languages. The central finding is that the Portuguese-English performance gap is task-dependent, not general. In diagnosis retrieval, English yields a consistent advantage across all models, with +7.5-12.1 accuracy points. This advantage disappears in differential diagnosis, exam recommendation, and treatment planning, where confidence intervals cross zero for most models and Portuguese completeness scores are marginally higher. Brazilian-endemic conditions proved easier than the full corpus, not harder, indicating that tropical presentations are adequately represented in current pre-training. Exam recommendation was the hardest task across all models and both languages, with F1 scores below 0.10, well below the differential diagnosis ceiling of 0.20-0.27.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces ClinicalBr, a bilingual benchmark of 2,892 parallel Portuguese-English clinical cases extracted from 28 SciELO journals spanning 18 specialties. Four models (MedGemma-27B, Sabiá-4, DeepSeek-R1, o3-mini) are evaluated on four tasks: diagnosis retrieval, differential diagnosis, exam recommendation, and treatment planning. The central claim is that the Portuguese-English performance gap is task-dependent rather than general: English shows a consistent advantage of +7.5–12.1 accuracy points only in diagnosis retrieval, while the gap disappears (confidence intervals cross zero) in the other three tasks, with Portuguese completeness scores sometimes marginally higher. Brazilian-endemic conditions are reported as easier than the full corpus.

Significance. If the evaluation pipeline is free of language-specific artifacts, the result would demonstrate that cross-lingual gaps in clinical LLMs are task-specific rather than uniform, with direct implications for model deployment in Portuguese-speaking clinical settings. The benchmark construction from real case reports and the finding that endemic conditions are not underrepresented are also potentially useful contributions to multilingual clinical NLP.

major comments (2)
  1. [Methods / Evaluation setup] The manuscript provides no description of how ground-truth labels were constructed for the four tasks (diagnosis lists, differential sets, exam recommendations, treatment plans) from the original SciELO case reports, nor of the annotation process, inter-annotator agreement, or whether the same procedure was applied identically to Portuguese and English versions. This is load-bearing for the central claim that the gap is task-dependent, because any language-specific bias in label extraction or scoring could artifactually produce an English advantage only on diagnosis retrieval.
  2. [Evaluation / Results] No information is given on the scoring rubrics, whether human or automated judges were used, how multi-label outputs were evaluated (e.g., exact match vs. partial credit), or the statistical procedure used to compute the reported confidence intervals. Without these details the reported F1 scores (exam recommendation <0.10, differential diagnosis 0.20–0.27) and the claim that intervals cross zero cannot be interpreted.
minor comments (2)
  1. [Abstract] The model name appears as 'Sabi\'a-4' in the abstract; confirm the correct spelling and citation.
  2. [Abstract] The abstract states that Brazilian-endemic conditions 'proved easier' but does not report the exact subset size or the statistical test used for this comparison.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and will revise the manuscript accordingly to improve methodological transparency.

read point-by-point responses

  1. Referee: [Methods / Evaluation setup] The manuscript provides no description of how ground-truth labels were constructed for the four tasks (diagnosis lists, differential sets, exam recommendations, treatment plans) from the original SciELO case reports, nor of the annotation process, inter-annotator agreement, or whether the same procedure was applied identically to Portuguese and English versions. This is load-bearing for the central claim that the gap is task-dependent, because any language-specific bias in label extraction or scoring could artifactually produce an English advantage only on diagnosis retrieval.

    Authors: We agree that the current description of ground-truth construction is insufficient for fully supporting the central claim. The manuscript's Methods section notes extraction from SciELO case reports but omits the required details on annotation guidelines, inter-annotator agreement, and language-identical application. In the revised manuscript we will add an expanded subsection that specifies the exact extraction procedure for each task's labels, the clinical annotation protocol and guidelines, inter-annotator agreement metrics, and explicit confirmation that the identical process was followed for both language versions. revision: yes

  2. Referee: [Evaluation / Results] No information is given on the scoring rubrics, whether human or automated judges were used, how multi-label outputs were evaluated (e.g., exact match vs. partial credit), or the statistical procedure used to compute the reported confidence intervals. Without these details the reported F1 scores (exam recommendation <0.10, differential diagnosis 0.20–0.27) and the claim that intervals cross zero cannot be interpreted.

    Authors: We acknowledge that the Evaluation section lacks the necessary implementation details. The manuscript reports F1 scores and confidence intervals but does not describe the rubrics, judge type, multi-label handling, or statistical method. In the revision we will expand this section to define the scoring rubrics for each task (including partial-credit rules for multi-label outputs), state whether automated or human evaluation was used, specify the multi-label metric, and detail the procedure (e.g., bootstrap) for the reported confidence intervals. revision: yes

Circularity Check

0 steps flagged

Empirical benchmark study with no derivations or self-referential structure

full rationale

The paper constructs a parallel Portuguese-English clinical case corpus from SciELO journals and reports direct accuracy/F1 measurements across four tasks for four LLMs. No equations, fitted parameters, predictions derived from inputs, uniqueness theorems, or self-citations appear in the provided text. The central claim (task-dependent language gap) is presented as an observed pattern from the evaluations rather than a reduction to any prior result or definition within the work itself. This is a standard empirical benchmark comparison whose results stand or fall on the dataset construction and scoring procedure, not on any internal circular chain.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claim rests on the assumption that the selected cases and tasks constitute a valid test of clinical LLM capability; no free parameters, invented entities, or non-standard axioms are introduced in the abstract.

pith-pipeline@v0.9.1-grok · 5791 in / 1142 out tokens · 18721 ms · 2026-06-27T21:38:33.087516+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 18 canonical work pages · 5 internal anchors

  1. [1]

    A Survey of Large Language Models,

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A Survey of Large Language Models,

  2. [2]

    Towards building multilingual language model for medicine.Nature Communications, 15(1):8384, September 2024

    Pengcheng Qiu, Chaoyi Wu, Xiaoman Zhang, Weixiong Lin, Haicheng Wang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards building multilingual language model for medicine.Nature Communications, 15(1):8384, September 2024

    2024

  3. [3]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushman, Blaise Agüera y Arcas, Dale Webster, Greg S. Corrado,...

    2023

  4. [4]

    MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering. InProceedings of the Conference on Health, Inference, and Learning, pages 248–260. PMLR, April 2022

    2022

  5. [5]

    Quantifying the reasoning abilities of LLMs on clinical cases.Nature Communications, 16(1):9799, November 2025

    Pengcheng Qiu, Chaoyi Wu, Shuyu Liu, Yanjie Fan, Weike Zhao, Zhuoxia Chen, Hongfei Gu, Chuanjin Peng, Ya Zhang, Yanfeng Wang, and Weidi Xie. Quantifying the reasoning abilities of LLMs on clinical cases.Nature Communications, 16(1):9799, November 2025

    2025

  6. [6]

    Diagnosis- arena: Benchmarking diagnostic reasoning for large language models,

    Yakun Zhu, Zhongzhen Huang, Linjie Mu, Yutong Huang, Wei Nie, Jiaji Liu, Shaoting Zhang, Pengfei Liu, and Xiaofan Zhang. DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models, May 2025. arXiv:2505.14107 [cs]

    work page arXiv 2025

  7. [7]

    Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. HealthBench: Evaluating Large Language Models Towards Improved Human Health, May 2025. arXiv:2505.08775 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding

    Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding, June 2025. arXiv:2501.18362 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark, November

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark, November

  10. [10]

    arXiv:2406.01574 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Alvorada-Bench: Can Language Models Solve Brazilian University Entrance Exams?, August

    Henrique Godoy. Alvorada-Bench: Can Language Models Solve Brazilian University Entrance Exams?, August

  12. [12]

    arXiv:2508.15835 [cs]

    work page arXiv

  13. [13]

    Class of LLMs: Benchmarking Large Language Models on the Brazilian National Medical Examination

    João Vitor Mariano Correia, Pedro Henrique Alves de Castro, Gabriel Lino Garcia, Pedro Henrique Paiola, and João Paulo Papa. Class of LLMs: Benchmarking Large Language Models on the Brazilian National Medical Examination. In Marlo Souza, Iria de Dios-Flores, Diana Santos, Larissa Freitas, Jackson Wilke da Cruz Souza, and Eugénio Ribeiro, editors,Proceedin...

    2026

  14. [14]

    Zero-shot performance of selected large language and multimodal models on the 2023 Brazilian Portuguese medical residency exam.Scientific Reports, March 2026

    César Augusto Madid Truyts, Amanda Gomes Rabelo, Gabriel Mesquita de Souza, Daniel Scaldaferri Lages, Adriano José Pereira, Uri Adrian Prync Flato, Eduardo Pontes dos Reis, Joaquim Edson Vieira, Paulo Sergio Panse Silveira, and Edson Amaro Junior. Zero-shot performance of selected large language and multimodal models on the 2023 Brazilian Portuguese medic...

    2023

  15. [15]

    Benchmarking open-source large language models on Portuguese Revalida multiple-choice questions

    João Victor Bruneti Severino, Pedro Angelo Basei de Paula, Matheus Nespolo Berger, Filipe Silveira Loures, Solano Amadori Todeschini, Eduardo Augusto Roeder, Maria Han Veiga, Murilo Guedes, and Gustavo Lenci Marques. Benchmarking open-source large language models on Portuguese Revalida multiple-choice questions. BMJ Health & Care Informatics, 32(1):e10119...

    2025

  16. [16]

    A Step Forward for Medical LLMs in Brazilian Portuguese: Establishing a Benchmark and a Strong Baseline

    Gabriel Lino Garcia, João Renato Ribeiro Manesco, Pedro Henrique Paiola, Pedro Henrique Crespan Ribeiro, Ana Lara Alves Garcia, and João Paulo Papa. A Step Forward for Medical LLMs in Brazilian Portuguese: Establishing a Benchmark and a Strong Baseline. In2025 IEEE 38th International Symposium on Computer- Based Medical Systems (CBMS), pages 214–219, June...

    2025

  17. [17]

    Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries

    Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, and Srijan Kumar. Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries. InProceedings of the ACM Web Conference 2024, WWW ’24, pages 2627–2638, New York, NY , USA, May 2024. Association for Computing Machinery

    2024

  18. [18]

    Desai, Emily Alsentzer, Leo Anthony Celi, Adam Rodman, Sebastian Schneeweiss, Jonathan H

    Jiageng Wu, Bowen Gu, Ren Zhou, Kevin Xie, Doug Snyder, Yixing Jiang, Valentina Carducci, Richard Wyss, Rishi J. Desai, Emily Alsentzer, Leo Anthony Celi, Adam Rodman, Sebastian Schneeweiss, Jonathan H. Chen, Santiago Romero-Brufau, Kueiyu Joshua Lin, and Jie Yang. BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice T...

    work page arXiv 2026

  19. [19]

    Large language model comparisons between English and Chinese query performance for cardiovascular prevention.Communications Medicine, 5(1):177, May 2025

    Hongwei Ji, Xiaofei Wang, Ching-Hui Sia, Jonathan Yap, Soo Teik Lim, Andie Hartanto Djohan, Yaowei Chang, Ning Zhang, Mengqi Guo, Fuhai Li, Zhi Wei Lim, Ya Xing Wang, Bin Sheng, Tien Yin Wong, Susan Cheng, Khung Keong Yeo, and Yih-Chung Tham. Large language model comparisons between English and Chinese query performance for cardiovascular prevention.Commu...

    2025

  20. [20]

    PubMedQA: A Dataset for Biomedi- cal Research Question Answering

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A Dataset for Biomedi- cal Research Question Answering. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors,Proceedings 14 ClinicalBr of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natura...

    2019

  21. [21]

    Association for Computational Linguistics

  22. [22]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams.arXiv preprint arXiv:2009.13081, 2020

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams, September 2020. arXiv:2009.13081 [cs]

    work page arXiv 2020

  23. [23]

    arXiv preprint arXiv:2506.22405 , year=

    Harsha Nori, Mayank Daswani, Christopher Kelly, Scott Lundberg, Marco Tulio Ribeiro, Marc Wilson, Xiaoxuan Liu, Viknesh Sounderajah, Jonathan Carlson, Matthew P. Lungren, Bay Gross, Peter Hames, Mustafa Suleyman, Dominic King, and Eric Horvitz. Sequential Diagnosis with Language Models, June 2025. arXiv:2506.22405 [cs] version: 1

    work page arXiv 2025

  24. [24]

    Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions, February 2024

    Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions, February 2024

    2024

  25. [25]

    Lawrence K. Q. Yan, Qian Niu, Ming Li, Yichao Zhang, Caitlyn Heqi Yin, Cheng Fei, Benji Peng, Ziqian Bi, Pohsun Feng, Keyu Chen, Tianyang Wang, Yunze Wang, Silin Chen, Ming Liu, Junyu Liu, Xinyuan Song, Riyang Bao, Zekun Jiang, and Ziyuan Qin. Large Language Model Benchmarks in Medical Tasks, November 2025. arXiv:2410.21348 [cs]

    work page arXiv 2025

  26. [26]

    Integrating Generative Artificial Intelligence into Patient-Centered Clinical Decision Support: A Viewpoint on Research and Practice Considerations

    Prashila Dullabh, Courtney Zott, Nicole Gauthreaux, Caroline Peterson, Kistein Monkhouse, and Dean F Sittig. Integrating Generative Artificial Intelligence into Patient-Centered Clinical Decision Support: A Viewpoint on Research and Practice Considerations

  27. [27]

    Enhancing LLMs with Smart Preprocessing for EHR Analysis, April 2025

    Yixiang Qu, Yifan Dai, Shilin Yu, Pradham Tanikella, Travis Schrank, Trevor Hackman, Didong Li, and Di Wu. Enhancing LLMs with Smart Preprocessing for EHR Analysis, April 2025. arXiv:2412.02868 [cs]

    work page arXiv 2025

  28. [28]

    Chen, Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J

    Sully F. Chen, Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J. Choi, Jin Vivian Lee, Amelia L. Chen, Pranav I. Warman, Rochelle T. Bitolas, Robert J. Steele, Daniel A. Alber, and Eric K. Oermann. LLM-assisted systematic review of large language models in clinical medicine.Nature Medicine, 32(3):1152–1159, March 2026

    2026

  29. [29]

    Digitization of healthcare sector: A study on privacy and security concerns.ICT Express, 9(4):571–588, August 2023

    Metty Paul, Leandros Maglaras, Mohamed Amine Ferrag, and Iman Almomani. Digitization of healthcare sector: A study on privacy and security concerns.ICT Express, 9(4):571–588, August 2023

    2023

  30. [30]

    Big data security and privacy in healthcare: A Review.Procedia Computer Science, 113:73–80, 2017

    Karim Abouelmehdi, Abderrahim Beni-Hssane, Hayat Khaloufi, and Mostafa Saadi. Big data security and privacy in healthcare: A Review.Procedia Computer Science, 113:73–80, 2017

    2017

  31. [31]

    High-Fidelity Pseudo-label Generation by Large Language Models for Training Robust Radiology Report Classifiers, May 2025

    Brian Wong and Kaito Tanaka. High-Fidelity Pseudo-label Generation by Large Language Models for Training Robust Radiology Report Classifiers, May 2025. arXiv:2505.01693 [cs]

    work page arXiv 2025

  32. [32]

    Langlotz, Jason Hom, Sergios Gatidis, John Pauly, and Akshay S

    Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerová, Nidhi Rohatgi, Poonam Hosamani, William Collins, Neera Ahuja, Curtis P. Langlotz, Jason Hom, Sergios Gatidis, John Pauly, and Akshay S. Chaudhari. Adapted large language models can...

    2024

  33. [33]

    A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs, April 2025

    Yihan Lin, Zhirong Bella Yu, and Simon Lee. A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs, April 2025

    2025

  34. [34]

    Tran, Daniel I

    Shreya Johri, Jaehwan Jeong, Benjamin A. Tran, Daniel I. Schlessinger, Shannon Wongvibulsin, Leandra A. Barnes, Hong-Yu Zhou, Zhuo Ran Cai, Eliezer M. Van Allen, David Kim, Roxana Daneshjou, and Pranav Rajpurkar. An evaluation framework for clinical use of large language models in patient interaction tasks.Nature Medicine, 31(1):77–86, January 2025

    2025

  35. [35]

    MedAgent-Pro: Towards Evidence-Based Multi- Modal Medical Diagnosis via Reasoning Agentic Workflow, March 2025

    Yueming Jin, Junde Wu, Chang Han Low, and Yueming Jin. MedAgent-Pro: Towards Evidence-Based Multi- Modal Medical Diagnosis via Reasoning Agentic Workflow, March 2025

    2025

  36. [36]

    Khaled Saab, Jan Freyberg, Chunjong Park, Tim Strother, Yong Cheng, Wei-Hung Weng, David G. T. Barrett, David Stutz, Nenad Tomasev, Anil Palepu, Valentin Liévin, Yash Sharma, Roma Ruparel, Abdullah Ahmed, Elahe Vedadi, Kimberly Kanada, Cian Hughes, Yun Liu, Geoff Brown, Yang Gao, Sean Li, S. Sara Mahdavi, James Manyika, Katherine Chou, Yossi Matias, Avina...

    work page arXiv 2025

  37. [37]

    MMEDAGENT-RL: OPTIMIZING MULTI-AGENT COL- LABORATION FOR MULTIMODAL MEDICAL REASON-

    Peng Xia, Jinglu Wang, Yibo Peng, Kaide Zeng, Zihan Dong, Xian Wu, Xiangru Tang, Hongtu Zhu, Yun Li, Linjun Zhang, Shujie Liu, Yan Lu, and Huaxiu Yao. MMEDAGENT-RL: OPTIMIZING MULTI-AGENT COL- LABORATION FOR MULTIMODAL MEDICAL REASON-. 2026. 15 Souza et al

    2026

  38. [38]

    AlpaCare:Instruction-tuned Large Language Models for Medical Application, March 2025

    Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Petzold. AlpaCare:Instruction-tuned Large Language Models for Medical Application, March 2025. arXiv:2310.14558 [cs]

    work page arXiv 2025

  39. [39]

    Suhana Bedi, Hejie Cui, Miguel Fuentes, Alyssa Unell, Michael Wornow, Juan M. Banda, Nikesh Kotecha, Timothy Keyes, Yifan Mai, Mert Oez, Hao Qiu, Shrey Jain, Leonardo Schettini, Mehr Kashyap, Jason Alan Fries, Akshay Swaminathan, Philip Chung, Fateme Nateghi, Asad Aali, Ashwin Nayak, Shivam Vedak, Sneha S. Jain, Birju Patel, Oluseyi Fayanju, Shreya Shah, ...

    work page arXiv 2025

  40. [40]

    Morley, and Luis Filipe Nakayama

    David Restrepo, Chenwei Wu, Zhengxu Tang, Zitao Shuai, Thao Nguyen Minh Phan, Jun-En Ding, Cong-Tinh Dao, Jack Gallifant, Robyn Gayle Dychiao, Jose Carlo Artiaga, André Hiroshi Bando, Carolina Pelegrini Barbosa Gracitelli, Vincenz Ferrer, Leo Anthony Celi, Danielle Bitterman, Michael G. Morley, and Luis Filipe Nakayama. Multi-OphthaLingua: A Multilingual ...

    2025

  41. [41]

    Firecrawl - The API to search, scrape, and interact with the web at scale

    Firecrawl. Firecrawl - The API to search, scrape, and interact with the web at scale

  42. [42]

    MedGemma Technical Report

    Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, Justin Chen, Fereshteh Mahvar, Liron Yatziv, Tiffany Chen, Bram Sterling, Stefanie Anna Baby, Susanna Maria Baby, Jeremy Lai, Samuel Schmidgall, Lu Yang, Kejia Chen, Per Bjornsson, Shashir Reddy, Ryan Br...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  43. [43]

    Sabiá-4 Technical Report, March 2026

    Thiago Laitz, Thales Sales Almeida, Hugo Abonizio, Roseval Malaquias Junior, Giovana Kerche Bonás, Marcos Piau, Celio Larcher, Ramon Pires, and Rodrigo Nogueira. Sabiá-4 Technical Report, March 2026. arXiv:2603.10213 [cs]

    work page arXiv 2026

  44. [44]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  45. [45]

    Introducing OpenAI o3 and o4-mini

  46. [46]

    Unified Medical Language System (UMLS)

  47. [47]

    [correct]

    Fangyu Liu, Ivan Vuli´c, Anna Korhonen, and Nigel Collier. Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking, May 2021. arXiv:2105.14398 [cs.CL]. 17 Souza et al. 7 Suplementary 7.1 Qualitative Case Study In this section, we are gonna walk one real case through each of the four tasks, contrasting the two strongest reas...

    work page arXiv 2021

  48. [48]

    Yes”. If the report does not primarily address the task of diagnosis/therapeutic planning, respond with “No

    [...] Surgical excision remains the treatment of choice for localized laryngeal amyloidosis. In this case, the recommended approach is: • Surgical Procedure: Under general anesthesia, perform [...] Judge scores (3-judge mean /5) Accuracy 3.67, Completeness 4.67, Clarity 5.00 Accuracy 5.00, Completeness 4.00, Clarity 5.00 Table 6: 22 ClinicalBr 8 Prompts 9...