Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DrVD-Bench, a 7,789-question benchmark across five imaging modalities, claims that state-of-the-art vision-language models can produce clinically plausible medical diagnoses without identifying the supporting lesion evidence.

desk verdict Solid new benchmark with a nice staged reasoning design; the headline 'overdiagnosis without understanding' claim is not supported by their own same-image data and needs rework. read the letter →

arxiv 2505.24173 v1 pith:YFUAIQLK submitted 2025-05-30 cs.CV

Tianhong Zhou , Yin Xu , Yingtao Zhu , Chuxi Xiao , Haiyang Bian , Lei Wei , Xuegong Zhang This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelsmedicalimagediagnosisclinicalreasoningbenchmarkvisualevidencegroundingzero-shotevaluationreportgenerationmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces DrVD-Bench, a benchmark of 7,789 image–question pairs that tests whether vision-language models (VLMs) follow the stepwise reasoning of a clinician—modality to anatomy to lesion to diagnosis—across CT, MRI, ultrasound, radiography, and pathology. The paper's central claim is that current VLMs show a sharp performance drop as reasoning complexity increases, and that many models can produce correct diagnoses without identifying the supporting lesion evidence, a phenomenon the authors call 'overdiagnosis without understanding.' The authors benchmark 19 models and argue that the benchmark's structured levels, erasure tasks, and multi-turn reasoning formats expose where models rely on shortcut correlations rather than grounded visual understanding. If the finding holds, high diagnostic accuracy in medical VLMs cannot be trusted as evidence of clinically sound reasoning.

What carries the argument

The load-bearing instrument is DrVD-Bench's five-level task hierarchy—image quality, basic information, anatomy, lesion, and clinical interpretation—paired with organ-erasure and lesion-erasure tasks that force models to rely on visible image content. The erasure-task design is the mechanism that exposes shortcut reliance: an AI-edited image with the lesion or organ removed tests whether a model's answer depends on the actual visual evidence, so a correct diagnosis on the erased image or a failed detection of the erasure demonstrates that the diagnosis is not grounded in the lesion. The reasoning-trajectory module's three prompting formats (Joint, Independent, Multi-turn) carry the secondary mechanism, showing that models depend on static global context rather than stepwise dialogue.

What would settle it

A paired-image experiment in which every image has both a lesion-localization question and a diagnosis question; if diagnosis accuracy no longer exceeds lesion accuracy when both are measured on the same image, the claimed disconnect would not hold.

Watch

Extended reading notes

Core claim

The central discovery is that state-of-the-art vision-language models, when evaluated on a benchmark organized like the clinical reasoning workflow, do not reason like human doctors: their accuracy declines monotonically as tasks move from basic image information to organs, lesions, and diagnosis, and several models score higher on final diagnosis than on lesion identification. The authors call this 'overdiagnosis without understanding' and attribute it to training data that pairs images with final reports but omits intermediate evidence. The paper also finds that models perform best when all reasoning questions are given at once (Joint QA) and worst when answers must be built turn by turn (Multi-turn QA), indicating difficulty maintaining a reasoning trajectory.

Load-bearing premise

The overdiagnosis-without-understanding result assumes that the diagnosis questions are comparable in difficulty and data familiarity to the lesion questions, even though the two scores come from different images in different datasets.

Editorial extensions

If this is right

  • Medical VLM accuracy on final diagnoses should not be read as evidence of visual grounding; deployment should require passing intermediate lesion-level checks.
  • Training pipelines should include explicit intermediate supervision (lesion localization, anatomy) rather than only image–report pairs, to reduce the diagnosis-without-evidence gap.
  • Models optimized for multi-turn clinical dialogue need better dialogue-state management, since stepwise prompting currently degrades performance.
  • Domain-optimized medical models can match or beat much larger general-purpose models, suggesting medical alignment is a cost-effective direction.
  • Report generation evaluation should include hallucination checks against image evidence, not just lexical or semantic similarity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The benchmark's lack of a human physician baseline means the 'reason like human doctors' question remains open; a natural extension is to run the same 7,789 questions with radiologists and pathologists to calibrate what level of stepwise consistency is actually human-like.
  • The erasure-task design could be turned into a diagnostic tool: if a model's confidence or answer changes when a lesion is removed, that sensitivity score may serve as a post-hoc measure of visual grounding for individual predictions.
  • The overdiagnosis finding suggests a testable training intervention: fine-tuning on lesion-level supervision before diagnosis should raise lesion accuracy and, if the paper's account is right, reduce the diagnosis-lesion gap.
  • Because Module 1 scores are computed on different images per level, a re-analysis that restricts to images with all levels annotated could either confirm or weaken the overdiagnosis result.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces DrVD-Bench, a multimodal benchmark for evaluating vision-language models on medical image diagnosis. The benchmark has three modules: Visual Evidence Comprehension (4,480 image-question pairs organized into a five-level hierarchy from image quality to clinical interpretation), Reasoning Trajectory Assessment (3,321 QA turns over 487 images in Independent, Joint, and Multi-turn formats), and Report Generation Evaluation (475 free-form clinical report questions). Nineteen VLMs, including general-purpose, proprietary, open-source, and medical-specific models, are evaluated zero-shot. The main empirical findings are that accuracy declines as reasoning level increases, that some models appear to achieve higher diagnostic accuracy than lesion-level accuracy (termed 'overdiagnosis without understanding'), that Joint QA outperforms Multi-turn QA, that reports contain hallucinations, and that specialized smaller models can be competitive.

Significance. The benchmark is a potentially useful community resource: it spans five modalities and 24 datasets, ships code and data, and attempts a clinically structured task hierarchy that goes beyond prior VQA benchmarks. The code and data release is a concrete strength, and the fine-grained per-level results could support future comparisons. However, the paper's headline interpretive claim—'overdiagnosis without understanding'—is not supported by the experiments as reported, and the central framing question ('reason like human doctors') is not operationalized with a human baseline. If the authors revise the analysis to be image-matched and add uncertainty quantification, the benchmark and its results could be a solid contribution to medical VLM evaluation.

major comments (4)
  1. [§5.1.2, Tables 4 and 8] The central claim of 'overdiagnosis without understanding' compares diagnosis accuracy with lesion-level accuracy computed on disjoint image sets. Section 3.2.2 states that in Module 1 each image is paired with a single QA, so the diagnosis and lesion accuracies in Table 4 and the CT example in §5.1.2 are not measured on the same images; they also differ in task priors, option counts, and underlying datasets (Table 2 and Appendix A.2). The same-image evidence already present in the paper runs in the opposite direction: Table 5, which reports per-image results for Module 2, shows Diagnosis ≤ Lesion for every model in every QA format (e.g., GPT-4o diagnosis 40/40/41 vs. lesion 45/45/54; Gemini 2.5 Pro diagnosis 48/35/55 vs. lesion 51/38/61). The authors should either reframe the claim as an unpaired task-difficulty observation or provide an image-matched analysis, for example using Module 2 data, that actually demonstrates diagnosis exceeding lesion identification on the same images.
  2. [Abstract and §5.1.2] The benchmark's title and stated purpose ask whether VLMs reason like human doctors, but no human clinician baseline is reported on any of the three modules. The observed level-wise decline in model accuracy is a statement about model behavior; by itself it does not show whether this pattern differs from human clinical reasoning, which is the paper's stated question. A human baseline on a representative subset (for example, a few hundred items from Module 1 and Module 2) is necessary to ground the 'reasoning like human doctors' framing and to calibrate what level of performance gap is clinically meaningful.
  3. [§4.2, Tables 4 and 13] The evaluation section states that all results are averaged over five independent runs, but no confidence intervals, standard errors, or significance tests are reported anywhere in the main text or appendix. Given the reported variation between runs (which motivated the five-run protocol), point estimates alone make it impossible to determine whether differences of a few percentage points—for example, GPT-4o at 64.6 vs. Doubao1.5-VisionPro at 64.7 in Table 13, or the Joint-vs-Multi-turn gaps in Table 5—are meaningful. The authors should provide uncertainty estimates (e.g., bootstrap confidence intervals over questions and runs) and, for headline claims such as the level-wise decline, a statistical test.
  4. [§3.2.3 and Table 8] The erasure tasks are used as evidence in §5.1.2 to argue that models fail to recognize missing lesion evidence, but the task construction may introduce confounds. The lesions are removed digitally via 'AI editing,' and below-random performance on lesion-erasure detection (e.g., GPT-o3 19%, Gemini 2.5 Pro 16% in Table 8) could reflect the inability to detect subtle inpainting artifacts rather than a failure of clinical evidence grounding. The authors should validate the erasure task, for instance by reporting human performance on the same items or showing that the task is not solvable by artifact detection alone; otherwise the interpretation of the below-random scores is ambiguous.
minor comments (5)
  1. [Table 4] The GPT-o3 row reads '6986 6641 48', which appears to be a formatting error merging columns; the table should be cleaned so that each numerical entry is separated and aligned.
  2. [Table 5] The table note says each cell shows Independent/Multi-turn/Joint accuracy, but several cells (e.g., Bodypart for GPT-4o and others) show only two values or a dash without an explanatory note. The caption should clarify the missing entries.
  3. [§3.2.2] The description of Module 1 as 'each image paired with a single QA' is easy to miss at first mention; the contrast with Module 2's multi-question-per-image design is load-bearing for the overdiagnosis claim and deserves an explicit statement earlier in the text.
  4. [§5.1.2] The term 'overdiagnosis without understanding' is memorable but clinically loaded; a more neutral phrase such as 'diagnosis-evidence accuracy gap' would better match the actual measurements and avoid overinterpreting cross-task comparisons.
  5. [Appendix B, Tables 8–12] The appendix tables are difficult to parse because the columns and entries are not cleanly separated in the text. The authors should ensure the final camera-ready version uses proper table formatting with unambiguous column headers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DrVD-Bench reports external measurements on a new benchmark, and the §5.1.2 inference concern is a validity issue, not a circular reduction.

full rationale

DrVD-Bench is an evaluation benchmark; the paper's claims are measurements of model outputs on newly constructed QA pairs. No parameter is fitted to a subset of data and then relabeled as a prediction, and no reported quantity is defined in terms of another quantity it is used to explain. The hierarchical five-level structure is an explicit design choice (Sections 3.1–3.2.2), and the 'overdiagnosis without understanding' claim in Section 5.1.2 is an inference from comparing diagnosis accuracy with lesion-level accuracy. Even if that comparison is not image-matched, because Module 1 pairs each image with a single QA (Section 3.2.2), and even if task difficulty or dataset composition could account for the gap, this is a validity/confounding concern rather than circularity: the gap is not forced by construction, and the same-image Module 2 results are reported in Table 5 rather than hidden. No load-bearing self-citation is present; references such as PubMedVision/HuatuoGPT-Vision are dataset and model citations, not author-generated justifications. The limitations section acknowledges that the structured design may not capture the variability of real clinical decision-making, further confirming that the benchmark's conclusions are empirical rather than definitional. Accordingly, no specific circular step can be exhibited with quoted evidence, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted parameters are used; design choices such as PSNR levels (15, 25, 35 dB) and answer-option counts (4 vs 8) are principled choices, not values fit to data. The load-bearing assumptions are domain assumptions about what clinical reasoning is and how to measure it, plus one comparability assumption across tasks. The paper invents no new physical or conceptual entities. The main risk is the comparability of accuracy across levels.

assumptions (3)
  • domain assumption Clinical diagnosis follows a fixed five-level linear trajectory from image quality to diagnosis.
    The entire task hierarchy (Levels 0-4) is built on this ordering. The paper itself concedes in Section 7 that "its structured task design may fail to capture the non-linearity and variability of real clinical decision-making." If real reasoning is non-linear, the benchmark measures a stylized version of clinical reasoning, not the real thing.
  • domain assumption Accuracy scores at different levels are comparable even though they come from different images.
    Module 1 pairs each image with a single question (Section 3.2.2), so lesion accuracy and diagnosis accuracy are computed on disjoint images and datasets. Section 5.1.2 infers overdiagnosis without understanding from this comparison, which assumes equal difficulty or matched distributions. This is unstated and load-bearing.
  • domain assumption Erasure tasks isolate reliance on visible evidence from memorized associations.
    The erasure tasks are engineered to "force models to rely on visible image content rather than memorized associations" (Section 3.1). This assumes that after digital removal of an organ or lesion, a correct identification of what is missing is a valid measure of visual grounding, and that failure is not caused by unrealistic editing artifacts. No human validation of the edited images' realism is reported beyond "manually verified to ensure artifact-free erasure."

how reviews work

0 comments
Cite this review

Pith. "Pith review of DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?." pith.science (2026). https://pith.science/paper/YFUAIQLK

@misc{pith2026250524173,
  author       = {Pith},
  title        = {Pith review of: DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFUAIQLK}},
  note         = {Machine review of arXiv:2505.24173}
}
read the original abstract

Vision-language models (VLMs) exhibit strong zero-shot generalization on natural images and show early promise in interpretable medical image analysis. However, existing benchmarks do not systematically evaluate whether these models truly reason like human clinicians or merely imitate superficial patterns. To address this gap, we propose DrVD-Bench, the first multimodal benchmark for clinical visual reasoning. DrVD-Bench consists of three modules: Visual Evidence Comprehension, Reasoning Trajectory Assessment, and Report Generation Evaluation, comprising a total of 7,789 image-question pairs. Our benchmark covers 20 task types, 17 diagnostic categories, and five imaging modalities-CT, MRI, ultrasound, radiography, and pathology. DrVD-Bench is explicitly structured to reflect the clinical reasoning workflow from modality recognition to lesion identification and diagnosis. We benchmark 19 VLMs, including general-purpose and medical-specific, open-source and proprietary models, and observe that performance drops sharply as reasoning complexity increases. While some models begin to exhibit traces of human-like reasoning, they often still rely on shortcut correlations rather than grounded visual understanding. DrVD-Bench offers a rigorous and structured evaluation framework to guide the development of clinically trustworthy VLMs.

Figures

Figures reproduced from arXiv: 2505.24173 by the authors.

Figure 1
Figure 1. Overview of the DrVD-Bench ∗Equal contribution. †Corresponding authors: bhy22@mails.tsinghua.edu.cn, weilei92@tsinghua.edu.cn Preprint. arXiv:2505.24173v1 [cs.CV] 30 May 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Hierarchical five-level evaluation framework for medical imaging diagnostics. Representa [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Report generation examples of Gemini 2.5 Pro. A and B represent high-scoring examples, [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (24 more)
Figure 4
Figure 4. Figure 4: Performance of VLMs on DrVD￾Bench visual evidence comprehension tasks across different scales and recencies. Yet scale and recency are not the whole story. While proprietary models dominate, open-source models like Qwen2.5-VL-72B perform compet￾itively despite having f…
Figure 5
Figure 5. Figure 5: Diagram for the composition of our benchmark, which contains 3 modules [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Diagram for the composition of Module 1. Module 1 spans all five modalities, and is [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Diagram for the composition of Module 2. Module 2 contains three parts: Independent QA, [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Performance of VLMs across five medical imaging modalities in visual evidence tasks, [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Normalized scores of report generation across five imaging modalities (sorted by overall [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: An example of an invalid answer, which is marked as wrong. The option high￾lighted in yellow is the correct answer [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 12
Figure 12. Figure 12: An example of Joint QA, in which the model answers four questions from different [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 13
Figure 13. Figure 13: Example for Artifact Recognition [PITH_FULL_IMAGE:figures/full_fig_p029_13.png]
Figure 14
Figure 14. Figure 14: Example for Noise Recognition 29 [PITH_FULL_IMAGE:figures/full_fig_p029_14.png]
Figure 15
Figure 15. Figure 15: Example for Modality Recognition [PITH_FULL_IMAGE:figures/full_fig_p030_15.png]
Figure 16
Figure 16. Figure 16: Example for View Recognition 30 [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: Example for Body Part Recognition [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: Example for Magnification Recognition 31 [PITH_FULL_IMAGE:figures/full_fig_p031_18.png]
Figure 19
Figure 19. Figure 19: Example for Stain Recognition [PITH_FULL_IMAGE:figures/full_fig_p032_19.png]
Figure 20
Figure 20. Figure 20: Example for Imaging Technique Recognition [PITH_FULL_IMAGE:figures/full_fig_p032_20.png]
Figure 21
Figure 21. Figure 21: Example for Organ Recognition [PITH_FULL_IMAGE:figures/full_fig_p033_21.png]
Figure 22
Figure 22. Figure 22: Example for Tissue Recognition 33 [PITH_FULL_IMAGE:figures/full_fig_p033_22.png]
Figure 23
Figure 23. Figure 23: Example for Organ Location [PITH_FULL_IMAGE:figures/full_fig_p034_23.png]
Figure 24
Figure 24. Figure 24: Example for Organ Erasure Recognition 34 [PITH_FULL_IMAGE:figures/full_fig_p034_24.png]
Figure 25
Figure 25. Figure 25: Example for Lesion Recognition [PITH_FULL_IMAGE:figures/full_fig_p035_25.png]
Figure 26
Figure 26. Figure 26: Example for Lesion Erasure Recognition 35 [PITH_FULL_IMAGE:figures/full_fig_p035_26.png]
Figure 27
Figure 27. Figure 27: Example for Morphology Description [PITH_FULL_IMAGE:figures/full_fig_p036_27.png]
Figure 28
Figure 28. Figure 28: Example for Diagnosis 36 [PITH_FULL_IMAGE:figures/full_fig_p036_28.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BabyVision: Visual Reasoning Beyond Language

    cs.CV 2026-01 conditional novelty 6.0 of 10

    On a 388-item benchmark of child-level visual puzzles, the best MLLM scores 49.7% versus 94.1% for adults, exposing a large gap in basic visual reasoning.

  2. 6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Across 22 vision-language models, accuracy on simple medical perception questions dropped from ~74% on typical anatomy to ~29% on rare anatomical variants, with errors aligning to textbook priors.

Reference graph

Works this paper leans on

103 extracted references · 40 canonical work pages · cited by 2 Pith papers

  1. [1]

    Abdin, J

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, J. R. Lee, Y . T. Lee, Y . Li, W. Liu, C. C. T. Mendes, A. Nguyen, E. Price, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, X. Wang, R. Ward, Y . Wu, D. Yu, C. Zhang, and Y . Zhang. Phi-4 technical report, 2024. URLhttps://arxiv.org/abs/2412.08905

    arXiv 2024

  2. [2]

    Al-Dhabyani, M

    W. Al-Dhabyani, M. Gomaa, H. Khaled, and A. Fahmy. Dataset of breast ultrasound images. Data in brief, 28:104863, 2020

    2020

  3. [3]

    Anouk Stein, C

    M. Anouk Stein, C. Wu, C. Carr, G. Shih, J. Dulkowski, kalpathy, L. Chen, L. Prevedello, M. Marc Kohli, M. McDonald, Peter, P. Culliton, S. H. MD, and T. Xia. Rsna pneumonia detection challenge. https://kaggle.com/competitions/ rsna-pneumonia-detection-challenge, 2018. Kaggle

    2018

  4. [4]

    Claude 3.7 sonnet

    Anthropic. Claude 3.7 sonnet. https://www.anthropic.com/claude/sonnet, 2024. Ac- cessed: 2025-05-08

    2024

  5. [5]

    Antonelli, A

    M. Antonelli, A. Reinke, S. Bakas, et al. The medical segmentation decathlon.Nature Communications, 2022. doi: 10.1038/s41467-022-30695-9

    doi:10.1038/s41467-022-30695-9 2022

  6. [6]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y . Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y . Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin. Qwen2.5-vl technical report, 2025. URLhttps://arxiv.org/abs/2502.13923

    arXiv 2025

  7. [7]

    Bakas, H

    S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby, J. B. Freymann, K. Farahani, and C. Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.Scientific data, 4(1):1–13, 2017

    2017

  8. [8]

    Bakas, M

    S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, R. T. Shinohara, C. Berger, S. M. Ha, M. Rozycki, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629, 2018

    arXiv 2018

Show all 103 references
  1. [9]

    Behboodi, F

    B. Behboodi, F. Carton, M. Chabanas, S. De Ribaupierre, O. Solheim, B. Munkvold, H. Rivaz, Y . Xiao, and I. Reinertsen. Resect-seg: Open access annotations of intra-operative brain tumor ultrasound images.arXiv preprint arXiv:2207.07494, 2022

    2022 arXiv

  2. [10]

    Bilic, P

    P. Bilic, P. Christ, H. B. Li, E. V orontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. H. Mamani, G. Chartrand, et al. The liver tumor segmentation benchmark (lits).Medical image analysis, 84:102680, 2023

    2023

  3. [11]

    Doubao vision: Advancing generalist visual understanding

    ByteDance. Doubao vision: Advancing generalist visual understanding. https://seed. bytedance.com/en/direction/vision, 2024. Accessed: 2025-05-08

    2024

  4. [12]

    D. B. L. C. Abdominal ultrasound images. https://www.kaggle.com/datasets/ darsh22blc1378/abdominal-ultrasound-images, 2023. Accessed: 2025-05-15

    2023

  5. [13]

    Cabral, D

    S. Cabral, D. Restrepo, Z. Kanjee, P. Wilson, B. Crowe, R.-E. Abdulnour, and A. Rodman. Clinical reasoning of a generative artificial intelligence model compared with physicians.JAMA Internal Medicine, 184(5):581–583, 2024

    2024

  6. [14]

    F. O. Catak, M. Kuzlu, and T. Patrick. Improving medical diagnostics with vision-language models: Convex hull-based uncertainty analysis, 2024. URLhttps://arxiv.org/abs/2412. 00056

    2024

  7. [15]

    J. Chen, C. Gui, R. Ouyang, A. Gao, S. Chen, G. H. Chen, X. Wang, R. Zhang, Z. Cai, K. Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024

    2024 arXiv

  8. [16]

    L. Chen, B. Li, S. Shen, J. Yang, C. Li, K. Keutzer, T. Darrell, and Z. Liu. Large language models are visual reasoning coordinators.Advances in Neural Information Processing Systems, 36:70115–70140, 2023

    2023

  9. [17]

    L. Chen, J. Li, X. Dong, P. Zhang, Y . Zang, Z. Chen, H. Duan, J. Wang, Y . Qiao, D. Lin, and F. Zhao. Are we on the right way for evaluating large vision-language models?, 2024. URL https://arxiv.org/abs/2403.20330. 10

    2024 arXiv

  10. [18]

    Q. Chen, R. Zhao, S. Wang, V . M. H. Phan, A. van den Hengel, J. Verjans, Z. Liao, M.-S. To, Y . Xia, J. Chen, Y . Xie, and Q. Wu. A survey of medical vision-and-language applications and their techniques, 2024. URLhttps://arxiv.org/abs/2411.12195

    2024 arXiv

  11. [19]

    X. Chen, Z. Wu, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, and C. Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025. URL https: //arxiv.org/abs/2501.17811

    2025 arXiv

  12. [20]

    Chitalia, S

    R. Chitalia, S. Pati, M. Bhalerao, S. Thakur, N. Jahani, J. Belenky, E. McDonald, J. Gibbs, D. Newitt, and N. Hylton. Expert tumor annotations and radiomic features for the ispy1/acrin 6657 trial data collection.The Cancer Imaging Archive, 2022

    2022

  13. [21]

    T. A. D’Antonoli, L. K. Berger, A. K. Indrakanti, N. Vishwanathan, J. Weiß, M. Jung, Z. Berkarda, A. Rau, M. Reisert, T. Küstner, et al. Totalsegmentator mri: Sequence-independent segmentation of 59 anatomical structures in mr images.arXiv preprint arXiv:2405.19492, 2024

    2024 arXiv

  14. [22]

    W. Ding, L. Li, J. Qiu, S. Wang, L. Huang, Y . Chen, S. Yang, and X. Zhuang. Aligning multi- sequence cmr towards fully automated myocardial pathology segmentation.IEEE Transactions on Medical Imaging, 2023

    2023

  15. [23]

    GLM, :, A

    T. GLM, :, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Zhang, D. Rojas, G. Feng, H. Zhao, H. Lai, H. Yu, H. Wang, J. Sun, J. Zhang, J. Cheng, J. Gui, J. Tang, J. Zhang, J. Sun, J. Li, L. Zhao, L. Wu, L. Zhong, M. Liu, M. Huang, P. Zhang, Q. Zheng, R. Lu, S. Duan, S. Zhang, S...

    2024 arXiv

  16. [24]

    H. Gong, J. Chen, G. Chen, H. Li, G. Li, and F. Chen. Thyroid region prior guided attention for ultrasound segmentation of thyroid nodules.Computers in Biology and Medicine, 155:106389, 2023

    2023

  17. [25]

    Gemini: Our most capable model, now with better plan- ning and reasoning

    Google DeepMind. Gemini: Our most capable model, now with better plan- ning and reasoning. https://blog.google/technology/google-deepmind/ gemini-model-thinking-updates-march-2025/, 2025. Accessed: 2025-05-08

    2025

  18. [26]

    Y . Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon. Domain-specific language model pretraining for biomedical natural language processing.ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23, 2021

    2021

  19. [27]

    Guo and D

    D. Guo and D. Terzopoulos. Prompting medical large vision-language models to diagnose pathologies by visual question answering.Machine Learning for Biomedical Imaging, 3 (March 2024):59–71, Mar. 2025. ISSN 2766-905X. doi: 10.59275/j.melba.2025-1a8b. URL http://dx.doi.org/10.59...

    2024 doi

  20. [28]

    I. E. Hamamci, S. Er, F. Almas, A. G. Simsek, S. N. Esirgun, I. Dogan, M. F. Dasdelen, B. Wittmann, E. Simsar, M. Simsar, et al. A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-shot detection of abnormalities.CoRR, 2024

    2024

  21. [29]

    Hartsock and G

    I. Hartsock and G. Rasool. Vision-language models for medical report generation and visual question answering: A review, 2024. URLhttps://arxiv.org/abs/2403.02469

    2024 arXiv

  22. [30]

    Heller, F

    N. Heller, F. Isensee, K. H. Maier-Hein, X. Hou, C. Xie, F. Li, Y . Nan, G. Mu, Z. Lin, M. Han, et al. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge.Medical Image Analysis, page 101821, 2020

    2020

  23. [31]

    Y . Hu, T. Li, Q. Lu, W. Shao, J. He, Y . Qiao, and P. Luo. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22170–22183, 2024

    2024

  24. [32]

    Huang, W

    I. Huang, W. Lin, M. J. Mirza, J. Hansen, S. Doveh, V . Butoi, R. Herzig, A. Arbelle, H. Kuehne, T. Darrell, et al. Conme: Rethinking evaluation of compositional reasoning for modern vlms. Advances in Neural Information Processing Systems, 37:22927–22946, 2024

    2024

  25. [33]

    Y . Ji, H. Bai, J. Yang, C. Ge, Y . Zhu, R. Zhang, Z. Li, L. Zhang, W. Ma, X. Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation.arXiv preprint arXiv:2206.08023, 2022

    2022 arXiv

  26. [34]

    Johnson, T

    A. Johnson, T. Pollard, R. Mark, S. Berkowitz, and S. Horng. Mimic-cxr database.PhysioNet10, 13026:C2JT1Q, 2024. 11

    2024

  27. [35]

    Kononenko

    I. Kononenko. Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in medicine, 23(1):89–109, 2001

    2001

  28. [36]

    S. S. Kunapuli and P. C. Bhallamudi. A review of deep learning models for medical diagnosis. Machine learning, big data, and IoT for Medical Informatics, pages 389–404, 2021

    2021

  29. [37]

    Y . Lai, J. Zhong, M. Li, S. Zhao, and X. Yang. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    2025

  30. [38]

    J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1):1–10, 2018

    2018

  31. [39]

    C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day, 2023. URLhttps://arxiv.org/abs/2306.00890

    2023 arXiv

  32. [40]

    L. Li, F. Wu, S. Wang, X. Luo, C. Martín-Isla, S. Zhai, J. Zhang, Y . Liu, Z. Zhang, M. J. Ankenbrand, et al. Myops: A benchmark of myocardial pathology segmentation combining three-sequence cardiac magnetic resonance images.Medical Image Analysis, 87:102808, 2023

    2023

  33. [41]

    M. Li, W. Cai, R. Liu, Y . Weng, X. Zhao, C. Wang, X. Chen, Z. Liu, C. Pan, M. Li, et al. Ffa-ir: Towards an explainable and reliable medical report generation benchmark. InThirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2), 2021

    2021

  34. [42]

    T. Lin, W. Zhang, S. Li, Y . Yuan, B. Yu, H. Li, W. He, H. Jiang, M. Li, X. Song, et al. Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation.arXiv preprint arXiv:2502.09838, 2025

    2025 arXiv

  35. [43]

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  36. [44]

    Liu, L.-M

    B. Liu, L.-M. Zhan, L. Xu, L. Ma, Y . Yang, and X.-M. Wu. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650–1654. IEEE, 2021

    2021

  37. [45]

    H. Liu, C. Li, Y . Li, and Y . J. Lee. Improved baselines with visual instruction tuning, 2024. URLhttps://arxiv.org/abs/2310.03744

    2024 arXiv

  38. [46]

    P. Liu, H. Han, Y . Du, H. Zhu, Y . Li, F. Gu, H. Xiao, J. Li, C. Zhao, L. Xiao, X. Wu, and S. K. Zhou. Deep learning to segment pelvic bones: large-scale ct datasets and baseline models. International Journal of Computer Assisted Radiology and Surgery, 16(5):749, 2021. doi: 1...

    2021 doi

  39. [47]

    Z. Liu, F. Fang, X. Feng, X. Du, C. Zhang, N. Wang, Q. Zhao, L. Fan, C. GAN, H. Lin, et al. Ii-bench: An image implication understanding benchmark for multimodal large language models.Advances in Neural Information Processing Systems, 37:46378–46480, 2024

    2024

  40. [48]

    J. Ma, Z. Yang, S. Kim, B. Chen, M. Baharoon, A. Fallahpour, R. Asakereh, H. Lyu, and B. Wang. Medsam2: Segment anything in 3d medical images and videos, 2025. URL https: //arxiv.org/abs/2504.03600

    2025 arXiv

  41. [49]

    B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y . Burren, N. Porz, J. Slotboom, R. Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats).IEEE transactions on medical imaging, 34(10):1993–2024, 2014

    1993

  42. [50]

    Nagar, S

    A. Nagar, S. Jaiswal, and C. Tan. Zero-shot visual reasoning by vision-language models: Benchmarking and analysis. In2024 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2024

    2024

  43. [51]

    H. T. Nguyen, H. H. Pham, N. T. Nguyen, H. Q. Nguyen, T. Q. Huynh, M. Dao, and V . Vu. Vindr-spinexr: A deep learning framework for spinal lesions detection and classification from radiographs. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th Inte...

    2021

  44. [52]

    Introducing o3 and o4 mini

    OpenAI. Introducing o3 and o4 mini. https://openai.com/index/ introducing-o3-and-o4-mini/, 2025. Accessed: 2025-05-08

    2025

  45. [53]

    Hurst, A

    OpenAI, A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, et al. Gpt-4o system card, 2024. URL https://arxiv.org/abs/2410.21276. 12

    2024 arXiv

  46. [54]

    Jaech, A

    OpenAI, A. Jaech, A. Kalai, A. Lerer, A. Richardson, et al. Openai o1 system card, 2024. URL https://arxiv.org/abs/2412.16720

    2024 arXiv

  47. [55]

    Panetta, R

    K. Panetta, R. Rajendran, A. Ramesh, S. P. Rao, and S. Agaian. Tufts dental database: a multimodal panoramic x-ray dataset for benchmarking diagnostic systems.IEEE journal of biomedical and health informatics, 26(4):1650–1659, 2021

    2021

  48. [56]

    Pedrosa, Guilherme, Carlos, Márcio, Patrícia, André, João, Eduardo, Isabel, António, and Aurélio

    J. Pedrosa, Guilherme, Carlos, Márcio, Patrícia, André, João, Eduardo, Isabel, António, and Aurélio. Lndb dataset, 2022. URL https://doi.org/10.5281/zenodo.7153205. 17th International Conference on Image Analysis and Recognition

    2022 doi

  49. [57]

    Pellegrini, M

    C. Pellegrini, M. Keicher, E. Özsoy, and N. Navab. Rad-restruct: A novel vqa benchmark and method for structured radiology reporting. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 409–419. Springer, 2023

    2023

  50. [58]

    Popescu, V

    M.-C. Popescu, V . E. Balas, L. Perescu-Popescu, and N. Mastorakis. Multilayer perceptron and neural networks.WSEAS Transactions on Circuits and Systems, 8(7):579–588, 2009

    2009

  51. [59]

    Z. Qin, H. Yi, Q. Lao, and K. Li. Medical image understanding with pretrained vision language models: A comprehensive study.arXiv preprint arXiv:2209.15517, 2022

    2022 arXiv

  52. [60]

    J. Qiu, L. Li, S. Wang, K. Zhang, Y . Chen, S. Yang, and X. Zhuang. Myops-net: Myocardial pathology segmentation with flexible combination of multi-sequence cmr images.Medical image analysis, 84:102694, 2023

    2023

  53. [61]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision, 2021. URLhttps://arxiv.org/abs/2103.00020

    2021 arXiv

  54. [62]

    Rahimzadeh, A

    M. Rahimzadeh, A. Attar, and S. M. Sakhaei. A fully automated deep learning-based network for detecting covid-19 from a new and large lung ct scan dataset.Biomedical Signal Processing and Control, page 102588, 2021. ISSN 1746-8094. doi: https://doi.org/10.1016/j.bspc.2021.1025...

    2021

  55. [63]

    Rajpurkar, J

    P. Rajpurkar, J. Irvin, A. Bagul, D. Ding, T. Duan, H. Mehta, B. Yang, K. Zhu, D. Laird, R. L. Ball, et al. Mura: Large dataset for abnormality detection in musculoskeletal radiographs.arXiv preprint arXiv:1712.06957, 2017

    2017 arXiv

  56. [64]

    Rister, D

    B. Rister, D. Yi, K. Shivakumar, T. Nobashi, and D. L. Rubin. Ct-org, a new dataset for multiple organ segmentation in computed tomography.Scientific Data, 7(1):381, 2020

    2020

  57. [65]

    Rodman and E

    A. Rodman and E. J. Topol. Is generative artificial intelligence capable of clinical reasoning? The Lancet, 405(10480):689, 2025

    2025

  58. [66]

    H. R. Roth, Z. Xu, C. Tor-Díez, R. S. Jacob, J. Zember, J. Molto, W. Li, S. Xu, B. Turkbey, E. Turkbey, et al. Rapid artificial intelligence solutions in a pandemic—the covid-19-20 lung ct lesion segmentation challenge.Medical image analysis, 82:102605, 2022

    2022

  59. [67]

    Royer, B

    C. Royer, B. Menze, and A. Sekuboyina. Multimedeval: A benchmark and a toolkit for evaluating medical vision-language models.arXiv preprint arXiv:2402.09262, 2024

    2024

  60. [68]

    A. Saha, J. S. Bosma, J. J. Twilt, B. van Ginneken, A. Bjartell, A. R. Padhani, D. Bonekamp, G. Villeirs, G. Salomon, G. Giannarini, et al. Artificial intelligence and radiologists in prostate cancer detection on mri (pi-cai): an international, paired, non-inferiority, confirm...

    2024

  61. [69]

    J. Shi. Structseg2019 gtv segmentation, 2023. URL https://dx.doi.org/10.21227/ h75x-gt46

    2023

  62. [70]

    A. L. Simpson, M. Antonelli, S. Bakas, M. Bilello, K. Farahani, B. van Ginneken, A. Kopp- Schneider, B. A. Landman, G. Litjens, B. Menze, O. Ronneberger, R. M. Summers, P. Bilic, P. F. Christ, R. K. G. Do, M. Gollub, J. Golia-Pernicka, S. H. Heckers, W. R. Jarnagin, M. K. McHu...

    2019

  63. [71]

    Singh, J

    C. Singh, J. P. Inala, M. Galley, R. Caruana, and J. Gao. Rethinking interpretability in the era of large language models, 2024. URLhttps://arxiv.org/abs/2402.01761

    2024 arXiv

  64. [72]

    Y . Sun, H. Wu, C. Zhu, S. Zheng, Q. Chen, K. Zhang, Y . Zhang, D. Wan, X. Lan, M. Zheng, J. Li, X. Lyu, T. Lin, and L. Yang. Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology, 2024. URL https://arxiv.org/abs/2401. 16355. 13

    2024

  65. [73]

    Szolovits, R

    P. Szolovits, R. S. Patil, and W. B. Schwartz. Artificial intelligence in medical diagnosis.Annals of internal medicine, 108(1):80–87, 1988

    1988

  66. [74]

    X. Wang, F. Wang, Y . Li, Q. Ma, S. Wang, B. Jiang, C. Li, and J. Tang. Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. arXiv preprint arXiv:2410.00379, 2024

    2024 arXiv

  67. [75]

    C. Wu, X. Zhang, Y . Zhang, Y . Wang, and W. Xie. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data, 2023. URL https://arxiv.org/ abs/2308.02463

    2023 arXiv

  68. [76]

    Z. Wu, L. Qiu, A. Ross, E. Akyürek, B. Chen, B. Wang, N. Kim, J. Andreas, and Y . Kim. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks, 2024. URLhttps://arxiv.org/abs/2307.02477

    2024 arXiv

  69. [77]

    Introducing grok-3: xai’s most advanced ai model yet

    xAI. Introducing grok-3: xai’s most advanced ai model yet. https://x.ai/news/grok-3,

  70. [78]

    P. Xia, Z. Chen, J. Tian, Y . Gong, R. Hou, Y . Xu, Z. Wu, Z. Fan, Y . Zhou, K. Zhu, et al. Cares: A comprehensive benchmark of trustworthiness in medical vision language models.Advances in Neural Information Processing Systems, 37:140334–140365, 2024

    2024

  71. [79]

    Y . Xiao, M. Fortin, G. Unsgård, H. Rivaz, and I. Reinertsen. Retrospective evaluation of cerebral tumors (resect): a clinical database of pre-operative mri and intra-operative ultrasound in low-grade glioma surgeries.Medical Physics, 44(7):3875–3882, 2017

    2017

  72. [80]

    K. Yan, X. Wang, L. Lu, and R. M. Summers. Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning.Journal of medical imaging, 5(3):036501–036501, 2018

    2018

  73. [81]

    Y . Yan, S. Wang, J. Huo, J. Ye, Z. Chu, X. Hu, P. S. Yu, C. Gomes, B. Selman, and Q. Wen. Position: Multimodal large language models can significantly advance scientific reasoning,

  74. [82]

    J. Yang, X. Ding, Z. Zheng, X. Xu, and X. Li. Graphecho: Graph-driven unsupervised domain adaptation for echocardiogram video segmentation, 2023. URL https://arxiv.org/abs/ 2309.11145

    2023 arXiv

  75. [83]

    J. Ye, J. Cheng, J. Chen, Z. Deng, T. Li, H. Wang, Y . Su, Z. Huang, J. Chen, L. Jiang, et al. Sa-med2d-20m dataset: Segment anything in 2d medical imaging with 20 million masks.arXiv preprint arXiv:2311.11969, 2023

    2023 arXiv

  76. [84]

    J. Ye, G. Wang, Y . Li, Z. Deng, W. Li, T. Li, H. Duan, Z. Huang, Y . Su, B. Wang, et al. Gmai-mmbench: A comprehensive multimodal evaluation benchmark towards general medical ai.Advances in Neural Information Processing Systems, 37:94327–94427, 2024

    2024

  77. [85]

    Yiming, Z

    X. Yiming, Z. Bowen, L. Xiaohong, W. Tao, J. Jinxiu, W. Shijie, L. Yufan, Z. Hongjun, L. Tong, S. Ye, J. Rui, W. Guangyu, R. Jie, and C. Ting. Annotated ultrasound liver images, Nov. 2022. URLhttps://doi.org/10.5281/zenodo.7272660

    2022 doi

  78. [86]

    C. H. Yoon, R. Torrance, and N. Scheinerman. Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned?Journal of Medical Ethics, 48(9):581–585, 2022

    2022

  79. [87]

    Zhang, Y

    G. Zhang, Y . Zhang, K. Zhang, and V . Tresp. Can vision-language models be a good guesser? exploring vlms for times and location reasoning. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 636–645, 2024

    2024

  80. [88]

    Zhang, J

    H. Zhang, J. Chen, F. Jiang, F. Yu, Z. Chen, J. Li, G. Chen, X. Wu, Z. Zhang, Q. Xiao, X. Wan, B. Wang, and H. Li. Huatuogpt, towards taming language model to be a doctor, 2023. URL https://arxiv.org/abs/2305.15075

    2023 arXiv

  81. [89]

    Zhang, J

    J. Zhang, J. Huang, S. Jin, and S. Lu. Vision-language models for vision tasks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  82. [90]

    Zhang, V

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi. Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  83. [91]

    Zhang, C

    X. Zhang, C. Wu, Z. Zhao, W. Lin, Y . Zhang, Y . Wang, and W. Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering.arXiv preprint arXiv:2305.10415, 2023

    2023 arXiv

  84. [92]

    B. N. Zhao, Z. Wang, X. JIANG, X. Luo, Y . Yang, B. Li, J. Alvarez-Valle, M. P. Lungren, D. Li, and L. Qiu. Benchmark dataset for radiology report generation with instructions and contexts. 14

  85. [93]

    X. Zhuang. Multivariate mixture model for myocardial segmentation combining multi-source images.IEEE transactions on pattern analysis and machine intelligence, 41(12):2933–2946, 2019. 15 A Technical Appendices and Supplementary Material A.1 Dataset Availability DrVD-Benchis pu...

    2019

  86. [96]

    What imaging modality is used in this image? Options: A. {} B. {} C. {} D. {}

  87. [97]

    Which organ appears to be abnormal in this image? Options: A. {} B. {} 24 C. {} D. {}

  88. [98]

    Based on the abnormal organ, what lesion or finding is most clearly visible? Options: A. {} B. {} C. {} D. {} E. {} F. {} G. {} H. {}

  89. [99]

    Considering all the above findings, what is the most likely diagnosis? Options: A. {} B. {} C. {} D. {} E. {} F. {} G. {} H. {} Instructions: Please reply with your four selected letters in order, separated by commas (e.g., A,C,B,A). Do not provide explanations. C.4 Multi-turn...

  90. [100]

    What imaging modality is used in this image? Assistant: CT User (Round 2):

    What imaging modality is used in this image? Options: A) CT B) MRI C) Ultrasound D) Radiography Assistant: A User: [Same image QA record]1. What imaging modality is used in this image? Assistant: CT User (Round 2):

  91. [101]

    Which organ is shown in this medical image? Assistant: Liver User (Round 3):

    Which organ is shown in this medical image? Options: A) Liver B) Pancreas C) Spleen D) Kidney Assistant: A User: [Same image QA record]2. Which organ is shown in this medical image? Assistant: Liver User (Round 3):

  92. [102]

    What lesion is visible in this image? Options: A) Target sign B) Hepatic steatosis C) Pancreatic pseudocyst D) Splenic infarct 25 Assistant: A User (Round 4):

  93. [103]

    no lung opacity

    What is the most likely diagnosis? Options: A) Crohn’s disease B) Acute pancreatitis C) Splenic rupture D) Diverticulitis C.5 Report Generation System prompt: Generate a clinical report based on the image. This is used solely for educational purposes. User prompt: Generate a c...

  94. [2024]

    Accessed: 2025-05-08

    2025

  95. [2025]

    URLhttps://arxiv.org/abs/2502.02871

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.