Pith. sign in

REVIEW 4 major objections 5 minor 6 cited by

CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read 3D chest CT agent beats baselines on reports and questions

desk verdict The architecture and ablations are solid and worth building on, but the QA evaluation is biased by train/test template overlap and a single untrained baseline, so the headline QA gains should not be taken at face value. read the letter →

arxiv 2505.16229 v1 pith:2YEE6EV3 submitted 2025-05-22 cs.CV

Yuren Mao , Wenyi Xu , Yuyang Qin , Yunjun Gao This is my paper · ORCID
classification cs.CV
keywords CTradiologyquestionanswering3DchestmultimodallargelanguagemodelLLMagentvisualtokencompressionLoRAfine-tuningreportgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CT-Agent is a multimodal LLM agent designed to answer questions about 3D chest CT volumes and to generate radiology reports. The paper argues that the two obstacles to good CT question answering are anatomical complexity and the difficulty of capturing relationships across hundreds of slices, and that both can be addressed by giving the agent a set of anatomy-specific tools. Each of ten anatomical regions gets its own lightweight LoRA adapter on a shared vision-language backbone, while a global-local token compression scheme reduces roughly 240 slices to about 64 visual tokens per slice without dropping the information needed for cross-slice reasoning. On CT-RATE and RadGenome-ChestCT the system reports clinical-efficacy F1 of 0.420 for report generation versus 0.261 for the best baseline, and QA F1 of 0.646 and 0.532 for presence and abnormality tasks versus 0.589 and 0.393 for LLaVA-CT. A sympathetic reader would take the paper to establish that decomposing a 3D medical volume by anatomy, rather than feeding the whole volume to one model, is a workable path to clinically useful CT question answering.

What carries the argument

The load-bearing object is the agent's action space: ten region-specific LoRA reasoning tools mounted on one frozen multimodal LLM backbone, each built on a two-stage hierarchical token compression. Global Token Aggregation applies a token-wise mixture-of-experts and slice-wise averaging to fold 240 slices into a compact global representation; Local Token Selection keeps the top attention-dominant tokens and merges the rest by key similarity, yielding 54 dominant plus 10 contextual tokens per slice. A planning module driven by an LLM classifies the task, identifies the anatomical region, rewrites free-form questions into templates, and routes the compressed visual tokens to the right LoRA plugin, with a prediction-guided retrieval mechanism pulling similar exemplar reports from memory. Together these components reduce the token count by roughly 75% while preserving semantic integrity, and that reduction is what makes whole-volume 3D reasoning feasible inside an LLM context window.

What would settle it

Run CT-Agent and LLaVA-CT on a held-out set of QA pairs written by radiologists rather than generated from templates, and have radiologists score the answers blind; if CT-Agent's F1 advantage over LLaVA-CT shrinks to near zero, the gains are artifacts of template overlap rather than better clinical reasoning.

Watch

Extended reading notes

Core claim

The paper's central claim is that CT-Agent consistently outperforms existing methods on both radiology report generation and region-guided question answering for 3D chest CT, on two public datasets. In report generation it reports a clinical-efficacy F1 of 0.420, with recall of 0.477, against a best baseline F1 of 0.261 (MS-VLM) and 0.221 for LLaVA-CT, while also improving most NLG metrics except BLEU-4. For question answering, over ten anatomical regions, it reports overall presence-detection F1 of 0.646 versus 0.589 for LLaVA-CT and abnormality-detection F1 of 0.532 versus 0.393. The authors attribute the gains to three mechanisms working together: the planning module that routes each query to the relevant anatomy, the ten region-specific LoRA plugins that specialize without retraining the backbone, and the dual-path token compression that keeps global context while trimming local redundancy. The paper frames the result as showing that anatomy-aware, agent-style reasoning is superior to treating the volume as a homogeneous input.

Load-bearing premise

The reported superiority rests on the assumption that the test questions and the exact-match clinical metric actually measure clinically meaningful performance, because the test QA pairs come from the same template-and-annotation scheme used for training and the adapted CE metric has not been validated against radiologist judgment.

Editorial extensions

If this is right

  • CT-Agent handles both radiology report generation and region-guided question answering through one planning module, so a single system can serve both clinical workflows.
  • Routing each query to a region-specific LoRA plugin removes anatomical-confusion errors, such as mistaking mediastinum findings for lung findings.
  • Compressing each of 240 slices to 64 tokens cuts the visual token budget by roughly 75%, making whole-volume 3D input feasible for a frozen vision-language backbone.
  • Prediction-guided exemplar retrieval improves report BLEU, ROUGE-L, and METEOR over static few-shot and zero-shot variants, indicating that memory-grounded prompting is part of the gain.
  • The modular design means existing regions can be improved or new regions added by training only that region's LoRA adapter, without retraining the shared backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test QA pairs are built from the same template-and-annotation scheme used to train the region tools, part of the measured gain likely reflects the query-rewriting step aligning user questions to the training distribution; a reader should not conclude that the agent reasons better on genuinely out-of-distribution questions.
  • The exact-match clinical-efficacy metric rewards the presence of predefined abnormality terms, so a system that produces template-consistent but clinically incomplete reports could score well, making a clinician-rated evaluation the natural next test.
  • The anatomy-first recipe is not tied to CT: the same planner-plus-LoRA-tools structure could be applied to MRI or whole-body PET by replacing the region taxonomy and the predefined query pool.
  • If the gains are real, the practical bottleneck for deploying such an agent shifts from model capacity to the quality and coverage of the region taxonomy and question templates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CT-Agent, a multimodal-LLM agent for 3D chest CT radiology report generation and region-guided question answering. The system combines a planning module built on Deepseek-v3, an action space of ten anatomy-specific LoRA-tuned reasoning tools, hierarchical global-local token compression, memory-guided few-shot exemplar retrieval, and query normalization. The authors evaluate on CT-RATE and RadGenome-ChestCT, reporting that CT-Agent outperforms existing methods on both report generation and question answering, with ablations attributed to planning, token compression, and retrieval components.

Significance. If the claims are validated, CT-Agent would be a useful contribution to 3D medical VQA: the anatomy-aware decomposition is a sensible response to the complexity of chest CT, the token compression strategy addresses a real computational bottleneck, and the use of public datasets with detailed implementation settings supports reproducibility. The ablations are also structured to test each component. However, the evaluation as reported cannot currently support the headline conclusions: the central report-generation table contains internally inconsistent precision/recall/F1 values, the QA benchmark is generated from the same template and GPT-label pool used for training CT-Agent while the sole baseline is not trained on that task, and the adapted Clinical Efficacy metric is not validated. These issues are load-bearing for the abstract's claim of consistent superiority.

major comments (4)
  1. [Table 1] The CE precision/recall/F1 triplets in Table 1 are internally inconsistent. For CT-Agent, P=0.423 and R=0.477 imply F1 = 2*0.423*0.477/(0.423+0.477) ≈ 0.448, not the reported 0.420. Similar mismatches appear for CT2Rep (0.355/0.132 implies 0.192, not 0.175) and M3D (0.407/0.009 implies 0.018, not 0.148). Since the paper's report-generation superiority claim rests on these numbers, the table must be corrected and the conclusions recomputed before the clinical-efficacy advantage can be assessed.
  2. [Sections 5.1.1, 4.3.1, 4.3.3, Appendix A] The QA evaluation is confounded by training/test distribution overlap. Section 5.1.1 states that the QA pair dataset is constructed by filling unified templates with RadGenome anatomical region names and deriving answers directly from RadGenome GPT annotations, and that the same constructed dataset provides both the 1,914,448 training pairs and the 119,200 test pairs. CT-Agent is explicitly trained on these pairs (Section 4.3.1), and its query-rewriting tool reformulates user questions into 'a standardized format aligned with the model's training distribution' (Section 4.3.3) before final answers are generated from a standardized template (Appendix A). The only QA baseline, LLaVA-CT, is described as fine-tuned directly on CT-RATE CT images and reports (Section 5.1.2), not on this QA task. Table 2 therefore conflates task-specific fine-tuning and query-template normalization with the agent architecture itself. Please retrain LLaVA-CT (and ideally a second baseline) on the same QA pairs, or evaluate on independently written, human-annotated questions with varied phrasing, and additionally report what fraction of outputs exactly match the templates.
  3. [Section 5.1.3] The 'improved Exact Match' adaptation of CT2Rep's Clinical Efficacy metric is introduced without validation. There is no comparison against the original CE metric, no clinician agreement study, and no analysis of how the lenient EM matching behaves on template-generated QA pairs. Given the very large QA gains in Table 2 (for example, Trachea & bronchi abnormality F1 rising from 0.018 for the baseline to 0.790 for CT-Agent), it is important to show that the metric is not merely rewarding template-consistent outputs. Please validate the metric on a human-annotated subset and report per-abnormality precision/recall rather than only a micro-averaged aggregate.
  4. [Tables 1-4] No confidence intervals, standard deviations, or significance tests are reported anywhere in the experiments. Several reported differences are very small (e.g., BLEU-3 +0.004 and METEOR +0.004 over the best baseline in Table 1), and some Table 2 region-level comparisons go in the opposite direction (e.g., Lung presence precision 0.973 vs. 0.962, Thyroid abnormality F1 0.540 vs. 0.453). The abstract's 'consistently outperforms' claim is therefore not statistically supported. At minimum, report variance or bootstrap confidence intervals for the main metrics, and significance tests for the differences that drive the conclusions.
minor comments (5)
  1. [Author affiliation] The affiliation line contains a typo: 'Hanghou' should be 'Hangzhou'.
  2. [Table 1 caption] The caption uses 'BLUE-4' instead of 'BLEU-4'; please correct this typo.
  3. [Equations (6) and (11)] Notation is inconsistent: the global aggregated token matrix is called Z_f in Eq. (6) but Z_global in Eq. (11); please align the symbols.
  4. [Appendix A] The query-rewriting prompt includes a size-estimation template ('What is the approximate size of the {abnormality} in the {region}?'), but Section 5.1.1 says the QA templates are only presence detection and abnormality identification; please clarify whether size questions are part of the evaluation set or only an auxiliary prompt template.
  5. [Table 2] The table reports overall and per-region metrics but no sample sizes per region or per task; please add the number of test questions for each anatomical region and task type.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the sole self-citation is non-load-bearing and the QA train/test overlap is a benchmark confound, not a circular derivation.

full rationale

CT-Agent is an empirical systems paper; there is no derivation chain in which a quantity is defined in terms of the quantity it is used to predict. The QA dataset construction in Section 5.1.1 (unified templates filled with RadGenome region names, answers derived from RadGenome GPT annotations) supplies both training and test pairs from the same template and annotation pool, and the inference pipeline (Sections 4.3.3 and Appendix A) rewrites user questions into the training-distribution format. This is a distributional-overlap and evaluation-confounding threat to the reported QA gains over LLaVA-CT, but it is not circular reasoning: the model still must map visual CT features to region-level findings, and the test labels are external to CT-Agent. The internal CE-F1 inconsistency in Table 1 is a correctness and reproducibility concern, not a circularity. The only self-citation is [33], a LoRA survey by the authors, cited in Section 3.3 to support LoRA as a general PEFT technique; no load-bearing claim in the paper depends on it. Accordingly, the paper receives a low score in the non-circular range (2) rather than a higher circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the reliability of RadGenome GPT labels, the validity of the CE metric, and the transferability of frozen CLIP features to CT slices, rather than on a formal derivation. Hyperparameters for token compression and retrieval are set by hand. No new physical entities are proposed; the framework is a composition of existing model components.

free parameters (5)
  • Dominant token count K = 54
    Section 5.1.4: each CT slice is compressed to 64 visual tokens, with 54 dominant tokens selected by attention scores; this hyperparameter is set by hand and affects representation capacity.
  • Contextual token count M = 10
    Section 5.1.4: 10 contextual tokens are produced by merging remaining tokens; this hyperparameter is set by hand.
  • Number of axial slices T = 240
    Section 5.1.4: each volume is decomposed into 240 axial slices; this fixed resampling choice discards information and influences all downstream token counts.
  • LoRA rank r = 16
    Section 5.1.4: LoRA adapters use rank 16 and scaling factor 16; adapter capacity is chosen by hand.
  • Few-shot retrieved count K' = 3
    Section 5.1.4: the retrieval tool selects the top-3 semantically similar cases; this number is chosen by hand.
assumptions (5)
  • domain assumption RadGenome-ChestCT GPT-generated pathological labels are accurate ground truth for abnormality labels.
    Section 5.1.1 uses these labels to construct 2,033,648 QA pairs and as answer labels for training and evaluation.
  • domain assumption Template-generated QA pairs from the same RadGenome label pool form a valid test distribution for clinical QA.
    Section 5.1.1 constructs train and test QA pairs from the same GPT annotations and unified question templates; the paper assumes this measures clinically meaningful generalization.
  • domain assumption The improved Exact Match CE metric over 18 CT2Rep abnormalities is a valid measure of clinical correctness.
    Section 5.1.3 introduces this unvalidated adaptation of CT2Rep's Clinical Efficacy metric; no comparison to the original implementation or to human judgments is provided.
  • domain assumption Frozen CLIP ViT-B/16 slice features preserve the anatomical and clinical information needed for CTQA.
    Sections 3.2 and 4.3.1 use CLIP ViT-B/16 as the sole visual encoder and train only the MoE, projector, and LoRA adapters.
  • domain assumption The Deepseek-v3 planner correctly classifies query type and anatomical region.
    Section 4.2 relies on the planner's parsing accuracy but reports no evaluation of planning errors or their downstream effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering." pith.science (2026). https://pith.science/paper/2YEE6EV3

@misc{pith2026250516229,
  author       = {Pith},
  title        = {Pith review of: CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YEE6EV3}},
  note         = {Machine review of arXiv:2505.16229}
}
read the original abstract

Computed Tomography (CT) scan, which produces 3D volumetric medical data that can be viewed as hundreds of cross-sectional images (a.k.a. slices), provides detailed anatomical information for diagnosis. For radiologists, creating CT radiology reports is time-consuming and error-prone. A visual question answering (VQA) system that can answer radiologists' questions about some anatomical regions on the CT scan and even automatically generate a radiology report is urgently needed. However, existing VQA systems cannot adequately handle the CT radiology question answering (CTQA) task for: (1) anatomic complexity makes CT images difficult to understand; (2) spatial relationship across hundreds slices is difficult to capture. To address these issues, this paper proposes CT-Agent, a multimodal agentic framework for CTQA. CT-Agent adopts anatomically independent tools to break down the anatomic complexity; furthermore, it efficiently captures the across-slice spatial relationship with a global-local token compression strategy. Experimental results on two 3D chest CT datasets, CT-RATE and RadGenome-ChestCT, verify the superior performance of CT-Agent.

Figures

Figures reproduced from arXiv: 2505.16229 by the authors.

Figure 1
Figure 1. Overall architecture of CT-Agent. It consists of three modules: planning module, [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of anatomy-aware reasoning tools. Given a set of axial CT slices, a frozen [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Semantic retrieval pipeline for few-shot prompting. The current case’s anatomy-level [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sentence-level comparison among the CT-Agent generated report, baseline generated re [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cheap Probes Predict Expensive Training in 3D-CT Vision--Language Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Disease-probe AUROC on frozen 3D-CT tokens predicts report-generation clinical micro-F1 across encoder×compression cells at r=0.95, ρ=0.89 (six cells, preliminary, one dataset).

  2. Policy-Driven CT-Agent: Modeling Phase-Aware Diagnostic Control for Clinically Consistent CT Reasoning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    An agent with structured CT evidence packages and guideline-guided control iteratively escalates imaging phases only when current evidence is judged insufficient for diagnosis.

  3. RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    RadAgent generates stepwise, tool-augmented chest CT reports with traceable decisions, improving accuracy, robustness, and adding a 37% faithfulness score absent in standard 3D VLMs.

  4. Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation

    cs.CR 2026-03 unverdicted novelty 6.0 of 10

    DiffVP turns scan-to-normal semantic discrepancies into learnable visual prefix tokens that guide an LLM to write more accurate, fine-grained 3D CT reports.

  5. CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation

    cs.AI 2025-10 conditional novelty 5.0 of 10

    A director-orchestrated agent with tool-output validation and adaptive expert teams reports state-of-the-art average accuracy on three chest X-ray interpretation benchmarks.

  6. A Comprehensive Survey of Electronic Health Record Modeling: From Deep Learning Approaches to Large Language Models

    cs.LG 2025-07 reject novelty 4.0 of 10

    A survey that taxonomizes EHR modeling research into data-centric, architectural, learning-focused, multimodal, and LLM-based categories, with datasets and metrics.

Reference graph

Works this paper leans on

41 extracted references · 19 canonical work pages · cited by 6 Pith papers

  1. [1]

    Computed tomography dose index and patient dose: They are not the same thing

    Cynthia H McCollough, Shuai Leng, Lifeng Yu, and Joel G Fletcher. Computed tomography dose index and patient dose: They are not the same thing. Radiology, 2015

    work page 2015

  2. [2]

    M. R. Bernsen, M. van Straten, G. Kotek, E. A. H. Warnert, J. C. Haeck, A. Ruggiero, P. A. Wielopolski, and G. P. Krestin. Computed tomography and magnetic resonance imaging. In Recent Results in Cancer Research. Springer, 2020

    work page 2020

  3. [3]

    Radsam: Segmenting 3d radiological images with a 2d promptable model

    Julien Khlaut, Elodie Ferreres, Daniel Tordjman, H´ el` ene Philippe, Tom Boeken, Pierre Manceron, and Corentin Dancette. Radsam: Segmenting 3d radiological images with a 2d promptable model. arXiv preprint arXiv:2504.20837 , 2025

    arXiv 2025

  4. [4]

    Explaining 3D Computed Tomography Classifiers with Counterfactuals

    Joseph Paul Cohen, Louis Blankemeier, and Akshay Chaudhari. Explaining 3d computed tomography classifiers with counterfactuals. arXiv preprint arXiv:2502.07156 , 2025

    work page Pith review arXiv 2025

  5. [5]

    Enhancing Single-Slice Segmentation with 3D-to-2D Unpaired Scan Distillation

    Xin Yu, Qi Yang, Han Liu, Ho Hin Lee, Yucheng Tang, Lucas W Remedios, Michael E Kim, Rendong Zhang, Shunxing Bao, Yuankai Huo, et al. Enhancing single-slice segmentation with 3d-to-2d unpaired scan distillation. arXiv preprint arXiv:2406.12254 , 2024

    work page Pith review arXiv 2024

  6. [6]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. NeurIPS, 2023

    work page 2023

  7. [7]

    Pmc-vqa: Visual instruction tuning for medical visual question answering

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415, 2023

    arXiv 2023

  8. [8]

    R2gengpt: Radiology report gen- eration with frozen llms

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. R2gengpt: Radiology report gen- eration with frozen llms. Meta-Radiology, 2023

    work page 2023

Show all 41 references
  1. [9]

    Generating radiology reports via memory-driven transformer

    Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. Generating radiology reports via memory-driven transformer. arXiv preprint arXiv:2010.16056 , 2020

    2010 arXiv

  2. [10]

    Bpi-mvqa: a bi-branch model for medical visual question answering

    Shengyan Liu, Xuejie Zhang, Xiaobing Zhou, and Jian Yang. Bpi-mvqa: a bi-branch model for medical visual question answering. BMC Medical Imaging , 2022. 17

    2022

  3. [11]

    Maira-1: A specialised large multimodal model for radiology report generation

    Stephanie L Hyland, Shruthi Bannur, Kenza Bouzid, Daniel C Castro, Mercy Ranjit, Anton Schwaighofer, Fernando P´ erez-Garc ´ ıa, Valentina Salvatelli, Shaury Srivastav, Anja Thieme, et al. Maira-1: A specialised large multimodal model for radiology report generation. arXiv pre...

    2023 arXiv

  4. [12]

    Xraygpt: Chest radiographs summarization using medical vision-language models

    Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Shahbaz Khan. Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971, 2023

    2023 arXiv

  5. [13]

    Ct2rep: Automated radiology report generation for 3d medical imaging

    Ibrahim Ethem Hamamci, Sezgin Er, and Bjoern Menze. Ct2rep: Automated radiology report generation for 3d medical imaging. In MICCAI, 2024

    2024

  6. [14]

    3d-ct-gpt: Generating 3d radiology reports through integration of large vision-language models

    Hao Chen, Wei Zhao, Yingli Li, Tianyang Zhong, Yisong Wang, Youlan Shang, Lei Guo, Junwei Han, Tianming Liu, Jun Liu, et al. 3d-ct-gpt: Generating 3d radiology reports through integration of large vision-language models. arXiv preprint arXiv:2409.19330 , 2024

    2024 arXiv

  7. [15]

    Vit3d alignment of llama3: 3d medical image report generation

    Siyou Li, Beining Xu, Yihao Luo, Dong Nie, and Le Zhang. Vit3d alignment of llama3: 3d medical image report generation. arXiv preprint arXiv:2410.08588 , 2024

    2024 arXiv

  8. [16]

    Ct-agrg: Auto- mated abnormality-guided report generation from 3d chest ct volumes

    Theo Di Piazza, Carole Lazarus, Olivier Nempont, and Loic Boussel. Ct-agrg: Auto- mated abnormality-guided report generation from 3d chest ct volumes. arXiv preprint arXiv:2408.11965, 2024

    2024

  9. [17]

    Benchmarking and boosting radiology report generation for 3d high- resolution medical images

    Che Liu, Zhongwei Wan, Yuqi Wang, Hui Shen, Haozhe Wang, Kangyu Zheng, Mi Zhang, and Rossella Arcucci. Benchmarking and boosting radiology report generation for 3d high- resolution medical images. arXiv preprint arXiv:2406.07146 , 2024

    2024 arXiv

  10. [18]

    Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation

    Changsun Lee, Sangjoon Park, Cheong-Il Shin, Woo Hee Choi, Hyun Jeong Park, and Jong Chul Lee. Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation. arXiv preprint arXiv:2412.13558 , 2024

    2024 arXiv

  11. [19]

    M3d: Advancing 3d medical image analysis with multi-modal large language models

    Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models. arXiv preprint arXiv:2404.00578 , 2024

    2024 arXiv

  12. [20]

    A survey of llm-based agents in medicine: How far are we from baymax? arXiv preprint arXiv:2502.11211, 2025

    Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Wenting Chen, Xiang Li, and Yixuan Yuan. A survey of llm-based agents in medicine: How far are we from baymax? arXiv preprint arXiv:2502.11211, 2025

    2025 arXiv

  13. [21]

    Pathfinder: A multi-modal multi-agent system for medical diagnostic decision-making applied to histopathol- ogy

    Fatemeh Ghezloo, Mehmet Saygin Seyfioglu, Rustin Soraki, Wisdom O Ikezogwo, Beibin Li, Tejoram Vivekanandan, Joann G Elmore, Ranjay Krishna, and Linda Shapiro. Pathfinder: A multi-modal multi-agent system for medical diagnostic decision-making applied to histopathol- ogy. arXi...

    2025 arXiv

  14. [22]

    Medagent-pro: Towards multi-modal evidence-based medical diagnosis via reasoning agentic workflow

    Ziyue Wang, Junde Wu, Chang Han Low, and Yueming Jin. Medagent-pro: Towards multi-modal evidence-based medical diagnosis via reasoning agentic workflow. arXiv preprint arXiv:2503.18968, 2025

    2025 arXiv

  15. [23]

    Mˆ 3builder: A multi-agent system for automated machine learning in medical imaging

    Jinghao Feng, Qiaoyu Zheng, Chaoyi Wu, Ziheng Zhao, Ya Zhang, Yanfeng Wang, and Weidi Xie. Mˆ 3builder: A multi-agent system for automated machine learning in medical imaging. arXiv preprint arXiv:2502.20301 , 2025. 18

    2025 arXiv

  16. [24]

    Pefomed: Parameter efficient fine-tuning of multimodal large language models for medical imaging.arXiv preprint arXiv:2401.02797, 2024

    Gang Liu, Jinlong He, Pengfei Li, Genrong He, Zhaolin Chen, and Shenjun Zhong. Pefomed: Parameter efficient fine-tuning of multimodal large language models for medical imaging.arXiv preprint arXiv:2401.02797, 2024

    2024 arXiv

  17. [25]

    Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis

    Jiancheng Yang, Rui Shi, and Bingbing Ni. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In ISBI, 2021

    2021

  18. [26]

    Medsegbench: A comprehensive benchmark for medical image segmentation in diverse data modalities

    Zeki Ku¸ s and Musa Aydin. Medsegbench: A comprehensive benchmark for medical image segmentation in diverse data modalities. Scientific Data , 2024

    2024

  19. [27]

    I. E. Hamamci, S. Er, F. Almas, et al. Developing generalist foundation models from a multimodal dataset for 3d computed tomography. arXiv preprint arXiv:2403.17834 , 2024

    2024

  20. [28]

    Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Jiayu Lei, Ya Zhang, Yanfeng Wang, and Weidi Xie. Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis. arXiv preprint arXiv:2404.16754, 2024

    2024 arXiv

  21. [29]

    Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification

    Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data , 2023

    2023

  22. [30]

    Medshapenet–a large-scale dataset of 3d medical shapes for computer vision

    Jianning Li, Zongwei Zhou, Jiancheng Yang, Antonio Pepe, Christina Gsaxner, Gijs Luijten, Chongyu Qu, Tiezheng Zhang, Xiaoxi Chen, Wenxuan Li, et al. Medshapenet–a large-scale dataset of 3d medical shapes for computer vision. Biomedical Engineering/Biomedizinische Technik, 2025

    2025

  23. [31]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021

    2021

  24. [32]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In ICLR, 2022

    2022

  25. [33]

    A survey on lora of large language models

    Yuren Mao, Yuhang Ge, Yijiang Fan, Wenyi Xu, Yu Mi, Zhonghao Hu, and Yunjun Gao. A survey on lora of large language models. Frontiers of Computer Science , 2025

    2025

  26. [34]

    Visionzip: Longer is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. arXiv preprint arXiv:2412.04467, 2024

    2024

  27. [35]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002

    2002

  28. [36]

    ROUGE: A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out , 2004

    2004

  29. [37]

    Trachea and Bronchi

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In ACL Eval Workshop , 2005. 19 A Prompt Template Task Classification Template: You are a medical-domain assistant that classifies the user’s intent...

    2005

  30. [38]

    What are the abnormalities in the {region}?

  31. [39]

    What is the approximate size of the {abnormality} in the {region}?

  32. [40]

    Where is the {abnormality} located in the image?

  33. [41]

    Trachea and both main bronchi are open

    Can {abnormality} be identified in the {region}? Input: User Question: {{user_question}} Target Anatomical Region: {{region}} Output Format: Rewritten Clinical Query: {{generated_question}} Answer Generation Template: 20 You are a medical-domain assistant. Your task is to answ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.