Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MedRegion-CT claims that a region-focused multimodal LLM—fed by global and region-representative visual tokens, pseudo-mask tokens, and quantitative patient attributes as text—generates more linguistically accurate and clinically accurate…

desk verdict Reasonable engineering contribution, but the SOTA claim is undercut by the paper's own Table 1 and an unfair baseline comparison. read the letter →

arxiv 2506.23102 v3 pith:QOLZPOJF submitted 2025-06-29 eess.IV cs.CV

Sunggu Kyung , Jinyoung Seo , Hyunseok Lim , Dongyeong Kim , Hyungbin Park , Jimin Sung , Jihyun Kim , Wooyoung Jo
show 2 more authors
Yoojin Nam Namkug Kim
This is my paper
classification eess.IVcs.CV
keywords CTreportgenerationmultimodallargelanguagemodel3Dcomputedtomographyregion-awarevisiontokensSlowFasttokenizationpseudo-maskguidancepatient-specificattributesRadGenome-Chest
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MedRegion-CT is a framework for generating radiology reports from 3D chest CT scans. The paper argues that prior models rely on global volume features and miss region-specific abnormalities, and that a region-focused multimodal large language model can fix this. To test that claim, the authors combine three inputs: global and region-representative visual tokens pooled from a frozen 2D encoder, segmentation-mask tokens derived from pseudo-masks of six anatomical regions, and quantitative patient attributes (organ volumes, lesion counts, diameters, locations) turned into text prompts. The LLM then writes a report for each region, and the six region reports are merged. On the RadGenome-Chest CT benchmark, the authors report state-of-the-art natural language and clinical accuracy metrics, with the full system beating every ablation variant.

What carries the argument

The load-bearing mechanism is the Region Representative (R2) Token Pooling in combination with the Mask-Driven Visual Extractor and the Patient-Specific Attribute Extractor. R2 Token Pooling merges one global token per slice (fast tokens, giving depth coverage) with spatially pooled tokens from slices that best show each of six anatomical regions (slow tokens, giving local detail), compressing D×T slice tokens to D+T vision tokens. The Mask-Driven Visual Extractor takes pseudo-masks from a universal segmentation model, pools mask-aligned visual features into mask tokens, and flattens mask geometry into spatial tokens; the Patient-Specific Attribute Extractor turns the same masks into deterministic quantitative text. The paper's argument is that these three token types—Tvision, Tseg, Tattr—together give the LLM global context, region-level attention, and patient-specific quantitative grounding that global-only encoders lack.

What would settle it

Run RadFM, MedBLIP, CT2Rep, and M3D on the RadGenome-Chest CT test split using their released checkpoints and native language decoders, without the paper's 48-hour A100 budget; if any baseline reaches or exceeds MedRegion-CT's BLEU-4 of 0.290 or CA-F1 of 0.450, the state-of-the-art claim is undercut.

Watch

Extended reading notes

Core claim

The paper's central discovery, stated on its own terms, is that jointly modeling global scan context and clinically meaningful regions—rather than global features alone—improves both the linguistic quality and diagnostic fidelity of automated 3D CT reports. MedRegion-CT reduces the 3D volume to D global tokens plus T region-representative tokens via R2 Token Pooling, where the region-representative slices are chosen by maximal pseudo-mask coverage; it then feeds mask tokens and spatial tokens from a masked visual extractor, together with deterministic patient-specific attribute text, into LLaMA3-8B. On RadGenome-Chest CT the full model achieves the best BLEU-4 (0.290), clinical-accuracy F1 (0.450), and GPT-4 evaluator score (48.837), while ablations removing mask guidance, attribute text, or region-centric sampling each drop the clinical-accuracy score. The authors therefore claim that region-centric cues, not just global features, are what let an MLLM produce clinically reliable CT reports.

Load-bearing premise

The central claim presupposes that the baselines, retrained under a 48-hour limit on two A100 GPUs with LLaMA3-8B swapped in as decoder, still represent their published capabilities; if that retraining understates them, the reported gains over prior work may be largely artifact.

Editorial extensions

If this is right

  • A frozen 2D encoder plus R2 Token Pooling can represent a whole 3D CT volume in D+T tokens instead of D×T, preserving inter-slice depth cues and region detail at tractable LLM context lengths.
  • Choosing slow-token slices by maximal pseudo-mask coverage outperforms LITA's uniform sampling, since the R2-only ablation beats the LITA ablation on clinical-accuracy metrics.
  • Removing either the mask tokens or the attribute text lowers clinical-accuracy F1 (from 0.450 to 0.430 and 0.425), meaning both region guidance and quantitative prompts contribute to clinical fidelity.
  • Structuring generation by six predefined anatomical regions and then merging the sub-reports is a viable way to produce full CT reports, matching or surpassing single-pass generation baselines.
  • Because segmentation tokens are placed at fixed positions in the prompt regardless of whether a mask is positive, the model can judge lesion presence directly from mask information rather than relying on prompt phrasing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not claimed in the paper: if the 48-hour, two-A100 retraining of the baselines understates their published checkpoints, the Table 1 margins could shrink or vanish; reproducing the baselines with their released weights would settle this.
  • Not claimed in the paper: the same region-splitting and slow-fast token scheme should transfer to other volumetric modalities, such as MRI, whenever a text-promptable segmentation model for the target anatomy is available.
  • Not claimed in the paper: since reports are produced region by region, a clinician could review or correct individual region paragraphs before the merge; the paper does not explore this interactive use.
  • Not claimed in the paper: the framework's ceiling is tied to segmentation quality, so any improvement in the universal segmenter (SAT) should directly raise the mask tokens' and attribute text's clinical informativeness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes MedRegion-CT, a multimodal large language model for 3D chest CT report generation. The method combines three components: (i) R2 Token Pooling, which creates global tokens from all slices and region-representative tokens from slices selected by pseudo-mask coverage; (ii) a Mask-Driven Visual Extractor that converts pseudo-masks into segmentation tokens; and (iii) a Patient Attribute Extractor that derives deterministic quantitative attributes (organ volumes, lesion counts/diameters/locations) from the masks and feeds them to the LLM as text prompts. The model is trained and evaluated on RadGenome-Chest CT, with generative reports split into six predefined anatomical regions. The paper reports quantitative comparisons against CT2Rep, RadFM, MedBLIP, and M3D, plus ablations removing each component. The central claim is that MedRegion-CT achieves state-of-the-art performance in both linguistic quality and clinical accuracy.

Significance. If the reported gains are real and the comparison is fair, the region-centric tokenization and explicit integration of deterministic mask-derived attributes would be a useful contribution to 3D CT report generation, an area with few open-source systems. Strengths of the paper include the public code release, the transparent description of the attribute extraction algorithm, and the systematic ablation of the proposed components. However, the paper's central claim is not established by the evidence as presented: Table 1 shows that the proposed method does not outperform all baselines on all headline metrics, and the baseline comparison is not like-for-like because MedRegion-CT receives segmentation masks and attribute prompts that the baselines do not see. These issues are load-bearing for the state-of-the-art conclusion.

major comments (5)
  1. [Abstract; Section 4.2, Table 1] The abstract and Section 4.2 claim that MedRegion-CT 'outperforms existing approaches in both linguistic quality and clinical accuracy' and achieves 'state-of-the-art performance.' Table 1 does not support this claim: M3D achieves higher ROUGE-L (0.391 vs 0.375) and METEOR (0.497 vs 0.494), and CT2Rep achieves a much higher GREEN score (0.374 vs 0.278). The claim should be restricted to BLEU-4, CA-F1, and GPT-4 score, or the comparison should be extended and analyzed to establish any overall superiority.
  2. [Section 3 (Eq. 1); Section 4.1; Supplementary Material] The comparison in Table 1 is not like-for-like. MedRegion-CT receives segmentation tokens Tseg and attribute text Tattr (Eq. 1) derived from pseudo-masks, while the four baselines receive only CT volumes. Because RadGenome-Chest CT provides organ-level SAT-generated masks (Section 4.1) and the Supplementary Material states that SAT was used 'to extract major lesion and anatomical masks when unavailable,' the model may be consuming the dataset's grounded annotations at test time. These same masks are used to link each ground-truth report sentence to anatomical regions, so the higher CA-F1 (0.450) and GPT-4 score (48.837) may reflect privileged region and attribute information rather than better text generation. The authors should either retrain the baselines with the same mask/attribute inputs, or evaluate MedRegion-CT with masks and attributes predicted purely at inference from CT volumes and report both settings.
  3. [Supplementary Material, Training and Implementation] The baseline results may not be representative of the methods being compared. All baselines were retrained under a 48-hour, two-A100-GPU budget with LLaMA3-8B substituted as the language decoder, whereas the published versions of these models used different backbones, training budgets, and decoders. The paper should report the published scores of the original models alongside the retrained scores, or provide evidence that the retrained baselines are representative; otherwise the state-of-the-art claim is not established against the actual prior art.
  4. [Section 4.2, Table 1] The dismissal of CT2Rep's GREEN score (0.374) as 'a biased result primarily due to overfitting to normal cases' is not supported by any reported evidence, such as a normal-case/abnormal-case breakdown of GREEN or CA-F1. Without such stratification, the highest GREEN score among all methods cannot simply be set aside; the authors should report per-case or stratified scores, or otherwise substantiate the overfitting explanation.
  5. [Section 4.3, Table 1] The ablation claim that the complete model 'consistently outperformed all ablation variants across various evaluation metrics' is contradicted by Table 1: Ablation-R2+Attr has higher BLEU-4 (0.293 vs 0.290) and Ablation-R2+Mask has higher METEOR (0.498 vs 0.494). The authors should soften this claim or add statistical/variance information to show that the differences are meaningful.
minor comments (5)
  1. [Section 2.4 heading] The heading 'Extention of Textual Prompts' contains a typo; it should be 'Extension of Textual Prompts.'
  2. [Figure 4 caption] In the caption, 'legion' should be 'lesion'; the code comment in Figure 4 repeats the same typo.
  3. [Table 1 caption] The caption reads 'Comparison of NLG Metrics and LM Based Metrics on the generated reports between MS-VLM and baseline methods,' but MS-VLM is not a method in Table 1; it should refer to MedRegion-CT or 'the proposed method.'
  4. [Section 3.1] The procedure for splitting ground-truth reports into the six predefined regions and later merging the generated region reports is not fully specified; the authors should clarify how sentence-to-region assignment is performed and how the final merged report is assembled, since this affects all reported metrics.
  5. [Section 4.1] The evaluation reports only single-run numbers without confidence intervals or significance tests; given the small differences in several metrics (e.g., METEOR 0.494 vs 0.497), the authors should consider reporting variance across seeds or runs.

Circularity Check

1 steps flagged · score 6.0 of 10

Clinical-accuracy claim is partly self-confirming: the model is fed mask tokens and patient attributes derived from the same SAT segmentation masks that RadGenome uses to ground and split the reference reports, while baselines receive CT only.

  1. other [Section 3 Eq. (1); Section 3.4; Section 4.1; Supplementary Material (Training and Implementation)]
    "R = LLM (Tvision, Tseg, Tattr, I) (1) ... medical information is extracted from the pseudo-masks using deterministic algorithms, which are then converted into patient-specific attribute text tokens Tattr. ... RadGenome-Chest CT provides detailed organ-level segmentation masks covering 197 anatomical regions, which were generated using SAT [53] ... we integrated the corresponding report sentences to create comprehensive region-specific reports, which we then used as ground truth text. ..."

    The generated report R is conditioned on Tseg and Tattr, and Tattr is a deterministic readout of pseudo-masks: lesion count, diameter, spatial location, and organ volumes. Those same quantities are exactly what the reference reports state, and RadGenome's reference sentences are split into six region-wise ground-truth reports using segmentation masks generated by the same SAT model. The supplementary material says SAT is used 'when unavailable,' implying that when masks are already provided (as in RadGenome) the provided masks are available as model inputs. Thus the model is not purely inferring lesion findings from CT; it is prompted with mask-derived facts that the ground-truth reports contain and that CA-F1 and GPT-4 scoring reward.

full rationale

This is an empirical systems paper rather than a mathematical derivation, so most engineering components (R2 token pooling, mask pooling, attribute extraction) are not circular reductions. No load-bearing self-citation chain, imported uniqueness theorem, or ansatz smuggling is present; the paper cites external prior work (LITA, MAIRA-SEG, SAT, RadGPT) for standard building blocks. The circular element is in the evaluation protocol: the model's extra inputs and the reference reports share the same underlying source of truth, namely RadGenome's SAT-generated segmentation masks. When Tattr directly encodes lesion count, diameter, location, and organ volumes, and those values are precisely what the reference reports contain and what the clinical-accuracy metrics check, the model's 'prediction' of those findings is at least partly a restatement of its input. The ablations do not fix this because they never give the baselines the same mask and attribute inputs. Separately, the SOTA wording is weakened by the paper's own Table 1, where M3D surpasses MedRegion-CT on ROUGE-L (0.391 vs 0.375) and METEOR (0.497 vs 0.494), and the 48-hour retraining protocol for baselines is a fairness concern; these are correctness risks, not circularity. Weighing these, I find one partially circular prediction step, scored 6.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical or theoretical entities are introduced. All components are existing neural modules and deterministic algorithms. The main assumptions are about the quality and representativeness of the segmentation masks, the report splitting procedure, and the fairness of the baseline comparison.

free parameters (2)
  • Number of representative slices s = Not specified in paper
    Controls the trade-off between global and region tokens. The paper says 'We select s slices from the CT' but never states the value; the final model uses 356 vision tokens.
  • Predefined six-region split = Lung, Large airways, Mediastinum, Heart and great vessels, Osseous structures, Upper Abdomen
    Chosen by the authors. Ground truth reports are split along this split, so training and evaluation depend on this design choice.
assumptions (3)
  • domain assumption SAT pseudo-masks accurately delineate the six clinical regions and lesions in test volumes.
    Sections 3.3 and 3.4 rely on SAT masks for region token sampling, mask tokens, and patient attributes. If masks are wrong, all three input streams degrade.
  • domain assumption Merging and splitting RadGenome grounded sentences into six region-wise reports preserves the clinical content of the original reports.
    Section 4.1 states 'For each of these six regions, we integrated the corresponding report sentences to create comprehensive region-specific reports.' If the split is lossy, training and evaluation targets are altered.
  • ad hoc to paper Baselines retrained under a 48-hour / two-A100-GPU budget with a swapped LLaMA3-8B decoder are representative of those methods.
    The Supplementary 'Comparison' paragraph describes the resource-constrained retraining. The state-of-the-art claim depends on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation." pith.science (2026). https://pith.science/paper/QOLZPOJF

@misc{pith2026250623102,
  author       = {Pith},
  title        = {Pith review of: Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOLZPOJF}},
  note         = {Machine review of arXiv:2506.23102}
}
read the original abstract

Current CT report generation frameworks predominantly rely on global feature representations, often failing to capture region-specific details and potentially missing certain abnormalities. To overcome this limitation, we propose MedRegion-CT, a region-focused multimodal large language model framework featuring three key innovations. First, we revisit the SlowFast strategy to jointly model global and fine-grained information and adapt it to the medical domain via a Region-based SlowFast Tokenizer that extracts tokens guided by clinically meaningful regions. Second, generated pseudo-masks guide the model to attend to diagnostically important anatomical regions, facilitating a systematic understanding of the overall scan context. Third, quantitative lesion information, including size, diameter, and spatial location, is encoded as structured textual prompts, enabling context-aware and clinically informed report generation. To enable rigorous evaluation, we validate our framework on multi-institutional structured report generation benchmarks. Experimental results demonstrate that MedRegion-CT achieves state-of-the-art performance, outperforming existing approaches in both linguistic quality and clinical accuracy. All code is publicly available at: https://github.com/babbu3682/MedRegion-CT.

Figures

Figures reproduced from arXiv: 2506.23102 by the authors.

Figure 1
Figure 1. Comprehensive radiological workflow for clinical di [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of MedRegion-CT. In R 2 token pooling, CT images processed in slice-wise independent manner undergo efficient feature extraction, yielding 3D vision tokens composed of global tokens and region representative tokens. Pseudo-masks are tokenized through R 2 pooling to correspond with Multi Level visual tokens, and these are subsequently combined via mask pooling. Spatial tokens are derived from pseudo-masks th… view at source ↗
Figure 3
Figure 3. Qualitative comparison of generated radiology reports for a patient with suspected COVID-19 pneumonia comparing previous [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The black text contains image information derived from [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression

    cs.CV 2026-07 conditional novelty 6.0 of 10

    ORCA compresses 3D CT tokens into organ-guided connected regions with sinusoidal centroid encoding, outperforming grid average and other compressors at matched budgets.

  2. Cheap Probes Predict Expensive Training in 3D-CT Vision--Language Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Disease-probe AUROC on frozen 3D-CT tokens predicts report-generation clinical micro-F1 across encoder×compression cells at r=0.95, ρ=0.89 (six cells, preliminary, one dataset).

Reference graph

Works this paper leans on

81 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [1]

    Modern diagnostic imaging technique applications and risk factors in the medical field: a review

    Shah Hussain, Iqra Mubeen, Niamat Ullah, Syed Shahab Ud Din Shah, Bakhtawar Abduljalil Khan, Muhammad Za- hoor, Riaz Ullah, Farhat Ali Khan, and Mujeeb A Sultan. Modern diagnostic imaging technique applications and risk factors in the medical field: a review. BioMed research in- ternational, 2022(1):5164970, 2022. 1

    2022

  2. [2]

    Evidence-based guideline for the written radiology re- port: Methods, recommendations and implementation chal- lenges

    Stacy K Goergen, Felicity J Pool, Tari J Turner, Jane E Grimm, Mark N Appleyard, Carmel Crock, Michael C Fa- hey, Michael F Fay, Nicholas J Ferris, Susan M Liew, et al. Evidence-based guideline for the written radiology re- port: Methods, recommendations and implementation chal- lenges. Journal of medical imaging and radiation oncology, 57(1):1–7, 2013. 1

    work page 2013

  3. [3]

    Early chest computed tomography scan to assist diagnosis and guide treatment decision for suspected community-acquired pneumonia

    Yann Erick Claessens, Marie-Pierre Debray, Florence Tubach, Anne Laure Brun, Blandine Rammaert, Pierre Haus- fater, Jean-Marc Naccache, Patrick Ray, Christophe Cho- quet, Marie France Carette, Charles Mayaud, Catherine Lep- ort, and Xavier Duval. Early chest computed tomography scan to assist diagnosis and guide treatment decision for suspected community-...

    work page

  4. [4]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  5. [5]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023

    2023

  6. [6]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  7. [7]

    Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024

    arXiv 2024

  8. [8]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 1

    arXiv 2025

Show all 81 references
  1. [9]

    Kevin Zhou, Hayit Greenspan, Christos Davatzikos, James S

    S. Kevin Zhou, Hayit Greenspan, Christos Davatzikos, James S. Duncan, Bram van Ginneken, Anant Madabhushi, Jerry L Prince, Daniel Rueckert, and Ronald M. Summers. A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highli...

  2. [10]

    Willemink, Wojciech A Koszek, Cailin Hardell, Jie Wu, Dominik Fleischmann, Hugh Harvey, Les R

    Martin J. Willemink, Wojciech A Koszek, Cailin Hardell, Jie Wu, Dominik Fleischmann, Hugh Harvey, Les R. Folio, Ronald M. Summers, D. Rubin, and Matthew P. Lungren. Preparing medical imaging data for machine learning. Radi- ology, page 192224, 2020. 2

    2020

  3. [11]

    A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero- shot detection of abnormalities

    Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Bastian Wittmann, Enis Sim- sar, Mehmet Simsar, et al. A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-...

    2024

  4. [12]

    Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Jiayu Lei, Ya Zhang, Yanfeng Wang, and Weidi Xie. Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis. arXiv preprint arXiv:2404.16754, 2024. 2, 6

    2024 arXiv

  5. [13]

    Towards generalist foundation model for radi- ology by leveraging web-scale 2d&3d medical data

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards generalist foundation model for radi- ology by leveraging web-scale 2d&3d medical data. arXiv preprint arXiv:2308.02463, 2023. 2, 3, 6

    2023 arXiv

  6. [14]

    Medblip: Bootstrapping language-image pre-training from 3d medical images and texts

    Qiuhui Chen and Yi Hong. Medblip: Bootstrapping language-image pre-training from 3d medical images and texts. In Proceedings of the Asian Conference on Computer Vision, pages 2404–2420, 2024. 3, 6, 2

    2024

  7. [15]

    M3d: Advancing 3d medical image analysis with multi-modal large language models

    Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models. arXiv preprint arXiv:2404.00578, 2024. 2, 3, 6

    2024 arXiv

  8. [16]

    Ct2rep: Automated radiology report generation for 3d medi- cal imaging

    Ibrahim Ethem Hamamci, Sezgin Er, and Bjoern Menze. Ct2rep: Automated radiology report generation for 3d medi- cal imaging. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 476–

  9. [17]

    Developing general- ist foundation models from a multimodal dataset for 3d com- puted tomography

    Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Omer Faruk Durugol, Bastian Wittmann, Tamaz Amiranashvili, et al. Developing general- ist foundation models from a multimodal dataset for 3d com- put...

    2024

  10. [18]

    Sedigheh Eslami, Christoph Meinel, and Gerard De Melo. Pubmedclip: How much does clip benefit visual question answering in the medical domain? In Findings of the As- sociation for Computational Linguistics: EACL 2023, pages 1181–1193, 2023. 2

    2023

  11. [19]

    Exploring scalable medical image encoders beyond text supervision

    Fernando P ´erez-Garc´ıa, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Matthew P Lungren, et al. Exploring scalable medical image encoders beyond text supervision. Nature Machine Int...

    2025

  12. [20]

    General pur- pose image encoder dinov2 for medical image registration

    Xinrui Song, Xuanang Xu, and Pingkun Yan. General pur- pose image encoder dinov2 for medical image registration. arXiv preprint arXiv:2402.15687, 2024. 2

    2024 arXiv

  13. [21]

    Next token prediction towards multi- modal intelligence: A comprehensive survey

    Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, et al. Next token prediction towards multi- modal intelligence: A comprehensive survey. arXiv preprint arXiv:2412.18619, 2024. 2

    2024 arXiv

  14. [22]

    Llava-med: Training a large language- and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language- and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36:28541–28564,

  15. [23]

    Xraygpt: Chest radiographs summarization using medical vision- language models

    Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullap- pilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Shahbaz Khan. Xraygpt: Chest radiographs summarization using medical vision- language models. arXiv preprint arXiv:2306.07971, 2023

    2023 arXiv

  16. [24]

    Med- flamingo: a multimodal medical few-shot learner

    Michael Moor, Qian Huang, Shirley Wu, Michihiro Ya- sunaga, Cyril Zakka, Yashodhara Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, and Jure Leskovec. Med- flamingo: a multimodal medical few-shot learner. ArXiv, abs/2307.15189, 2023

    2023 arXiv

  17. [25]

    R2gengpt: Radiology report generation with frozen llms

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. R2gengpt: Radiology report generation with frozen llms. Meta-Radiology, 1(3):100033, 2023

    2023

  18. [26]

    Biomedgpt: A unified and generalist biomed- ical generative pre-trained transformer for vision, language, and multimodal tasks

    Kai Zhang, Jun Yu, Eashan Adhikarla, Rong Zhou, Zhiling Yan, Yixin Liu, Zhengliang Liu, Lifang He, Brian Davison, Xiang Li, et al. Biomedgpt: A unified and generalist biomed- ical generative pre-trained transformer for vision, language, and multimodal tasks. arXiv e-prints , p...

    2023

  19. [27]

    Chexagent: Towards a foun- dation model for chest x-ray interpretation

    Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Mag- dalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, et al. Chexagent: Towards a foun- dation model for chest x-ray interpretation. arXiv prepr...

    2024 arXiv

  20. [28]

    Maira-seg: Enhancing radiology report gen- eration with segmentation-aware multimodal large language models

    Harshita Sharma, Valentina Salvatelli, Shaury Srivastav, Kenza Bouzid, Shruthi Bannur, Daniel C Castro, Maximil- ian Ilse, Sam Bond-Taylor, Mercy Prasanna Ranjit, Fabian Falck, et al. Maira-seg: Enhancing radiology report gen- eration with segmentation-aware multimodal large l...

    2024 arXiv

  21. [29]

    Towards gen- eralist biomedical ai

    Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaeker- mann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, et al. Towards gen- eralist biomedical ai. Nejm Ai, 1(3):AIoa2300138, 2024

    2024

  22. [30]

    Medtrinity-25m: A large-scale multimodal dataset with multigranular annotations for medicine

    Yunfei Xie, Ce Zhou, Lang Gao, Juncheng Wu, Xianhang Li, Hong-Yu Zhou, Sheng Liu, Lei Xing, James Zou, Cihang Xie, et al. Medtrinity-25m: A large-scale multimodal dataset with multigranular annotations for medicine. arXiv preprint arXiv:2408.02900, 2024. 2

    2024 arXiv

  23. [31]

    A dataset of clinically generated visual questions and answers about radiology images

    Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1–10, 2018. 2

    2018

  24. [32]

    Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317, 2019

    2019

  25. [33]

    Pmc-vqa: Vi- sual instruction tuning for medical visual question answer- ing

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Vi- sual instruction tuning for medical visual question answer- ing. arXiv preprint arXiv:2305.10415, 2023. 2

    2023 arXiv

  26. [34]

    Benchmarking and boosting radiology report generation for 3d high-resolution medical images

    Che Liu, Zhongwei Wan, Yuqi Wang, Hui Shen, Haozhe Wang, Kangyu Zheng, Mi Zhang, and Rossella Arcucci. Benchmarking and boosting radiology report generation for 3d high-resolution medical images. arXiv preprint arXiv:2406.07146, 2024. 3

    2024 arXiv

  27. [35]

    Generatect: Text-conditional generation of 3d chest ct volumes

    Ibrahim Ethem Hamamci, Sezgin Er, Enis Simsar, An- jany Kumar Sekuboyina, Chinmay Prabhakar, Alperen Tez- can, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Furkan Al- mas, Irem Dougan, Muhammed Furkan Dasdelen, Hadrien Reynaud, Sarthak Pati, Christian Bluethgen, Mehmet Kemal Ozde...

    2023

  28. [36]

    Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation

    Changsun Lee, Sangjoon Park, Cheong-Il Shin, Woo Hee Choi, Hyun Jeong Park, Jeong Eun Lee, and Jong Chul Ye. Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation. arXiv preprint arXiv:2412.13558, 2024. 3

    2024 arXiv

  29. [37]

    Med-2e3: A 2d-enhanced 3d medical multimodal large language model

    Yiming Shi, Xun Zhu, Ying Hu, Chenyi Guo, Miao Li, and Ji Wu. Med-2e3: A 2d-enhanced 3d medical multimodal large language model. arXiv preprint arXiv:2411.12783 ,

  30. [38]

    Lita: Language instructed temporal-localization assistant

    De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, and Jan Kautz. Lita: Language instructed temporal-localization assistant. ArXiv, abs/2403.19046, 2024. 3, 5, 6

    2024 arXiv

  31. [39]

    Slowfast-llava: A strong training-free baseline for video large language models

    Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin De- hghan. Slowfast-llava: A strong training-free baseline for video large language models. ArXiv, abs/2407.15841, 2024

    2024 arXiv

  32. [40]

    Text-to-edit: Controllable end-to-end video ad creation via multimodal llms

    Dabing Cheng, Haosen Zhan, Xingchen Zhao, Guisheng Liu, Zemin Li, Jinghui Xie, Zhao Song, Weiguo Feng, and Bingyue Peng. Text-to-edit: Controllable end-to-end video ad creation via multimodal llms. ArXiv, abs/2501.05884,

    arXiv

  33. [41]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6201–6210, 2018. 3

    2019

  34. [42]

    Gpt4roi: In- struction tuning large language model on region-of-interest

    Shilong Zhang, Pei Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. Gpt4roi: In- struction tuning large language model on region-of-interest. ArXiv, abs/2307.03601, 2023. 3

    2023 arXiv

  35. [43]

    Groma: Localized visual tokenization for grounding multimodal large language models

    Chuofan Ma, Yi Jiang, Jiannan Wu, Zehuan Yuan, and Xiao- juan Qi. Groma: Localized visual tokenization for grounding multimodal large language models. ArXiv, abs/2404.13013, 2024

    2024 arXiv

  36. [44]

    Regiongpt: Towards region understanding vision lan- guage model

    Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision lan- guage model. 2024 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 13796–13806,

    2024

  37. [45]

    A refer-and- ground multimodal large language model for biomedicine

    Xiaoshuang Huang, Haifeng Huang, Lingdong Shen, Yehui Yang, Fangxin Shang, Junwei Liu, and Jia Liu. A refer-and- ground multimodal large language model for biomedicine. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2024. 3

    2024

  38. [46]

    Topol, and Pranav Rajpurkar

    Hong-Yu Zhou, Subathra Adithan, Juli ´an Nicol ´as Acosta, Eric J. Topol, and Pranav Rajpurkar. A generalist learner for multifaceted medical image interpretation. ArXiv, abs/2405.07988, 2024

    2024 arXiv

  39. [47]

    Interpretable bilin- gual multimodal large language model for diverse biomedi- cal tasks

    Lehan Wang, Haonan Wang, Honglong Yang, Jiaji Mao, Ze- hong Yang, Jun Shen, and Xiaomeng Li. Interpretable bilin- gual multimodal large language model for diverse biomedi- cal tasks. ArXiv, abs/2410.18387, 2024. 3

    2024 arXiv

  40. [48]

    Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Anton Schwaighofer, Sam Bond-Taylor, Maximilian Ilse, Fernando P’erez-Garc’ia, Valentina Salvatelli, Harshita Sharma, Fe- lix Meissen, Mercy Prasanna Ranjit, Shaury Srivastav, Julia Gong, Fabian Falck, Ozan Oktay, Anja Thieme, Ma...

    2024 arXiv

  41. [49]

    Interactive and explainable region-guided radiol- ogy report generation

    Tim Tanida, Philip M ¨uller, Georgios Kaissis, and Daniel Rueckert. Interactive and explainable region-guided radiol- ogy report generation. 2023 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 7433– 7442, 2023. 3

    2023

  42. [50]

    An inclusive task-aware frame- work for radiology report generation

    Lin Wang, Munan Ning, Donghuan Lu, Dong Wei, Yefeng Zheng, and Jie lian Chen. An inclusive task-aware frame- work for radiology report generation. In International Conference on Medical Image Computing and Computer- Assisted Intervention, 2022. 3

    2022

  43. [51]

    Osprey: Pixel un- derstanding with visual instruction tuning

    Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. Osprey: Pixel un- derstanding with visual instruction tuning. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28202–28211, 2023. 3, 5

    2024

  44. [52]

    Large language model with region-guided referring and grounding for ct report generation

    Zhixuan Chen, Yequan Bie, Haibo Jin, and Hao Chen. Large language model with region-guided referring and grounding for ct report generation. ArXiv, abs/2411.15539, 2024. 3

    2024 arXiv

  45. [53]

    One model to rule them all: Towards universal segmentation for medical images with text prompts

    Ziheng Zhao, Yao Zhang, Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. One model to rule them all: Towards universal segmentation for medical images with text prompts. ArXiv, abs/2312.17183, 2023. 3, 6, 2

    2023 arXiv

  46. [54]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in nat- ural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hi- roaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in nat- ural language processing. ACM Computing Surveys, 55:1 – 35, 2021. 3

    2021

  47. [55]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji rong Wen. Evaluating object hallucination in large vision-language models. In Conference on Empirical Methods in Natural Language Processing, 2023. 3

    2023

  48. [56]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9568–9578, 2024. 3

    2024

  49. [57]

    Medical image understanding with pretrained vision language mod- els: A comprehensive study

    Ziyuan Qin, Huahui Yi, Qicheng Lao, and Kang Li. Medical image understanding with pretrained vision language mod- els: A comprehensive study. ArXiv, abs/2209.15517, 2022. 4

    2022 arXiv

  50. [58]

    Promptmrg: Diagnosis-driven prompts for medical report generation

    Haibo Jin, Haoxuan Che, Yi-Mou Lin, and Haoxing Chen. Promptmrg: Diagnosis-driven prompts for medical report generation. ArXiv, abs/2308.12604, 2023. 4

    2023 arXiv

  51. [59]

    Dia-llama: Towards large language model-driven ct report generation

    Zhixuan Chen, Luyang Luo, Yequan Bie, and Hao Chen. Dia-llama: Towards large language model-driven ct report generation. ArXiv, abs/2403.16386, 2024. 4

    2024 arXiv

  52. [60]

    Graham, An- drea J

    Chun-Hsiao Yeh, Jiayun Wang, Andrew D. Graham, An- drea J. Liu, Bo Tan, Yubei Chen, Yi Ma, and Meng C. Lin. Insight: A multi-modal diagnostic pipeline using llms for oc- ular surface disease diagnosis. ArXiv, abs/2410.00292, 2024. 4, 5

    arXiv

  53. [61]

    Bassi, Mehmet Can Yavuz, Kang Wang, Xi- aoxi Chen, Wenxuan Li, Sergio Decherchi, Andrea Cav- alli, Yang Yang, Alan L

    Pedro R.A.S. Bassi, Mehmet Can Yavuz, Kang Wang, Xi- aoxi Chen, Wenxuan Li, Sergio Decherchi, Andrea Cav- alli, Yang Yang, Alan L. Yuille, and Zongwei Zhou. Radgpt: Constructing 3d image-text tumor datasets. ArXiv, abs/2501.04678, 2025. 4, 5

    2025 arXiv

  54. [62]

    Jour- nal club: Structured radiology reports are more complete and more effective than unstructured reports

    Peter Andrew Marcovici and George Albert Taylor. Jour- nal club: Structured radiology reports are more complete and more effective than unstructured reports. AJR. Ameri- can journal of roentgenology, 203 6:1265–71, 2014. 5

    2014

  55. [63]

    Simiele, Valeria Pe- ruzzi, Martina Bortolot, Annarita Tullio, Giuseppe Como, Chiara Zuiani, and Rossano Girometti

    Lorenzo Cereser, Francesco Cortiula, C. Simiele, Valeria Pe- ruzzi, Martina Bortolot, Annarita Tullio, Giuseppe Como, Chiara Zuiani, and Rossano Girometti. Assessing the impact of structured reporting on learning how to report lung cancer staging ct: A triple cohort study on i...

    2024

  56. [64]

    Castro, Anton Schwaighofer, Matthew P

    Fernando P’erez-Garc’ia, Harshita Sharma, Sam Bond- Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Matthew P. Lungren, Maria Teodora Wetscherek, Noel Codella, Stephanie L. Hyland, Javier Alvarez-Valle, and Oza...

    arXiv

  57. [65]

    Automatic quan- titative computed tomography measurement of longitudinal lung volume loss in interstitial lung diseases

    Salim Aymeric Si-Mohamed, Mouhamad Nasser, Marion Colevray, Olivier Nempont, Pierre Lartaud, Anna Sesilia Vlachomitrou, Thomas Broussaud, Kais Ahmad, Julie Tr- aclet, Vincent Cottin, and Loic Boussel. Automatic quan- titative computed tomography measurement of longitudinal lun...

    2022

  58. [66]

    Consistency of radiologists in identifying pulmonary nodules based on low-dose computed tomogra- phy

    Shuai Ming, Wei Yang, Sijia Cui, Shuai Huang, and Xi- angyang Gong. Consistency of radiologists in identifying pulmonary nodules based on low-dose computed tomogra- phy. Journal of thoracic disease, 11 7:2973–2980, 2019. 6

    2019

  59. [67]

    Renal incidental find- ings on computed tomography

    Hans Jonas Meyer, Alina Pfeil, Dominik Schramm, An- dreas Gunter Bach, and Alexey Surov. Renal incidental find- ings on computed tomography. Medicine, 96, 2017. 6

    2017

  60. [68]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin, editors, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , ...

    2002

  61. [69]

    ROUGE: A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Com- putational Linguistics. 6

    2004

  62. [70]

    Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments. In IEEvaluation@ACL, 2005. 6

    2005

  63. [71]

    Chang, Amilcare Gentili, and Chun-Nan Hsu

    An Yan, Julian McAuley, Xing Lu, Jiang Du, Eric Y . Chang, Amilcare Gentili, and Chun-Nan Hsu. Radbert: Adapting transformer-based language models to radiology. Radiology. Artificial intelligence, 4 4:e210258, 2022. 6

    2022

  64. [72]

    Moseley, Curtis P

    Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Edward Michalson, Michael E. Moseley, Curtis P. Langlotz, Ak- shay S. Chaudhari, and Jean-Benoit Delbrouck. Green: Gen- erative radiology report evaluation and error notation.ArX...

    2024 arXiv

  65. [73]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aur´elien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation langu...

    2023 arXiv

  66. [74]

    Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    J. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. ArXiv, abs/2106.09685, 2021. 2

    2021 arXiv

  67. [75]

    Fixing weight decay reg- ularization in adam

    Ilya Loshchilov and Frank Hutter. Fixing weight decay reg- ularization in adam. ArXiv, abs/1711.05101, 2017. 2

    2017 arXiv

  68. [76]

    Zero: Memory optimizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–16, 2019. 2 MedRegion-CT: Region-Foc...

    2019

  69. [78]

    ] do 25: volume ← sitk.GetArrayFromImage(organ).sum() 26: end for 27: for legion = [N odule, Cyst, Ef f usion,

    Patient Attribution Extractor 1: function GET DIAMETER( mask) 2: mask array ← 3: sitk.GetArrayFromImage(mask) 4: labeled mask ← 5: skimage.measure.label(mask array) 6: regions ← 7: skimage.measure.regionprops(labeled mask) 8: dmts ← ∅ 9: for rg in regions do 10: mnz, mny, mnx,...

  70. [79]

    We preprocess the 3D CT images using consis- tent min-max normalization and resize the 3D images to Figure 5

    Training and Implementation Training. We preprocess the 3D CT images using consis- tent min-max normalization and resize the 3D images to Figure 5. The black text contains image information derived from CT slices, the blue text indicates region-specific segmentation to- kens l...

  71. [80]

    The MLLM training consists of two steps

    as our LLM base model and load its pre-trained param- eters. The MLLM training consists of two steps. Initially, we froze the vision encoder and LLM while fine-tuning the Multimodal Connector with an image-report dataset, apply- ing a batch size of 4 × 8, learning rate of 10−4...

  72. [81]

    Comparison

    The implementation is performed in PyTorch, and train- ing is conducted in parallel on NVIDIA A100 GPUs with 80GB memory. Comparison. For baseline comparisons, we compare three state-of-the-art open-source 3D methods capable of generating CT reports: RadFM [13], Med-BLIP [14],...

  73. [486]

    Springer, 2024. 3, 6, 2

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.