Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training

T0 review · 3 major / 5 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Structured diagnostic supervision from reports, not model scale, is what makes 3D radiology vision-language models work under tight compute.

desk verdict Solid empirical recipe: MUST supervision + small ResNet-18 hits strong zero-shot CT numbers and real efficiency; LLM-label risk is real but not fatal given the external checks. read the letter →

arxiv 2603.06467 v3 pith:HYE7TXP3 submitted 2026-03-06 cs.CV

Yingtai Li , Shuai Ming , Qiuli Wang , Mingyue Zhao , Hongchun Zhang , Yuhe Tian , Haoran Lai , Rongsheng Wang
show 8 more authors
Rui Zhou Rundong Wang Yujia Li Zhiyang He Xiaodong Tao Wei Chen Wei Wei Shaohua Kevin Zhou
This is my paper · ORCID
classification cs.CV
keywords radiologyfoundationmodelsvision-languagepre-training3DCTMRIsupervisiondesignzero-shotdiagnosisresource-efficienttrainingLLMlabelextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Radiology foundation models have mostly copied the scale-first recipe of natural-image vision-language pre-training, but 3D CT and MRI volumes are scarcer, reports are noisier, and hospitals need local adaptation under privacy and GPU limits. GreenRFM argues that the missing ingredient is not a larger backbone but better use of the supervision already latent in routine reports. The authors convert free-text reports into structured present/absent/uncertain diagnostic labels with large language models, then apply four principles they call MUST: more distilled labels, ubiquitous supervision of every encoder and the alignment space, semantic-enforcing unimodal pre-training before alignment, and task-aligning design choices that match how diagnosis is actually done. With a standard 3D ResNet-18 they reach zero-shot CT-RATE AUC 84.8 after 24 GPU-hours on one 24 GB card (or 4 hours on 6 GB), transfer across private institutional cohorts and to musculoskeletal MRI, and show that the same local retraining lifts performance on a held-out hospital set. The claim is that diagnosis-centered supervision, not parameter count, is the practical path to resource-efficient and locally adaptable radiology representations.

What carries the argument

MUST supervision: LLM-distilled present/absent/uncertain labels used first for independent unimodal classification of image and text encoders, then for contrastive alignment that keeps a shared diagnostic classifier and drops L2 normalization so feature magnitude can encode confidence.

What would settle it

Train the identical pipeline on a large cohort whose labels are produced solely by multiple radiologists under a locked protocol that never uses an LLM; if zero-shot AUC and external transfer then collapse relative to the LLM-label version, the silver-standard premise is false.

Watch

Extended reading notes

Core claim

Converting noisy radiology reports into structured diagnostic labels and injecting that supervision into every stage of a two-stage pipeline lets a lightweight 3D ResNet-18 match or beat far larger scale-first models on zero-shot abnormality diagnosis, cross-modal retrieval, and private-cohort transfer while training in hours rather than thousands of GPU-hours.

Load-bearing premise

The method assumes that LLM-extracted silver-standard labels from multi-institutional reports are accurate and unbiased enough to serve as the main training signal, without baking in generator artifacts that fail under human-curated or out-of-distribution labeling.

Editorial extensions

If this is right

  • Hospitals can retrain or adapt a competitive radiology foundation model on a single consumer GPU without shipping data off-site.
  • The same four supervision principles transfer from chest and abdominal CT to multi-sequence musculoskeletal MRI without architecture redesign.
  • Local retraining on an institutional cohort can raise macro-AUC by more than ten points while remaining computationally feasible.
  • The aligned space can be used for secondary clinical endpoints such as microvascular-invasion prediction and treatment-response analysis that improve on established clinical scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If silver-standard labels are sufficient, the historical bottleneck of gold-standard annotation for 3D radiology may be largely removable for many diagnostic categories.
  • Task-aligning choices such as removing L2 normalization may generalize to other medical contrastive settings where confidence is clinically meaningful.
  • The data-efficiency curve suggests further gains from modest additional volumes once the supervision pipeline is fixed, rather than from larger vision transformers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that radiology vision–language foundation models need not follow the scale-first recipe of natural-image CLIP-style pre-training. Instead it introduces GreenRFM, a two-stage supervision-centric pipeline organized around four MUST principles (More-distilled LLM-extracted ternary diagnostic labels, Ubiquitous classification heads on vision/text/alignment, Semantic-enforcing independent unimodal pre-training, and Task-aligning architectural choices such as GAP and no L2 normalization). With a standard 3D ResNet-18 (≈33 M parameters) the method reaches zero-shot CT-RATE AUC 84.8 after 24 GPU-hours on one 24 GB GPU (lightweight variant: 6 GB / 4 h), outperforms larger literature baselines on CT-RATE, RAD-ChestCT and Merlin, transfers zero-shot to two private multi-protocol CT cohorts, and shows principle-level gains on private musculoskeletal MRI when both models are trained from scratch on the same splits. Ablations and data-scaling curves are supplied to attribute gains to the supervision design rather than model size.

Significance. If the empirical claims hold, the work supplies concrete evidence that carefully engineered diagnostic supervision can replace orders-of-magnitude more compute and parameters for 3D radiology VLMs, directly addressing the equity and carbon-cost barriers that currently confine foundation-model development to a few well-resourced labs. The multi-institution evaluation (>200 k volumes, six sites, two modalities), public code release, explicit lightweight configuration, and component-wise ablations constitute reproducible, falsifiable contributions that the community can build on. The result would shift research attention from architectural scaling to supervision design and would make local hospital adaptation of RFMs practically feasible.

major comments (3)
  1. [LLM-distilled diagnostic labels; Supp. Table S4 / Fig. S1] Supp. Table S4 and Fig. S1 (and the corresponding paragraph in “LLM-distilled diagnostic labels…”): strict-match Cohen’s κ with Merlin’s human labels is only 0.10; high agreement (κ = 0.94) appears only after discarding all uncertain cases. Because the same ternary taxonomy and LLM pipeline define both the training targets (including the shared classifier) and the zero-shot evaluation prompts, residual generator-specific biases could be absorbed into both sides of the comparison. A random human re-annotation of a few hundred CT-RATE training reports (or a sensitivity analysis that systematically flips uncertain labels) is required to substantiate the claim that the silver-standard labels are sufficiently unbiased for the central “MUST-supervision-beats-scale” conclusion.
  2. [Abstract; Results (Generalization to private clinical benchmarks)] Abstract (and Introduction) state that “computationally feasible retraining raises macro-AUC from 70.5 to 82.1” on a local institutional cohort. No corresponding experiment, data split, training protocol, or table appears in the Results or Methods sections (only zero-shot numbers for AH-Chest/AH-Abd are given in Table 2). Either the full local-adaptation results must be added or the claim must be removed from the abstract.
  3. [Extension to MRI modality; Table 4] Table 4 and the section “Extension to MRI modality”: both GreenRFM and the MRCLIP baseline are trained from scratch on the private AH-Knee / AH-Spine splits (n_val = 372 / 921). The experiment therefore demonstrates that the MUST principles transfer across modalities, not that a CT-pretrained foundation model transfers. With such small validation sets the foundation-model claim for MRI remains provisional; either zero-shot transfer of the CT checkpoint or a clear statement that the MRI results are principle-level only is needed.
minor comments (5)
  1. [Figure 1; Introduction] Figure 1 panel C and the surrounding text claim “orders of magnitude” reductions in compute and CO2e relative to VoCo; the comparison mixes different hardware (RTX 3090 vs H800) and data regimes. A normalized GPU-hour or FLOPs table would make the efficiency claim more precise.
  2. [Task-aligning supervision; Eq. (2)] Eq. (2) and the surrounding paragraph assert that removing L2 normalization preserves “confidence or severity.” No quantitative analysis of embedding-norm versus radiologist confidence or disease severity is supplied; a short correlation plot or ablation would strengthen the geometric argument.
  3. [Table 1] Table 1 reports literature numbers for VoCo, Uniferum, BrgSA etc. without re-evaluation under a common prompt set or preprocessing pipeline. A footnote acknowledging possible protocol mismatch would be helpful.
  4. [Figure 3; Methods] Several figure captions (Fig. 3e–f, Fig. 4) use “time-weighted” loss without defining the weighting scheme; a one-sentence clarification in Methods would aid reproducibility.
  5. [Throughout] Minor typographical inconsistencies appear (e.g., “T wo-stage”, “L2” vs “L 2”, “AnHui” vs “Anhui”). A final proof-reading pass is recommended.

Circularity Check

1 steps flagged · score 1.0 of 10

Empirical methods paper with independent external benchmarks; only minor non-load-bearing self-citation to a preliminary study by overlapping authors.

  1. self citation load bearing [Introduction, paragraph on framework origin]
    "Our framework builds on our preliminary study [20] and substantially expands it through theoretical formalization and large-scale validation."

    Citation [20] shares authors; it supplies the seed idea of LLM-distilled labels. However the present paper re-implements, ablate, and re-validates on >200 k volumes and external sets, so the citation is not load-bearing for the SOTA or efficiency claims.

full rationale

GreenRFM is an empirical vision-language pre-training paper whose central claims (MUST supervision yields SOTA zero-shot AUC 84.8 on CT-RATE with a 33 M ResNet-18 and 24 GPU-h, plus transfer to Merlin, RAD-ChestCT, private CT/MRI cohorts) rest on held-out evaluation, ablations (two-stage vs joint, diagnostic labels vs visual descriptions, pooling/L2 choices), and external human-curated or dual-LLM-consensus sets. There is no mathematical derivation, uniqueness theorem, fitted parameter re-used as prediction, or self-definitional loop. The sole self-citation ([20], preliminary study by overlapping authors) is acknowledged as the starting point that is substantially expanded and re-validated at scale; it does not force the results. Taxonomy overlap between training labels and zero-shot prompts is intentional task-alignment, not circularity by construction, because metrics are computed on unseen volumes and independent labeling protocols. Residual LLM-label noise is a correctness/risk issue, not circularity. Score 1 reflects only the minor self-citation; core claims are independently supported.

Assumptions & free parameters 6 free parameters · 6 assumptions · 3 invented entities

The central efficiency and accuracy claims rest on standard deep-learning practice plus several domain assumptions about report-to-label distillation quality, the superiority of diagnostic over descriptive supervision for zero-shot diagnosis, and architectural consistency between pre-training and inference. Free parameters are ordinary training hyperparameters and design choices, not physical constants. No new physical entities are postulated; ‘MUST’ and ‘silver-standard labels’ are methodological constructs.

free parameters (6)
  • AdamW learning rates (image / text / alignment) = 1e-4 / 1e-5 / 1e-5
    Set to 1e-4, 1e-5, 1e-5 respectively; chosen by authors and load-bearing for reported convergence in 5+5 epochs.
  • batch size = 10
    Fixed at 10 for both stages on CT-RATE and Merlin; affects contrastive alignment quality and memory claims.
  • shared-classifier loss weight λ
    Balances L_CLIP against classification heads in stage 2; value not exhaustively swept in the main text but affects the final 84.8 AUC.
  • training epochs (stage 1 + stage 2) = 5 + 5
    5 + 5 epochs claimed for convergence; efficiency narrative depends on this short schedule.
  • Lite stem stride/kernel (4,4,4) = (4,4,4)
    Hand-chosen downsampling for 6 GB VRAM variant; directly enables the laptop-training claim.
  • abnormality taxonomy size (18 / 30 / MRI-specific) = 18 (chest), 30 (abd), 13/8 (MRI)
    Fixed label sets define what ‘diagnostic supervision’ means; choice is clinical/ad hoc and shapes all metrics.
assumptions (6)
  • domain assumption LLM-extracted present/absent/uncertain labels from free-text reports are a sufficiently faithful and scalable proxy for clinical diagnostic supervision.
    Core of ‘More distilled supervision’; justified by Merlin agreement after dropping uncertain cases and by noise-robustness literature, but imperfect under strict match.
  • domain assumption Strong unimodal discriminative features must be learned before cross-modal alignment (two-stage semantic-enforcing training beats joint multi-task).
    Stated as Principle 3; supported by loss curves in Fig. 3e–f but remains an empirical design choice, not a theorem.
  • domain assumption Diagnostic concept labels are better pre-training targets for zero-shot diagnosis than visual-description labels extracted from the same reports.
    Task-aligning principle; ablation in Fig. 4c (84.8 vs 81.3 AUC) supports it for this metric suite.
  • ad hoc to paper Removing L2 normalization and using global average pooling throughout preserves confidence geometry and matches zero-shot inference.
    Architectural consistency claims; ablations show drops when L2 or alternative pooling is used, but the geometric interpretation is interpretive.
  • domain assumption Standard contrastive + BCE losses with equal positive/negative weighting and the listed preprocessing windows are adequate for multi-label 3D CT/MRI.
    Ordinary ML practice assumed throughout Methods; not re-derived.
  • domain assumption Kinetics-initialized 3D ResNet-18 is a fair, non-specialized backbone for comparing against billion-parameter RFMs.
    Used to argue that gains come from supervision design rather than architecture; reasonable but still a modeling choice.
invented entities (3)
  • MUST supervision principles (More distilled, Ubiquitous, Semantic-enforcing, Task-aligning)
    purpose: Organize and name the four design rules that convert reports into supervision for every model component.
    Methodological framing rather than a physical entity; independent evidence is the ablation suite, not an external measurement.
  • Silver-standard LLM diagnostic labels (ternary 1/0/-1 vectors)
    purpose: Provide scalable structured targets replacing raw report text and expensive manual annotation.
    Constructed by prompting; partial external check on Merlin, dual-LLM filter on private data. Not independently measured pathology ground truth.
  • GreenRFM / GreenRFM-L model family
    purpose: Concrete resource-efficient RFM instantiations of the MUST pipeline.
    Engineering artifact defined by the paper; evaluated on public and private benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training." pith.science (2026). https://pith.science/paper/HYE7TXP3

@misc{pith2026260306467,
  author       = {Pith},
  title        = {Pith review of: GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYE7TXP3}},
  note         = {Machine review of arXiv:2603.06467}
}
read the original abstract

Radiology foundation models (RFMs) have largely inherited the scale-first recipe of natural-image vision--language pre-training. This recipe is difficult to deploy in 3D radiology, where training corpora are smaller, reports vary across institutions, and receiving hospitals often need local adaptation under privacy and compute constraints. We ask whether routine radiology reports can instead be converted into auditable diagnostic supervision that shapes the image encoder, text encoder, aligned space, and local-adaptation procedure. We develop GreenRFM, a supervision-centric pre-training framework organized around four empirical principles: More distilled, Ubiquitous, Semantic-enforcing, and Task-aligning (MUST) supervision. These principles convert noisy reports into structured diagnostic signals and use them to learn discriminative unimodal encoders plus an aligned image--text space for diagnosis-centered multimodal use. GreenRFM requires 24 GPU-hours on a single 24GB GPU (lightweight variant: 6GB VRAM, 4~hours) and reaches a zero-shot CT-RATE AUC of 84.8. Evaluations using more than 200,000 volumes from six institutions and two modalities show transfer to private clinical cohorts and to musculoskeletal MRI. On a local institutional cohort, computationally feasible retraining raises macro-AUC from 70.5 to 82.1. The aligned space also improves hepatocellular-carcinoma microvascular-invasion prediction and trans-arterial chemoembolization response analysis over established clinical scores. These results support supervision-centric pre-training as a practical route to resource-efficient, locally adaptable, diagnosis-centered radiology vision--language representations.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ORACLE-CT: Anatomy-Aware Support Pooling for CT Classification

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    ORACLE-CT improves CT classification performance by using anatomy-specific support pooling based on multi-organ segmentation, showing gains in AUROC on internal and external datasets.

Reference graph

Works this paper leans on

76 extracted references · 8 linked inside Pith · cited by 1 Pith paper

  1. [1]

    End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.Nature medicine, 25(6):954–961, 2019

    Diego Ardila, Atilla P Kiraly, Sujeeth Bharadwaj, Bokyung Choi, Joshua J Reicher, Lily Peng, Daniel Tse, Mozziyar Etemadi, Wenxing Ye, Greg Corrado, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.Nature medicine, 25(6):954–961, 2019

    2019

  2. [2]

    Large-scale pancreatic cancer detection via non-contrast ct and deep learning.Nature medicine, 29(12):3033–3043, 2023

    Kai Cao, Yingda Xia, Jiawen Yao, Xu Han, Lukas Lambert, Tingting Zhang, Wei Tang, Gang Jin, Hui Jiang, Xu Fang, et al. Large-scale pancreatic cancer detection via non-contrast ct and deep learning.Nature medicine, 29(12):3033–3043, 2023

    2023

  3. [3]

    Ai-based diagnosis of acute aortic syndrome from noncontrast ct.Nature Medicine, pages 1–13, 2025

    Yujian Hu, Yilang Xiang, Yan-Jie Zhou, Yangyan He, Dehai Lang, Shifeng Yang, Xiaolong Du, Chunlan Den, Youyao Xu, Gaofeng Wang, et al. Ai-based diagnosis of acute aortic syndrome from noncontrast ct.Nature Medicine, pages 1–13, 2025

    2025

  4. [4]

    Screening and diagnosis of cardiovascular disease using artificial intelligence-enabled cardiac magnetic resonance imaging.Nature Medicine, pages 1–10, 2024

    Yan-Ran Wang, Kai Yang, Yi Wen, Pengcheng Wang, Yuepeng Hu, Yongfan Lai, Yufeng Wang, Kankan Zhao, Siyi Tang, Angela Zhang, et al. Screening and diagnosis of cardiovascular disease using artificial intelligence-enabled cardiac magnetic resonance imaging.Nature Medicine, pages 1–10, 2024

    2024

  5. [5]

    Rsna 2023 abdominal trauma ai challenge: Review and outcomes.Radiology: Artificial Intelligence, 7(1):e240334, 2024

    Sebastiaan Hermans, Zixuan Hu, Robyn L Ball, Hui Ming Lin, Luciano M Prevedello, Ferco H Berger, Ibrahim Yusuf, Jeffrey D Rudie, Maryam Vazirabad, Adam E Flanders, et al. Rsna 2023 abdominal trauma ai challenge: Review and outcomes.Radiology: Artificial Intelligence, 7(1):e240334, 2024

    2023

  6. [6]

    A generative model uses healthy and diseased image pairs for pixel-level chest x-ray pathology localization.Nature Biomedical Engineering, pages 1–13, 2025

    Kaiming Dong, Yuxiao Cheng, Kunlun He, and Jinli Suo. A generative model uses healthy and diseased image pairs for pixel-level chest x-ray pathology localization.Nature Biomedical Engineering, pages 1–13, 2025

    2025

  7. [7]

    A vision–language foundation model for the generation of realistic chest x-ray images.Nature Biomedical Engineering, 9(4):494–506, 2025

    Christian Bluethgen, Pierre Chambon, Jean-Benoit Delbrouck, Rogier Van Der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P Langlotz, and Akshay S Chaudhari. A vision–language foundation model for the generation of realistic chest x-ray images.Nature Biomedical Engineering, 9(4):494–506, 2025

    2025

  8. [8]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 19

    2022

Show all 76 references
  1. [9]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–...

    2022

  2. [10]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  3. [11]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  4. [12]

    A pan-organ vision-language model for generalizable 3d ct representations.medRxiv, 2025

    Cameron Beeche, Joonghyun Kim, Hamed Tavolinejad, Bingxin Zhao, Rakesh Sharma, Jeffrey Duda, James Gee, Farouk Dako, Anurag Verma, Colleen Morse, et al. A pan-organ vision-language model for generalizable 3d ct representations.medRxiv, 2025

    2025

  5. [13]

    A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-shot detection of abnormalities.CoRR, 2024

    Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Bastian Wittmann, Enis Simsar, Mehmet Simsar, et al. A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-sh...

    2024

  6. [14]

    Simcrop: Radiograph representation learning with similarity-driven cross-granularity pre-training

    Rongsheng Wang, Fenghe Tang, Qingsong Yao, Rui Yan, Xu Zhang, Zhen Huang, Haoran Lai, Zhiyang He, Xiaodong Tao, Zihang Jiang, et al. Simcrop: Radiograph representation learning with similarity-driven cross-granularity pre-training. InInternational Conference on Medical Image C...

    2025

  7. [15]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  8. [16]

    Beyond the failures: Rethinking foundation models in pathology.arXiv preprint arXiv:2510.23807, 2025

    Hamid R Tizhoosh. Beyond the failures: Rethinking foundation models in pathology.arXiv preprint arXiv:2510.23807, 2025

    2025 arXiv

  9. [17]

    Multi-modal masked autoencoders for medical vision-and-language pre-training

    Zhihong Chen, Yuhao Du, Jinpeng Hu, Yang Liu, Guanbin Li, Xiang Wan, and Tsung-Hui Chang. Multi-modal masked autoencoders for medical vision-and-language pre-training. In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, and Shuo Li, editors,Medical Image Computing an...

    2022

  10. [18]

    Large-scale 3d medical image pre-training with geometric context priors.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Linshan Wu, Jiaxin Zhuang, and Hao Chen. Large-scale 3d medical image pre-training with geometric context priors.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  11. [19]

    Foundation models in radiology: What, how, why, and why not.Radiology, 314(2):e240597, 2025

    Magdalini Paschali, Zhihong Chen, Louis Blankemeier, Maya Varma, Alaa Youssef, Christian Bluethgen, Curtis Langlotz, Sergios Gatidis, and Akshay Chaudhari. Foundation models in radiology: What, how, why, and why not.Radiology, 314(2):e240597, 2025

    2025

  12. [20]

    More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the llm era

    Yingtai Li, Haoran Lai, Xiaoqian Zhou, Shuai Ming, Wenxin Ma, Wei Wei, and Shaohua Kevin Zhou. More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the llm era. InInternational Conference on Medical Image Computing and Computer- Ass...

    2025

  13. [21]

    A reality check of vision-language pre- training in radiology: Have we progressed using text? InInternational Conference on Information Processing in Medical Imaging, pages 294–309

    Julio Silva-Rodríguez, Jose Dolz, and Ismail Ben Ayed. A reality check of vision-language pre- training in radiology: Have we progressed using text? InInternational Conference on Information Processing in Medical Imaging, pages 294–309. Springer, 2025

    2025

  14. [22]

    Contrastive learning of medical visual representations from paired images and text

    Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D Manning, and Curtis P Langlotz. Contrastive learning of medical visual representations from paired images and text. InMachine learning for healthcare conference, pages 2–25. PMLR, 2022

    2022

  15. [23]

    Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020. 20

    2020

  16. [24]

    Multi-task learning as a bargaining game.arXiv preprint arXiv:2202.01017, 2022

    Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as a bargaining game.arXiv preprint arXiv:2202.01017, 2022

    2022 arXiv

  17. [25]

    Magface: A universal representation for face recognition and quality assessment

    Qiang Meng, Shichao Zhao, Zhida Huang, and Feng Zhou. Magface: A universal representation for face recognition and quality assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14225–14234, 2021

    2021

  18. [26]

    Normface: L2 hypersphere embedding for face verification

    Feng Wang, Xiang Xiang, Jian Cheng, and Alan L Yuille. Normface: L2 hypersphere embedding for face verification. InProceedings of the 25th ACM international conference on Multimedia, pages 1041–1049, 2017

    2017

  19. [27]

    Merlin: A vision language foundation model for 3d computed tomography.Research Square, pages rs–3, 2024

    Louis Blankemeier, Joseph Paul Cohen, Ashwin Kumar, Dave Van Veen, Syed Jamal Safdar Gardezi, Magdalini Paschali, Zhihong Chen, Jean-Benoit Delbrouck, Eduardo Reis, Cesar Truyts, et al. Merlin: A vision language foundation model for 3d computed tomography.Research Square, page...

    2024

  20. [28]

    Large language model-based uncertainty-adjusted label extraction for artificial intelligence model development in upper extremity radiography.European Radiology, pages 1–12, 2025

    Hanna Kreutzer, Anne-Sophie Caselitz, Thomas Dratsch, Daniel Pinto dos Santos, Christiane Kuhl, Daniel Truhn, and Sven Nebelung. Large language model-based uncertainty-adjusted label extraction for artificial intelligence model development in upper extremity radiography.Europe...

    2025

  21. [29]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. InProceedings of the AAAI c...

    2019

  22. [30]

    Deep learning is robust to massive label noise.arXiv preprint arXiv:1705.10694, 2017

    David Rolnick, Andreas Veit, Serge Belongie, and Nir Shavit. Deep learning is robust to massive label noise.arXiv preprint arXiv:1705.10694, 2017

    2017 arXiv

  23. [31]

    The class imbalance problem: A systematic study

    Nathalie Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intelligent data analysis, 6(5):429–449, 2002

    2002

  24. [32]

    A systematic study of the class imbalance problem in convolutional neural networks.Neural networks, 106:249–259, 2018

    Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks.Neural networks, 106:249–259, 2018

    2018

  25. [33]

    Large-scale and fine-grained vision-language pre-training for enhanced ct image understanding

    Zhongyi Shui, Jianpeng Zhang, Weiwei Cao, Sinuo Wang, Ruizhe Guo, Le Lu, Ling Zhang, Tingbo Liang, Lin Yang, Xianghua Ye, and Qi Zhang. Large-scale and fine-grained vision-language pre-training for enhanced ct image understanding. InThe Thirteenth International Conference on L...

    2025

  26. [34]

    Making the most of text semantics to improve biomedical vision–language processing

    Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, et al. Making the most of text semantics to improve biomedical vision–language processing. InEuropean ...

    2022

  27. [35]

    Openclip.Zenodo, 2021

    Gabriel Ilharco, Mitchell Wortsman, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, et al. Openclip.Zenodo, 2021

    2021

  28. [36]

    A multimodal biomedical foundation model trained from fifteen million image–text pairs.NEJM AI, 2(1):AIoa2400640, 2025

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. A multimodal biomedical foundation model trained from fifteen million image–text pairs.NEJM AI, 2(1):AIoa2400640, 2025

    2025

  29. [37]

    Dcformer: Efficient 3d vision-language modeling with decomposed convolutions.arXiv preprint arXiv:2502.05091, 2025

    Gorkem Can Ates, Yu Xin, Kuang Gong, and Wei Shao. Dcformer: Efficient 3d vision-language modeling with decomposed convolutions.arXiv preprint arXiv:2502.05091, 2025

    2025 arXiv

  30. [38]

    X2ct-clip: Enable multi- abnormality detection in computed tomography from chest radiography via tri-modal contrastive learning.arXiv preprint arXiv:2503.02162, 2025

    Jianzhong You, Yuan Gao, Sangwook Kim, and Chris Mcintosh. X2ct-clip: Enable multi- abnormality detection in computed tomography from chest radiography via tri-modal contrastive learning.arXiv preprint arXiv:2503.02162, 2025

    2025 arXiv

  31. [39]

    Bootstrapping chest ct image understanding by distilling knowledge from x-ray expert models

    Weiwei Cao, Jianpeng Zhang, Yingda Xia, Tony CW Mok, Zi Li, Xianghua Ye, Le Lu, Jian Zheng, Yuxing Tang, and Ling Zhang. Bootstrapping chest ct image understanding by distilling knowledge from x-ray expert models. InProceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  32. [40]

    Towards scalable language-image pre-training for 3d medical imaging.arXiv preprint arXiv:2505.21862, 2025

    Chenhui Zhao, Yiwei Lyu, Asadur Chowdury, Edward Harake, Akhil Kondepudi, Akshay Rao, Xinhai Hou, Honglak Lee, and Todd Hollon. Towards scalable language-image pre-training for 3d medical imaging.arXiv preprint arXiv:2505.21862, 2025

    2025

  33. [41]

    Boosting vision semantic density with anatomy normality modeling for medical vision-language pre-training

    Weiwei Cao, Jianpeng Zhang, Zhongyi Shui, Sinuo Wang, Zeli Chen, Xi Li, Le Lu, Xianghua Ye, Qi Zhang, Tingbo Liang, et al. Boosting vision semantic density with anatomy normality modeling for medical vision-language pre-training. InProceedings of the IEEE/CVF International Con...

    2025

  34. [42]

    Bridged semantic alignment for zero-shot 3d medical image diagnosis.arXiv preprint arXiv:2501.03565, 2025

    Haoran Lai, Zihang Jiang, Qingsong Yao, Rongsheng Wang, Zhiyang He, Xiaodong Tao, Wei Wei, Weifu Lv, and S Kevin Zhou. Bridged semantic alignment for zero-shot 3d medical image diagnosis.arXiv preprint arXiv:2501.03565, 2025

    2025

  35. [43]

    Unified supervision for vision-language modeling in 3d computed tomography

    Hao-Chih Lee, Zelong Liu, Hamza Ahmed, Spencer Kim, Sean Huver, Vishwesh Nath, Zahi A Fayad, Timothy Deyer, and Xueyan Mei. Unified supervision for vision-language modeling in 3d computed tomography. InProceedings of the IEEE/CVF International Conference on Computer Vision, pa...

    2025

  36. [44]

    On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    Rishi Bommasani. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  37. [45]

    Pillar-0: A new frontier for radiology foundation models.arXiv preprint arXiv:2511.17803, 2025

    Kumar Krishna Agrawal, Longchao Liu, Long Lian, Michael Nercessian, Natalia Harguindeguy, Yufu Wu, Peter Mikhael, Gigin Lin, Lecia V Sequist, Florian Fintelmann, et al. Pillar-0: A new frontier for radiology foundation models.arXiv preprint arXiv:2511.17803, 2025

    2025

  38. [46]

    The kinetics human action video dataset.arXiv preprint arXiv:1705.06950, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset.arXiv preprint arXiv:1705.06950, 2017

    2017 arXiv

  39. [47]

    Scaling supervision for free: Leveraging universal segmentation models for enhanced medical image diagnosis

    Yingtai Li, Shuai Ming, Haoran Lai, Fenghe Tang, Wei Wei, and S Kevin Zhou. Scaling supervision for free: Leveraging universal segmentation models for enhanced medical image diagnosis. In Submitted to Medical Imaging with Deep Learning, 2025. under review

    2025

  40. [48]

    Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intelligence, 5(5):e230024, 2023

    Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang, et al. Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intel...

    2023

  41. [49]

    Machine-learning-based multiple abnormality prediction with large-scale chest computed tomography volumes.Medical image analysis, 67:101857, 2021

    Rachel Lea Draelos, David Dov, Maciej A Mazurowski, Joseph Y Lo, Ricardo Henao, Geoffrey D Rubin, and Lawrence Carin. Machine-learning-based multiple abnormality prediction with large-scale chest computed tomography volumes.Medical image analysis, 67:101857, 2021

    2021

  42. [50]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  43. [51]

    Doubao: A large language model by bytedance.https://www.doubao.com, 2023

    ByteDance. Doubao: A large language model by bytedance.https://www.doubao.com, 2023. Accessed: June 2024

    2023

  44. [52]

    high attenuation

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. InProceedings of the IEEE international conference on computer vision, pages 618–626, ...

    2017

  45. [53]

    High-attenuation focus (density_high)

  46. [54]

    Low-attenuation focus (density_low)

  47. [55]

    Mixed-attenuation focus (density_mixed) - Morphology patterns -

  48. [56]

    Nodular opacity (morphology_nodular)

  49. [57]

    Patchy opacity (morphology_patchy)

  50. [58]

    Linear/stripe-like opacity (morphology_linear)

  51. [59]

    Reticular/network-like opacity (morphology_reticular) - Distribution patterns -

  52. [60]

    Focal distribution (distribution_focal)

  53. [61]

    Diffuse distribution (distribution_diffuse)

  54. [62]

    Bilateral symmetric distribution (distribution_bilateral_symmetric)

  55. [63]

    Example: 0,1,0,0,0,1,0,1,0,0,0 G

    Unilateral distribution (distribution_unilateral) Output format: exactly eleven comma-separated binary digits (0 or 1) matching the order above, with no additional text or punctuation. Example: 0,1,0,0,0,1,0,1,0,0,0 G. LLM prompts and code for diagnostic label extraction To en...

  56. [64]

    aortic_valve_calcification

  57. [65]

    coronary_calcification

  58. [66]

    surgically_absent_gallbladder

  59. [67]

    abdominal_aortic_aneurysm

  60. [68]

    biliary_ductal_dilation

  61. [69]

    - Use 1 if the finding is explicitly present

    fracture Labeling rules - Output a single line of 30 comma-separated values. - Use 1 if the finding is explicitly present. - Use 0 if the finding is explicitly ruled out or absent. - Use -1 if the report is ambiguous, uncertain, or lacks information. Only return the comma-sepa...

  62. [70]

    arterial wall calcification

  63. [71]

    pericardial effusion

  64. [72]

    coronary artery wall calcification

  65. [73]

    pulmonary fibrotic sequela

  66. [74]

    mosaic attenuation pattern

  67. [75]

    peribronchial thickening

  68. [76]

    - Use 1 if the finding is explicitly present

    interlobular septal thickening Labeling rules - Output a single line of 18 comma-separated values. - Use 1 if the finding is explicitly present. - Use 0 if the finding is explicitly ruled out or absent. - Use -1 if the report is ambiguous, uncertain, or lacks information. Only...

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.