Pith. sign in

REVIEW 8 cited by

MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.06542 v1 pith:OE422LH6 submitted 2024-10-09 eess.IV cs.CV

Noel C. F. Codella , Ying Jin , Shrey Jain , Yu Gu , Ho Hin Lee , Asma Ben Abacha , Alberto Santamaria-Pang , Will Guyman
show 23 more authors
Naiteek Sangani Sheng Zhang Hoifung Poon Stephanie Hyland Shruthi Bannur Javier Alvarez-Valle Xue Li John Garrett Alan McMillan Gaurav Rajguru Madhu Maddi Nilesh Vijayrania Rehaan Bhimai Nick Mecklenburg Rupal Jain Daniel Holstein Naveen Gaur Vijay Aski Jenq-Neng Hwang Thomas Lin Ivan Tarapov Matthew Lungren Mu Wei
This is my paper · ORCID
classification eess.IVcs.CV
keywords medimageinsightmedicalsotaimagingothersearchachievesacross
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this work, we present MedImageInsight, an open-source medical imaging embedding model. MedImageInsight is trained on medical images with associated text and labels across a diverse collection of domains, including X-Ray, CT, MRI, dermoscopy, OCT, fundus photography, ultrasound, histopathology, and mammography. Rigorous evaluations demonstrate MedImageInsight's ability to achieve state-of-the-art (SOTA) or human expert level performance across classification, image-image search, and fine-tuning tasks. Specifically, on public datasets, MedImageInsight achieves SOTA in CT 3D medical image retrieval, as well as SOTA in disease classification and search for chest X-ray, dermatology, and OCT imaging. Furthermore, MedImageInsight achieves human expert performance in bone age estimation (on both public and partner data), as well as AUC above 0.9 in most other domains. When paired with a text decoder, MedImageInsight achieves near SOTA level single image report findings generation with less than 10\% the parameters of other models. Compared to fine-tuning GPT-4o with only MIMIC-CXR data for the same task, MedImageInsight outperforms in clinical metrics, but underperforms on lexical metrics where GPT-4o sets a new SOTA. Importantly for regulatory purposes, MedImageInsight can generate ROC curves, adjust sensitivity and specificity based on clinical need, and provide evidence-based decision support through image-image search (which can also enable retrieval augmented generation). In an independent clinical evaluation of image-image search in chest X-ray, MedImageInsight outperformed every other publicly available foundation model evaluated by large margins (over 6 points AUC), and significantly outperformed other models in terms of AI fairness (across age and gender). We hope releasing MedImageInsight will help enhance collective progress in medical imaging AI research and development.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Semantically Calibrated Evidence Composition for CT Vision-Language Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SCOPE composes organ-level CT evidence under whole-volume context and calibrates it with diagnostic-summary supervision, reaching 85.0/72.2 macro AUC on CT-RATE and RadChestCT.

  2. A Scale-adaptive Vision Model Links C. elegans Neuronal Morphology to Behavior for Neurotoxicity Assessment

    cs.CE 2026-07 conditional novelty 6.0 of 10

    A scale-adaptive vision model links C. elegans neuron morphology to behavior, predicts dopamine-dependent deficits, and identifies the benzimidazole moiety as a neurotoxicity determinant.

  3. Curia: A Multi-Modal Foundation Model for Radiology

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A DINOv2 vision transformer pre-trained on 200M unlabeled CT/MRI slices matches or beats prior foundation models and radiology residents on a new 19-task benchmark.

  4. Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification

    cs.CV 2025-12 conditional novelty 5.0 of 10

    A frozen 2D vision foundation model with lightweight LoRA adapters and attention-based slice fusion achieves state-of-the-art 3D medical image classification across 12 tasks with about 1M trainable parameters per task.

  5. Grounding Multimodal Large Language Models with Quantitative Skin Attributes: A Retrieval Study

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Fine-tuning Qwen2-VL to predict 16 quantitative skin attributes yields embeddings that retrieve images matching a query in both appearance and the target attribute.

  6. Consistent Point Matching

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Adding a round-trip consistency heuristic and multi-neighbor voting to Point Matching improves matching accuracy on four medical imaging datasets and beats a supervised baseline on Deep Lesion Tracking.

  7. Fairness and Robustness of CLIP-Based Models for Chest X-rays

    cs.CV 2025-07 conditional novelty 5.0 of 10

    CLIP-based chest X-ray models show age-related performance gaps and rely on chest drains as a shortcut, while sensitive attributes remain decodable from their embeddings.

  8. From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine

    cs.AI 2025-02 conditional novelty 3.0 of 10

    A PRISMA-ScR scoping review of 144 studies finds the field shifting from text-only LLMs to multimodal AI in medicine, with evaluation and data diversity still the main bottlenecks.

Pith tools