{"paper":{"title":"Advancing Multimodal Medical Capabilities of Gemini","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI","cs.CL","cs.LG"],"primary_cat":"cs.CV","authors_text":"Akshay Goel, Andrew Sellergren, Arnav Agharwal, Atilla Kiraly, Christopher Kelly, Chuck Lau, Cory McLean, Daniel Golden, Daniel Tse, Dave Steiner, David Fleet, David G. T. Barrett, Ellery Wulczyn, Eric Wang, Farhad Hormozdiari, Faruk Ahmed, Fayaz Jamil, Greg Corrado, Gregory Sorensen, Ira Ktena, Joelle Barral, Jorge Cuadros, Katherine Chou, Kendall Park, Khaled Saab, Lin Yang, Mozziyar Etemadi, Nick George, Rory Pilgrim, Ryutaro Tanno, Shawn Xu, Shekoofeh Azizi, Shravya Shetty, Shruthi Prabhakara, Siyuan Qiao, S. M. Ali Eslami, Sreenivasa Raju Kalidindi, S. Sara Mahdavi, Tao Tu, Theo Guidroz, Tiam Jaroensri, Timo Kohlberger, Wei-Hung Weng, Yang Wang, Yossi Matias, Yuchen Zhou, Yun Liu","submitted_at":"2024-05-06T04:44:22Z","abstract_excerpt":"Many clinical tasks require an understanding of specialized data, such as medical images and genomics, which is not typically found in general-purpose large multimodal models. Building upon Gemini's multimodal models, we develop several models within the new Med-Gemini family that inherit core capabilities of Gemini and are optimized for medical use via fine-tuning with 2D and 3D radiology, histopathology, ophthalmology, dermatology and genomic data. Med-Gemini-2D sets a new standard for AI-based chest X-ray (CXR) report generation based on expert evaluation, exceeding previous best results ac"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2405.03162","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2405.03162/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}