REVIEW 9 cited by
Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Models (LLMs) have introduced a new era of proficiency in comprehending complex healthcare and biomedical topics. However, there is a noticeable lack of models in languages other than English and models that can interpret multi-modal input, which is crucial for global healthcare accessibility. In response, this study introduces Qilin-Med-VL, the first Chinese large vision-language model designed to integrate the analysis of textual and visual data. Qilin-Med-VL combines a pre-trained Vision Transformer (ViT) with a foundational LLM. It undergoes a thorough two-stage curriculum training process that includes feature alignment and instruction tuning. This method enhances the model's ability to generate medical captions and answer complex medical queries. We also release ChiMed-VL, a dataset consisting of more than 1M image-text pairs. This dataset has been carefully curated to enable detailed and comprehensive interpretation of medical data using various types of images.
Forward citations
Cited by 9 Pith papers
-
Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation
A new Vietnamese PET/CT-report dataset improves medical VLM report generation and VQA, but clinical F1 scores remain modest.
-
Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Research
A systematic review of brain imaging foundation models covering 86 models and 161 datasets, with a performance tournament, dataset atlas, and duplicated-data warnings.
-
Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback
UMed-LVLM uses GPT-4V-generated abnormality data and abnormal-aware rewards to improve medical image diagnosis and localization.
-
GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI
GMAI-VL-5.5M is a new 5.5M-sample medical image-text dataset built from 219 datasets via GPT-4o annotation-guided generation, and GMAI-VL is a three-stage LLaVA-style model reporting SOTA numbers, though the evaluatio...
-
MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration
MAM, a role-specialized multi-agent LLM framework with discussion, voting, and web retrieval, reports higher diagnostic accuracy than single models on ten multimodal medical datasets.
-
Advancing Vision-based Human Action Recognition: Exploring Vision-Language CLIP Model for Generalisation in Domain-Independent Tasks
CLIP misclassifies human actions under masking, and the proposed class-specific noise augmentation is claimed to help but lacks quantitative validation.
-
Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
A survey that classifies vision-language model adaptation for medical imaging into five strategies across eleven tasks, with challenges and future directions.
-
Multimodal Large Language Models for Medicine: A Comprehensive Survey
A comprehensive review cataloging medical MLLMs, their uses in report generation, diagnosis, and treatment, and the challenges of accuracy, hallucination, fairness, privacy, and deployment.
-
Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions
This is a broad survey of medical multimodal foundation models that categorizes datasets, proxy tasks, vision-language models, and clinical applications without introducing new results.
Discussion (0). Continue with ORCID to comment.