REVIEW 8 cited by
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Adapting a language model into a specific domain, a.k.a `domain adaption', is a common practice when specialized knowledge, e.g. medicine, is not encapsulated in a general language model like Llama2. The challenge lies in the heterogeneity of data across the two training stages, as it varies in languages, genres, or formats. To tackle this and simplify the learning protocol, we propose to transform heterogeneous data, from the both pre-training and supervised stages, into a unified, simple input-output pair format. We validate the new protocol in the domains where proprietary LLMs like ChatGPT perform relatively poorly, such as Traditional Chinese Medicine. The developed model, HuatuoGPT-II, has shown state-of-the-art performance in Chinese medicine domain on a number of benchmarks, e.g. medical licensing exams. It even outperforms proprietary models like ChatGPT and GPT-4 in some aspects, especially in Traditional Chinese Medicine. Expert manual evaluations further validate HuatuoGPT-II's advantages over existing LLMs. Notably, HuatuoGPT-II was benchmarked in a fresh Chinese National Medical Licensing Examination where it achieved the best performance, showcasing not only its effectiveness but also its generalization capabilities.
Forward citations
Cited by 8 Pith papers
-
ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
A cascaded multi-encoder medical MLLM with native 3D fusion and RoI-grounded report metrics claims SOTA on most 2D/3D medical benchmarks and highest radiologist report rankings.
-
DentalBench: Benchmarking and Advancing LLMs Capability for Bilingual Dentistry Understanding
A new bilingual dental QA benchmark and corpus reveals large performance gaps in LLMs for dentistry, and shows that domain adaptation with the corpus improves accuracy.
-
Training-free LLM Merging for Multi-task Learning
Hi-Merging merges two task-specialized LLMs by pruning and scaling delta vectors at model and layer level, reporting gains over prior merging and multi-task fine-tuning on English and Chinese MCQA and QA tasks.
-
Enhancing Medical Dialogue Generation through Knowledge Refinement and Dynamic Prompt Adjustment
MedRef combines variational knowledge refinement, entity-action prediction, and dynamic prompt adjustment to improve medical dialogue generation on MedDG and KaMed.
-
C-MIG: Multi-view Information Gain-based Retrieval-Augmented Generation for Clinical Diagnosis Reasoning
C-MIG uses multi-view information gain from retrieved documents and refinements to supervise RAG-RL for clinical diagnosis, claiming top performance on four medical benchmarks.
-
DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment
A data-processing and privacy-preserving deployment framework claims state-of-the-art Chinese medical LLM accuracy and a 27% reduction in training-data leakage.
-
Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL
Medical QA accuracy improves substantially through reinforcement learning with a binary correct-answer reward alone, without supervised fine-tuning on distilled reasoning traces.
-
Survey of Specialized Large Language Model
A survey of 24 specialized LLMs (2022-2025) claims a shift from domain fine-tuning to native architectures, but the synthesis is undermined by citation errors and selection bias.
Discussion (0). Sign in to comment.