{"id":"d2863107-7551-48d5-9e1e-9bb2bd6cd11f","arxiv_id":"2506.18512","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"MedTVT-R1 integrates ECG, CXR, and lab data with a modality perception layer and GRPO-based reinforcement fine-tuning, claiming improved multi-disease diagnosis, but the evidence is weakened by unfair baselines and answer-conditioned training data.","lead":"A new multimodal large language model combines ECG, chest X-ray, and lab data to diagnose multiple diseases, trained on a dataset of GPT-4o-generated question-answer pairs. The paper reports state-of-the-art results, but the evaluation compares against untrained models and the reasoning chains are generated from known answers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The superiority claim in Table 1 is confounded: MedTVT-R1 is fine-tuned on MedTVT-QA, while all eight baselines are evaluated zero-shot, so the comparison cannot isolate model or architecture quality.","rationale":"The reader's weakest assumption about GPT-4o label-conditioned CoE is a serious concern for interpretability, but the strongest claim in Section 4.2 is about outperforming leading MLLMs in NLG and clinical metrics. That comparative claim is undercut directly by the missing control; even if the CoE were perfectly valid, Table 1 would still not establish SOTA because no baseline was fine-tuned on the same task. The proposed check isolates the training-protocol confound. This does not dispute the dataset's potential value; it disputes the performance comparison and the architectural attribution.","tokens_in":23178,"tokens_out":4101,"duration_ms":43177,"concrete_test":"Fine-tune the strongest zero-shot baseline, Qwen2.5-VL-3B-Instruct, on the same MedTVT-QA training split with an identical LoRA rank, PT/SFT epochs, and GRPO/Jaccard RFT settings (or at minimum SFT on disease-level QA + RFT), then re-run Table 1's disease-level evaluation on the held-out 375 samples. If its F1/AUC reaches or exceeds MedTVT-R1's 0.5190/0.6554, the claimed superiority is not established; if it remains at zero-shot levels, the architecture claims gain support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 and Table 1 compare MedTVT-R1 (LLaMA3.2-1B + LoRA, PT+SFT+RFT on 8,331 MedTVT-QA training samples) against eight MLLMs that are evaluated zero-shot via SWIFT. No baseline receives the MedTVT-QA training data, the CoE formatting, or the GRPO/Jaccard reward. The headline 'outperforms these leading models' is therefore a comparison of a task-tuned model against untuned models, not a comparison of architectures or training methods. The same confound affects the physiological-level results in Table 2. Even the ablation rows (w/o PT, w/o RFT) are still fine-tuned on the evaluation task, so they do not control for the fact that the baselines are zero-shot. A fair test must give the same training intervention to at least the strongest baseline; otherwise the reported F1/AUC gains (0.5190/0.6554 vs. best baseline 0.1995/0.5053) may be entirely due to seeing the training distribution, not to MPL, CMHA/CAO, or Jaccard RFT. This is the load-bearing gap for the paper's central empirical claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MedTVT-R1, a multimodal large language model that integrates ECG time series, chest X-ray images, and blood-test tabular data for multi-disease diagnosis and interpretable reasoning. The authors construct MedTVT-QA, an instruction dataset derived from MIMIC-IV with GPT-4o-generated physiological-level reports and disease-level 'Chain of Evidence' reasoning, and train a LLaMA3.2-1B backbone with a modality perception layer (CMHA and CAO) and GRPO-based reinforcement fine-tuning with a Jaccard reward. The paper claims state-of-the-art performance against eight existing MLLMs in both disease-level diagnosis and physiological-level understanding, with ablation studies supporting the contributions of pre-training, RFT, MPL components, and each modality.","tokens_in":23399,"tokens_out":2690,"duration_ms":27286,"significance":"If the claims held, the work would be a useful step toward interpretable multimodal medical diagnosis: the dataset construction across three heterogeneous modalities is a genuine contribution, the MPL architecture is a plausible design, and the open release of dataset and code supports reproducibility. The ablations of CMHA, CAO, and modality missingness are informative. However, the two load-bearing claims—that MedTVT-R1 outperforms existing MLLMs and that its generated reasoning reflects diagnostic inference—are undermined by a circular training-target construction and by an evaluation that compares a fine-tuned model against zero-shot baselines. These issues affect the central contribution and would require substantial rework of the dataset and evaluation protocol to resolve.","major_comments":[{"comment":"The Chain of Evidence target is generated by GPT-4o after being given the ground-truth disease set and explicit instruction to 'prove the reasons why the patient has the diseases I provided' and to 'not mention any diseases other than those I provided'. The resulting text is therefore a post-hoc justification conditioned on the answer, not an inference from the multimodal inputs. Training the model to reproduce this text and then evaluating its diagnostic agreement against the same labels measures how well the model can reproduce the training distribution, not the quality of medical reasoning. This is a load-bearing circularity for the paper's central claim of interpretable multi-disease reasoning.","section":"§3.1 Disease-QA Prompt"},{"comment":"The headline superiority claim is confounded: MedTVT-R1 is trained on MedTVT-QA through PT, SFT, and RFT, while all eight baseline MLLMs are evaluated zero-shot via SWIFT. No baseline receives the same fine-tuning intervention, the Chain-of-Evidence formatting, or the GRPO/Jaccard reward. The reported F1/AUC gains (0.5190/0.6554 vs. the best zero-shot baseline 0.1995/0.5053) therefore may reflect task-specific training rather than architectural or methodological superiority. A fair comparison requires at least the strongest baseline to be fine-tuned on the same training data; otherwise the claim 'outperforms these leading models' is not supported.","section":"§4.2, Table 1"},{"comment":"The physiological-level comparison has the same confound as Table 1: the test QA pairs for ECG, CXR, and LAB are drawn from MedTVT-QA, and MedTVT-R1 was pre-trained and SFT-tuned on this exact dataset, while the baselines are zero-shot. In addition, the test set is only 375 samples, and the paper provides no error bars, confidence intervals, or multiple-seed runs for the RL/SFT training. With high-variance training procedures and a small test set, the reported differences cannot be judged statistically reliable. The evaluation protocol should report variance and significance.","section":"§4.2, Table 2; §4.1 Metrics"},{"comment":"The qualitative example in Figure 4 is presented as evidence of evidence-based reasoning, but the reasoning text closely follows the style and content of the GPT-4o-generated Chain-of-Evidence targets, which were produced from the answer labels. The limitation section acknowledges data scarcity and missing modalities but does not mention the circular construction of the reasoning targets or the zero-shot baseline comparison. Given that these are the two central limitations of the empirical study, they should be disclosed prominently rather than omitted.","section":"§4.3, Figure 4; Appendix E"}],"minor_comments":[{"comment":"The conversion of ECG signals to images and LAB tabular data to text for the baseline MLLMs is described only briefly; the exact input format, resolution, and prompt templates used for baselines should be specified to ensure comparability and reproducibility.","section":"§4.1 Training Details"},{"comment":"The disease distribution bar chart lists numbers (e.g., 3893, 3081, 6456, 3298, etc.) that do not match the ICD-10 subtype counts reported in Table 5; please reconcile these statistics.","section":"Figure 2(b)"},{"comment":"There is a typo: 'MIMIX-IV-CXR-report' should be 'MIMIC-IV-CXR-report'.","section":"Appendix B, Figure 5"},{"comment":"The CXR-QA example appears to be generated from a text report rather than from the image; clarify whether the physiological-level CXR QA pairs are based on image features, report text, or both.","section":"Appendix C, CXR-QA Example"},{"comment":"The claim that MedTVT-QA is 'the first medical instruction dataset' with these three modalities is strong; please contrast with existing multimodal medical instruction datasets to justify this claim.","section":"Introduction"}],"recommendation":"reject","confidential_remarks":"The two load-bearing issues—the answer-conditioned Chain of Evidence construction and the zero-shot baseline comparison—are fundamental to the manuscript's central claims. The first cannot be fixed by a local edit because the dataset itself is built on a circular target; the second would require re-running the entire evaluation with fine-tuned baselines. The dataset release and MPL architecture may still be of interest to the community, but the current manuscript does not support its stated conclusions. I would encourage the authors to address these issues and resubmit as a revised study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. First, the MedTVT-QA dataset is a real contribution: 8,706 patient-level combinations of ECG, CXR, and lab values with physiological and disease-level QA pairs, built from MIMIC-IV and filtered for evidence across modalities. That is not nothing. Second, the central empirical claim – that MedTVT-R1 outperforms leading MLLMs on multi-disease diagnosis – is not backed by the experiments as reported.\n\nThe paper does several things well. The dataset construction is careful: they check temporal alignment (ECG/labs within 24h, CXR within 24–72h), group lab values into seven physiological categories, map ICD-10 codes to seven disease families, and have professionals review the generated text. The architecture is also a reasonable integration of known pieces: modality encoders, a cyclic attention layer for cross-modal fusion, a contribution-aware weighting operator, and GRPO with a Jaccard reward for multi-label set overlap. The ablations show each piece helps, and the modality-drop experiments indicate all three modalities contribute.\n\nNow the soft spots, in proportion. The biggest is the comparison in Tables 1 and 2. MedTVT-R1 is pre-trained, SFT'd, and RFT'd on MedTVT-QA; the eight baselines are evaluated zero-shot, merely shown the ECG image, CXR image, and lab text. They were never fine-tuned on the task, never given the Chain-of-Evidence format, never trained with the Jaccard reward. So the F1 and AUC gaps (0.5190 vs 0.1995 best baseline) may reflect task tuning, not architectural superiority. The ablation rows are also all fine-tuned on the task, so they do not fix the confound. A fair comparison would fine-tune at least the strongest baseline on the same data.\n\nSecond, the reasoning evaluation has a circularity problem. The Disease-QA prompt gives GPT-4o the ground-truth disease list and tells it to 'prove the reasons why the patient has the diseases I provided' and 'must not mention any diseases other than those I provided.' The model is then trained to reproduce that rationalization, and the evaluation measures agreement with the same labels. So the reported reasoning quality is partly a measure of how well the model imitates a post-hoc justification, not how well it independently infers diseases from the data. The authors do not acknowledge this limitation in Appendix E.\n\nThird, the test set is 375 samples with no error bars, and the physiological-level results in Table 2 are likewise zero-shot comparisons. The qualitative example in Figure 6 shows the baselines struggling, but that is illustrative, not evidence of superiority.\n\nIs this a serious paper? Yes, for the dataset and the honest engineering. The limitations are real but the core data resource could be useful to the community. Worth a serious peer review, but the authors should be pushed to add fine-tuned baselines and external clinical evaluation before acceptance.","headline":"A genuinely new three-modality medical QA dataset and a sensible integration pipeline, but the headline performance claim is not supported because every baseline is zero-shot while the proposed model is trained on the task.","tokens_in":23918,"tokens_out":1020,"would_cite":false,"duration_ms":12525,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a 1-billion-parameter model trained to fuse ECG, chest X-ray, and laboratory data can reason through and diagnose seven diseases jointly, beating larger multimodal models on accuracy and language quality.","keywords":["multimodal large language model","multi-disease diagnosis","chain of evidence","GRPO reinforcement fine-tuning","Jaccard reward","ECG time series","chest X-ray imaging","laboratory tabular data"],"falsifier":"Regenerate the disease-level training data with the identical prompts but without the {result_diseases} field, so that GPT-4o must infer each patient's diseases from the three modality reports on its own; then train the identical pipeline on that data and measure diagnostic F1 on the same 375-case test split. If the score collapses toward the zero-shot baselines, the Chain of Evidence was leaking the answer labels and the model learned post-hoc justification rather than evidence-based inference. A supplementary check: strip the <think> reasoning block from the model's outputs at inference; a large drop in answer-set accuracy would indicate the reasoning text itself is doing label-carrying work.","tokens_in":22967,"feed_emoji":"🩺","tokens_out":14078,"duration_ms":117373,"temperature":0.7,"pith_summary":"The paper claims that a 1-billion-parameter multimodal language model can combine three heterogeneous clinical data streams—electrocardiogram (ECG) time series, chest X-ray (CXR) images, and laboratory test tables—to reason through and jointly diagnose seven diseases, and that it beats larger general-purpose multimodal models on both language quality and clinical accuracy. The claim rests on three ingredients: a new instruction dataset, MedTVT-QA, whose answers are physiological explanations plus a GPT-4o-written 'Chain of Evidence' citing corroborating findings from each modality for every disease in the patient's set; a modality perception layer that cross-attends the three streams and adaptively weights each modality's contribution; and reinforcement fine-tuning in which a Jaccard reward scores the overlap between predicted and true disease sets. On a 375-case test split the model reports 0.519 F1 and 0.655 AUC, about 2.6 times the F1 of the best zero-shot baseline. If the numbers hold, interpretable comorbidity reasoning does not require a large model: a small one can be taught to triangulate evidence across time series, images, and tables.","feed_headline":"1B-parameter model diagnoses seven diseases from three data types","feed_subtitle":"Combining ECG, chest X-ray, and lab data, the small model reaches 0.52 F1 where rivals stall near 0.20.","key_machinery":"Three named components carry the argument. First, MedTVT-QA, an instruction dataset of 8,706 multimodal patient data combinations whose GPT-4o-written answers come in two layers: per-modality physiological explanations and a disease-level Chain of Evidence, a written justification that must cite corroborating findings from ECG, CXR, and lab data for each disease in a given set. Second, the Modality Perception Layer (MPL), a fusion block with two parts: Cyclic Multi-Head Attention, in which each modality's features cyclically take turns as query, key, and value so the three streams exchange information, and a Contribution-Aware Operator, a sigmoid gate that learns to weight each modality's contribution per diagnostic question (ECG more heavily for coronary artery disease, for instance). Third, the Jaccard Reward, the intersection-over-union of the predicted and ground-truth disease sets, which is used inside GRPO to push the model toward complete and precise sets of diseases rather than a single label.","core_discovery":"MedTVT-R1 is presented as the first MLLM framework that integrates ECG, CXR, and laboratory data for interpretable multi-disease diagnosis rather than single-modality, single-disease classification. The authors build MedTVT-QA from MIMIC-IV records that keep clinical temporal order (ECG and lab tests from the first 24 hours of admission, chest X-rays from 24 to 72 hours), and they ask GPT-4o to expand physiological labels into long-form explanations and then to write a Chain of Evidence that justifies the patient's ICD-derived disease set using mutually corroborating findings from all three modalities. The model encodes each modality separately, fuses the streams through the modality perception layer, and is post-trained with GRPO, whose Jaccard reward measures intersection-over-union between predicted and ground-truth disease sets. In experiments against eight open multimodal models ranging from 1B to 8B parameters, MedTVT-R1 reports the highest scores on both language-quality metrics (BLEU, METEOR, ROUGE, BERTScore) and clinical metrics (precision, recall, F1, AUC), and it also leads on physiological-level description for each single modality. The ablation results are read by the authors as showing that physiological pre-training, the perception layer, all three modalities, and the reinforcement stage each contribute to the margin.","pith_inferences":["Because the Chain of Evidence was written by GPT-4o after being handed the true disease list and told to prove exactly those diseases and no others, the dataset teaches retroductive justification rather than open-ended inference; a direct test is to regenerate the training data without revealing the disease labels and check whether the model's diagnostic F1 survives.","A clinically motivated variant of the Jaccard reward could weight severe conditions (for example, sepsis) more heavily than mild ones, changing the model's error profile toward triage priorities.","The Contribution-Aware Operator's learned weights could double as an explainability artifact—showing a clinician which modality drove a diagnosis—and a natural study is whether presenting those weights improves clinician trust or helps catch model errors.","The paper's limitation section flags missing modalities; medical history is the most natural addition, since the ICD-based ground-truth labels themselves encode information a clinician would have drawn from history the model never sees."],"forward_implications":["A 1B-parameter model with LoRA can produce structured, interpretable, multi-disease diagnostic text that beats much larger zero-shot multimodal models on this task, suggesting that model scale is not the binding constraint for this kind of clinical reasoning.","The Jaccard reward generalizes to any multi-label diagnosis setting: any task whose answer is a set of conditions can be scored by set overlap and optimized with GRPO.","The modality perception layer yields a per-case weighting of ECG, CXR, and lab contributions, so the model can in principle indicate which data source most strongly supports each diagnosis.","The paper's ablations imply that removing any single modality degrades performance, with ECG removal hurting most, so the three streams carry complementary rather than redundant evidence.","The three-stage recipe—physiological pre-training, disease-level supervised fine-tuning, then GRPO—transfers to other multimodal clinical inputs, such as alternative imaging modalities or additional time series."],"supporting_citations":[{"why":"GPT-4o is the engine that writes both the physiological-level reports and the disease-level Chain of Evidence answers, so the entire dataset's content rests on it.","marker":"[2]"},{"why":"DeepSeek-R1 supplies the GRPO method and the verifiable-reward training recipe that the paper adapts with its Jaccard reward.","marker":"[19]"},{"why":"Symile provides the patient-level alignment used to collect the temporally consistent ECG, CXR, and lab combinations, and it also serves as the lab-data encoder.","marker":"[49]"},{"why":"MIMIC-IV-ECG is the source of the electrocardiogram signals.","marker":"[16]"},{"why":"MIMIC-IV is the source of the laboratory test results and hospital admission records.","marker":"[25]"},{"why":"MIMIC-CXR-JPG is the source of the chest X-ray images.","marker":"[26]"},{"why":"MIMIC-IV-ECG-EXT-ICD supplies the ICD-10 discharge diagnoses that define the seven disease categories and the ground-truth disease sets.","marker":"[54]"},{"why":"LLaMA 3.2 provides the 1B-parameter language backbone that is fine-tuned with LoRA.","marker":"[17]"},{"why":"ECGFM-KED supplies the pre-trained ECG encoder used for feature extraction.","marker":"[58]"}],"fun_headline_variants":["Multimodal LLM fuses ECG, X-ray, labs for multi-disease diagnosis","1B-parameter model diagnoses multiple diseases from three data types","MedTVT-R1: Integrating ECG, CXR, and labs for multi-disease reasoning","Small model beats larger rivals by fusing three medical modalities","Chain-of-evidence reasoning across ECG, X-ray, and lab data for diagnosis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a Chain of Evidence written by GPT-4o after being given the correct disease list is genuine diagnostic reasoning, so that training on it teaches the model to reason from the data rather than to restate labels it was told to justify.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal LLM fuses ECG, X-ray, labs for multi-disease diagnosis","1B-parameter model diagnoses multiple diseases from three data types","MedTVT-R1: Integrating ECG, CXR, and labs for multi-disease reasoning","Small model beats larger rivals by fusing three medical modalities","Chain-of-evidence reasoning across ECG, X-ray, and lab data for diagnosis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000648,"raw_usage":{"total_tokens":3017,"prompt_tokens":1029,"completion_tokens":1988,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":1886}},"tokens_in":645,"tokens_out":1988,"duration_ms":14336,"temperature":1.0,"reasoning_tokens":1886,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:47:23.177539+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Regenerate the disease-level training data with the identical prompts but without the {result_diseases} field, so that GPT-4o must infer each patient's diseases from the three modality reports on its own; then train the identical pipeline on that data and measure diagnostic F1 on the same 375-case test split. If the score collapses toward the zero-shot baselines, the Chain of Evidence was leaking the answer labels and the model learned post-hoc justification rather than evidence-based inference. A supplementary check: strip the <think> reasoning block from the model's outputs at inference; a large drop in answer-set accuracy would indicate the reasoning text itself is doing label-carrying work.","supporting_citations":[{"cited_title":"Saporta, A","cited_arxiv_id":null,"evidence_quote":"Symile provides the patient-level alignment used to collect the temporally consistent ECG, CXR, and lab combinations, and it also serves as the lab-data encoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MIMIC-IV-ECG is the source of the electrocardiogram signals."},{"cited_title":"Johnson, L","cited_arxiv_id":null,"evidence_quote":"MIMIC-IV is the source of the laboratory test results and hospital admission records."},{"cited_title":"Strodthoff, J","cited_arxiv_id":null,"evidence_quote":"MIMIC-IV-ECG-EXT-ICD supplies the ICD-10 discharge diagnoses that define the seven disease categories and the ground-truth disease sets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ECGFM-KED supplies the pre-trained ECG encoder used for feature extraction."}],"review_version":2}