REVIEW 4 major objections 4 minor 1 cited by
TMI-CLNet: Triple-Modal Interaction Network for Chronic Liver Disease Prognosis From Imaging, Clinical, and Radiomic Data Fusion
T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read The paper claims a triple-modal network fusing CT images, radiomic features, and clinical data predicts chronic liver disease prognosis with 83.12% accuracy, outperforming existing unimodal and multimodal baselines.
desk verdict A sensible triple-modal fusion architecture for liver prognosis whose reported gains are likely inflated by tuning a loss weight on the same cross-validation folds; the method deserves a look, but the evaluation needs to be redone. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is a three-part interaction loop. The Intra-Modality Aggregation (IMA) module is a 16-head multi-head self-attention block that consolidates each modality's own features before fusion. The Triple-Modal Cross-Attention Fusion (TCAF) module computes, for each modality $i$, cross-attention against the adjacent modality $j = i \bmod 3 + 1$ through $F^i_{\text{hidden}} = \operatorname{softmax}(Q^i K^{j\top}/\sqrt{d_k}) V^j$, concatenates the attended output with the original modality features, and concatenates the three results into a global representation. The Triple-Modal Feature Fusion (TMFF) loss is a weighted sum of pairwise Similarity Distribution Matching losses, $L_{\text{multi}} = \lambda (L^{it}_{\text{SDM}} + L^{rt}_{\text{SDM}})/2 + (1-\lambda) L^{ir}_{\text{SDM}}$, added to the cross-entropy task loss. The claim is that these three components together remove intra-modality redundancy, capture cross-modal relationships, and align the three feature spaces so that a 1D DenseNet-121 classification head can predict good versus bad prognosis.
What would settle it
Re-run the released code on the same 184 patients but with the $\lambda$ values chosen on a held-out validation fold or an external cohort; if TMI-CLNet no longer beats HFBSurv by roughly 3.67 percentage points in accuracy and 0.0208 in AUC, the paper's central claim would be refuted.
Extended reading notes
Core claim
The paper's central claim is that TMI-CLNet, a triple-modal network for CT imaging, radiomic features, and clinical information, achieves the best prognosis classification on its liver disease dataset, and that each of its three interaction components contributes to the gain. The Intra-Modality Aggregation module uses 16-head self-attention to reduce redundancy inside each modality; the Triple-Modal Cross-Attention Fusion module computes, for each modality, attention over the adjacent modality's key-value pairs in a cycle and concatenates the attended features with the original modality features; and the Triple-Modal Feature Fusion loss aligns image-text, radiomics-text, and image-radiomics pairwise similarity distributions. In the head-to-head comparison, TMI-CLNet reaches 83.12% accuracy, 84.38% precision, 74.28% recall, 0.7805 F1, and 0.8223 AUC, exceeding every unimodal and multimodal baseline on all metrics. The ablation results attribute the largest gains to using IMA and TCAF together (accuracy increase of 7.64 percentage points) and to the TMFF loss with weight $\lambda = 0.6$ (accuracy increase of 6.45 percentage points and AUC increase of 0.0687 over no TMFF loss).
Load-bearing premise
The whole reported advantage rests on the assumption that 5-fold cross-validation on a single private 184-patient cohort, with the TMFF weight $\lambda$ chosen by sweeping on those same folds, gives an unbiased estimate of the model's true predictive advantage.
Editorial extensions
If this is right
- If TMI-CLNet's reported gains are real, fusing CT, radiomics, and clinical data is better than any single modality for this prognosis task, with recall improving from 64.32% for the best baseline to 74.28%.
- The ablation results imply that neither cross-attention nor self-attention alone is enough: adding both IMA and TCAF raises accuracy by 7.64 percentage points over the no-module baseline.
- The TMFF loss contributes independently: setting $\lambda = 0.6$ raises accuracy by 6.45 percentage points and AUC by 0.0687 over omitting the loss.
- Because the method outperforms HFBSurv and MMD, which were designed for heterogeneous and incomplete multimodal medical data, the architecture is a plausible template for other three-modality medical prediction tasks.
Reading between the lines
- Editorial inference: the core design cycles one step to the next modality rather than computing all pairwise attention maps, reducing the number of cross-modal interactions from six to three; on tasks with more modalities this linear cycle may scale better, but whether it loses information versus full pairwise attention is untested here.
- Editorial inference: the reported improvements should be re-checked with nested cross-validation or a separate test fold, because the $\lambda$ sweep and model selection happen on the same five folds; a reader can test this by re-running the released code with a fixed hold-out split.
- Editorial inference: the framework likely transfers to other prognosis tasks with the same data triad of imaging, quantitative features, and clinical text, such as other organ cancers, but the 184-patient single-center cohort makes the effect size uncertain until multi-center validation appears.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TMI-CLNet, a triple-modal network that fuses CT images, radiomic features, and clinical information for chronic liver disease prognosis. The architecture consists of an Intra-Modality Aggregation (IMA) module based on multi-head self-attention, a Triple-modal Cross-Attention Fusion (TCAF) module with cyclic one-direction cross-attention among the three modalities, and a Triple-Modal Feature Fusion (TMFF) loss adapted from the Similarity Distribution Matching loss. On a private 184-patient liver prognosis dataset, the method reports 83.12% accuracy, 0.7805 F1, and 0.8223 AUC, outperforming several unimodal and multimodal baselines. The paper includes ablations on modality configuration, the IMA/TCAF modules, and the TMFF loss weight lambda, plus Grad-CAM visualizations.
Significance. If the reported gains are reliable, the combination of within-modality aggregation, cyclic cross-attention, and pairwise distribution-matching alignment would be a useful template for triple-modal medical prognosis. The paper provides code, runs all comparisons on the same protocol, and includes multiple ablation studies, which are strengths. The architecture itself is internally consistent and the loss derivation in Eq. (5) is a straightforward extension of the cited SDM loss, so there is no circular reasoning in the method. However, the central empirical claim of significant superiority is not currently established because the evaluation protocol selects the TMFF weight on the same folds that produce the final metrics, reports no error bars or statistical tests, and uses a single small private cohort. These issues are load-bearing for the paper's main contribution and need to be addressed before the claim can be accepted.
major comments (4)
- [Section 3.2, Table 4 and Eq. (5)] The TMFF loss weight lambda is tuned on the same 5-fold cross-validation used to report the final results. Table 4 sweeps lambda over {0.2, 0.4, 0.6, 0.8}, selects 0.6 because it gives the best accuracy and AUC, and then those selected-fold numbers are presented as the final performance. This is a selection-on-the-test-set protocol that can inflate the reported gains. Please provide a nested or held-out validation procedure, or clearly separate model selection from evaluation, and report per-fold results with confidence intervals. This is essential for the headline claim of 83.12% accuracy and 0.8223 AUC.
- [Section 3.2, Table 1 and abstract] The statement that TMI-CLNet 'significantly outperforms existing state-of-the-art' methods is not supported by any statistical analysis. The dataset has only 184 patients, the reported improvements over HFBSurv are 3.67 percentage points in accuracy and 0.0208 in AUC, and no standard errors, confidence intervals, or significance tests are provided. With five folds and this sample size, the AUC difference is within the range of plausible sampling noise. Please add statistical testing (e.g., DeLong test for AUC differences, McNemar for accuracy), per-fold results, and error bars, or soften the significance claim accordingly.
- [Section 3.2, Tables 3 and 4] The ablation baselines are inconsistent between the two tables. Table 3 reports that removing both IMA and TCAF yields ACC 75.48%, whereas Table 4 reports that removing TMFF loss yields ACC 76.61%. If both rows correspond to the same configuration (full IMA+TCAF without the TMFF loss), the discrepancy of 1.13 percentage points needs an explanation. If the configurations differ in another way, that difference should be stated explicitly. As written, the component-wise gains attributed to the modules versus the loss are not cleanly separable.
- [Section 3.1 and Section 4] External validity is a central concern. All conclusions rest on a single private 184-patient cohort with 5-fold cross-validation and no held-out test set or multi-center validation. The authors themselves list multi-center validation as future work in Section 4, but the abstract and Section 3.2 state that the method 'significantly outperforms' alternatives without acknowledging this limitation. Please either add external validation or substantially temper the generalizability claims in the abstract and conclusion.
minor comments (4)
- [Abstract and Section 4] There are typographical and grammatical issues, including the capitalized 'We' after 'However' in the abstract and 'infomation' in the conclusion. A careful proofread is needed.
- [Section 3.1, Dataset description] The clinical information is described as 'clinical text' in the methodology, but the dataset description lists structured numerical data (gender, height, BMI, CBC, and fat analysis). Please clarify how this structured information is converted into the text modality and what the BioBERT input looks like.
- [Section 3.1, Implementation Details] The classification head is described as a 'pre-trained 1D DenseNet-121,' but no details are given about its input dimension, output dimension, or how the fused global feature vector is reshaped for this head. A short explanation or architecture diagram would improve reproducibility.
- [Section 2.3, Eq. (5)] The paper states that alpha in Eq. (6) is set to 1, but no ablation or sensitivity analysis for alpha is reported. Since the TMFF loss weight lambda is heavily analyzed, a brief note on alpha would complete the hyperparameter discussion.
Circularity Check
No significant circularity: the network is an empirical construction, the loss is a weighted SDM-style objective, and the only self-citations are background references that do not carry the argument.
full rationale
The paper does not present a formal derivation chain from first principles to a predicted result; it is an empirical architecture paper. The loss function in Eqs. (3)-(6) is a weighted composition of the externally defined SDM loss [7] and cross-entropy, and no equation is defined in terms of the reported accuracy or AUC. The TCAF module (Eqs. (1)-(2)) is standard cross-attention with concatenation, not a self-referential construction. The only self-citations are refs. [2] and [4], which are used as background examples of multimodal and cross-modal networks; they are not invoked to justify the IMA, TCAF, or TMFF designs, nor to forbid alternatives. The hyperparameter sweep for lambda in Table 4 is performed on the same five CV folds used to report the final metrics, which is a legitimate evaluation-protocol risk and could inflate the headline numbers, but it is not circular in the derivation sense: the reported 83.12% accuracy is an empirical training outcome, not an algebraic consequence of the chosen lambda value. Thus, while the evaluation may be optimistic, the architecture and loss are not circularly defined, and the central empirical claim does not reduce to a fitted parameter or a self-citation chain.
Assumptions & free parameters
free parameters (2)
- lambda (TMFF loss weight) =
0.6 (best in ablation sweep)
- alpha (overall loss weight) =
1 (typical value)
assumptions (4)
- domain assumption Pretrained 3D ResNet-50 and BioBERT provide useful, transferable features for chronic liver disease CT and clinical text.
- domain assumption The private 184-patient single-center dataset with 109 good and 75 bad prognoses is representative enough for a 5-fold cross-validation comparison.
- ad hoc to paper The cyclic one-direction cross-attention in Eq. (1), where each modality attends to the next in a ring, captures the necessary inter-modal relationships.
- domain assumption The SDM loss formulation transfers from text-image retrieval to the three medical modalities.
Cite this review
Pith. "Pith review of TMI-CLNet: Triple-Modal Interaction Network for Chronic Liver Disease Prognosis From Imaging, Clinical, and Radiomic Data Fusion." pith.science (2026). https://pith.science/paper/N5QVW75Z
@misc{pith2026250200695,
author = {Pith},
title = {Pith review of: TMI-CLNet: Triple-Modal Interaction Network for Chronic Liver Disease Prognosis From Imaging, Clinical, and Radiomic Data Fusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/N5QVW75Z}},
note = {Machine review of arXiv:2502.00695}
}
read the original abstract
Chronic liver disease represents a significant health challenge worldwide and accurate prognostic evaluations are essential for personalized treatment plans. Recent evidence suggests that integrating multimodal data, such as computed tomography imaging, radiomic features, and clinical information, can provide more comprehensive prognostic information. However, modalities have an inherent heterogeneity, and incorporating additional modalities may exacerbate the challenges of heterogeneous data fusion. Moreover, existing multimodal fusion methods often struggle to adapt to richer medical modalities, making it difficult to capture inter-modal relationships. To overcome these limitations, We present the Triple-Modal Interaction Chronic Liver Network (TMI-CLNet). Specifically, we develop an Intra-Modality Aggregation module and a Triple-Modal Cross-Attention Fusion module, which are designed to eliminate intra-modality redundancy and extract cross-modal information, respectively. Furthermore, we design a Triple-Modal Feature Fusion loss function to align feature representations across modalities. Extensive experiments on the liver prognosis dataset demonstrate that our approach significantly outperforms existing state-of-the-art unimodal models and other multi-modal techniques. Our code is available at https://github.com/Mysterwll/liver.git.
Forward citations
Cited by 1 Pith paper
-
A Consensus Algorithm for Second-Order Systems Evolving on Lie Groups
The paper claims a configuration-only consensus algorithm for second-order systems on Lie groups, but the attached manuscript text is an unrelated medical-imaging paper, leaving the claim unverifiable.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION Chronic liver disease poses a significant threat to human health and safety. Chronic hepatitis caused by viral in- fections of hepatitis B virus (HBV) and hepatitis C virus (HCV) may progress to cirrhosis and hepatocellular carci- noma (HCC) [1]. Therefore, providing clinicians with timely and accurate prognostic results to guide early interv...
-
[2]
METHODOLOGY The overall architecture of our network, as illustrated in Fig- ure 1, consists of three main components: a feature extraction module, a multi-modal interaction module, and a classifica- arXiv:2502.00695v1 [cs.CV] 2 Feb 2025 Fig. 1: Overview of our proposed TMI-CLNet. Given the CT scans, radiomic features, and clinical text, we feed them into ...
work page Pith review arXiv 2025
-
[3]
EXPERIMENTS 3.1. Experimental Setup Dataset. Our study utilized a private liver prognosis dataset provided by a partner hospital, comprising 184 patients with Table 1 : Comparison with other methods using different modality configurations. Method Modality ACC (%) Precision (%) Recall (%) F1 Score AUC Resnet-50 [8] Image 76.67 80.69 58.36 0.6718 0.7354 ViT...
-
[4]
CONCLUSION This study presented TMI-CLNet, which integrates CT imag- ing, radiomic features, and clinical infomation to provide early prognosis assessments for patients with chronic liver disease. By introducing the TCAF module and the TMFF loss function, the proposed model can address the heterogeneity among different modalities, thus achieving remarkabl...
-
[5]
COMPLIANCE WITH ETHICAL STANDARDS This study was conducted in accordance with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Longgang Central Hospital of Shenzhen (2024.5.8/No.2024052)
work page 2024
-
[6]
ACKNOWLEDGMENTS This work was supported by the Open Project Program of the State Key Laboratory of CAD&CG (No.A2410), Zhejiang University, Zhejiang Provincial Natural Science Foundation of China (No.LY21F020017), National Natu- ral Science Foundation of China (No.61702146, 62076084), GuangDong Basic and Applied Basic Research Foundation (No.2022A151511057...
-
[7]
An update on viral hepatitis b and c in mexico: Advances and pitfalls in eradication strategies,
Marina Campos-Valdez, Manuel Alejandro Castro- Garc´ıa, Martha Elo ´ısa Ramos-M ´arquez, Carmen Mag- dalena Gurrola-D ´ıaz, Adriana Mar ´ıa Salazar-Montes, and Laura Ver ´onica S ´anchez-Orozco, “An update on viral hepatitis b and c in mexico: Advances and pitfalls in eradication strategies,” Microorganisms, vol. 12, no. 7, pp. 1368, 2024
work page 2024
-
[8]
Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Qing Wu, Xiang Wan, Lihua Li, and Changmiao Wang, “Ich-prnet: A cross- modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism,” Neural Networks, p. 107096, 2025
work page 2025
Show all 21 references
-
[9]
A multi-modal deep neural network for multi-class liver cancer diagnosis,
Rayyan Azam Khan, Minghan Fu, Brent Burbridge, Yi- gang Luo, and Fang-Xiang Wu, “A multi-modal deep neural network for multi-class liver cancer diagnosis,” Neural Networks, vol. 165, pp. 553–561, 2023
2023
-
[10]
Ich- pro: Intracerebral hemorrhage prognosis classification via joint-attention fusion-based 3d cross-modal net- work,
Xinlei Yu, Xinyang Li, Ruiquan Ge, Shibin Wu, Ahmed Elazab, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Taosheng Xu, Xiang Wan, and Changmiao Wang, “Ich- pro: Intracerebral hemorrhage prognosis classification via joint-attention fusion-based 3d cross-modal net- work,” in 2024 IEE...
2024
-
[11]
Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?,
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh, “Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2018, pp. 6546–6555
2018
-
[12]
Biobert: a pre-trained biomedical language representa- tion model for biomedical text mining,
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang, “Biobert: a pre-trained biomedical language representa- tion model for biomedical text mining,” Bioinformatics, vol. 36, pp. 1234 – 1240, 2019
2019
-
[13]
Cross-modal implicit rela- tion reasoning and aligning for text-to-image person re- trieval,
Ding Jiang and Mang Ye, “Cross-modal implicit rela- tion reasoning and aligning for text-to-image person re- trieval,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2787–2797
2023
-
[14]
Deep residual learning for image recognition,
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778
2016
-
[15]
An image is worth 16x16 words: Transformers for image recognition at scale,
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint ar...
2010 arXiv
-
[16]
Radiomics-based machine learning models can detect pancreatic cancer on prediagnostic cts at a substantial lead time prior to clinical diagnosis.,
Sovanlal Mukherjee, Anurima Patra, Hala Khasawneh, Panagiotis Korfiatis, Naveen Rajamohan, Garima Suman, Shounak Majumder, Ananya Panda, Matthew P. Johnson, N.B. Larson, Darryl E. Wright, Timothy L. Kline, Joel G. Fletcher, Suresh T. Chari, and Ajit H. Goenka, “Radiomics-based...
2022
-
[17]
Random forests,
L. Breiman, “Random forests,” Machine Learning, vol. 45, pp. 5–32, 2001
2001
-
[18]
Fully automated hybrid approach to predict the idh mutation status of gliomas via deep learning and ra- diomics,
Yoon Seong Choi, Sohi Bae, Jong Hee Chang, Seok- Gu Kang, Se Hoon Kim, Jinna Kim, Tyler Hyungtaek Rim, Seung Hong Choi, Rajan Jain, and Seung-Koo Lee, “Fully automated hybrid approach to predict the idh mutation status of gliomas via deep learning and ra- diomics,” Neuro-oncol...
2021
-
[19]
Hf- bsurv: hierarchical multimodal fusion with factorized bilinear models for cancer survival prediction,
Ruiqing Li, Xingqi Wu, Ao Li, and Minghui Wang, “Hf- bsurv: hierarchical multimodal fusion with factorized bilinear models for cancer survival prediction,” Bioin- formatics, vol. 38, no. 9, pp. 2587–2594, 2022
2022
-
[20]
Survival prediction of brain cancer with incomplete radiology, pathology, genomic, and demographic data,
Can Cui, Han Liu, Quan Liu, Ruining Deng, Zuhayr Asad, Yaohong Wang, Shilin Zhao, Haichun Yang, Ben- nett A Landman, and Yuankai Huo, “Survival prediction of brain cancer with incomplete radiology, pathology, genomic, and demographic data,” in International Con- ference on Med...
2022
-
[21]
Grad-cam: Visual explanations from deep net- works via gradient-based localization,
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra, “Grad-cam: Visual explanations from deep net- works via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vi- sion, 2017, pp...
2017
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.