REVIEW 3 major objections 6 minor 59 references
ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A new dataset fuses ultrasound images, exam tables, and report text to classify six ovarian tumor types, reaching 85% accuracy.
desk verdict Useful new multimodal ovarian ultrasound dataset, but the benchmark numbers are only as good as the undisclosed train/test split; needs major revision before the 85% claim can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Triplet Hierarchical Offset Attention Mechanism (THOAM), a two-stage cross-attention fusion module. Image and table features are fused first through scaled dot-product attention, the result is attended once more with text features, and the final output is concatenated with the original visual feature map and passed to a linear classifier. This is the component the paper credits with reducing fusion loss and improving relevance and complementarity among the three modalities.
What would settle it
Train the same ViTaL-Net under a patient-disjoint split—grouping all slices of each patient together and assigning whole patients to train or test—and compare accuracy and AUC to the reported 85.59% and 0.95. A large drop would show that the reported multimodal gain depends on seeing slices of the same tumor in both training and testing.
Extended reading notes
Core claim
The central claim is that a six-way ovarian tumor classification task can be performed from three complementary clinical modalities—2D ultrasound images, structured examination values, and free-text ultrasound reports—and that an attention-based hierarchical fusion of these modalities outperforms image-only and simple-concatenation baselines. The authors report that ViTaL-Net reaches 85.59% overall accuracy and a 0.95 mean AUC across the six pathology classes, with per-class accuracy above 90% for the two most common types. They also state that this is the first dataset of its kind to combine visual, tabular, and linguistic modalities specifically for multi-pathology ovarian tumor recognition.
Load-bearing premise
The load-bearing assumption is that the train/test split keeps all ultrasound slices of a single patient on one side; the paper never states this split protocol, and if patients are split across train and test, the model could memorize patient-specific slices and the reported accuracy would be optimistic.
Editorial extensions
If this is right
- Adding tabular and linguistic data to ultrasound images is the main driver of the reported gain: accuracy rises from 74.54% with visual data only to 85.59% with all three modalities.
- The multi-pathology setting is feasible: the model distinguishes six tumor types, not just benign versus malignant, with class-specific AUC values mostly at or above 0.9.
- The heterogeneous 'Other' category currently degrades performance, so the paper excludes it from the benchmark and identifies handling it as future work.
- Attention-based fusion beats direct concatenation on the same three modalities (85.59% vs 75.88% accuracy), supporting the claim that THOAM reduces information loss during fusion.
Reading between the lines
- The dataset split protocol is not described; if ultrasound slices from the same patient appear in both training and test sets, the reported 85.59% accuracy and 0.95 AUC are likely optimistic relative to a patient-disjoint evaluation.
- Because all images come from one scanner at one hospital, a natural test is whether ViTaL-Net's gains persist on data from other institutions or devices; the tabular and text modalities may partly encode institution-specific reporting style.
- The tabular modality's contribution could be isolated further by ablating individual markers such as CA-125, which would reveal whether the gain comes from broad tabular structure or from a few high-signal tumor markers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ViTaL, an ovarian-tumor dataset of 496 patients containing 2,216 2D ultrasound images, tabular examination data, and ultrasound report text, labeled with six pathological categories plus an 'Other' category. It also proposes ViTaL-Net, a multimodal classifier using MobileNet, TabNet, and BERT backbones fused through a Triplet Hierarchical Offset Attention Mechanism (THOAM). The authors report 85.59% accuracy and 0.95 AUC on six-way classification, claim a 14.5% improvement over prior methods, and provide ablations across modality combinations and fusion strategies. The dataset and code are promised publicly.
Significance. If the evaluation is valid, ViTaL is a valuable resource: it is among very few ovarian ultrasound datasets that pair images with per-patient tabular and textual data, it releases code, and it is ethically approved. The modality ablation in Table 5 and the attention-vs-concatenation comparison in Table 6 are useful checks of the proposed fusion. However, the central benchmark claim is not yet supported because the train/test split protocol is unspecified, the six-class benchmark is defined by post-hoc exclusion of the 'Other' category, and several reported numbers are inconsistent across tables. The dataset contribution itself is plausible, but the performance claims need substantially stronger evaluation before the benchmark can be adopted.
major comments (3)
- [Sections 5.1, 5.2, 3.1] The paper never specifies how the 2,216 ultrasound images from 496 patients are partitioned into training and test sets. This is load-bearing because Section 3.1 states that multiple scans with different viewing angles are selected per patient; if the split is at image level rather than patient level, slices from the same tumor can appear in both training and test, which can inflate the reported 85.59% accuracy and 0.95 AUC. Please state the split unit (patient or image), the number of patients/images in each split, the random seed or cross-validation scheme, and whether validation data were used for model selection.
- [Tables 3, 5, 6 and Section 5.3] The reported accuracy is internally inconsistent: Table 3 lists THOAM accuracy as 83.08%, while Tables 5 and 6 report 85.59% for the same attention-fusion configuration. Additionally, Section 5.3 describes Table 3 as image-unimodal, yet the table includes THOAM and other multimodal fusion methods. Please clarify which experiments are unimodal, which are multimodal, how the claimed 14.5% improvement over the best competitor is computed, and whether all comparison methods used the same modality set and the same data split.
- [Section 6.2 and Table 6] The benchmark excludes the 'Other' category after observing that including it lowers accuracy to 70.39%, so the six-class task and the headline 85.59% figure are defined post hoc rather than prospectively. Because the dataset actually contains seven categories, the paper should either justify this exclusion with a pre-specified criterion, report results with and without 'Other' including per-class metrics, or redefine the task to cover all collected categories. The current presentation risks overstating the model's practical diagnostic performance.
minor comments (6)
- [Section 3.1] The sentence 'we have collected 30 cases from patients aged 6 to 23' should clarify whether these are cases or patients, and how this subset is flagged in the released data.
- [Equations (4), (7)] There are typos in the text surrounding these equations: 'Sof tmax' should be 'Softmax', 'Concact' should be 'Concatenate', and 's a scaling factor' should be 'a scaling factor'.
- [Table 4] The SEN and SPE columns run together without clear spacing, making the table difficult to read; formatting should be cleaned.
- [Section 5.2.1] The software name 'MMpretrain' should be spelled consistently (e.g., MMPretrain) and the reference/version details should be provided.
- [Figures 2, 3, 4] The captions refer to the 'horizontal axis of the table' where they mean the horizontal axis of the figure; please correct.
- [Dataset release] The paper should state the dataset license, the de-identification process, and any data-use restrictions to accompany the public release.
Circularity Check
No circular derivation found: the paper reports an empirical benchmark with training and evaluation on its own dataset; the main threats (split protocol, post-hoc Other exclusion) concern validity, not circularity.
full rationale
The paper's central claims are empirical: they construct ViTaL, train ViTaL-Net with THOAM, and report accuracy/AUC on the ViTaL test partition (Tables 3–6). No equation in Sections 4.3–4.4 defines a predicted quantity in terms of a fitted input: the cross-attention fusion (Eqs. 3–7) and the linear decoder (Eq. 8) are standard trainable modules, and the reported 85.59% accuracy is a measured evaluation result, not a fitted parameter renamed as a prediction. The normalization formulas (Eqs. 10–12) are preprocessing steps and do not encode the classification outcome. The only self-citation is to MMOTU [57] as a prior ovarian ultrasound dataset; it appears in a literature comparison (Table 1) and is not load-bearing for the proposed method or the benchmark numbers. The known weaknesses of the paper—no explicit patient-level train/test split (Section 5.1–5.2), the post-hoc exclusion of the 'Other' category (Section 6.2), and evaluation only on the authors' own dataset—are substantive threats to generalization and external validity, but they are not circularity in the sense of a derivation reducing to its inputs by construction. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Train/validation/test split assignment =
not reported
assumptions (3)
- domain assumption Each ultrasound image contains exactly one tumor type, and the pathology label for that image is correct.
- domain assumption Tumor marker values and ultrasound report text provide information complementary to the images for the six-way classification.
- domain assumption Multiple ultrasound slices from the same patient can be treated as independent samples during training and evaluation.
Cite this review
Pith. "Pith review of ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition." pith.science (2026). https://pith.science/paper/Z2M64WVE
@misc{pith2026250704383,
author = {Pith},
title = {Pith review of: ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/Z2M64WVE}},
note = {Machine review of arXiv:2507.04383}
}
read the original abstract
Ovarian tumor, as a common gynecological disease, can rapidly deteriorate into serious health crises when undetected early, thus posing significant threats to the health of women. Deep neural networks have the potential to identify ovarian tumors, thereby reducing mortality rates, but limited public datasets hinder its progress. To address this gap, we introduce a vital ovarian tumor pathological recognition dataset called \textbf{ViTaL} that contains \textbf{V}isual, \textbf{T}abular and \textbf{L}inguistic modality data of 496 patients across six pathological categories. The ViTaL dataset comprises three subsets corresponding to different patient data modalities: visual data from 2216 two-dimensional ultrasound images, tabular data from medical examinations of 496 patients, and linguistic data from ultrasound reports of 496 patients. It is insufficient to merely distinguish between benign and malignant ovarian tumors in clinical practice. To enable multi-pathology classification of ovarian tumor, we propose a ViTaL-Net based on the Triplet Hierarchical Offset Attention Mechanism (THOAM) to minimize the loss incurred during feature fusion of multi-modal data. This mechanism could effectively enhance the relevance and complementarity between information from different modalities. ViTaL-Net serves as a benchmark for the task of multi-pathology, multi-modality classification of ovarian tumors. In our comprehensive experiments, the proposed method exhibited satisfactory performance, achieving accuracies exceeding 90\% on the two most common pathological types of ovarian tumor and an overall performance of 85\%. Our dataset and code are available at https://github.com/GGbond-study/vitalnet.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
X. Yang, X. Xi, K. Wang, et al., Triple-attention interaction network for breast tumor classification based on multi-modality images, Pattern Recognition (PR) 139 (2023) 109526
work page 2023
-
[2]
D. Hao, M. Ahsan, T. Salim, A. Duarte-Rojo, D. Esmaeel, Y. Zhang, D. Arefan, S. Wu, A self-training teacher-student model with an auto- matic label grader for abdominal skeletal muscle segmentation, Artificial Intelligence in Medicine 132 (2022) 102366
work page 2022
- [3]
- [4]
-
[5]
K. Tan, W. Huang, X. Liu, J. Hu, S. Dong, A multi-modal fusion frame- work based on multi-task correlation learning for cancer prognosis pre- diction, Artificial Intelligence in Medicine 126 (2022) 102260
work page 2022
-
[6]
H. Chereda, A. Leha, T. Beißbarth, Stable feature selection utilizing graph convolutional neural network and layer-wise relevance propaga- tion for biomarker discovery in breast cancer, Artificial intelligence in medicine 151 (2024) 102840
work page 2024
- [7]
-
[8]
J. H. Kim, H.-W. Cho, E. Y. Park, et al., Prognostic value of ca125 kinetics, half-life, and nadir in the treatment of epithelial ovarian can- cer: A systematic review and meta-analysis, International Journal of Gynecological Cancer 33 (12) (2023) 1913–1920
work page 2023
Show all 59 references
-
[9]
A. A. Bidgoli, S. Rahnamayan, T. Dehkharghanian, A. Riasatian, S. Kalra, M. Zaveri, C. J. Campbell, A. Parwani, L. Pantanowitz, H. R. Tizhoosh, Evolutionary deep feature selection for compact representa- tion of gigapixel images in digital pathology, Artificial Intelligence in...
2022
-
[10]
X. Xu, H. Wang, Y. Lu, H. Zhang, T. Tan, F. Xu, J. Lei, Joint segmen- tation of retinal layers and fluid lesions in optical coherence tomography with cross-dataset learning, Artificial Intelligence in Medicine 162 (2025) 103096
2025
-
[11]
Idrisoglu, A
A. Idrisoglu, A. L. Dallora, A. Cheddad, P. Anderberg, A. Jakobsson, J. S. Berglund, Copdvd: Automated classification of chronic obstruc- tive pulmonary disease on a new collected and evaluated voice dataset, Artificial Intelligence in Medicine 156 (2024) 102953
2024
-
[12]
W. Hu, Z. Guan, N. Cheng, A. Zhang, Y. Liu, T. Wang, B. Lei, Multi- modal fusion network integrating imaging and clinical tabular data for alzheimer’s disease classification, in: 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), IEEE, 2025, pp. 1–4
2025
-
[13]
S. Woo, J. Park, J.-Y. Lee, I. S. Kweon, Cbam: Convolutional block attention module, in: Proceedings of the European conference on com- puter vision (ECCV), 2018, pp. 3–19
2018
-
[14]
S. Li, L. Lin, Y. Huang, P. Cheng, X. Tang, Text-guided foundation model adaptation for long-tailed medical image classification, in: 2024 IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, 2024, pp. 1–5
2024
-
[15]
M. Tan, Q. Le, Efficientnetv2: Smaller models and faster training, in: International conference on machine learning, PMLR, 2021, pp. 10096– 10106. 27
2021
-
[16]
Dosovitskiy, L
A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)
2020 arXiv
-
[17]
G. C. Ates, P. Mohan, E. Celik, Dual cross-attention for medical im- age segmentation, Engineering Applications of Artificial Intelligence 126 (2023) 107139
2023
-
[18]
Parvaiz, M
A. Parvaiz, M. A. Khalid, R. Zafar, H. Ameer, M. Ali, M. M. Fraz, Vision transformers in medical computer vision—a contemplative retro- spection, Engineering Applications of Artificial Intelligence 122 (2023) 106126
2023
-
[19]
X. Yu, H. Guo, Y. Yuan, W. Guo, X. Yang, H. Xu, Y. Kong, Y. Zhang, H. Zheng, S. Li, An improved medical image segmentation framework with channel-height-width-spatial attention module, Engineering Appli- cations of Artificial Intelligence 136 (2024) 108751
2024
-
[20]
Y. Li, M. E. H. Daho, P.-H. Conze, et al., A review of deep learning- based information fusion techniques for multimodal medical image clas- sification, Computers in Biology and Medicine (2024) 108635
2024
-
[21]
W. Fan, H. Peng, R. Li, et al., U-bilateral attention gate nested u- transformers for medical image segmentation, Engineering Applications of Artificial Intelligence 154 (2025) 110880
2025
-
[22]
J.-H. You, C. Lee, Comparative analysis of mechanical behavior pre- diction models for abdominal aortic wall, Engineering Applications of Artificial Intelligence 142 (2025) 109899
2025
-
[23]
Meidani, F
Z. Meidani, F. Atoof, Z. Mobarak, et al., Development of clinical- guideline-based mobile application and its effect on head ct scan uti- lization in neurology and neurosurgery departments, BMC Medical In- formatics and Decision Making 22 (1) (2022) 106
2022
-
[24]
X. Tang, Z. Cao, W. Zhang, et al., Follow sonographers’ visual scan- path: Adjusting cnn model for diagnosing gout from musculoskeletal ultrasound, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2024, pp. 611–620. 28
2024
-
[25]
Cao-Van, T
K. Cao-Van, T. C. Minh, H. M. Tan, et al., Prediction of heart failure using voting ensemble learning models and novel data normalization techniques, Engineering Applications of Artificial Intelligence 154 (2025) 110888
2025
-
[26]
Sivakumar, A
R. Sivakumar, A. Penkova, Enhancing glaucoma detection through multi-modal integration of retinal images and clinical biomarkers, Engi- neering Applications of Artificial Intelligence 143 (2025) 110010
2025
-
[27]
T. A. Fahim, F. B. Alam, K. T. Ahmmed, Ovanet: Dual attention mechanism based new deep learning framework for diagnosis and classi- fication of ovarian cancer subtypes from histopathological images, IEEE Access (2024)
2024
-
[28]
X. Hou, N. Innab, S. Alahmari, et al., Explainable deep learning model with the internet of medical devices for early lung abnormality detection, Engineering Applications of Artificial Intelligence 153 (2025) 110961
2025
-
[29]
K. Wang, F. Zheng, L. Cheng, H.-N. Dai, Q. Dou, J. Qin, Breast cancer classification from digital pathology images via connectivity-aware graph transformer, IEEE Transactions on Medical Imaging (2024)
2024
-
[30]
W. Tang, H. Deng, Z. Huang, et al., Medical image segmentation method based on full perceived dynamic network, Engineering Applications of Artificial Intelligence 142 (2025) 109867
2025
-
[31]
Q. Rong, W. Wu, Z. Lu, et al., Decision-level fusion classification of ovarian ct benign and malignant tumors based on radiomics and deep learning of dual views, IEEE Access (2024)
2024
-
[32]
X. Yang, H. Li, Y. Wang, X. Liang, et al., Contrastive rendering with semi-supervised learning for ovary and follicle segmentation from 3d ultrasound, Medical Image Analysis 73 (2021) 102134
2021
-
[33]
R. T. Narra, N. Singhal, N. S. Narayan, et al., Automated ovarian vol- ume quantification in transvaginal ultrasound, in: 2018 IEEE 15th In- ternational Symposium on Biomedical Imaging (ISBI 2018), 2018, pp. 1513–1516. 29
2018
-
[34]
Huang, Z
G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely con- nected convolutional networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708
2017
-
[35]
H. Li, J. Fang, S. Liu, et al., Cr-unet: A composite network for ovary and follicle segmentation in ultrasound images, IEEE journal of biomedical and health informatics 24 (4) (2019) 974–983
2019
-
[36]
D. S. Wanderley, C. B. Carvalho, A. Domingues, et al., End-to-end ovarian structures segmentation, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, 2019, pp. 681–689
2019
-
[37]
H. Wang, C. Liu, Z. Zhao, et al., Application of deep convolutional neural networks for discriminating benign, borderline, and malignant serous ovarian tumors from ultrasound images, Frontiers in oncology 11 (2021) 770683
2021
-
[38]
Zhang, X
X. Zhang, X. Zhou, M. Lin, J. Sun, Shufflenet: An extremely efficient convolutional neural network for mobile devices, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848–6856
2018
-
[39]
J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in: Proceed- ings of the IEEE conference on computer vision and pattern recognition (CVPR), 2018, pp. 7132–7141
2018
-
[40]
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, Mobilenets: Efficient convolu- tional neural networks for mobile vision applications, arXiv preprint arXiv:1704.04861 (2017)
2017 arXiv
-
[41]
Zhang, H
Y. Zhang, H. Jiang, Y. Miura, et al., Contrastive learning of medical visual representations from paired images and text, in: Machine learning for healthcare conference (MLHC), 2022, pp. 2–25
2022
-
[42]
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), 2021, pp. 10012–10022. 30
2021
-
[43]
Oktay, J
O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Mis- awa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz, et al., At- tention u-net: Learning where to look for the pancreas, arXiv preprint arXiv:1804.03999 (2018)
2018 arXiv
-
[44]
J. Fan, J. Liu, Q. Chen, et al., Accurate ovarian cyst classification with a lightweight deep learning model for ultrasound images, IEEE Access 11 (2023) 110681–110691
2023
-
[45]
Bahdanau, K
D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473 (2014)
2014 arXiv
-
[46]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, et al., Attention is all you need, Advances in neural information processing systems 30 (2017)
2017
-
[47]
Hsu, Y.-J
S.-T. Hsu, Y.-J. Su, C.-H. Hung, et al., Automatic ovarian tumors recog- nition system based on ensemble convolutional neural network with ul- trasound imaging, BMC Medical Informatics and Decision Making 22 (1) (2022) 298
2022
-
[48]
S. Reka, T. Suriya Praba, M. Mukesh Prasanna, et al., Automated high precision pcos detection through a segment anything model on super res- olution ultrasound ovary images, Scientific Reports 15 (1) (2025) 16832
2025
-
[49]
Xiang, Y
H. Xiang, Y. Xiao, F. Li, et al., Development and validation of an inter- pretable model integrating multimodal information for improving ovar- ian cancer diagnosis, Nature Communications 15 (1) (2024) 2681
2024
-
[50]
Lab, Ovatus-v1: Ovarian tumor ultrasound image dataset (2025)
S. Lab, Ovatus-v1: Ovarian tumor ultrasound image dataset (2025). URL https://sigm-seee.github.io/datasets/Ovarian.html
2025
-
[51]
Z. Li, Y. Li, Q. Li, et al., Lvit: language meets vision transformer in medical image segmentation, IEEE transactions on medical imaging (TMI) 43 (1) (2023) 96–107
2023
-
[52]
Huang, L
S.-C. Huang, L. Shen, M. P. Lungren, et al., Gloria: A multimodal global-local representation learning framework for label-efficient medi- cal image recognition, in: Proceedings of the IEEE/CVF international conference on computer vision (CVPR), 2021, pp. 3942–3951. 31
2021
-
[53]
Y. Yue, Z. Li, Medmamba: Vision mamba for medical image classifica- tion, arXiv preprint arXiv:2403.03849 (2024)
2024 arXiv
-
[54]
C. Wu, Y. Wang, F. Wang, Deep learning for ovarian tumor classifica- tion with ultrasound images, in: Advances in Multimedia Information Processing(PCM), 2018, pp. 395–406
2018
-
[55]
K. He, X. Zhang, S. Ren, et al., Deep residual learning for image recog- nition, in: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp. 770–778
2016
-
[56]
Hampton, Cancer genome atlas, Jama 296 (16) (2006) 1958–1958
T. Hampton, Cancer genome atlas, Jama 296 (16) (2006) 1958–1958
2006
-
[57]
Q. Zhao, S. Lyu, W. Bai, et al., Mmotu: a multi-modality ovarian tumor ultrasound image dataset for unsupervised cross-domain semantic segmentation, arXiv preprint arXiv:2207.06799 (2022)
2022 arXiv
-
[58]
F. Bray, M. Laversanne, H. Sung, et al., Global cancer statistics 2022: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries, CA: a cancer journal for clinicians 74 (3) (2024) 229– 263
2024
-
[59]
Ronneberger, P
O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Medical image computing and computer-assisted intervention (MICCAI)), 2015, pp. 234–241. 32
2015
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.