REVIEW 4 major objections 5 minor 31 references
Medical-Knowledge Driven Multiple Instance Learning for Classifying Severe Abdominal Anomalies on Prenatal Ultrasound
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Medical-knowledge prompts guide case-level MIL to classify fetal abdominal anomalies without standard-plane localization, reaching 85.12% weighted accuracy.
desk verdict The dataset and clinical framing are genuinely useful, but the 'medical-knowledge' feature selection mechanism is under-specified to the point of being unverified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rests on three mechanisms. MoAE (mixture-of-attention-experts) treats attention heads as experts and lets each image token activate the top-k most relevant heads via a router, so different sonographic planes can recruit different attention patterns. MFS (medical-knowledge-driven feature selection) encodes category definitions and signs with PubMedBERT, computes a dot-product similarity between each image token and the prompt embeddings, and uses an adaptive threshold to keep only the most anomaly-relevant images; the threshold is scaled by a coefficient beta that increases progressively across training stages. PPL (prompt-based prototype learning) is a loss term that treats the prompt embeddings as prototypes, pulling selected image tokens of the true class closer and pushing them away from other classes. These mechanisms together are what the paper claims carry the 85.12% weighted accuracy result.
What would settle it
A control experiment that replaces the medical-knowledge prompts with random or non-medical texts (same syntax, different content) and measures whether the reported weighted accuracy changes would directly test the knowledge-driven claim. A second test would check whether the images retained by the MFS overlap with the manually annotated abnormal frames in the dataset more than randomly selected images do.
Extended reading notes
Core claim
The authors' central claim is that replacing the standard two-step pipeline — locate standard planes, then classify individual images — with a bag-of-images MIL pipeline yields superior case-level diagnosis of fetal abdominal anomalies. The load-bearing idea is that image tokens from an exam can be routed through shared and top-k selected attention heads (MoAE), then filtered by similarity to medical-knowledge prompts (MFS), and finally trained with a prototype-pulling loss (PPL) so that the selected frames are the ones that matter for the case label. With these modules added to a TransMIL-style baseline, the paper reports an overall weighted accuracy of 85.12%, with sensitivity gains of 3.13 percentage points on duodenal atresia, 9.38 on gastroschisis, and 5 on renal agenesis over the second-best competitor. The authors take this as evidence that automated case-level screening is feasible even when severe wall defects make standard planes unidentifiable.
Load-bearing premise
The MFS module multiplies image-token features and PubMedBERT prompt features directly, without any learned projection or alignment step, so the medical-knowledge selection is only meaningful if those two heterogeneous embedding spaces are already comparable.
Editorial extensions
If this is right
- Screening tools for prenatal abdominal anomalies could shift from image-level to case-level, reducing the need for fine-grained standard-plane annotations.
- The method's largest sensitivity gains occur on exactly the defects that break standard-plane localization (gastroschisis +9.38 points), suggesting it copes with severe cases.
- Medical-knowledge prompts can act as self-supervision for frame selection in weakly supervised ultrasound bags, without image-level anomaly labels.
- Adding MoAE, MFS, and PPL to the TransMIL baseline raises weighted accuracy by 5.59 points in the reported ablation.
Reading between the lines
- The paper does not test whether replacing the medical-knowledge prompts with random or mismatched text changes accuracy, so the specific contribution of medical knowledge over mere prompt-conditioned selection remains an open, testable question.
- The dot-product comparison between image features and PubMedBERT features presumes compatibility that the paper does not establish with a learned projection; if a simple alignment layer changed the result, the MFS design would need revision.
- Because the MFS threshold is set by a coefficient beta whose schedule is not fully specified, reproducing the method on a new dataset will require fixing that schedule and checking sensitivity to it.
- If the selection mechanism generalizes, the same prompt-based MIL recipe could be applied to other fetal organ systems (brain, heart) where severe malformations also defeat standard-plane pipelines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multiple instance learning (MIL) framework for case-level classification of severe fetal abdominal anomalies from prenatal ultrasound image pools, without relying on standard-plane localization. The method combines a mixture-of-attention-experts module (MoAE), a medical-knowledge-driven feature selection module (MFS) that uses frozen PubMedBERT prompt embeddings to select relevant images via dot-product similarity, and a prompt-based prototype learning (PPL) loss that pulls image tokens toward the ground-truth category prompt. Experiments on a multi-center dataset of 2,419 cases (24,748 images, 6 categories) report a weighted accuracy of 85.12%, which the authors state outperforms nine MIL baselines. The paper also includes an ablation study and qualitative visualizations of selected images.
Significance. If the reported results are reliable, the work addresses a clinically meaningful gap: automated case-level screening of severe abdominal anomalies without prior standard-plane detection, which is particularly relevant for abdominal wall defects where standard planes are often unobtainable. The main strengths are the relatively large multi-center dataset, the use of case-level supervision only, the comparison against nine baselines, and the public release of code. However, the central contribution (MFS) rests on an underspecified and potentially invalid feature-space alignment between image and text embeddings, and the empirical claims lack statistical support (no confidence intervals, p-values, or multiple runs). The significance of the paper therefore hinges on whether these technical gaps can be resolved.
major comments (4)
- [Section 2, Equations (6) and (7)] The dot product in Eq. (6) and Eq. (7) between image tokens x'_i and medical-knowledge prompts is not well-defined as written. The image tokens come from a ResNet18 encoder (typically 512-dimensional), while PubMedBERT embeddings are typically 768-dimensional; no projection, linear layer, or alignment mechanism is described that would place these features in a common space. Because MFS is the paper's central contribution, the selection threshold and the PPL loss are only meaningful if the similarity computation is valid. The authors must specify the projection (its dimensionality, initialization, and training objective) or use a pre-aligned multimodal encoder; otherwise the reported gains could stem from arbitrary norm or scale artifacts rather than medical knowledge.
- [Section 2, Equation (6) and implementation details] The threshold in Eq. (6) depends on a coefficient beta described only as 'increases progressively at each stage,' with no initial value, schedule, or stage boundaries. Since the threshold directly controls which images are selected for classification and for the PPL loss, the sensitivity of the method to beta must be reported (e.g., a figure or table over beta values), and the exact schedule must be stated. Without this, the ablation results in Table 2 cannot be reproduced or interpreted.
- [Section 3, Table 1] The claim that the proposed method outperforms state-of-the-art competitors is not uniformly supported by the reported class-wise numbers. For example, Omphalocele sensitivity is 87.50% for the proposed method versus 93.42% for TransMIL, and MCDK sensitivity is 80.36% versus 85.71% for DSMIL and AbMIL. The overall weighted accuracy advantage (85.12% vs 81.23%) is presented without confidence intervals, p-values, or repeated-run variability. The authors should provide statistical significance tests (e.g., bootstrapping or paired tests over multiple seeds) and discuss the class-wise trade-offs; otherwise the headline claim of consistent superiority is not established.
- [Section 3, Table 2 and Section 2] The attribution of improvements to MFS and PPL is confounded by two issues. First, Table 2 shows non-monotonic class-level changes: e.g., adding MoAE and PPL to MFS decreases MCDK sensitivity from 85.71% to 80.36%, and adding PPL alone to MFS decreases RA sensitivity from 72.50% to 75.00% (which actually increases) but the paper does not analyze these fluctuations. Second, the PPL loss in Eq. (7) uses the ground-truth label Cls as the positive prototype, so the training is supervised, not 'self-supervised' as claimed in the abstract and Section 2. The authors should clarify this terminology and provide an analysis of which module drives gains for each class.
minor comments (5)
- [Throughout] The manuscript contains multiple typos and formatting errors: 'preformed' should be 'performed' (Section 3), 'DoudenalAtresia' in Fig. 4, stray spaces in 'F etal', 'R enal Agenesis', and the abstract line breaks. These should be corrected.
- [Section 2, Equation (5)] The notation in Eq. (5) is ambiguous: the range of j, the relationship between h_s and h, and the meaning of Top-K are not fully specified. Please define all symbols explicitly.
- [Section 2, MFS module] The prompt template is described only as '{Cls, definition, signs}' with no concrete examples of the text used for each category. Providing the actual prompts (or at least one full example) is essential for reproducibility and for assessing the claimed medical-knowledge grounding.
- [Figure 2] Figure 2 contains placeholder '???' markers that should be replaced with the actual module names and symbols.
- [References] Some references are incomplete or incorrectly attributed: Ref. [20] is cited as 'Kaito et al.' in the text but the reference list gives 'Shiku, K., et al.'; Ref. [13] is cited as 'IBMIL' but the list entry is 'Lin, T., Yu, Z.,...' without the method name. Please verify all citations.
Circularity Check
No circularity found: the claimed gains are held-out test-set comparisons, and the medical-knowledge modules use external definitions plus standard supervised training, not the test labels as outputs.
full rationale
The paper's central claim is an empirical comparison against nine MIL baselines on a held-out test set, split at the case level with the category distribution maintained. No parameter fitted on the training set is renamed as a prediction: the reported sensitivities, F1 scores, and weighted accuracy are evaluated on cases not used for training. The PPL loss in Eqs. (7)-(8) uses ground-truth class labels during training to pull image tokens toward the corresponding prompt prototype; this is standard supervised learning, not a circular prediction. The medical prompts are hand-authored from external medical definitions and are not derived from the test labels, and the model could fail to align visual features with them; the reported improvements are therefore empirical findings rather than artifacts of definition. The MFS dot-product computation between ResNet18 image features and PubMedBERT prompt embeddings is under-specified, since no projection or shared embedding space is described and the dimensions may be incompatible; this is a reproducibility and correctness risk, not a circularity. Similarly, the unspecified schedule for the threshold coefficient beta is an engineering gap, not circular reasoning. The self-citations in the references are related-work citations and are not load-bearing for the proposed modules or the evaluation. There is no uniqueness theorem imported from the authors, no ansatz smuggled in through a citation, and no known result renamed as new. The main contribution is an empirical framework with a held-out evaluation, so the derivation chain is not circular.
Assumptions & free parameters
free parameters (4)
- Threshold coefficient beta =
not specified (described as progressively increasing)
- MoAE balancing coefficients alpha1 and alpha2 =
not specified
- Medical prompt templates =
not disclosed
- Number of shared heads hs and top-K =
not specified
assumptions (4)
- domain assumption MIL assumption: each case is a bag; case label applies if at least one image contains the anomaly
- ad hoc to paper Image and text features can be meaningfully compared by dot product in a shared space
- domain assumption The adaptive threshold keeps relevant anomaly images while discarding irrelevant ones
- domain assumption Expert-provided case labels in the multicenter dataset are accurate
Cite this review
Pith. "Pith review of Medical-Knowledge Driven Multiple Instance Learning for Classifying Severe Abdominal Anomalies on Prenatal Ultrasound." pith.science (2026). https://pith.science/paper/3KLCEWSV
@misc{pith2026250701401,
author = {Pith},
title = {Pith review of: Medical-Knowledge Driven Multiple Instance Learning for Classifying Severe Abdominal Anomalies on Prenatal Ultrasound},
year = {2026},
howpublished = {\url{https://pith.science/paper/3KLCEWSV}},
note = {Machine review of arXiv:2507.01401}
}
read the original abstract
Fetal abdominal malformations are serious congenital anomalies that require accurate diagnosis to guide pregnancy management and reduce mortality. Although AI has demonstrated significant potential in medical diagnosis, its application to prenatal abdominal anomalies remains limited. Most existing studies focus on image-level classification and rely on standard plane localization, placing less emphasis on case-level diagnosis. In this paper, we develop a case-level multiple instance learning (MIL)-based method, free of standard plane localization, for classifying fetal abdominal anomalies in prenatal ultrasound. Our contribution is three-fold. First, we adopt a mixture-of-attention-experts module (MoAE) to weight different attention heads for various planes. Secondly, we propose a medical-knowledge-driven feature selection module (MFS) to align image features with medical knowledge, performing self-supervised image token selection at the case-level. Finally, we propose a prompt-based prototype learning (PPL) to enhance the MFS. Extensively validated on a large prenatal abdominal ultrasound dataset containing 2,419 cases, with a total of 24,748 images and 6 categories, our proposed method outperforms the state-of-the-art competitors. Codes are available at:https://github.com/LL-AC/AAcls.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
In: Pro- ceedings of the IEEE/CVF Conference on CVPR
Chen, W., Si, C., et al.: Semantic prompt for few-shot image recognition. In: Pro- ceedings of the IEEE/CVF Conference on CVPR. pp. 23581–23591 (2023)
work page 2023
-
[2]
Journal of Imaging Informatics in Medicine pp
Ciobanu, S.G., Enache, I.A., Iovoaica-R˘ amescu, C., Berbecaru, E.I.A., et al.: Au- tomatic identification of fetal abdominal planes from ultrasound images based on deep learning. Journal of Imaging Informatics in Medicine pp. 1–8 (2025)
work page 2025
-
[3]
ACM Transactions on Com- puting for Healthcare (HEALTH) 3(1), 1–23 (2021)
Gu, Y., Tinn, R., Cheng, H., Lucas, M., et al.: Domain-specific language model pre- training for biomedical natural language processing. ACM Transactions on Com- puting for Healthcare (HEALTH) 3(1), 1–23 (2021)
work page 2021
-
[4]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Huang, Y., Yang, X., Huang, X., Zhou, X., Chi, H., Dou, H., Hu, X., Wang, J., Deng, X., Ni, D.: Fourier test-time adaptation with multi-level consistency for robust classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 221–231. Springer (2023)
work page 2023
-
[5]
In: International conference on machine learning
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International conference on machine learning. pp. 2127–2136. PMLR (2018)
work page 2018
-
[6]
In: Handbook of Medical Image Computing and Computer Assisted Intervention, pp
Ilse, M., Tomczak, J.M., Welling, M.: Deep multiple instance learning for digital histopathology. In: Handbook of Medical Image Computing and Computer Assisted Intervention, pp. 521–546. Elsevier (2020)
work page 2020
-
[7]
Ad- vances in Neural Information Processing Systems 35, 20689–20702 (2022)
Javed, S.A., Juyal, D., Padigela, H., Taylor-Weiner, A., Yu, L., Prakash, A.: Ad- ditive mil: Intrinsically interpretable multiple instance learning for pathology. Ad- vances in Neural Information Processing Systems 35, 20689–20702 (2022)
work page 2022
-
[8]
arXiv preprint arXiv:2410.11842 (2024)
Jin, P., Zhu, B., Yuan, L., Yan, S.: Moh: Multi-head attention as mixture-of-head attention. arXiv preprint arXiv:2410.11842 (2024)
arXiv 2024
Show all 31 references
-
[9]
data from two birth-defect registries
Laurichesse Delmas, H., et al.: Congenital unilateral renal agenesis: prevalence, pre- natal diagnosis, associated anomalies. data from two birth-defect registries. Birth defects research 109(15), 1204–1211 (2017)
2017
-
[10]
In: Proceedings of the IEEE/CVF conference on CVPR
Li, B., et al.: Dual-stream multiple instance learning network for whole slide im- age classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF conference on CVPR. pp. 14318–14328 (2021)
2021
-
[11]
In: Medical Image Computing and Computer Assisted Intervention: 24th International Conference, Strasbourg, France, September 27– October 1, 2021, Proceedings, Part VIII 24
Li, H., Yang, F., et al.: Dt-mil: deformable transformer for multi-instance learning on histopathological image. In: Medical Image Computing and Computer Assisted Intervention: 24th International Conference, Strasbourg, France, September 27– October 1, 2021, Proceedings, Part ...
2021
-
[12]
Ultrasound in Obstetrics & Gynecology 59(3), 304–316 (2022)
Lin, M., He, X., Guo, H., He, M., Zhang, L., Xian, J., Lei, T., Xu, Q., Zheng, J., Feng, J., et al.: Use of real-time artificial intelligence in detection of abnormal image patterns in standard sonographic reference planes in screening for fetal intracranial malformations. Ult...
2022
-
[13]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Lin, T., Yu, Z., Hu, H., Xu, Y., Chen, C.W.: Interventional bag multi-instance learning on whole-slide pathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19830–19839 (2023)
2023
-
[14]
In: International Work- shop on Machine Learning in Medical Imaging
Liu, Z., Zhu, X., Han, T., Huang, Y., Wang, J., Liu, L., Wang, F., Ni, D., Gou, Z., Yang, X.: Mitral regurgitation recogniton based on unsupervised out-of- distribution detection with residual diffusion amplification. In: International Work- shop on Machine Learning in Medical...
2024
-
[15]
Frontiers in Surgery 9, 891896 (2022)
Pechriggl, E., Blumer, M., et al.: Embryology of the abdominal wall and associated malformations—a review. Frontiers in Surgery 9, 891896 (2022)
2022
-
[16]
arXiv preprint arXiv:2501.02000 (2025)
Qi, Y., Cai, J., Lu, J., Xiong, R., Chen, R., Zheng, L., Ma, D.: Multi-center study on deep learning-assisted detection and classification of fetal central nervous system anomalies using ultrasound imaging. arXiv preprint arXiv:2501.02000 (2025)
2025 arXiv
-
[17]
In: International conference on machine learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., et al.: Learning transfer- able visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
2021
-
[18]
Journal of pediatric urology 10(4), 645–649 (2014)
Sarhan, O., Alghanbar, M., Alsulaihim, A., et al.: Multicystic dysplastic kidney: impact of imaging modality selection on the initial management and prognosis. Journal of pediatric urology 10(4), 645–649 (2014)
2014
-
[19]
Advances in neural information processing systems 34, 2136–2147 (2021)
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., et al.: Transmil: Trans- former based correlated multiple instance learning for whole slide image classifica- tion. Advances in neural information processing systems 34, 2136–2147 (2021)
2021
-
[20]
arXiv preprint arXiv:2411.14750 (2024)
Shiku, K., Nishimura, K., Suehiro, D., Tanaka, K., Bise, R.: Ordinal multiple- instance learning for ulcerative colitis severity estimation with selective aggregated transformer. arXiv preprint arXiv:2411.14750 (2024)
2024 arXiv
-
[21]
In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., Titov, I.: Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 5797–5808 (2019)
2019
-
[22]
In: Medical Image Computing and Computer Assisted Intervention– MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23
Wang, J., Miao, J., Yang, X., Li, R., Zhou, G., Huang, Y., Lin, Z., Xue, W., Jia, X., Zhou, J., et al.: Auto-weighting for breast cancer classification in multimodal ultrasound. In: Medical Image Computing and Computer Assisted Intervention– MICCAI 2020: 23rd International Con...
2020
-
[23]
Medical Image Analysis 83, 102673 (2023)
Wang, X., Tang, F., Chen, H., Cheung, C.Y., Heng, P.A.: Deep semi-supervised multiple instance learning with self-correction for dme classification from oct im- ages. Medical Image Analysis 83, 102673 (2023)
2023
-
[24]
arXiv preprint arXiv:2210.10163 (2022)
Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. arXiv preprint arXiv:2210.10163 (2022)
2022 arXiv
-
[25]
International journal of computer assisted radiology and surgery 15, 1303–1312 (2020)
Xie, B., Lei, T., Wang, N., Cai, H., et al.: Computer-aided diagnosis for fetal brain ultrasound images using deep convolutional neural networks. International journal of computer assisted radiology and surgery 15, 1303–1312 (2020)
2020
-
[26]
BMC Public Health 25, 449 (2025)
Xie, X., et al.: Global birth prevalence of major congenital anomalies: a systematic review and meta-analysis. BMC Public Health 25, 449 (2025)
2025
-
[27]
In: International Conference on Medical Im- age Computing and Computer-Assisted Intervention
Yang, S., et al.: Mambamil: Enhancing long sequence modeling with sequence re- ordering in computational pathology. In: International Conference on Medical Im- age Computing and Computer-Assisted Intervention. pp. 296–306. Springer (2024) MIL for Classifying Abdominal Anomalie...
2024
-
[28]
In: Proceedings of the IEEE/CVF conference on CVPR
Yao, H., Zhang, R., Xu, C.: Visual-language prompt tuning with knowledge-guided context optimization. In: Proceedings of the IEEE/CVF conference on CVPR. pp. 6757–6767 (2023)
2023
-
[29]
arXiv preprint arXiv:2303.00915 (2023)
Zhang, S., Xu, Y., Usuyama, N., et al.: Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 (2023)
2023 arXiv
-
[30]
arXiv preprint arXiv:2408.15032 (2024)
Zhang, Y., Zhang, X., Wang, J., Yang, Y., Peng, T., Tong, C.: Mamba2mil: State space duality based multiple instance learning for computational pathology. arXiv preprint arXiv:2408.15032 (2024)
2024 arXiv
-
[31]
In: AMIA Annual Symposium Proceedings
Zhuang, L., Ivezic, V., et al.: Patient-level thyroid cancer classification using at- tention multiple instance learning on fused multi-scale ultrasound image features. In: AMIA Annual Symposium Proceedings. vol. 2023, p. 1344 (2024)
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.