REVIEW 3 major objections 5 minor 21 references
Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A frozen pathology vision-language feature extractor plus a transformer-based aggregator beats fine-tuned ImageNet CNNs for Ewing sarcoma diagnosis on tissue microarrays.
desk verdict A plausible and useful benchmark of frozen PLIP + TransMIL for sarcoma TMA classification, but the unstated split unit and missing error bars keep the headline 3-point gain from being fully convincing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is PLIP, a contrastive vision-language model pretrained on paired histopathology image-caption data, used frozen to encode 256-by-256 pixel patches from tissue-microarray cores into embeddings. The other half is TransMIL, a transformer-based multiple-instance-learning aggregator that takes those patch embeddings, models spatial and morphological correlations between them, and produces a core-level embedding that a classifier maps to one of four sarcoma classes. The combination matters because it lets the model exploit in-domain visual-language knowledge without updating the backbone, so training only touches the aggregator and the final classifier.
What would settle it
Re-split the cores so that no two cores from the same patient or tissue microarray lie on opposite sides of the training/test boundary and recompute test accuracy; if the PLIP plus TransMIL accuracy drops to or below the fine-tuned VGG16 level, or falls well below 90.5%, the reported advantage is an artifact of split leakage.
Extended reading notes
Core claim
On its own test split, the paper's central discovery is that the best configuration is a frozen PLIP feature extractor feeding a TransMIL transformer aggregator, which correctly classifies 90.5% of cores among Ewing sarcoma, chondrosarcoma, GIST, and rhabdomyosarcoma, with an F1 score of 0.903. This outperforms VGG16 fine-tuned with BGAP (87.5% accuracy) and frozen VGG16 with BGAP (83.7%). The authors attribute the gain to using vision-language supervision on in-domain histopathology data, which produces better separated core embeddings, and to the transformer aggregator's use of spatial and morphological correlations among patches. They also report a sharp efficiency advantage: 2.6 million trainable parameters and 0.0155 seconds per training step for PLIP plus TransMIL versus 29.7 million parameters and 0.112 seconds for fine-tuned VGG16 with BGAP.
Load-bearing premise
The accuracy numbers rest on the assumption that the 60/15/25 split keeps every core from the same patient or tissue-microarray block entirely on one side of the split; the paper does not say this, and if cores from one tumor block appear in both training and test, the model could be memorizing staining and scanner artifacts instead of sarcoma features.
Editorial extensions
If this is right
- Pathology-specific vision-language pretraining can replace ImageNet transfer learning for tissue-microarray classification, giving better accuracy while keeping the feature extractor frozen.
- The choice of aggregator matters: with PLIP features, the transformer-based TransMIL aggregator beats average pooling, max pooling, and attention-based MIL on the validation set.
- The pipeline needs 2.6 million trainable parameters instead of 29.7 million and about 0.0155 seconds instead of 0.112 seconds per training step, making it practical with limited annotated histopathology data.
- The fine-tuned VGG16 baseline loses accuracy from validation to test, while the frozen-PLIP pipeline keeps its advantage, suggesting supervised fine-tuning overfits this small cohort.
Reading between the lines
- Editorial inference: if the split is truly patient- or tissue-microarray-disjoint, the same frozen-feature-plus-transformer recipe should transfer to other small round cell tumors, where annotated biopsies are scarce and whole-slide images are the practical input.
- Editorial inference: a direct test of the mechanism would swap PLIP for another in-domain pathology vision-language model, such as CONCH, and check whether the accuracy gain persists; the paper's argument predicts it should.
- Editorial inference: the compute comparison is reported at batch size one on a single GPU type, so the speed advantage should be re-measured at deployment batch sizes and on CPU or edge hardware before drawing conclusions about clinical cost.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multiple-instance learning (MIL) pipeline for classifying Ewing sarcoma versus three morphologically similar sarcomas (chondrosarcoma, GIST, rhabdomyosarcoma) from digitized tissue microarray cores. It compares frozen pathology vision-language features (PLIP) with ImageNet-pretrained VGG16 features, both frozen and fine-tuned, combined with several aggregators (BGAP, BGMP, attention, TransMIL). On a single 60/15/25 split of the dataset, the authors report that PLIP features with a TransMIL aggregator reach 90.5% test accuracy, outperforming fine-tuned VGG16 with BGAP at 87.5% accuracy, while using substantially fewer trainable parameters (2.6M vs 29.7M) and lower per-step compute (0.0155 s vs 0.112 s). The paper concludes that frozen in-domain vision-language features plus a trained transformer aggregator are both more accurate and more efficient than fine-tuning a supervised CNN on this task.
Significance. If the reported results hold, the paper makes a useful empirical contribution: it shows that a frozen pathology-specific vision-language feature extractor can outperform fine-tuned ImageNet features in a small, imbalanced histopathology dataset, with large savings in training cost. The ablation across aggregators is informative, and reporting computational cost and trainable parameters is a strength. The main significance, however, depends on the test split being independent at the patient or TMA level and on the accuracy gap being statistically reliable; neither is currently established. The paper also does not release code or data, which limits reproducibility for a result whose validity hinges on evaluation protocol details.
major comments (3)
- [Section 3 and Section 5.1] The split unit is not specified. The dataset is described in terms of TMAs and cores, but Section 5.1 states only that the database was partitioned into 60% training, 15% validation, and 25% test proportions. If cores from the same TMA or the same patient appear in both training and test, the model can memorize staining, sectioning, and scanner artifacts, which would inflate the reported 90.5% accuracy and invalidate the comparison between PLIP and VGG16. Please specify whether the split was performed at the patient, TMA, or core level, report the number of patients or TMAs in each partition, and, ideally, provide a patient-exclusive or TMA-exclusive split to confirm the results.
- [Table 2 and Section 5.3] The central comparison rests on a single split with no confidence intervals, error bars, or significance tests. With roughly 487 test cores (25% of 1947 cores), the 90.5% versus 87.5% accuracy difference corresponds to only about 15 test cores, which could easily arise from sampling noise. Please report bootstrap confidence intervals, per-class confidence intervals, or repeated cross-validation results, and state whether the PLIP advantage is statistically significant.
- [Section 5.2 and Figure 4] The overfitting claim for fine-tuned VGG16 is supported only by a qualitative statement that its validation performance decreased on the test set. The paper does not report validation accuracies, training curves, early-stopping criteria, or the number of epochs for any configuration. Without this information, the reader cannot assess whether the fine-tuned VGG result is a robust baseline or an artifact of incomplete training or hyperparameter selection. Please report the validation results corresponding to Figure 4 in numeric form and describe the training protocol in enough detail to reproduce it.
minor comments (5)
- [Figure 3 caption] The caption states "the five neoplasms under study," but Table 1 lists four classes (EWING, COND, GIST, RHABDO); please correct the caption to four.
- [Section 4, Equation (1)] The symbol K is used in the loss normalization but is not defined in the problem formulation; please define K as the number of classes.
- [Section 5.1] The learning rate is reported as "between 1e-5 and 5e-5," but the paper does not specify which configuration used which value; please report the learning rate for each feature extractor and aggregator combination.
- [Section 5.1] The class-weight scheme is described only as weights inversely proportional to the number of instances per class; please provide the exact formula or the weight values used for each class.
- [Table 2 and Figure 5] Per-class sensitivity and precision would make the comparison more informative, especially given the class imbalance; the confusion matrix in Figure 5 is small and the numeric values are not reproduced in the text.
Circularity Check
No circularity: the accuracy comparison is an empirical benchmark against external pretrained feature extractors, and the reported test metrics are measured outputs rather than quantities defined by the paper's own equations.
full rationale
The paper's central claim is an empirical comparison of feature extractors and aggregators on a held-out test set: frozen PLIP with TransMIL reaches 90.5% accuracy versus 87.5% for fine-tuned VGG16 with BGAP. These numbers are test-set measurements, not derived quantities. The loss in Eq. (1) is standard categorical cross-entropy, and no parameter is fitted to the test labels and then reported as a prediction. The feature extractors are external pretrained models (ImageNet VGG16 and PLIP trained on OpenPath), and the aggregators (BGAP, TransMIL) are taken from cited external work. The only apparent self-citation, Ref. [16], supports the general bag-embedding MIL framing and does not carry the accuracy claim, so it is not load-bearing. The ambiguity about whether the train/validation/test split was performed by core, TMA, or patient is a real validity risk for generalization, but it is not a circularity: it concerns possible information leakage in evaluation, not equivalence between the paper's inputs and outputs. No definitional reduction, fitted-input-called-prediction, or self-citation chain forces the reported results, so the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (5)
- Learning rate =
1e-5 to 5e-5 (per configuration)
- Train/validation/test split ratio =
60/15/25
- Class weight scheme =
Inversely proportional to instance count per class
- TransMIL hyperparameters =
Not reported
- Training epochs / early stopping =
Not reported
assumptions (4)
- domain assumption Bag-embedding MIL assumption
- domain assumption PLIP feature transferability
- domain assumption Histological labels are correct
- domain assumption Exclusion of non-informative cores is unbiased
Cite this review
Pith. "Pith review of Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma." pith.science (2026). https://pith.science/paper/4OZTHZ7A
@misc{pith2026250108042,
author = {Pith},
title = {Pith review of: Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma},
year = {2026},
howpublished = {\url{https://pith.science/paper/4OZTHZ7A}},
note = {Machine review of arXiv:2501.08042}
}
read the original abstract
Ewing's sarcoma (ES), characterized by a high density of small round blue cells without structural organization, presents a significant health concern, particularly among adolescents aged 10 to 19. Artificial intelligence-based systems for automated analysis of histopathological images are promising to contribute to an accurate diagnosis of ES. In this context, this study explores the feature extraction ability of different pre-training strategies for distinguishing ES from other soft tissue or bone sarcomas with similar morphology in digitized tissue microarrays for the first time, as far as we know. Vision-language supervision (VLS) is compared to fully-supervised ImageNet pre-training within a multiple instance learning paradigm. Our findings indicate a substantial improvement in diagnostic accuracy with the adaption of VLS using an in-domain dataset. Notably, these models not only enhance the accuracy of predicted classes but also drastically reduce the number of trainable parameters and computational costs.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Z. Burningham, M. Hashibe, L. Spector, and J. D. Schiffman, “The epidemiology of sarcoma,” Clinical sarcoma research, vol. 2, no. 1, pp. 1–16, 2012
work page 2012
-
[2]
Ewing tumour: incidence, prognosis and treatment options,
M. Paulussen, B. Fr¨ ohlich, and H. J¨ urgens, “Ewing tumour: incidence, prognosis and treatment options,” Paediatric drugs, vol. 3, pp. 899–913, 2001
work page 2001
-
[3]
N. Esiashvili, M. Goodman, and R. B. Marcus, “Changes in incidence and survival of ewing sarcoma patients over the past 3 decades: Surveillance epidemiology and end results data,” Journal of pediatric hematology/oncology , vol. 30, no. 6, pp. 425–430, 2008
work page 2008
-
[4]
S. Consalvo, F. Hinterwimmer, J. Neumann, M. Steinborn, M. Salzmann, F. Seidl, U. Lenze, C. Knebel, D. Rueckert, and R. H. Burgkart, “Two-phase deep learning algorithm for detection and differentiation of ewing sarcoma and acute osteomyeli- tis in paediatric radiographs,” Anticancer Research, vol. 42, no. 9, pp. 4371–4380, 2022
work page 2022
-
[5]
Machine learning for rhabdomyosarcoma histopathology,
A. O. Frankel, M. Lathara, C. Y. Shaw, O. Wogmon, J. M. Jackson, M. M. Clark, N. Eshraghi, S. E. Keenen, A. D. Woods, R. Purohit et al. , “Machine learning for rhabdomyosarcoma histopathology,” Modern Pathology, vol. 35, no. 9, pp. 1193– 1203, 2022
work page 2022
-
[6]
X. Zhang, S. Wang, E. R. Rudzinski, S. Agarwal, R. Rong, D. A. Barkauskas, O. Daescu, L. F. Cline, R. Venkatramani, Y. Xie et al. , “Deep learning of rhab- domyosarcoma pathology images for classification and survival outcome predic- tion,” The American Journal of Pathology , vol. 192, no. 6, pp. 917–925, 2022. 10 A. Pastor-Naranjo et al
work page 2022
-
[7]
D. Milewski, H. Jung, G. T. Brown, Y. Liu, B. Somerville, C. Lisle, M. Ladanyi, E. R. Rudzinski, H. Choo-Wosoba, D. A. Barkauskas et al., “Predicting molecular subtype and survival of rhabdomyosarcoma patients using deep learning of h&e images: A report from the children’s oncology group,” Clinical Cancer Research, vol. 29, no. 2, pp. 364–378, 2023
work page 2023
-
[8]
Deep learning based automated tool for cancer diagnosis from bone histopathology images,
S. R. Vandana B.S, “Deep learning based automated tool for cancer diagnosis from bone histopathology images,” International Conference on Intelligent Technologies (CONIT), 2021
work page 2021
Show all 21 references
-
[9]
Learning transferable visual models from natural language supervision,
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763
2021
-
[10]
Scaling up visual and vision-language representation learning with noisy text supervision,
C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in International conference on machine learning. PMLR, 2021, pp. 4904–4916
2021
-
[11]
Contrastive learning of medical visual representations from paired images and text,
Y. Zhang, H. Jiang, Y. Miura, C. D. Manning, and C. P. Langlotz, “Contrastive learning of medical visual representations from paired images and text,” inMachine Learning for Healthcare Conference. PMLR, 2022, pp. 2–25
2022
-
[12]
Towards a visual-language foundation model for computational pathology,
M. Y. Lu, B. Chen, D. F. Williamson, R. J. Chen, I. Liang, T. Ding, G. Jaume, I. Odintsov, A. Zhang, L. P. Leet al., “Towards a visual-language foundation model for computational pathology,” arXiv preprint arXiv:2307.12914 , 2023
2023 arXiv
-
[13]
A visual– language foundation model for pathology image analysis using medical twitter,
Z. Huang, F. Bianchi, M. Yuksekgonul, T. J. Montine, and J. Zou, “A visual– language foundation model for pathology image analysis using medical twitter,” Nature medicine, vol. 29, no. 9, pp. 2307–2316, 2023
2023
-
[14]
Detection of breast cancer from whole slide histopathological images using deep multiple instance cnn,
K. Das, S. Conjeti, J. Chatterjee, and D. Sheet, “Detection of breast cancer from whole slide histopathological images using deep multiple instance cnn,” IEEE Ac- cess, vol. 8, pp. 213 502–213 511, 2020
2020
-
[15]
Self-learning for weakly supervised gleason grading of local patterns,
J. Silva-Rodriguez, A. Colomer, J. Dolz, and V. Naranjo, “Self-learning for weakly supervised gleason grading of local patterns,” IEEE journal of biomedical and health informatics, vol. 25, no. 8, pp. 3094–3104, 2021
2021
-
[16]
An attention-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in whole slide images,
R. Del Amor, L. Launet, A. Colomer, A. Moscard´ o, A. Mosquera-Zamudio, C. Mon- teagudo, and V. Naranjo, “An attention-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in whole slide images,” Artificial intelligence in medicine, vol. 121, p. 102197, 2021
2021
-
[17]
Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution,
Y. Zhao, F. Yang, Y. Fang, H. Liu, N. Zhou, J. Zhang, J. Sun, S. Yang, B. Menze, X. Fan et al. , “Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution,” in Proceedings of the IEEE/CVF Conference on Comp...
2020
-
[18]
Classification of volu- metric images using multi-instance learning and extreme value theorem,
R. Tennakoon, G. Bortsova, S. Ørting, A. K. Gostar, M. M. Wille, Z. Saghir, R. Hoseinnezhad, M. de Bruijne, and A. Bab-Hadiashar, “Classification of volu- metric images using multi-instance learning and extreme value theorem,” IEEE Transactions on Medical Imaging, vol. 39, no....
2019
-
[19]
Attention-based deep multiple instance learning,
M. Ilse, J. M. Tomczak, and M. Welling, “Attention-based deep multiple instance learning,” in 35th International Conference on Machine Learning (ICML) , 2018
2018
-
[20]
Clinical- grade computational pathology using weakly supervised deep learning on whole slide images,
G. Campanella, M. G. Hanna, L. Geneslaw, A. Miraflor, V. Werneck Krauss Silva, K. J. Busam, E. Brogi, V. E. Reuter, D. S. Klimstra, and T. J. Fuchs, “Clinical- grade computational pathology using weakly supervised deep learning on whole slide images,” Nature Medicine, vol. 25,...
2019
-
[21]
Transmil: Transformer based correlated multiple instance learning for whole slide image classification,
Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji et al., “Transmil: Transformer based correlated multiple instance learning for whole slide image classification,” Advances in neural information processing systems , vol. 34, pp. 2136–2147, 2021
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.