Pith. sign in

REVIEW 3 major objections 5 minor 21 references

Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read A frozen pathology vision-language feature extractor plus a transformer-based aggregator beats fine-tuned ImageNet CNNs for Ewing sarcoma diagnosis on tissue microarrays.

desk verdict A plausible and useful benchmark of frozen PLIP + TransMIL for sarcoma TMA classification, but the unstated split unit and missing error bars keep the headline 3-point gain from being fully convincing. read the letter →

arxiv 2501.08042 v1 pith:4OZTHZ7A submitted 2025-01-14 cs.CV cs.AI

classification cs.CVcs.AI
keywords Ewingsarcomatissuemicroarraysmultipleinstancelearningvision-languagepre-trainingpathologyfoundationmodelhistopathologyclassificationtransformeraggregation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a frozen vision-language model trained on pathology images and captions can serve as a patch-level feature extractor for diagnosing Ewing sarcoma on tissue microarrays, and that this beats the standard recipe of fine-tuning an ImageNet-pretrained CNN. The authors compare PLIP features against frozen and fine-tuned VGG16 features inside a multiple-instance learning setup, aggregating patch embeddings into a core-level prediction with a transformer-based aggregator. On their test set the PLIP pipeline reaches 90.5% accuracy and 91.4% precision versus 87.5% for fine-tuned VGG16, while cutting trainable parameters from 29.7 million to 2.6 million and per-step training time from 0.112 seconds to 0.0155 seconds. If true, this means in-domain vision-language pretraining plus a trainable aggregator is both more accurate and much cheaper than supervised fine-tuning for this diagnostic task.

What carries the argument

The load-bearing object is PLIP, a contrastive vision-language model pretrained on paired histopathology image-caption data, used frozen to encode 256-by-256 pixel patches from tissue-microarray cores into embeddings. The other half is TransMIL, a transformer-based multiple-instance-learning aggregator that takes those patch embeddings, models spatial and morphological correlations between them, and produces a core-level embedding that a classifier maps to one of four sarcoma classes. The combination matters because it lets the model exploit in-domain visual-language knowledge without updating the backbone, so training only touches the aggregator and the final classifier.

What would settle it

Re-split the cores so that no two cores from the same patient or tissue microarray lie on opposite sides of the training/test boundary and recompute test accuracy; if the PLIP plus TransMIL accuracy drops to or below the fine-tuned VGG16 level, or falls well below 90.5%, the reported advantage is an artifact of split leakage.

Watch

Extended reading notes

Core claim

On its own test split, the paper's central discovery is that the best configuration is a frozen PLIP feature extractor feeding a TransMIL transformer aggregator, which correctly classifies 90.5% of cores among Ewing sarcoma, chondrosarcoma, GIST, and rhabdomyosarcoma, with an F1 score of 0.903. This outperforms VGG16 fine-tuned with BGAP (87.5% accuracy) and frozen VGG16 with BGAP (83.7%). The authors attribute the gain to using vision-language supervision on in-domain histopathology data, which produces better separated core embeddings, and to the transformer aggregator's use of spatial and morphological correlations among patches. They also report a sharp efficiency advantage: 2.6 million trainable parameters and 0.0155 seconds per training step for PLIP plus TransMIL versus 29.7 million parameters and 0.112 seconds for fine-tuned VGG16 with BGAP.

Load-bearing premise

The accuracy numbers rest on the assumption that the 60/15/25 split keeps every core from the same patient or tissue-microarray block entirely on one side of the split; the paper does not say this, and if cores from one tumor block appear in both training and test, the model could be memorizing staining and scanner artifacts instead of sarcoma features.

Editorial extensions

If this is right

  • Pathology-specific vision-language pretraining can replace ImageNet transfer learning for tissue-microarray classification, giving better accuracy while keeping the feature extractor frozen.
  • The choice of aggregator matters: with PLIP features, the transformer-based TransMIL aggregator beats average pooling, max pooling, and attention-based MIL on the validation set.
  • The pipeline needs 2.6 million trainable parameters instead of 29.7 million and about 0.0155 seconds instead of 0.112 seconds per training step, making it practical with limited annotated histopathology data.
  • The fine-tuned VGG16 baseline loses accuracy from validation to test, while the frozen-PLIP pipeline keeps its advantage, suggesting supervised fine-tuning overfits this small cohort.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the split is truly patient- or tissue-microarray-disjoint, the same frozen-feature-plus-transformer recipe should transfer to other small round cell tumors, where annotated biopsies are scarce and whole-slide images are the practical input.
  • Editorial inference: a direct test of the mechanism would swap PLIP for another in-domain pathology vision-language model, such as CONCH, and check whether the accuracy gain persists; the paper's argument predicts it should.
  • Editorial inference: the compute comparison is reported at batch size one on a single GPU type, so the speed advantage should be re-measured at deployment batch sizes and on CPU or edge hardware before drawing conclusions about clinical cost.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a multiple-instance learning (MIL) pipeline for classifying Ewing sarcoma versus three morphologically similar sarcomas (chondrosarcoma, GIST, rhabdomyosarcoma) from digitized tissue microarray cores. It compares frozen pathology vision-language features (PLIP) with ImageNet-pretrained VGG16 features, both frozen and fine-tuned, combined with several aggregators (BGAP, BGMP, attention, TransMIL). On a single 60/15/25 split of the dataset, the authors report that PLIP features with a TransMIL aggregator reach 90.5% test accuracy, outperforming fine-tuned VGG16 with BGAP at 87.5% accuracy, while using substantially fewer trainable parameters (2.6M vs 29.7M) and lower per-step compute (0.0155 s vs 0.112 s). The paper concludes that frozen in-domain vision-language features plus a trained transformer aggregator are both more accurate and more efficient than fine-tuning a supervised CNN on this task.

Significance. If the reported results hold, the paper makes a useful empirical contribution: it shows that a frozen pathology-specific vision-language feature extractor can outperform fine-tuned ImageNet features in a small, imbalanced histopathology dataset, with large savings in training cost. The ablation across aggregators is informative, and reporting computational cost and trainable parameters is a strength. The main significance, however, depends on the test split being independent at the patient or TMA level and on the accuracy gap being statistically reliable; neither is currently established. The paper also does not release code or data, which limits reproducibility for a result whose validity hinges on evaluation protocol details.

major comments (3)
  1. [Section 3 and Section 5.1] The split unit is not specified. The dataset is described in terms of TMAs and cores, but Section 5.1 states only that the database was partitioned into 60% training, 15% validation, and 25% test proportions. If cores from the same TMA or the same patient appear in both training and test, the model can memorize staining, sectioning, and scanner artifacts, which would inflate the reported 90.5% accuracy and invalidate the comparison between PLIP and VGG16. Please specify whether the split was performed at the patient, TMA, or core level, report the number of patients or TMAs in each partition, and, ideally, provide a patient-exclusive or TMA-exclusive split to confirm the results.
  2. [Table 2 and Section 5.3] The central comparison rests on a single split with no confidence intervals, error bars, or significance tests. With roughly 487 test cores (25% of 1947 cores), the 90.5% versus 87.5% accuracy difference corresponds to only about 15 test cores, which could easily arise from sampling noise. Please report bootstrap confidence intervals, per-class confidence intervals, or repeated cross-validation results, and state whether the PLIP advantage is statistically significant.
  3. [Section 5.2 and Figure 4] The overfitting claim for fine-tuned VGG16 is supported only by a qualitative statement that its validation performance decreased on the test set. The paper does not report validation accuracies, training curves, early-stopping criteria, or the number of epochs for any configuration. Without this information, the reader cannot assess whether the fine-tuned VGG result is a robust baseline or an artifact of incomplete training or hyperparameter selection. Please report the validation results corresponding to Figure 4 in numeric form and describe the training protocol in enough detail to reproduce it.
minor comments (5)
  1. [Figure 3 caption] The caption states "the five neoplasms under study," but Table 1 lists four classes (EWING, COND, GIST, RHABDO); please correct the caption to four.
  2. [Section 4, Equation (1)] The symbol K is used in the loss normalization but is not defined in the problem formulation; please define K as the number of classes.
  3. [Section 5.1] The learning rate is reported as "between 1e-5 and 5e-5," but the paper does not specify which configuration used which value; please report the learning rate for each feature extractor and aggregator combination.
  4. [Section 5.1] The class-weight scheme is described only as weights inversely proportional to the number of instances per class; please provide the exact formula or the weight values used for each class.
  5. [Table 2 and Figure 5] Per-class sensitivity and precision would make the comparison more informative, especially given the class imbalance; the confusion matrix in Figure 5 is small and the numeric values are not reproduced in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the accuracy comparison is an empirical benchmark against external pretrained feature extractors, and the reported test metrics are measured outputs rather than quantities defined by the paper's own equations.

full rationale

The paper's central claim is an empirical comparison of feature extractors and aggregators on a held-out test set: frozen PLIP with TransMIL reaches 90.5% accuracy versus 87.5% for fine-tuned VGG16 with BGAP. These numbers are test-set measurements, not derived quantities. The loss in Eq. (1) is standard categorical cross-entropy, and no parameter is fitted to the test labels and then reported as a prediction. The feature extractors are external pretrained models (ImageNet VGG16 and PLIP trained on OpenPath), and the aggregators (BGAP, TransMIL) are taken from cited external work. The only apparent self-citation, Ref. [16], supports the general bag-embedding MIL framing and does not carry the accuracy claim, so it is not load-bearing. The ambiguity about whether the train/validation/test split was performed by core, TMA, or patient is a real validity risk for generalization, but it is not a circularity: it concerns possible information leakage in evaluation, not equivalence between the paper's inputs and outputs. No definitional reduction, fitted-input-called-prediction, or self-citation chain forces the reported results, so the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper contributes an empirical comparison and introduces no new parameters, axioms, or entities beyond standard deep learning choices. The listed free parameters are the hyperparameters and split choices that the reported accuracy depends on but that the paper does not justify or vary systematically.

free parameters (5)
  • Learning rate = 1e-5 to 5e-5 (per configuration)
    Chosen per configuration without a reported sweep or criterion; optimization behavior and results depend on it.
  • Train/validation/test split ratio = 60/15/25
    Arbitrary split proportions; no cross-validation or patient-level splitting is described, so variance and potential leakage are uncontrolled.
  • Class weight scheme = Inversely proportional to instance count per class
    Hand-selected to address imbalance; no comparison to unweighted or other weighting schemes.
  • TransMIL hyperparameters = Not reported
    Number of transformer layers, heads, and dropout are not given, and aggregator capacity affects the headline accuracy.
  • Training epochs / early stopping = Not reported
    Training duration is absent from the implementation details, so the reported results cannot be tied to a reproducible stopping rule.
assumptions (4)
  • domain assumption Bag-embedding MIL assumption
    The paper assumes a core-level label arises from the combination of all patch embeddings, not from any single positive patch, following reference [16] in Section 4.
  • domain assumption PLIP feature transferability
    The method assumes features learned from OpenPath image-caption pairs transfer to TMA histology without any fine-tuning; this is tested on only one private dataset.
  • domain assumption Histological labels are correct
    The dataset uses genetically confirmed diagnoses as ground truth, but the paper does not describe independent pathology review or adjudication of the TMA cores.
  • domain assumption Exclusion of non-informative cores is unbiased
    The paper states some cores were non-informative due to artifacts or necrosis, but does not describe criteria or how this exclusion might interact with class balance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma." pith.science (2026). https://pith.science/paper/4OZTHZ7A

@misc{pith2026250108042,
  author       = {Pith},
  title        = {Pith review of: Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4OZTHZ7A}},
  note         = {Machine review of arXiv:2501.08042}
}
read the original abstract

Ewing's sarcoma (ES), characterized by a high density of small round blue cells without structural organization, presents a significant health concern, particularly among adolescents aged 10 to 19. Artificial intelligence-based systems for automated analysis of histopathological images are promising to contribute to an accurate diagnosis of ES. In this context, this study explores the feature extraction ability of different pre-training strategies for distinguishing ES from other soft tissue or bone sarcomas with similar morphology in digitized tissue microarrays for the first time, as far as we know. Vision-language supervision (VLS) is compared to fully-supervised ImageNet pre-training within a multiple instance learning paradigm. Our findings indicate a substantial improvement in diagnostic accuracy with the adaption of VLS using an in-domain dataset. Notably, these models not only enhance the accuracy of predicted classes but also drastically reduce the number of trainable parameters and computational costs.

Figures

Figures reproduced from arXiv: 2501.08042 by the authors.

Figure 1
Figure 1. (a) Extraction process of cores from digitized microarrays (TMAs); (b) chon￾drosarcoma (COND), (c) Ewing’s sarcoma, (d) gastrointestinal stroma (GIST) and (e) Rhabdomyosarcoma (RHABDO) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Method overview. Multi-class classification of different sarcomas under a mul￾tiple instance learning paradigm. In A., feature extraction is described using a CNN￾based model, whereas in B., it is based on visual language models. Patch-level feature extraction. In the MIL paradigm, the initial step in￾volves patch-level feature extraction. Typically, convolutional neural networks (CNN) architectures, such as VGG16 p… view at source ↗
Figure 3
Figure 3. TSNE visualization of features embeddings. Core embedding representation from patch features extracted with VGG16 (a) and PLIP (b) for the five neoplasms under study. Note that the core embedding in both cases was calculated by averaging all the patch embedding. We explore various embedding aggregation strategies within the MIL frame￾work: batch global average pooling (BGAP), max pooling (BGMP), attention mechanisms… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of different embedding aggregators at the core level in the valida￾tion set [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrix using the test set. (a). VGG16 trained using BGAP as the embedding aggregator. (b) VGG16 frozen using BGAP as embedding aggregation. (c) PLIP frozen using TransMIL as embedding aggregation. 7 Fundings This work has received funding from the Spanish Min…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

  1. [1]

    The epidemiology of sarcoma,

    Z. Burningham, M. Hashibe, L. Spector, and J. D. Schiffman, “The epidemiology of sarcoma,” Clinical sarcoma research, vol. 2, no. 1, pp. 1–16, 2012

  2. [2]

    Ewing tumour: incidence, prognosis and treatment options,

    M. Paulussen, B. Fr¨ ohlich, and H. J¨ urgens, “Ewing tumour: incidence, prognosis and treatment options,” Paediatric drugs, vol. 3, pp. 899–913, 2001

  3. [3]

    Changes in incidence and survival of ewing sarcoma patients over the past 3 decades: Surveillance epidemiology and end results data,

    N. Esiashvili, M. Goodman, and R. B. Marcus, “Changes in incidence and survival of ewing sarcoma patients over the past 3 decades: Surveillance epidemiology and end results data,” Journal of pediatric hematology/oncology , vol. 30, no. 6, pp. 425–430, 2008

  4. [4]

    Two-phase deep learning algorithm for detection and differentiation of ewing sarcoma and acute osteomyeli- tis in paediatric radiographs,

    S. Consalvo, F. Hinterwimmer, J. Neumann, M. Steinborn, M. Salzmann, F. Seidl, U. Lenze, C. Knebel, D. Rueckert, and R. H. Burgkart, “Two-phase deep learning algorithm for detection and differentiation of ewing sarcoma and acute osteomyeli- tis in paediatric radiographs,” Anticancer Research, vol. 42, no. 9, pp. 4371–4380, 2022

  5. [5]

    Machine learning for rhabdomyosarcoma histopathology,

    A. O. Frankel, M. Lathara, C. Y. Shaw, O. Wogmon, J. M. Jackson, M. M. Clark, N. Eshraghi, S. E. Keenen, A. D. Woods, R. Purohit et al. , “Machine learning for rhabdomyosarcoma histopathology,” Modern Pathology, vol. 35, no. 9, pp. 1193– 1203, 2022

  6. [6]

    Deep learning of rhab- domyosarcoma pathology images for classification and survival outcome predic- tion,

    X. Zhang, S. Wang, E. R. Rudzinski, S. Agarwal, R. Rong, D. A. Barkauskas, O. Daescu, L. F. Cline, R. Venkatramani, Y. Xie et al. , “Deep learning of rhab- domyosarcoma pathology images for classification and survival outcome predic- tion,” The American Journal of Pathology , vol. 192, no. 6, pp. 917–925, 2022. 10 A. Pastor-Naranjo et al

  7. [7]

    Predicting molecular subtype and survival of rhabdomyosarcoma patients using deep learning of h&e images: A report from the children’s oncology group,

    D. Milewski, H. Jung, G. T. Brown, Y. Liu, B. Somerville, C. Lisle, M. Ladanyi, E. R. Rudzinski, H. Choo-Wosoba, D. A. Barkauskas et al., “Predicting molecular subtype and survival of rhabdomyosarcoma patients using deep learning of h&e images: A report from the children’s oncology group,” Clinical Cancer Research, vol. 29, no. 2, pp. 364–378, 2023

  8. [8]

    Deep learning based automated tool for cancer diagnosis from bone histopathology images,

    S. R. Vandana B.S, “Deep learning based automated tool for cancer diagnosis from bone histopathology images,” International Conference on Intelligent Technologies (CONIT), 2021

Show all 21 references
  1. [9]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

  2. [10]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in International conference on machine learning. PMLR, 2021, pp. 4904–4916

  3. [11]

    Contrastive learning of medical visual representations from paired images and text,

    Y. Zhang, H. Jiang, Y. Miura, C. D. Manning, and C. P. Langlotz, “Contrastive learning of medical visual representations from paired images and text,” inMachine Learning for Healthcare Conference. PMLR, 2022, pp. 2–25

  4. [12]

    Towards a visual-language foundation model for computational pathology,

    M. Y. Lu, B. Chen, D. F. Williamson, R. J. Chen, I. Liang, T. Ding, G. Jaume, I. Odintsov, A. Zhang, L. P. Leet al., “Towards a visual-language foundation model for computational pathology,” arXiv preprint arXiv:2307.12914 , 2023

  5. [13]

    A visual– language foundation model for pathology image analysis using medical twitter,

    Z. Huang, F. Bianchi, M. Yuksekgonul, T. J. Montine, and J. Zou, “A visual– language foundation model for pathology image analysis using medical twitter,” Nature medicine, vol. 29, no. 9, pp. 2307–2316, 2023

  6. [14]

    Detection of breast cancer from whole slide histopathological images using deep multiple instance cnn,

    K. Das, S. Conjeti, J. Chatterjee, and D. Sheet, “Detection of breast cancer from whole slide histopathological images using deep multiple instance cnn,” IEEE Ac- cess, vol. 8, pp. 213 502–213 511, 2020

  7. [15]

    Self-learning for weakly supervised gleason grading of local patterns,

    J. Silva-Rodriguez, A. Colomer, J. Dolz, and V. Naranjo, “Self-learning for weakly supervised gleason grading of local patterns,” IEEE journal of biomedical and health informatics, vol. 25, no. 8, pp. 3094–3104, 2021

  8. [16]

    An attention-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in whole slide images,

    R. Del Amor, L. Launet, A. Colomer, A. Moscard´ o, A. Mosquera-Zamudio, C. Mon- teagudo, and V. Naranjo, “An attention-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in whole slide images,” Artificial intelligence in medicine, vol. 121, p. 102197, 2021

  9. [17]

    Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution,

    Y. Zhao, F. Yang, Y. Fang, H. Liu, N. Zhou, J. Zhang, J. Sun, S. Yang, B. Menze, X. Fan et al. , “Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution,” in Proceedings of the IEEE/CVF Conference on Comp...

  10. [18]

    Classification of volu- metric images using multi-instance learning and extreme value theorem,

    R. Tennakoon, G. Bortsova, S. Ørting, A. K. Gostar, M. M. Wille, Z. Saghir, R. Hoseinnezhad, M. de Bruijne, and A. Bab-Hadiashar, “Classification of volu- metric images using multi-instance learning and extreme value theorem,” IEEE Transactions on Medical Imaging, vol. 39, no....

  11. [19]

    Attention-based deep multiple instance learning,

    M. Ilse, J. M. Tomczak, and M. Welling, “Attention-based deep multiple instance learning,” in 35th International Conference on Machine Learning (ICML) , 2018

  12. [20]

    Clinical- grade computational pathology using weakly supervised deep learning on whole slide images,

    G. Campanella, M. G. Hanna, L. Geneslaw, A. Miraflor, V. Werneck Krauss Silva, K. J. Busam, E. Brogi, V. E. Reuter, D. S. Klimstra, and T. J. Fuchs, “Clinical- grade computational pathology using weakly supervised deep learning on whole slide images,” Nature Medicine, vol. 25,...

  13. [21]

    Transmil: Transformer based correlated multiple instance learning for whole slide image classification,

    Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji et al., “Transmil: Transformer based correlated multiple instance learning for whole slide image classification,” Advances in neural information processing systems , vol. 34, pp. 2136–2147, 2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.