Pith. sign in

REVIEW 3 major objections 5 minor 21 references

Scalable Whole Slide Image Representation Using K-Mean Clustering and Fisher Vector Aggregation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read This paper proposes that clustering patch embeddings and encoding each cluster with a Fisher vector yields a compact whole-slide representation that matches or beats attention-based baselines on four pathology classification tasks.

desk verdict The K-means + Fisher-vector idea is a reasonable incremental twist on DFVC, but test-set hyperparameter tuning and an underspecified FV equation sink the reported accuracy claims. read the letter →

arxiv 2501.12085 v1 pith:MNBFAGTD submitted 2025-01-21 cs.CV cs.AIcs.LG

classification cs.CVcs.AIcs.LG
keywords wholeslideimageK-meansclusteringFishervectorpatch-basedfeaturesHER2scoringEGFRmutationpredictionlymphnodemetastasisdigitalpathology
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to turn a gigapixel whole slide image into one compact vector for classification: cut the slide into patches, embed each patch with a pre-trained network, group the embeddings with K-means, and summarize each cluster with a Fisher vector computed from a Gaussian mixture model. The cluster Fisher vectors are concatenated and fed to a classifier. The authors claim this captures both local tissue appearance and global slide-level structure, and they report that it outperforms attention-based multiple-instance learning and other baselines on HER2 score prediction, HER2 status classification, EGFR mutation prediction, and lymph node metastasis detection. A sympathetic reader would care because a compact, permutation-invariant slide representation would remove the need to train on all patches jointly and would make gigapixel pathology images tractable for standard classifiers.

What carries the argument

The key machinery is the per-cluster Fisher vector. After K-means groups patch embeddings into K=10 clusters, each cluster's descriptor distribution is modeled with a Gaussian mixture model of m=5 centers, and the Fisher vector function, defined in the paper as scaled first and second moments of each embedding relative to the cluster centers with fixed scale constants c_j and ĉ_j and fixed mixture parameters π_m=0.2 and σ_m=0.1, summarizes the cluster. Concatenating these per-cluster Fisher vectors yields the slide-level representation, which is then classified by an attention multi-instance learning block or other classifier. The role of this machinery is to compress thousands of patch embeddings into a fixed-length vector that is permutation invariant and preserves both the identity of tissue patterns through cluster membership and their variation through the moments.

What would settle it

Compute the standard Fisher vector as the gradient of the GMM log-likelihood with respect to the mixture parameters for the same clusters, replace the paper's formula with it, and rerun the Warwick and TCGA-BRCA experiments. If accuracy does not change, the special formula is not the source of the result; if accuracy drops, the published formula is doing the work and needs a derivation. Also, shuffle cluster assignments before computing the Fisher vectors: if performance stays, clustering is not contributing.

Watch

Extended reading notes

Core claim

The central claim is that clustering patch embeddings before Fisher-vector aggregation makes a whole-slide representation both more informative and more scalable than pooling all patches through an attention mechanism. On the Warwick HER2 challenge the method reaches 0.72 accuracy for three-class scoring and 0.80 accuracy for HER2+/HER2- classification; on TCGA-BRCA HER2 status it reaches 0.86 accuracy; on TCGA-LUAD EGFR mutation prediction 0.84 accuracy; and on CAMELYON17 metastasis detection 0.77 accuracy, in each case matching or exceeding the best compared baseline with the same or a lighter feature extractor. The authors interpret these results as evidence that per-cluster distribution summaries preserve the heterogeneity of tissue while a single concatenated vector remains manageable for a classifier.

Load-bearing premise

The whole representation depends on the specific Fisher-vector formula and the fixed GMM parameters in Section 2; if that formula is not a faithful summary of each cluster's patch distribution, the method is not actually doing Fisher-vector aggregation and its reported gains would need a different explanation.

Editorial extensions

If this is right

  • A single compact vector per slide means classifier training can be done with ordinary fully connected heads rather than patch-level attention, lowering memory use for large cohorts.
  • Because the pipeline is agnostic to the patch encoder, improvements in pre-trained encoders should transfer directly to slide-level accuracy without re-architecting the aggregation.
  • The method addresses multiple diagnostic tasks (HER2 scoring, mutation prediction, metastasis detection) with the same aggregation recipe, suggesting one representation can serve several clinical questions.
  • Using K-means clusters to group semantically similar regions gives a natural way to inspect which tissue patterns drive a prediction, since each cluster corresponds to a set of patches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed GMM constants and the nonstandard Fisher vector formula suggest the method may work as a cluster-weighted moment pooling rather than a true probabilistic Fisher vector; renaming it and fitting the GMM would make the contribution easier to verify and compare.
  • Because the representation is permutation invariant and compact, it should extend to other gigapixel imaging domains, such as satellite or full-section electron microscopy, wherever patch-level features can be clustered.
  • A testable extension is to replace K-means with soft clustering or with a variational autoencoder's latent clusters; if accuracy holds, the essential ingredient is the per-cluster moment aggregation rather than the specific algorithm.
  • The reported gains over attention baselines might be partially attributable to the stronger feature extractors used in the proposed pipeline (e.g., RegNetY, MoCo-v2); an apples-to-apples comparison holding the backbone fixed would identify how much the aggregation alone contributes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a scalable whole slide image (WSI) classification pipeline. Patches extracted from a WSI are embedded with a pretrained CNN/transformer, the patch embeddings are grouped with K-means clustering, and each cluster is summarized by a custom 'Fisher vector' computed from the patch embeddings. These per-cluster vectors are concatenated into a single WSI-level representation and classified by an attention-based multiple instance learning (AMIL) block. Experiments are reported on Warwick HER2 scoring and HER2 status, TCGA-BRCA HER2 status, TCGA-LUAD EGFR mutation prediction, and CAMELYON17 metastasis detection. The abstract claims that the method 'captures local and global tissue structures and yields robust performance ... demonstrating superior accuracy and scalability compared to other approaches.'

Significance. The topic is timely, and the idea of combining clustering with Fisher-vector-style codewords for WSI classification is plausible and worth exploring. The paper includes a relatively broad evaluation across four datasets and compares with several baselines, which is a strength. However, the evaluation protocol is compromised by an explicit statement that hyperparameters were chosen using the test dataset, and the Fisher vector equation is mathematically underspecified and is not a standard Fisher vector. These issues are load-bearing because the reported accuracy gains in Tables 1–4 are the sole evidence for the central claim. If the evaluation were redone with a clean validation protocol and a correctly defined Fisher vector, the method could become a useful contribution, but as presented the evidence does not support the claimed superiority.

major comments (3)
  1. [Section 3, 'Hyperparameters' paragraph] The text states: 'These hyperparameters are decided based on performance for test dataset.' This is an explicit admission that test-set labels were used to select k, m, π_m, σ_m, feature scaling range, jitter level, mixup alpha, and the backbone per task. As a result, the accuracy, AUC, and F1 numbers in Tables 1–4 may reflect overfitting to the test sets rather than genuine generalization performance. The central claim of the abstract is unsupported without a clean model-selection protocol (e.g., a held-out validation set or nested cross-validation) and re-reporting of all test metrics after fixing hyperparameters without looking at test labels.
  2. [Section 2, Fisher vector equation] The Fisher vector function is not the standard Fisher vector (the gradient of the log-likelihood with respect to GMM parameters). The constants c_j and ĉ_j are never defined, the responsibilities s_ij are never defined, and fixing π_m=0.2 and σ_m=0.1 as scalar constants is inconsistent with fitting a Gaussian mixture model. The notation '1/n N∑ i=1' also mixes n and N. Because this equation is the core of the proposed representation, the method is not reproducible and its behavior cannot be assessed. Please provide a complete, standard definition of the Fisher vector (e.g., following Sánchez et al., 'Image classification with the Fisher vector: Theory and practice') or derive the formula from a specified probabilistic model.
  3. [Tables 1–3 and Section 3, 'Ablation and validation studies'] In Tables 1, 2, and 3, the proposed method uses a different backbone feature extractor (EfficientNetV2-S, RegNetY-3.2GF, MoCo-v2, respectively) than the AMIL or Anand et al. baselines (ResNet-50 in each table). The reported accuracy differences could therefore be caused by the backbone rather than by the K-means and Fisher-vector aggregation. To support the claim that the proposed aggregation method is superior, the authors should compare against baselines using the same backbone features, or include an ablation that holds the backbone fixed while varying only the aggregation method. Table 4 is better in this respect but does not rescue the earlier tables.
minor comments (5)
  1. [Section 2, equation display] The displayed formula '1/n N∑ i=1 FV(...)' uses both n and N without defining either; this appears to be a typo and should be corrected.
  2. [Section 2, 'permutation-invariant classifier'] The text says the concatenated vector is 'input into a permutation-invariant classifier,' but MLP, Swin Tiny, and ConvNeXt are not permutation-invariant, while AMIL is. Please clarify which classifier is used for the reported results.
  3. [Section 3, hyperparameter selection] The elbow method is mentioned as determining the optimal number of clusters, but the next sentence sets k=10 for all datasets; please explain whether the elbow method was used or whether k was set a priori.
  4. [Reference [20]] The SimCLR reference lists 'Tianyang Chen' as the first author; the correct first author is Ting Chen (Chen, T., Kornblith, S., Norouzi, M., Hinton, G.).
  5. [Table 3 and references [16,18]] The comparison with Sekhar et al. [18] uses the same MoCo-v2 backbone as the proposed method, but the differences between the two methods are not described; adding a brief explanation of how the proposed approach differs from that prior work would strengthen the comparison.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported test-set superiority is undermined because hyperparameters and encoders were explicitly selected on the test data, making the headline accuracy claim a fitted result rather than an independent prediction.

  1. fitted input called prediction [Section 3 (Datasets, Experiments, and Results), hyperparameter/ablation paragraph after Table 4]
    "For our experiments, we set k = 10clusters across all datasets. Fisher vector encoding was applied with five m = 5 centers (performed ablation study with m=10,15 and 3 but found best at m=5), with encoding parameters πm = 0.2 and σm = 0.1 as in [5]. These hyperparameters are decided based on performance for test dataset."

    The central claim is superior accuracy on the Warwick, TCGA-BRCA, TCGA-LUAD, and CAMELYON17 test sets (Tables 1-4). Those same test sets were used to choose k, m, π, σ, augmentation settings, feature encoders, and classifiers ('best-performing encoder was selected for each task'; 'decided based on performance for test dataset'). The test-set numbers reported as predictions are therefore the output of a selection procedure optimized on those very labels; the comparison is not an independent evaluation. This is fitted-input-called-prediction: the fitted hyperparameters determine the reported test predictions, so the claimed superiority is constructed from the evaluation criterion itself.

full rationale

The paper's derivation chain is empirical rather than formal. The only load-bearing circular step is the explicit use of the test dataset to decide hyperparameters and per-task encoders, after which the same test datasets are reported as the evaluation. This invalidates the abstract's 'superior accuracy' claim as independent evidence. The Fisher-vector formula deviates from the standard definition and leaves constants (c_j, ĉ_j) unspecified, but that is an interpretability/correctness issue, not a circularity. Self-references to prior work by the same group ([8], [16], [18]) appear only as baselines or related datasets and do not carry the argument. With a clean validation-based model-selection protocol, the method could be genuinely superior; as written, the headline results are statistically forced by test-set fitting, hence score 6 rather than 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The method's performance depends on a large number of ad hoc choices (k, m, π, σ, data augmentation, per-task backbone), most of which were selected using the test set. No new entities are introduced.

free parameters (8)
  • k (number of K-means clusters) = 10
    Set to 10 across all datasets despite claiming the elbow method determines it; chosen on test performance.
  • m (number of GMM/FV centers) = 5
    Ablation over 3, 10, 15 found 5 best; decision based on test dataset performance.
  • π_m (mixture weight parameter) = 0.2
    Stated as in [5], but for 5 components these sum to 1; not derived.
  • σ_m (standard deviation parameter) = 0.1
    Taken from [5]; no justification for this fixed value.
  • feature scaling range = 0.9 to 1
    Augmentation parameter; no justification.
  • jitter level = 0.01
    Augmentation parameter; no justification.
  • mixup alpha = 0.2
    Augmentation parameter; no justification.
  • backbone per task = EfficientNetV2-S / RegNetY-3.2GF / MoCo-v2
    Best-performing encoder selected per task based on test performance.
assumptions (5)
  • ad hoc to paper ImageNet-pretrained features transfer to histopathology patches without fine-tuning.
    Used throughout Section 3 with no validation that this transfer holds for H&E slides.
  • domain assumption Fixed 512x512 patches capture diagnostically relevant local and global structure.
    Assumed in Section 2; patch size is not varied or justified.
  • domain assumption K-means clustering on the embeddings yields meaningful tissue clusters.
    Assumed in Section 2; no qualitative or quantitative evidence of cluster semantics.
  • ad hoc to paper The Fisher vector equation in Section 2 is a valid encoding of the cluster distribution.
    The formula uses ad hoc constants and lacks the standard GMM gradient derivation.
  • ad hoc to paper Selected hyperparameters generalize across datasets.
    Parameters were chosen on the test set, so generalization is assumed without evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable Whole Slide Image Representation Using K-Mean Clustering and Fisher Vector Aggregation." pith.science (2026). https://pith.science/paper/MNBFAGTD

@misc{pith2026250112085,
  author       = {Pith},
  title        = {Pith review of: Scalable Whole Slide Image Representation Using K-Mean Clustering and Fisher Vector Aggregation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MNBFAGTD}},
  note         = {Machine review of arXiv:2501.12085}
}
read the original abstract

Whole slide images (WSIs) are high-resolution, gigapixel sized images that pose significant computational challenges for traditional machine learning models due to their size and heterogeneity.In this paper, we present a scalable and efficient methodology for WSI classification by leveraging patch-based feature extraction, clustering, and Fisher vector encoding. Initially, WSIs are divided into fixed size patches, and deep feature embeddings are extracted from each patch using a pre-trained convolutional neural network (CNN). These patch-level embeddings are subsequently clustered using K-means clustering, where each cluster aggregates semantically similar regions of the WSI. To effectively summarize each cluster, Fisher vector representations are computed by modeling the distribution of patch embeddings in each cluster as a parametric Gaussian mixture model (GMM). The Fisher vectors from each cluster are concatenated into a high-dimensional feature vector, creating a compact and informative representation of the entire WSI. This feature vector is then used by a classifier to predict the WSI's diagnostic label. Our method captures local and global tissue structures and yields robust performance for large-scale WSI classification, demonstrating superior accuracy and scalability compared to other approaches.

Figures

Figures reproduced from arXiv: 2501.12085 by the authors.

Figure 1
Figure 1. Patches from WSIs are encoded into embeddings, clustered via K-means, and transformed into Fisher vectors [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

  1. [1]

    Understanding of a convolutional neural network

    Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. Understanding of a convolutional neural network. In International Conference on Engineering and Technology (ICET), pages 1–6. IEEE, 2017

  2. [2]

    Transformers in vision: A survey

    Salman Khan, Muzammal Naseer, Munawar Hayat, and et al. Transformers in vision: A survey. ACM Computing Surveys (CSUR), 54(10s):1–41, 2022

  3. [3]

    Least squares quantization in pcm

    Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on information theory, 28(2):129–137, 1982

  4. [4]

    Deep fisher vector coding for whole slide image classification

    Amir Akbarnejad, Nilanjan Ray, and Gilbert Bigras. Deep fisher vector coding for whole slide image classification. In IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 243–246, 2021

  5. [5]

    K. S. Arun, V . K. Govindan, and S. D. Madhu Kumar. Enhanced bag of visual words representations for content based image retrieval: a comparative study. Artificial Intelligence Review, 53(3):1615–1653, 2020

  6. [6]

    Her2 challenge contest: A detailed assessment of automated her2 scoring algorithms in whole slide images of breast cancer tissues

    Talha Qaiser, Abhik Mukherjee, Chaitanya PB, Sai Munugoti, and et al. Her2 challenge contest: A detailed assessment of automated her2 scoring algorithms in whole slide images of breast cancer tissues. Histopathology, 72, 05 2017

  7. [7]

    The cancer genome atlas (tcga) - breast invasive carcinoma (brca)

    Genomic Data Commons. The cancer genome atlas (tcga) - breast invasive carcinoma (brca). https://portal. gdc.cancer.gov/projects/TCGA-BRCA, 2024. Accessed: 2024-10-07

  8. [8]

    Egfr mutation prediction of lung biopsy images using deep learning, 2023

    Ravi Kant Gupta, Shivani Nandgaonkar, Nikhil Cherian Kurian, Swapnil Rane, and et al. Egfr mutation prediction of lung biopsy images using deep learning, 2023

Show all 21 references
  1. [9]

    From detection of individual metastases to classification of lymph node status at the patient level: The camelyon17 challenge

    Peter Bandi, Oscar Geessink, Quirine Manson, Marcory van Dijk, and et al. From detection of individual metastases to classification of lymph node status at the patient level: The camelyon17 challenge. IEEE Transactions on Medical Imaging, PP:1–1, 2018

  2. [10]

    Efficient quality control of whole slide pathology images with human-in-the-loop training

    Abhijeet Patil, Harsh Diwakar, Jay Sawant, Nikhil Cherian Kurian, and et al. Efficient quality control of whole slide pathology images with human-in-the-loop training. Journal of Pathology Informatics, 14:100306, 2023. 4

  3. [11]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yuankai Qi, and et al. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10012– 10022, 2021

  4. [12]

    Attention-based deep multiple instance learning

    Maxim Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. In Proceedings of the International Conference on Machine Learning (ICML), pages 2127–2136. PMLR, 2018

  5. [13]

    Convnext: Revisiting convolutional neural networks for visual recognition

    Zhuang Liu, Jiangfeng Wang, Xie Si, and et al. Convnext: Revisiting convolutional neural networks for visual recognition. arXiv preprint arXiv:2201.03545, 2022

  6. [14]

    Deep residual learning for image recognition

    S Jian, H Kaiming, R Shaoqing, and Z Xiangyu. Deep residual learning for image recognition. InIEEE Conference on Computer Vision & Pattern Recognition, pages 770–778, 2016

  7. [15]

    Mingxing Tan and Quoc V . Le. Efficientnetv2: Smaller models and faster training.CoRR, abs/2104.00298, 2021

  8. [16]

    Deep learning to estimate human epidermal growth factor receptor 2 status from hematoxylin and eosin-stained breast tissue images.Journal of Pathology Informatics, 11(1):19, 2020

    Deepak Anand, Nikhil Cherian Kurian, Amit Sethi, and et al. Deep learning to estimate human epidermal growth factor receptor 2 status from hematoxylin and eosin-stained breast tissue images.Journal of Pathology Informatics, 11(1):19, 2020

  9. [17]

    Girshick, Kaiming He, and Piotr Dollár

    Ilija Radosavovic, Raj Prateek Kosaraju, Ross B. Girshick, Kaiming He, and Piotr Dollár. Designing network design spaces. CoRR, abs/2003.13678, 2020

  10. [18]

    Her2 and fish status prediction in breast biopsy h&e-stained images using deep learning

    Ardhendu Sekhar, Vrinda Goel, Garima Jain, Abhijeet Patil, and et al. Her2 and fish status prediction in breast biopsy h&e-stained images using deep learning. arXiv preprint arXiv:2408.13818, 2024

  11. [19]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020

  12. [20]

    A simple framework for contrastive learning of visual representations

    Tianyang Chen, Simon Kornblith, Mohammad Noroozi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning (ICML), pages 1597–1607. PMLR, 2020

  13. [21]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 5

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.