REVIEW 4 major objections 5 minor 33 references
Uncertainty Awareness Enables Efficient Labeling for Cancer Subtyping in Digital Pathology
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Using an evidential-uncertainty score to pick which images to label, a cancer-subtyping model reaches state-of-the-art accuracy with 1–10% of annotations.
desk verdict A useful but under-powered combination of evidential uncertainty and active learning for pathology; the headline 9%-labels SOTA claim needs multiple seeds and a proper acquisition-function comparison before it can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is an evidential-uncertainty score built on the theory of belief functions and subjective logic. Each image passes through the network to produce a non-negative evidence vector $e_k$ for $k=1,\dots,K$ classes; belief masses are $b_k = e_k/S$ and the total uncertainty is $u = K/S$, where $S = \sum_k e_k + 1$. These quantities define a Dirichlet distribution over class probabilities with parameters $\alpha_k = e_k + 1$. The training loss minimizes the expected sum-of-squares error over that Dirichlet and adds an annealed Kullback–Leibler term that suppresses evidence for wrong classes. The resulting uncertainty score serves two roles: it flags predictions the model is likely to get wrong, and it is the acquisition function that selects which unlabeled patches a pathologist should label next.
What would settle it
Run the same out-domain UA-AL protocol on NCT100k with five different random seeds for the initial 1% labeled set and for the ordering of the uncertainty-ranked queries. If the reported 9%-label accuracy of about 97% varies by more than a few points, or drops below the 100%-label baseline, the claimed SOTA result is not robust to the initial seed. A second, cheaper check: replace the evidential uncertainty score with Shannon entropy as the acquisition function while keeping everything else fixed; if entropy matches or beats evidential uncertainty, the specific evidence-based ranking is not what drives the gain.
Extended reading notes
Core claim
The central discovery is that an evidential-uncertainty score—computed by replacing the classifier's softmax with a ReLU evidence vector and fitting a Dirichlet distribution over class probabilities—is an effective acquisition function for active learning in histopathology. The so-called UA-SimCLRv2 model uses the total uncertainty $u = K/S$, with $S$ the sum of class evidence plus one, to rank unlabeled patches and ask a pathologist to label the most uncertain ones each round. Starting from a random 1% annotated seed, the authors report that this querying strategy reaches parity with the state of the art at 2–3% labels and exceeds it at 9% labels on the NCT100k dataset, outperforming even the model trained on all 100% of labels. On the binary PCam dataset, the model is also state-of-the-art at 10% and 1% label settings, and the paper interprets this as evidence that uncertainty-aware selection, not just the self-supervised backbone, is what makes low-label training work.
Load-bearing premise
The evidential uncertainty score reliably ranks how valuable each unlabeled image is for reducing the model's error, so that labeling the top 1% most uncertain images is a sound proxy for information gain.
Editorial extensions
If this is right
- On both benchmark datasets, UA-AL reaches accuracy comparable to or better than fully supervised state-of-the-art models using only 1–10% of the labels.
- In the out-domain pre-training setting on NCT100k, the model surpasses the previous best published classifier with 9% labels, and even beats the 100%-label baseline.
- Uncertainty-aware querying consistently beats random label selection, and the gap grows as more labels are added.
- The uncertainty score correlates with prediction error, meaning the same mechanism that drives labeling also tells a clinician when to distrust a prediction.
Reading between the lines
- Extending the paper's logic: the evidential score has not been tested against simpler acquisition functions such as Shannon entropy or BALD; a swap test would show whether the evidence-based ranking is the active ingredient rather than the self-supervised features.
- The single-seed evaluation on NCT100k leaves open the possibility that the 9%-label SOTA result depends on the particular initial 1% labeled set; a multi-seed repeat would settle that.
- The paper itself notes that at 1% labels UA-SimCLRv2 underperforms SimCLRv2; a natural extension is a hybrid acquisition function that mixes uncertainty with diversity-based sampling to avoid querying redundant patches.
- If the ranking is robust, the same uncertainty-guided querying should transfer to whole-slide image classification by using the fine-tuned encoder inside a multiple-instance learning framework, which the authors propose as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes UA-SimCLRv2, a SimCLRv2-based classifier augmented with evidential deep learning uncertainty, and uses the resulting uncertainty score as an acquisition function for active learning on two histopathology patch datasets (PCam and NCT-CRC-HE-100K). The authors report that with 1–10% of labels, particularly with 9% of out-domain NCT100k labels, UA-AL reaches 97.01% accuracy, surpassing quoted SOTA values and the 100%-label baseline. They also present t-SNE and histogram analyses linking high uncertainty to incorrect predictions. The core claim is that uncertainty-aware selection outperforms random selection and reduces annotation cost.
Significance. If the central claim held, the method would be practically valuable for digital pathology, where expert labels are expensive. The paper contributes a clear comparison against random sampling, a held-out test evaluation, and public code. However, the significance is currently limited by the lack of repeated runs and by the absence of comparisons to established AL acquisition functions, which leaves both the robustness of the headline result and the mechanism behind it insufficiently established.
major comments (4)
- [Section 4.2, Table 5] The headline result—out-domain UA-AL reaching 97.01% at 9% labels—is reported for a single run with no error bars or seed variance. The random baselines are strongly non-monotonic (e.g., out-domain Sim-rnd accuracy: 94.34, 91.67, 94.62 at 1%, 3%, 10%; in-domain U.Sim-rnd: 91.70, 94.59, 93.23 at 1%, 2%, 3%), which suggests substantial run-to-run variance or unstable fine-tuning; a single favorable trajectory could therefore drive the reported advantage over random selection and SOTA. Please provide multiple seeds and confidence intervals for the AL curves.
- [Sections 2.2 and 3.5, Eq. (2)] The acquisition function is total evidential uncertainty u = K/S, which combines aleatoric and epistemic uncertainty, yet the paper motivates the method by reducing epistemic uncertainty (Section 2.2). High-u samples may be intrinsically noisy or uninformative, and no comparison is made against entropy, BALD, margin sampling, or other standard acquisition functions. As written, the results do not establish that the evidential uncertainty score, rather than the overall AL loop, is responsible for the gains.
- [Tables 4 and 5, Section 4.2] The SOTA comparison relies on accuracy numbers quoted from [18] and [23] that were obtained under different, unspecified training protocols and possibly different evaluation splits. The claim that 9% of labels 'surpasses the SOTA' (97.01 vs 96.55 for HistoSSL-Res) is therefore not a controlled comparison; a direct re-implementation of the baselines under the authors' training pipeline is needed, or at least a precise statement of protocol differences.
- [Abstract and Section 4] The abstract's claim of '1-10% of strategically selected annotations' and the introduction's 'parity with 2-3%' are not uniformly supported: at 1% labels, UA-AL equals random selection (both 91.70 in-domain and 92.34 out-domain) and is below plain SimCLRv2 random (94.27/94.34). The benefit appears only after several iterations, so the stated range and level of improvement should be qualified accordingly.
minor comments (5)
- [Eq. (3)] The product index inside the Dirichlet density is written with i, which collides with the sample index i; please use a separate class index (e.g., k) to avoid confusion.
- [Abstract and Introduction] The phrase 'improve the precision and efficiency' should read 'improves'; also, the 1-10% label range is stated inconsistently with the 2% and 9% figures that actually appear in the results and discussion.
- [Figure 2 caption] The caption states UA-AL achieves comparable results to SOTA with 3% labels, but Table 5 shows 96.41% vs 96.55% at that point; 'comparable' is acceptable only if qualified to indicate the small gap.
- [Section 4.2] The statement that UA-training 'consistently outperformed' random selection is not true at 1% labels in Table 5, where the UA-AL and U.Sim-rnd columns are identical; please rephrase to describe the region of improvement.
- [Tables 3 and 4] Several baseline results are marked with asterisks indicating quoted numbers, but the table notes do not specify the source paper's training details (e.g., pre-training data, fine-tuning epochs, augmentation); this limits reproducibility of the comparison.
Circularity Check
No significant circularity: the uncertainty-guided label selection is an empirical active-learning result evaluated on held-out data.
full rationale
The paper's central claims are empirical comparisons, not derivations that reduce to their inputs. The uncertainty score u = K/S from Eq. (2) is a definition imported from evidential deep learning [25]; it is not fitted to the target result. The active-learning loop selects images using the current model's uncertainty and retrains on the queried labels, but performance is measured on held-out test sets (Tables 3-5), so the reported accuracy is not logically entailed by the selection criterion. The SOTA comparisons are against external published results quoted from [18] and [23], and the random-label baselines provide an independent empirical contrast rather than a fitted identity. There are no load-bearing self-citations: the methodological foundation is the external EDL formulation of Sensoy et al. [25], and the paper does not invoke an author-specific uniqueness theorem or prior result to force its choice. The lack of multiple seeds, the absence of entropy/BALD acquisition baselines, and the single fixed 1% query batch are empirical robustness concerns about whether the headline result is reproducible, but they do not constitute circularity under the standard definition used here.
Assumptions & free parameters
free parameters (3)
- label fraction per AL iteration =
1%
- KL annealing coefficient schedule =
lambda_t = min(1, t/10)
- number of AL iterations =
10
assumptions (4)
- domain assumption The Dirichlet-evidence loss from [25] yields well-calibrated epistemic uncertainty estimates for histopathology images.
- ad hoc to paper The most uncertain images (highest u) are the most informative to label for improving classification accuracy.
- domain assumption Pre-training on a different histopathology dataset (out-domain) provides a fair and useful initialization for the target dataset.
- domain assumption The test labels of PCam and NCT100k are accurate and the test distributions match the training distribution.
Cite this review
Pith. "Pith review of Uncertainty Awareness Enables Efficient Labeling for Cancer Subtyping in Digital Pathology." pith.science (2026). https://pith.science/paper/DSLQRPSN
@misc{pith2026250611439,
author = {Pith},
title = {Pith review of: Uncertainty Awareness Enables Efficient Labeling for Cancer Subtyping in Digital Pathology},
year = {2026},
howpublished = {\url{https://pith.science/paper/DSLQRPSN}},
note = {Machine review of arXiv:2506.11439}
}
read the original abstract
Machine-learning-assisted cancer subtyping is a promising avenue in digital pathology. Cancer subtyping models, however, require careful training using expert annotations so that they can be inferred with a degree of known certainty (or uncertainty). To this end, we introduce the concept of uncertainty awareness into a self-supervised contrastive learning model. This is achieved by computing an evidence vector at every epoch, which assesses the model's confidence in its predictions. The derived uncertainty score is then utilized as a metric to selectively label the most crucial images that require further annotation, thus iteratively refining the training process. With just 1-10% of strategically selected annotations, we attain state-of-the-art performance in cancer subtyping on benchmark datasets. Our method not only strategically guides the annotation process to minimize the need for extensive labeled datasets, but also improves the precision and efficiency of classifications. This development is particularly beneficial in settings where the availability of labeled data is limited, offering a promising direction for future research and application in digital pathology.
Figures
Reference graph
Works this paper leans on
-
[18]
Histossl: Self-supervised representation learning for classi- fying histopathology images
Xu Jin, Teng Huang, Ke Wen, Mengxian Chi, and Hong An. Histossl: Self-supervised representation learning for classi- fying histopathology images. Mathematics, 11(1):110, 2022. 7, 8
work page 2022
-
[23]
Self-distillation augmented masked autoencoders for histopathological im- age classification
Yang Luo, Zhineng Chen, and Xieping Gao. Self-distillation augmented masked autoencoders for histopathological im- age classification. arXiv preprint arXiv:2203.16983 , 2022. 1, 2, 6, 7
arXiv 2022
-
[1]
Emerg- ing properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 6, 7
2021
-
[2]
Jacob Carse and Stephen McKenna. Active learning for patch-based digital pathology using convolutional neural networks to reduce annotation costs. In Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15, pages 20–27. Springer, 2019. 1, 2
work page 2019
-
[3]
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 2, 6, 7
work page 2020
-
[4]
Big self-supervised mod- els are strong semi-supervised learners
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised mod- els are strong semi-supervised learners. Advances in neural information processing systems, 33:22243–22255, 2020. 1, 2, 6, 7
work page 2020
-
[5]
Exploring simple siamese rep- resentation learning
Xinlei Chen and Kaiming He. Exploring simple siamese rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 15750–15758, 2021. 2
work page 2021
-
[6]
An empirical study of training self-supervised vision transformers
Xinlei Chen*, Saining Xie*, and Kaiming He. An empirical study of training self-supervised vision transformers. arXiv preprint arXiv:2104.02057, 2021. 6, 7
arXiv 2021
Show all 33 references
-
[7]
Self super- vised contrastive learning for digital histopathology
Ozan Ciga, Tony Xu, and Anne Louise Martel. Self super- vised contrastive learning for digital histopathology. Ma- chine Learning with Applications, 7:100198, 2022. 1, 2
2022
-
[8]
Upper and lower probabilities induced by a multivalued mapping
Arthur P Dempster et al. Upper and lower probabilities induced by a multivalued mapping. Classic works of the Dempster-Shafer theory of belief functions , 219(2):57–72,
-
[9]
Uncertainty-informed deep learn- ing models enable high-confidence predictions for digital histopathology
James M Dolezal, Andrew Srisuwananukorn, Dmitry Karpeyev, Siddhi Ramesh, Sara Kochanny, Brittany Cody, Aaron S Mansfield, Sagar Rakshit, Radhika Bansal, Melanie C Bois, et al. Uncertainty-informed deep learn- ing models enable high-confidence predictions for digital histopatho...
2022
-
[10]
Breast cancer histopathological image classification via deep active learning and confidence boosting
Baolin Du, Qi Qi, Han Zheng, Yue Huang, and Xinghao Ding. Breast cancer histopathological image classification via deep active learning and confidence boosting. InInterna- tional Conference on Artificial Neural Networks, pages 109–
-
[11]
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning , pages 1050–1059. PMLR, 2016. 2
2016
-
[12]
Learning representations by predicting bags of visual words
Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick P´erez, and Matthieu Cord. Learning representations by predicting bags of visual words. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6928–6938, 2020. 2
2020
-
[13]
Bootstrap your own latent-a new approach to self-supervised learning
Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...
2020
-
[14]
Reliability-aware contrastive self-ensembling for semi-supervised medical image classifi- cation
Wenlong Hang, Yecheng Huang, Shuang Liang, Baiying Lei, Kup-Sze Choi, and Jing Qin. Reliability-aware contrastive self-ensembling for semi-supervised medical image classifi- cation. In International Conference on Medical Image Com- puting and Computer-Assisted Intervention, pa...
2022
-
[15]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 2, 5, 6, 7
2022
-
[16]
Distill- ing the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 2
2015 arXiv
-
[17]
Re- ducing the annotation cost of whole slide histology images using active learning
Xu Jin, Hong An, Jue Wang, Ke Wen, and Zheng Wu. Re- ducing the annotation cost of whole slide histology images using active learning. In Proceedings of the 2021 3rd In- ternational Conference on Image Processing and Machine Vision, pages 47–52, 2021. 1, 2
2021
-
[19]
Generalising bayes’ theorem in subjective logic
Audun Jøsang. Generalising bayes’ theorem in subjective logic. In MFI, pages 462–469, 2016. 4
2016
-
[20]
100,000 histological images of human colorectal cancer and healthy tissue
Jakob Nikolas Kather, Niels Halama, and Alexander Marx. 100,000 histological images of human colorectal cancer and healthy tissue. Zenodo10, 5281, 2018. 3
2018
-
[21]
Continuous mul- tivariate distributions–vol
S Kotz, N Balakrishnan, and NL Johnson. Continuous mul- tivariate distributions–vol. 1, john wiley & sons, new york,
-
[22]
Simple and scalable predictive uncertainty estima- tion using deep ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estima- tion using deep ensembles. Advances in neural information processing systems, 30, 2017. 2
2017
-
[24]
Self-supervised learning of pretext-invariant representations
Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6707–6717, 2020. 2
2020
-
[25]
Eviden- tial deep learning to quantify classification uncertainty
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Eviden- tial deep learning to quantify classification uncertainty. Ad- vances in neural information processing systems , 31, 2018. 1, 2, 4
2018
-
[26]
Fusdom: Combining in-domain and out-of-domain knowl- edge for continuous self-supervised learning
Ashish Seth, Sreyan Ghosh, S Umesh, and Dinesh Manocha. Fusdom: Combining in-domain and out-of-domain knowl- edge for continuous self-supervised learning. arXiv preprint arXiv:2312.13026, 2023. 1
2023 arXiv
-
[27]
From theories to queries: Active learning in practice
Burr Settles. From theories to queries: Active learning in practice. In Active learning and experimental design work- shop in conjunction with AISTATS 2010, pages 1–18. JMLR Workshop and Conference Proceedings, 2011. 1
2010
-
[28]
What makes for good views for contrastive learning? Advances in neural informa- tion processing systems, 33:6827–6839, 2020
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in neural informa- tion processing systems, 33:6827–6839, 2020. 2
2020
-
[29]
Rotation equivariant cnns for digital pathology
Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Co- hen, and Max Welling. Rotation equivariant cnns for digital pathology. In Medical Image Computing and Computer As- sisted Intervention–MICCAI 2018: 21st International Con- ference, Granada, Spain, September 16-20, 2018,...
2018
-
[30]
Fast dropout training
Sida Wang and Christopher Manning. Fast dropout training. In international conference on machine learning, pages 118–
-
[31]
Transpath: Transformer-based self-supervised learning for histopatho- logical image classification
Xiyue Wang, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Junzhou Huang, Wei Yang, and Xiao Han. Transpath: Transformer-based self-supervised learning for histopatho- logical image classification. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th...
2021
-
[32]
Hyperparameter ensembles for robustness and un- certainty quantification
Florian Wenzel, Jasper Snoek, Dustin Tran, and Rodolphe Jenatton. Hyperparameter ensembles for robustness and un- certainty quantification. Advances in Neural Information Processing Systems, 33:6514–6527, 2020. 2
2020
-
[33]
Barlow twins: Self-supervised learning via redundancy reduction
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Ma- chine Learning, pages 12310–12320. PMLR, 2021. 2
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.