Pith. sign in

REVIEW 3 major objections 6 minor 69 references

HistoEncoder: a digital pathology foundation model for prostate cancer

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-training a Transformer on 48 million prostate-tissue tiles yields image features that outperform natural-image pre-training on prostate cancer tasks, even without fine-tuning.

desk verdict Useful prostate pathology encoder with released code and weights, but the headline ImageNet comparison is confounded by cancer-classifier-based pre-training tile selection. read the letter →

arxiv 2411.11458 v2 pith:2LAICRK7 submitted 2024-11-18 eess.IV cs.AIcs.CV

Joona Pohjonen , Abderrahim-Oussama Batouche , Antti Rannikko , Kevin Sandeman , Andrew Erickson , Esa Pitkanen , Tuomas Mirtti This is my paper · ORCID
classification eess.IVcs.AIcs.CV
keywords digitalpathologyfoundationmodelself-supervisedlearningprostatecancerDINOXCiTtransferhistomics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a self-supervised model pre-trained on prostate tissue images captures histology far better than the same model pre-trained on natural images, and that this advantage is large enough to matter in practice. The authors train two XCiT vision transformers with the DINO self-supervised objective on 48 million tiles from prostate biopsies and prostatectomy sections, then compare them against identical models pre-trained on ImageNet. Across several prostate cancer classification datasets, the prostate-pretrained models achieve higher AUROC scores with far fewer fine-tuned parameters and far less training data; even a parameter-free KNN classifier on their features beats a fully fine-tuned ImageNet model. The same features also cluster into interpretable histological patterns, enabling automatic annotation of large slide datasets and adding prognostic information to Gleason, CAPRA-S, and MSKCC-S nomograms for prostate-cancer-specific death.

What carries the argument

HistoEncoder is an XCiT (cross-covariance image transformer), a vision-transformer backbone with linear complexity in token count, trained with DINO, a self-supervised self-distillation objective, on 48 million 640x640 tissue tiles from 1,307 patients in the HelsinkiProstate dataset. The pre-training set is balanced by a cancer classifier from the authors' earlier work: all 16 million tiles with predicted cancer score $\hat{y} > 0.2$ are kept, alongside 32 million randomly sampled tiles with $\hat{y} \le 0.2$. Downstream, the encoder is used in three ways: as a frozen feature extractor for KNN classification, as a partially fine-tuned backbone for classifiers, and as a source of tile-level features that are clustered into per-patient histological pattern fractions for survival models. The feature clustering is the load-bearing operation for the annotation and histomics use cases.

What would settle it

Take the same 48M-tile pool, drop the $\hat{y} > 0.2$ filter, randomly sample 48M tiles from all tissue, and re-run the DINO/XCiT pre-training with identical hyperparameters; then compare KNN AUROC on Karolinska and Radboud. If the unfiltered model's advantage over the ImageNet baseline largely disappears, the tile-selection filter, not histology-specific learning, drove the reported result.

Watch

Extended reading notes

Core claim

The paper's central claim is that self-supervised pre-training on prostate tissue yields a feature extractor that beats natural-image pre-training for prostate cancer tasks, decisively and data-efficiently. The strongest demonstration is that a KNN classifier fit to prostate-s features, with no fine-tuning and no learned classifier, outperforms a fully fine-tuned natural-s model on every evaluation dataset. Under limited fine-tuning of the last transformer blocks, prostate-s reaches higher AUROC than natural-s with orders of magnitude fewer parameters and with as little as one-thousandth of the training tiles. The feature space is also semantically coherent: benign versus cancerous epithelium and different Gleason grades form separated clusters, and these clusters can be used to annotate more than two million PANDA tiles at over 90 percent purity after inspecting only 256 clusters. Finally, the paper shows that patient-level cluster fractions add clinically meaningful prognostic signal to Gleason grade, CAPRA-S, and MSKCC-S nomograms for prostate-cancer-specific death, forming the basis of two real workflows.

Load-bearing premise

The reported advantage over ImageNet rests on the assumption that the cancer-score-based selection of pre-training tiles does not leak cancer label information into the learned features; if the selection filter is itself a weak cancer classifier, the comparison is not purely about domain-specific pre-training.

Editorial extensions

If this is right

  • Prostate cancer classification models can be built with a few hundred annotated tiles, matching or exceeding ImageNet-initialized models trained on the full PANDA datasets.
  • Large unlabelled slide collections can be annotated automatically: clustering HistoEncoder features labels more than two million PANDA tiles with over 90 percent purity after reviewing only 256 cluster centres.
  • Adding histology cluster fractions to Gleason grade, CAPRA-S, or MSKCC-S improves concordance, time-dependent AUC, and net benefit for prostate-cancer-specific death prediction.
  • Fine-tuning HistoEncoder for clinical tasks is feasible on a single laptop, lowering the computational barrier for building clinical software tools.
  • The authors state that evaluation against tissue-agnostic pathology foundation models is a necessary next step, which will isolate the value of organ-specific pre-training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cleaner test of the domain-specific pre-training claim would pre-train on randomly sampled tiles without the cancer-score filter; if the advantage shrinks, part of the foundation model value actually comes from the curation step rather than from self-supervised learning of histology.
  • If the domain-specific advantage holds up, the same DINO/XCiT recipe likely transfers to other organ sites, making organ-level pathology foundation models a general pattern rather than a prostate-specific result.
  • The survival analysis is single-centre with no external validation, a limitation the paper itself acknowledges, so the six selected clusters should be treated as hypothesis-generating until replicated in an independent cohort.
  • Comparing HistoEncoder not only to ImageNet but also to pan-cancer histopathology models would clarify whether the benefit is prostate-specific or merely any histology beats natural images.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces HistoEncoder, a prostate pathology foundation model built by pre-training XCiT-S12 and XCiT-M24 backbones with DINO on 48 million tissue tile images derived from Helsinki University Hospital slides. The central claims are that these prostate-pretrained encoders outperform natural-image DINO counterparts in prostate cancer classification, even with a zero-training KNN classifier or with orders of magnitude less fine-tuning data; that learned feature clusters enable automatic annotation of large-scale datasets; and that patient-level cluster fractions improve prostate-cancer-specific mortality prediction over Gleason grade, CAPRA-S, and MSKCC-S nomograms. The evaluation includes external datasets (PESO, Karolinska, Radboud) and internal Helsinki datasets, with code made publicly available.

Significance. If the central attribution claim is correct, the paper demonstrates a concrete and reproducible path to data-efficient, domain-specific foundation models in digital pathology, with potentially high clinical value for resource-limited groups. The public code release, the use of external datasets for classification evaluation, the five-fold repeated experiments, and the candid statement about the lack of external validation for survival analysis are strengths. However, the headline claims currently rest on a comparison that is confounded by the label-dependent construction of the pre-training dataset and on benchmarks that overlap with pre-training, so the significance is not yet established.

major comments (3)
  1. [§2.3.1, §3.1] The construction of HelsinkiProstate is label-dependent, which undermines the claim that HistoEncoder learns useful histology 'without any annotated information' and confounds the prostate-s versus natural-s comparison. Section 2.3.1 states that all 16 million tiles with predicted cancer score ŷ > 0.2 from the Pohjonen et al. classifier are kept, together with 32 million randomly sampled tiles with ŷ ≤ 0.2. That classifier was trained on pathologist-annotated prostate cancer data, so its predictions encode expert label information, and the resulting pre-training corpus is a 1:2 cancer-enriched, classifier-selected sample rather than an uncurated sample of prostate tissue. DINO organizes features around the dominant visual axis separating these two curated groups, and the KNN classifier used in Section 3.1 is particularly sensitive to such an axis. The external PESO, Karolinska, and Radboud results do not rule out this explanation, because the same label-filtered encoder is used in all of them. To support the attribution, the authors should provide an ablation that pre-trains on an unselected or score-stratified but non-thresholded subset of the same tile pool, or otherwise demonstrate that the label-filter is not the driver of the reported advantages.
  2. [Table 1, §2.3.2, Fig. 2, Fig. S6] Helsinki30 and Helsinki60 are reported as evaluation benchmarks even though Section 2.3.2 and Table 1 explicitly state that these datasets are part of HelsinkiProstate, the pre-training set. The high AUROC scores on these datasets therefore reflect test-on-train overlap and are not evidence of generalization. They should be removed from the main comparisons or reported only as memorization checks, with the external datasets carrying the generalization claims.
  3. [§2.4.1, §3.3] The survival analysis selects six out of 32 clusters based on a 'parameter importance analysis' on the full HelsinkiTMA cohort before performing 1,000 random stratified splits. This selection uses outcome information from the same cohort that is later split into training and test sets, yielding optimistically biased concordance, time-dependent AUC, and net benefit estimates. The paper correctly notes the lack of an external validation cohort, but it does not address the within-cohort selection circularity. The authors should either pre-specify the cluster selection protocol, use nested cross-validation in which cluster selection is performed inside each training fold, or restrict the reported results to an external cohort.
minor comments (6)
  1. [Abstract, Table 1] The abstract says '48 million prostate tissue tile images' while Table 1 reports 898.4 million tiles for HelsinkiProstate before the classifier-based selection; please clarify that 48 million is the number of tiles retained after the score-thresholding and random sampling step.
  2. [§3.2, Fig. 4] The text reports that 45.6% of all tile images, 53.3% of cancerous epithelium, and 42.4% of benign epithelium and stroma fall in high-purity clusters, but the corresponding labels in Figure 4a appear to be ordered differently and one value is printed as 53.6% in the figure. Please reconcile the text and figure numbers.
  3. [§2.3.2] Since Helsinki30 and Helsinki60 come from the same Pohjonen et al. work [13] that supplied the cancer classifier used for pre-training selection, the relationship between the classifier, these datasets, and HelsinkiProstate should be stated explicitly to avoid confusion.
  4. [§3.3] The heading 'Predicing prostate cancer-specific mortality' contains a typo; it should read 'Predicting'.
  5. [§4, References [57-59]] The Discussion states that previous foundation models for histopathology were 'typically pre-trained with natural images' and cites [57-59], but those references are for ResNet, Inception, and Swin architectures rather than foundation-model training pipelines; please cite the actual ImageNet-pretrained models used in pathology transfer-learning studies.
  6. [§2.4] The fine-tuning section does not specify some details needed for reproducibility, such as the choice of optimizer, learning rate schedule, number of epochs, and whether KNN features are L2-normalized before distance computation; please add these details or point to the code configuration.

Circularity Check

3 steps flagged · score 6.0 of 10

Label-filtered pretraining via a self-cited cancer classifier and outcome-informed cluster selection make key comparisons partially circular; the 'annotation-free' claim is contradicted by construction.

  1. fitted input called prediction [Section 2.3.1, Training dataset]
    "all remaining tile images are run through a prostate cancer classifier model from [13]. To create a balanced training dataset, all 16 million tile images with a prediction score ŷ > 0.2, and 32 million randomly sampled tile images with score ŷ ≤ 0.2 were selected to comprise the HelsinkiProstate dataset."

    The pretraining corpus is constructed by thresholding a supervised prostate-cancer classifier (the authors' own [13]). This injects a cancer/non-cancer label signal into the 'self-supervised' training distribution before DINO training. The later Discussion claim that the model learns 'without any annotated information on tissue types being available during training' is therefore contradicted by construction: the annotation signal enters through the tile-selection filter. The downstream 'prediction' of prostate cancer by prostate-s, including the KNN result, is partly a re-expression of that same label signal rather than a clean demonstration of domain-specific self-supervised learning.

  2. fitted input called prediction [Section 2.4.1, Prostate cancer-specific death survival models]
    "From the 32 patient-level clusters, six clusters are selected based on a parameter importance analysis. All models are then trained on the patient Gleason grade, CAPRA-S or MSKCC-S, with or without HistoEncoder cluster features. Each model is trained 1,000 times, where 25% of the samples are set aside as a test set using stratified random splits."

    The six cluster features are selected using outcome information from the entire HelsinkiTMA cohort before any of the 1,000 random 25% test splits are created. Thus the test folds are not independent of feature selection: the same outcome data that determined which clusters are prognostic is then used to evaluate the model's concordance, time-dependent AUC, and net benefit. The reported improvement of HistoEncoder-augmented survival models is therefore an optimistic in-sample evaluation, not an out-of-sample prediction. The paper itself concedes 'the lack of an external validation cohort in the survival analysis', confirming that no independent check of this selection exists.

1 more flagged steps
  1. other [Section 2.3.2, Evaluation datasets; Table 1]
    "Helsinki30 and Helsinki60 datasets [13] contain whole slide images ... These datasets are also part of the training (HelsinkiProstate) dataset (table 1)."

    The prostate-s encoder was pretrained on tiles from these exact patients, so evaluating cancer classification on Helsinki30 and Helsinki60 is not an external test. Any advantage over natural-s on these datasets can reflect pretraining exposure to the same slides rather than transferable histology features. The paper nonetheless reports these as evaluation datasets in Fig. 2 and Supplementary Fig. 6, so the claim that prostate-s 'significantly outperform[s] natural-s based models in all evaluation datasets' is partly circular by construction for these two datasets.

full rationale

HistoEncoder is primarily an empirical evaluation paper rather than a formal derivation, so most of its claims rest on measured AUROC and concordance values. The main circularity risk is in the construction of the pretraining corpus. HelsinkiProstate is built by thresholding the authors' own prostate-cancer classifier [13] at score >0.2 and mixing those tiles with random low-score tiles (§2.3.1). This injects a supervised cancer/non-cancer signal into the 'self-supervised' corpus, contradicting the Discussion's claim that the model learns without annotated information on tissue types. The prostate-s vs natural-s gap, including the KNN result, is thus not a clean demonstration of domain-specific SSL; it is partly a re-expression of the label signal used to curate the corpus, and [13] is a self-citation that is not independently verified in the paper. The survival analysis has a separate partial circularity: six of the 32 clusters are selected by parameter importance on the full HelsinkiTMA cohort before any random splits, so the test folds are not independent of feature selection; the paper explicitly concedes there is no external validation cohort. A further evaluation confound is that Helsinki30 and Helsinki60 are part of the HelsinkiProstate pretraining set yet are reported as evaluation datasets. Despite these issues, the external PESO, Karolinska, and Radboud results are independent of the pretraining corpus, so the paper is not wholly circular; the central classification comparison is confounded rather than mathematically forced. Overall score 6 reflects one prediction that reduces to outcome-informed selection plus a central claim whose 'annotation-free' attribution is contradicted by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

All premises are standard ML modeling choices; the main burden is the selection of training tiles by a prior classifier and the in-cohort feature selection for survival models.

free parameters (5)
  • Survival cluster count = 32
    Number of HistoEncoder clusters used to create patient-level histomics features in Section 2.4.1; chosen by the authors, affects survival model inputs.
  • Selected survival clusters = 6 of 32
    Section 2.4.1 selects six clusters based on a parameter importance analysis on the full HelsinkiTMA cohort before validation.
  • KNN k = 20
    K-nearest neighbour classifier uses k=20 (Section 2.4), a hand-set hyperparameter for zero-shot evaluation.
  • Cox penalizer and L1-ratio = 0.001 and 0.5
    Penalized Cox models use penalizer=0.001 and L1-ratio=0.5 (Section 2.4.1).
  • Training tile selection threshold = y_hat > 0.2
    Section 2.3.1 keeps all tiles above this predicted cancer score from the classifier in [13] and randomly samples below it; this shapes the pre-training distribution.
assumptions (4)
  • domain assumption DINO self-supervised pre-training on 48M prostate tiles produces features that separate histologically meaningful patterns.
    The cluster-purity and KNN results depend on this; no labels were used, but the tile selection itself came from a supervised classifier.
  • domain assumption Tile-level histology can be summarized at patient level by cluster fractions.
    Section 2.2.2 aggregates tile cluster memberships into patient-level proportions for survival models.
  • domain assumption The prostate cancer classifier from Pohjonen et al. [13] used to filter pre-training tiles generalizes to the HelsinkiProstate cohort.
    Section 2.3.1 relies on this prior same-author model to balance cancer and non-cancer tiles.
  • ad hoc to paper Random stratified splits within HelsinkiTMA provide unbiased validation after cluster selection on the same cohort.
    Section 2.4.1 selects six clusters based on parameter importance on full data, then evaluates with 1,000 random splits; no external cohort is used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HistoEncoder: a digital pathology foundation model for prostate cancer." pith.science (2026). https://pith.science/paper/2LAICRK7

@misc{pith2026241111458,
  author       = {Pith},
  title        = {Pith review of: HistoEncoder: a digital pathology foundation model for prostate cancer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2LAICRK7}},
  note         = {Machine review of arXiv:2411.11458}
}
read the original abstract

Foundation models are trained on massive amounts of data to distinguish complex patterns and can be adapted to a wide range of downstream tasks with minimal computational resources. Here, we develop a foundation model for prostate cancer digital pathology called HistoEncoder by pre-training on 48 million prostate tissue tile images. We demonstrate that HistoEncoder features extracted from tile images with similar histological patterns map closely together in the feature space. HistoEncoder outperforms models pre-trained with natural images, even without fine-tuning or with 1000 times less training data. We describe two use cases that leverage the capabilities of HistoEncoder by fine-tuning the model with a limited amount of data and computational resources. First, we show how HistoEncoder can be used to automatically annotate large-scale datasets with high accuracy. Second, we combine histomics with commonly used clinical nomograms, significantly improving prostate cancer-specific death survival models. Foundation models such as HistoEncoder can allow organizations with limited resources to build effective clinical software tools without needing extensive datasets or significant amounts of computing.

Figures

Figures reproduced from arXiv: 2411.11458 by the authors.

Figure 1
Figure 1. An overview of the HistoEncoder workflow. HistoEncoder utilizes the cross-covariance transformer (XCiT) as the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 7
Figure 7. Helsinki30 and Helsinki60 datasets [13] contain whole slide images (normal size and whole mounts) from 30 and 60 patients who have undergone radical prostatectomy at the Helsinki Uni￾versity Hospital between the years 2014 and 2021. All slide images in both datasets have been annotated by pathologists, and classified as cancerous or benign. These datasets are also part of the training (HelsinkiProstate) dataset (tab… view at source ↗
Figure 2
Figure 2. AUROC scores for the prostate cancer classifiers fine [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: UMAP representation of the features extracted from [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 4
Figure 4. Figure 4: Cluster label purities for different labels in the Karolinska and Radboud datasets. A significant proportion of tile images in these datasets are contained in clusters with greater than 90% label purity for cancerous vs benign tissue (a), stroma vs. epithelium tissue (…
Figure 5
Figure 5. Figure 5: Head-to-head concordance comparisons between 1,000 stratified random splits ( [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 56 canonical work pages

  1. [1]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015

    2015

  2. [2]

    Deep learning in histopathology: the path to the clinic

    Jeroen Van der Laak, Geert Litjens, and Francesco Ciompi. Deep learning in histopathology: the path to the clinic. Nature medicine, 27(5):775–784, 2021

    work page 2021

  3. [4]

    Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology

    Kaustav Bera, Kurt A Schalper, David L Rimm, Vamsidhar Velcheti, and Anant Madabhushi. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nature reviews Clinical oncology, 16(11):703– 715, 2019

    work page 2019

  4. [5]

    Simko, Sandy DeVries, Emmalyn Chen, Edward M

    Andre Esteva, Jean Feng, Douwe van der Wal, Shih-Cheng Huang, Jeffry P. Simko, Sandy DeVries, Emmalyn Chen, Edward M. Schaeffer, Todd M. Morgan, Yilun Sun, Ami- rata Ghorbani, Nikhil Naik, Dhruv Nathawani, Richard Socher, Je ff M. Michalski, Mack Roach, Thomas M. Pisansky, Jedidiah M. Monson, Farah Naz, James Wal- lace, Michelle J. Ferguson, Jean-Paul Bah...

    work page 2022

  5. [6]

    Spratt, Siyi Tang, Yilun Sun, and et al

    Daniel E. Spratt, Siyi Tang, Yilun Sun, and et al. Artifi- cial intelligence predictive model for hormone therapy use in prostate cancer. PREPRINT (Version 1) available at Research Square, 2023. 21 April 2023

    work page 2023

  6. [7]

    De- velopment and validation of a deep learning algorithm for improving gleason scoring of prostate cancer

    Kunal Nagpal, Davis Foote, Yun Liu, Po-Hsuan Cameron Chen, Ellery Wulczyn, Fraser Tan, Niels Olson, Jenny L Smith, Arash Mohtashamian, James H Wren, et al. De- velopment and validation of a deep learning algorithm for improving gleason scoring of prostate cancer. NPJ digital medicine, 2(1):1–10, 2019

    work page 2019

  7. [8]

    Stain-transforming cycle- consistent generative adversarial networks for improved segmentation of renal histopathology

    Thomas de Bel, Meyke Hermsen, Jesper Kers, Jeroen van der Laak, and Geert Litjens. Stain-transforming cycle- consistent generative adversarial networks for improved segmentation of renal histopathology. In International Conference on Medical Imaging with Deep Learning–Full Paper Track, 2018

    work page 2018

  8. [9]

    Ar- tificial intelligence-based breast cancer nodal metastasis detection: Insights into the black box for pathologists

    Yun Liu, Timo Kohlberger, Mohammad Norouzi, George E Dahl, Jenny L Smith, Arash Mohtashamian, Niels Olson, Lily H Peng, Jason D Hipp, and Martin C Stumpe. Ar- tificial intelligence-based breast cancer nodal metastasis detection: Insights into the black box for pathologists. Archives of pathology & laboratory medicine, 143(7):859– 868, 2019

    work page 2019

Show all 69 references
  1. [10]

    To- wards machine learned quality control: A benchmark for sharpness quantification in digital pathology

    Gabriele Campanella, Arjun R Rajanna, Lorraine Corsale, Peter J Schüffler, Yukako Yagi, and Thomas J Fuchs. To- wards machine learned quality control: A benchmark for sharpness quantification in digital pathology. Computer- ized Medical Imaging and Graphics, 65:142–151, 2018

    2018

  2. [11]

    Association between surgical skin markings in dermoscopic images and diagnostic perfor- mance of a deep learning convolutional neural network for melanoma recognition

    Julia K Winkler, Christine Fink, Ferdinand Toberer, Alexander Enk, Teresa Deinlein, Rainer Hofmann- Wellenhof, Luc Thomas, Aimilios Lallas, Andreas Blum, Wilhelm Stolz, et al. Association between surgical skin markings in dermoscopic images and diagnostic perfor- mance of a de...

    2019

  3. [12]

    From development to deployment: dataset shift, causality, and shift-stable models in health ai

    Adarsh Subbaswamy and Suchi Saria. From development to deployment: dataset shift, causality, and shift-stable models in health ai. Biostatistics, 21(2):345–352, 2020

    2020

  4. [13]

    Augment like there’s no tomorrow: Consistently performing neural networks for medical imaging

    Joona Pohjonen, Carolin Stürenberg, Atte Föhr, Reija Randen-Brady, Lassi Luomala, Jouni Lohi, Esa Pitkänen, Antti Rannikko, and Tuomas Mirtti. Augment like there’s no tomorrow: Consistently performing neural networks for medical imaging. arXiv preprint arXiv:2206.15274, 2022

    2022 arXiv

  5. [14]

    Using pre-training can improve model robustness and uncertainty

    Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pages 2712–2721. PMLR, 2019

    2019

  6. [15]

    Srinidhi, Ozan Ciga, and Anne L

    Chetan L. Srinidhi, Ozan Ciga, and Anne L. Martel. Deep neural network models for computational histopathology: A survey. Medical Image Analysis, 67:101813, 2021

    2021

  7. [16]

    High-accuracy prostate cancer pathol- ogy using deep learning

    Yuri Tolkach, Tilmann Dohmgörgen, Marieta Toma, and Glen Kristiansen. High-accuracy prostate cancer pathol- ogy using deep learning. Nature Machine Intelligence , 2(7):411–418, 2020

    2020

  8. [17]

    Triage-driven diagnosis of barrett’s esoph- agus for early detection of esophageal adenocarcinoma using deep learning

    Marcel Gehrung, Mireia Crispin-Ortuzar, Adam G Berman, Maria O’Donovan, Rebecca C Fitzgerald, and Florian Preprint – HistoEncoder: a digital pathology foundation model for prostate cancer 9 Markowetz. Triage-driven diagnosis of barrett’s esoph- agus for early detection of esop...

    2021

  9. [18]

    Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning approach

    William Lotter, Abdul Rahman Diab, Bryan Haslam, Jiye G Kim, Giorgia Grisot, Eric Wu, Kevin Wu, Jorge Onieva Onieva, Yun Boyer, Jerrold L Boxerman, et al. Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning...

    2021

  10. [19]

    Ai-based pathology predicts origins for cancers of un- known primary

    Ming Y Lu, Tiffany Y Chen, Drew FK Williamson, Melissa Zhao, Maha Shady, Jana Lipkova, and Faisal Mahmood. Ai-based pathology predicts origins for cancers of un- known primary. Nature, 594(7861):106–110, 2021

    2021

  11. [20]

    Federated learning for predict- ing histological response to neoadjuvant chemotherapy in triple-negative breast cancer

    Jean Ogier du Terrail, Armand Leopold, Clément Joly, Constance Béguier, Mathieu Andreux, Charles Maussion, Benoît Schmauch, Eric W Tramel, Etienne Bendjebbar, Mikhail Zaslavskiy, et al. Federated learning for predict- ing histological response to neoadjuvant chemotherapy in tr...

    2023

  12. [21]

    An annotation-free whole-slide training approach to pathological classifica- tion of lung cancer types using deep learning

    Chi-Long Chen, Chi-Chung Chen, Wei-Hsiang Yu, Szu- Hua Chen, Yu-Chan Chang, Tai-I Hsu, Michael Hsiao, Chao-Yuan Yeh, and Cheng-Yu Chen. An annotation-free whole-slide training approach to pathological classifica- tion of lung cancer types using deep learning. Nature communicat...

    2021

  13. [22]

    Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning

    Yongju Lee, Jeong Hwan Park, Sohee Oh, Kyoungseob Shin, Jiyu Sun, Minsun Jung, Cheol Lee, Hyojin Kim, Jin-Haeng Chung, Kyung Chul Moon, et al. Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning. Nature Biom...

    2022

  14. [23]

    Artificial intelligence in histopathology: enhancing cancer research and clinical oncology

    Artem Shmatko, Narmin Gha ffari Laleh, Moritz Ger- stung, and Jakob Nikolas Kather. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. Nature cancer, 3(9):1026–1038, 2022

    2022

  15. [24]

    Deep learning- enabled assessment of cardiac allograft rejection from en- domyocardial biopsies

    Jana Lipkova, Tiffany Y Chen, Ming Y Lu, Richard J Chen, Maha Shady, Mane Williams, Jingwen Wang, Zahra Noor, Richard N Mitchell, Mehmet Turan, et al. Deep learning- enabled assessment of cardiac allograft rejection from en- domyocardial biopsies. Nature medicine, 28(3):575–582, 2022

    2022

  16. [25]

    Fast and scalable search of whole-slide images via self-supervised deep learning

    Chengkuan Chen, Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Andrew J Schaumberg, and Faisal Mah- mood. Fast and scalable search of whole-slide images via self-supervised deep learning. Nature Biomedical Engi- neering, 6(12):1420–1434, 2022

    2022

  17. [26]

    Data-efficient and weakly supervised computational pathol- ogy on whole-slide images

    Ming Y Lu, Drew FK Williamson, Ti ffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathol- ogy on whole-slide images. Nature biomedical engineer- ing, 5(6):555–570, 2021

    2021

  18. [27]

    Transfusion: Understanding transfer learning for medical imaging

    Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understanding transfer learning for medical imaging. Advances in neural information processing systems, 32, 2019

    2019

  19. [28]

    Masked autoencoders are scalable vision learners, 2021

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Pi- otr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners, 2021

    2021

  20. [29]

    Simmim: A simple framework for masked image modeling, 2022

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling, 2022

    2022

  21. [30]

    ibot: Image bert pre- training with online tokenizer, 2022

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre- training with online tokenizer, 2022

    2022

  22. [31]

    Emerging properties in self-supervised vision transformers, 2021

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jé- gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers, 2021

    2021

  23. [32]

    Dinov2: Learning robust visual features without supervi- sion, 2023

    Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion, 2023

    2023

  24. [33]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bern- stein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models, 2022

    2022

  25. [34]

    Unsupervised pre-training of image features on non-curated data, 2019

    Mathilde Caron, Piotr Bojanowski, Julien Mairal, and Ar- mand Joulin. Unsupervised pre-training of image features on non-curated data, 2019

    2019

  26. [35]

    Self-supervised pretraining of visual features in the wild, 2021

    Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, and Piotr Bo- janowski. Self-supervised pretraining of visual features in the wild, 2021

    2021

  27. [36]

    Vision models are more robust and fair when pretrained on uncurated images without supervision, 2022

    Priya Goyal, Quentin Duval, Isaac Seessel, Mathilde Caron, Ishan Misra, Levent Sagun, Armand Joulin, and Piotr Bojanowski. Vision models are more robust and fair when pretrained on uncurated images without supervision, 2022

    2022

  28. [37]

    Sslp: Spatial guided self-supervised learning on pathological images

    Jiajun Li, Tiancheng Lin, and Yi Xu. Sslp: Spatial guided self-supervised learning on pathological images. In Medi- cal Image Computing and Computer Assisted Intervention– MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings...

    2021

  29. [38]

    Improving self- supervised learning with hardness-aware dynamic curricu- lum learning: An application to digital pathology

    Chetan L Srinidhi and Anne L Martel. Improving self- supervised learning with hardness-aware dynamic curricu- lum learning: An application to digital pathology. In Pro- ceedings of the IEEE /CVF International Conference on Computer Vision, pages 562–571, 2021

    2021

  30. [39]

    Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge

    Wouter Bulten, Kimmo Kartasalo, Po-Hsuan Cameron Chen, Peter Ström, Hans Pinckaers, Kunal Nagpal, Yuan- nan Cai, David F Steiner, Hester van Boven, Robert Vink, et al. Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge. Nature med...

    2022

  31. [40]

    Peso: Prostate epithelium segmentation Preprint – HistoEncoder: a digital pathology foundation model for prostate cancer 10 on h&e-stained prostatectomy whole slide images, 2018

    Wouter Bulten, Péter Bándi, Jeffrey Hoven, Rob van de Loo, Johannes Lotz, Nick Weiss, Jeroen van der Laak, Bram van Ginneken, Christina Hulsbergen-van de Kaa, and Geert Litjens. Peso: Prostate epithelium segmentation Preprint – HistoEncoder: a digital pathology foundation mode...

    2018

  32. [41]

    The 2014 international society of urological pathology (isup) consen- sus conference on gleason grading of prostatic carcinoma

    Jonathan I Epstein, Lars Egevad, Mahul B Amin, Brett De- lahunt, John R Srigley, and Peter A Humphrey. The 2014 international society of urological pathology (isup) consen- sus conference on gleason grading of prostatic carcinoma. The American journal of surgical pathology, 40...

    2014

  33. [42]

    Xcit: Cross-covariance image transformers, 2021

    Alaaeldin El-Nouby, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, and Hervé Jegou. Xcit: Cross-covariance image transformers, 2021

    2021

  34. [43]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  35. [44]

    Histoprep: Prepro- cessing large medical images for machine learning made easy! https: //github.com/jopo666/HistoPrep, 2022

    Joona Pohjonen and Valeria Ariotta. Histoprep: Prepro- cessing large medical images for machine learning made easy! https: //github.com/jopo666/HistoPrep, 2022

    2022

  36. [46]

    The capra-s score: a straightforward tool for improved prediction of outcomes after radical prostatectomy.Cancer, 117(22):5039–5046, 2011

    Matthew R Cooperberg, Joan F Hilton, and Peter R Carroll. The capra-s score: a straightforward tool for improved prediction of outcomes after radical prostatectomy.Cancer, 117(22):5039–5046, 2011

    2011

  37. [47]

    https://www.mskcc.org/nomograms/prostate/ post_op/coefficients

    Dynamic prostate cancer nomogram: Coe fficients. https://www.mskcc.org/nomograms/prostate/ post_op/coefficients. Accessed: 2023-05-02

    2023

  38. [48]

    Nonparametric es- timation from incomplete observations

    Edward L Kaplan and Paul Meier. Nonparametric es- timation from incomplete observations. Journal of the American statistical association, 53(282):457–481, 1958

    1958

  39. [49]

    Randaugment: Practical automated data augmen- tation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmen- tation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020

    2020

  40. [50]

    A Buslaev, A Parinov, E Khvedchenya, V . I. Iglovikov, and A. A. Kalinin. Albumentations: fast and flexible image augmentations. ArXiv e-prints, 2018

    2018

  41. [51]

    Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

    Andrew J Vickers, Ben Van Calster, and Ewout W Steyer- berg. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. bmj, 352, 2016

    2016

  42. [52]

    Zhuang, F

    Fa Wang, Z. Zhuang, F. Gao, and et al. TMO-Net: an explainable pretrained multi-omics model for multi-task learning in oncology. Genome Biology, 25:149, 2024

    2024

  43. [53]

    Truhn, J

    D. Truhn, J. N. Eckardt, D. Ferber, et al. Large language models and multimodal foundation models for precision oncology. npj Precision Oncology, 8:72, 2024

    2024

  44. [54]

    M. A. Wójcik. Foundation models in healthcare: Oppor- tunities, biases and regulatory prospects in europe. In Andrea K˝o, Enrico Francesconi, Gabriele Kotsis, A Min Tjoa, and Ismail Khalil, editors, Electronic Government and the Information Systems Perspective. EGOVIS 2022, Le...

    2022

  45. [55]

    A comprehensive sur- vey of foundation models in medicine

    Wasif Khan, Seowung Leem, Kyle B See, Joshua K Wong, Shaoting Zhang, and Ruogu Fang. A comprehensive sur- vey of foundation models in medicine. arXiv preprint arXiv:2406.10729, 2024

    2024 arXiv

  46. [56]

    Transfer learning for medical image classification: a literature review

    Hee E Kim, Alejandro Cosa-Linan, Nandhini Santhanam, Mahboubeh Jannesari, Mate E Maros, and Thomas Gans- landt. Transfer learning for medical image classification: a literature review. BMC medical imaging, 22(1):69, 2022

    2022

  47. [57]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  48. [58]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015

    2015

  49. [59]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  50. [60]

    A foundation model for clinical-grade computational pathology and rare cancers detection

    Eugene V orontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature medicine,...

    2024

  51. [61]

    A pathology foundation model for cancer diagnosis and prognosis prediction

    Xiyue Wang, Junhan Zhao, Eliana Marostica, Wei Yuan, Jietian Jin, Jiayu Zhang, Ruijiang Li, Hongping Tang, Kan- ran Wang, Yu Li, et al. A pathology foundation model for cancer diagnosis and prognosis prediction. Nature, pages 1–9, 2024

    2024

  52. [62]

    Predicting bio- chemical recurrence of prostate cancer with artificial intel- ligence

    Hans Pinckaers, Jolique van Ipenburg, Jonathan Melamed, Angelo De Marzo, Elizabeth A Platz, Bram van Ginneken, Jeroen van der Laak, and Geert Litjens. Predicting bio- chemical recurrence of prostate cancer with artificial intel- ligence. Communications Medicine, 2(1):64, 2022

    2022

  53. [63]

    Deep learning in cancer pathology: a new generation of clinical biomarkers

    Amelie Echle, Niklas Timon Rindtorff, Titus Josef Brinker, Tom Luedde, Alexander Thomas Pearson, and Jakob Niko- las Kather. Deep learning in cancer pathology: a new generation of clinical biomarkers. British journal of can- cer, 124(4):686–696, 2021

    2021

  54. [64]

    Computer extracted features from initial h&e tissue biop- sies predict disease progression for prostate cancer patients on active surveillance

    Sacheth Chandramouli, Patrick Leo, George Lee, Robin Elliott, Christine Davis, Guangjing Zhu, Pingfu Fu, Jonathan I Epstein, Robert Veltri, and Anant Madabhushi. Computer extracted features from initial h&e tissue biop- sies predict disease progression for prostate cancer pati...

    2020

  55. [65]

    A systematic analysis of deep learning in genomics and histopathology for precision oncology

    Michaela Unger and Jakob Nikolas Kather. A systematic analysis of deep learning in genomics and histopathology for precision oncology. BMC Medical Genomics, 17(1):48, 2024. Preprint – HistoEncoder: a digital pathology foundation model for prostate cancer 11

    2024

  56. [66]

    Visualization and analysis of gene expression in tissue sections by spatial transcriptomics

    Patrik L Ståhl, Fredrik Salmén, Sanja Vickovic, Anna Lundmark, José Fernández Navarro, Jens Magnusson, Stefania Giacomello, Michaela Asp, Jakub O Westholm, Mikael Huss, et al. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science,...

    2016

  57. [67]

    A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics

    Hong-Yu Zhou, Yizhou Yu, Chengdi Wang, Shu Zhang, Yuanxu Gao, Jia Pan, Jun Shao, Guangming Lu, Kang Zhang, and Weimin Li. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nature biomedi- cal engineering, 7(...

    2023

  58. [68]

    Artificial intelligence for digital and com- putational pathology

    Andrew H Song, Guillaume Jaume, Drew FK Williamson, Ming Y Lu, Anurag Vaidya, Tiffany R Miller, and Faisal Mahmood. Artificial intelligence for digital and com- putational pathology. Nature Reviews Bioengineering , 1(12):930–949, 2023

    2023

  59. [69]

    Foundation models for generalist medical artificial intelligence

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259–265, 2023

    2023

  60. [70]

    Towards a general-purpose foundation model for computational pathology

    Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Andrew H Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30(3):850– 862, 2024. Preprint – Hi...

    2024

  61. [2022]

    Springer International Publishing

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.