Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Are the Latent Representations of Foundation Models for Pathology Invariant to Rotation?

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pathology foundation models are not inherently rotation-invariant; training with rotation augmentation significantly improves their alignment on TCGA-KIRC.

desk verdict Useful 12-model benchmark showing rotation-augmented pathology FMs align better under patch rotation, but the rotation pipeline and t-test details are under-specified enough that the headline should be read as provisional. read the letter →

arxiv 2412.11938 v1 pith:3JLWMUKV submitted 2024-12-16 eess.IV cs.CV

Matouš Elphick , Samra Turajlic , Guang Yang This is my paper · ORCID
classification eess.IVcs.CV
keywords foundationmodelsdigitalpathologyrotationinvarianceself-supervisedlearninglatentrepresentationscomputationalhistopathologymutualk-nearestneighbours
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the latent representations that pathology foundation models produce are stable when a patch is rotated, and what determines that stability. It compares twelve self-supervised models on patches from TCGA-KIRC, measuring alignment between non-rotated and rotated representations with mutual k-nearest neighbours and cosine distance. The central empirical claim is that models trained with rotation augmentation are significantly more rotation-invariant: the augmented group beats the non-augmented group on both metrics with p < 0.0001. This matters because these encoders are used for tumour detection and grading on whole slide images, where tissue orientation is not controlled, and a representation that shifts with rotation can quietly degrade downstream decisions.

What carries the argument

The central objects are two representational-alignment metrics applied between a patch and its rotated copies: mutual k-nearest neighbours (m-kNN), the fraction of a given patch's k=10 nearest neighbours, measured by Euclidean distance in latent space, that are shared between the non-rotated and rotated representation sets; and cosine distance, the average angular mismatch between a patch's latent vector and its rotated vector. Rotation augmentation during self-supervised training is the explanatory mechanism: the paper contrasts the twelve models by whether their training included rotated inputs. The underlying hypothesis is that the transformer architecture provides no rotational inductive bias, so stable representations must be learned rather than inherited.

What would settle it

Recompute the same m-kNN and cosine distance after cropping away the border regions created by rotation, or after padding and filling with tissue-like content, and see whether the large gap between rotation-augmented and non-augmented models persists; if it shrinks, the reported invariance difference is partly a corner-artifact effect.

Watch

Extended reading notes

Core claim

On 256x256 patches from TCGA-KIRC whole slide images, the paper finds that rotational invariance is not a shared property of pathology foundation models. Mean mutual k-nearest neighbour scores across rotation angles range from 0.53 for Virchow to 0.85 for PathDino, and mean cosine distance ranges from 0.016 for Hibou-L to 0.145 for Phikon2. When the twelve models are split by whether rotation augmentation was used during self-supervised training, the two groups separate cleanly: rotation-augmented models have significantly smaller cosine distance (t = -8.88, p < 0.0001) and significantly higher m-kNN (t = 6.91, p < 0.0001) than models trained without it. Alignment is poorest at 45, 135, 225 and 315 degrees, which the paper attributes to slight differences in the patch corners introduced by rotation. The authors propose that because transformers lack a rotational inductive bias, rotation augmentation is needed for the encoder to learn invariance.

Load-bearing premise

The rankings depend on rotated patches being genuine rotations of the same tissue, but the paper does not say how border pixels were handled, and it attributes the worst scores at diagonal angles to corner differences, so empty-corner artifacts could be doing some of the work.

Editorial extensions

If this is right

  • Rotation augmentation should be treated as a necessary component of self-supervised pretraining for pathology foundation models, since the transformer architecture on its own does not confer rotational invariance.
  • The two-metric protocol (m-kNN at k=10 plus cosine distance across 15-degree rotations) gives practitioners a label-free way to check a model's orientation robustness before deploying it.
  • On this benchmark, rotation-augmented models such as PathDino and Hibou-L are the safer defaults for downstream tasks where slide orientation is uncontrolled.
  • Even the best models lose alignment at diagonal angles, so no evaluated model is fully rotation-invariant; residual corner artifacts remain a known source of variation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the invariance gap survives when rotated patches are cropped to remove border effects; if the diagonal-angle dip is mostly corners, the ranking between augmented and non-augmented groups could shift under a border-free protocol.
  • Because the benchmark uses one dataset (TCGA-KIRC) and only five tissue contours per slide, the clean separation between groups should be replicated across cancer types and magnifications before it is treated as a general law of pathology foundation models.
  • The same alignment metrics could be run on other transformations, such as flips, small translations, or stain perturbations, to produce a cheap multi-axis robustness profile before clinical deployment.
  • The results suggest a practical selection rule: reject a pathology foundation model if its rotated-patch alignment is poor, even if its benchmark accuracy looks good.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper evaluates the rotational invariance of latent representations from twelve pathology foundation models by comparing embeddings of original and rotated H&E patches from TCGA-KIRC WSIs. Invariance is quantified with mutual k-nearest neighbours (m-kNN) and cosine distance across rotations from 0° to 360° in 15° increments. The authors report that models trained with rotation augmentation achieve significantly better (i.e., more invariant) scores on both metrics, with t = −8.88 and t = 6.91, and conclude that rotation augmentation is necessary to learn rotation invariance in transformer-based pathology models. The paper includes a code repository and presents results as figures and heatmaps.

Significance. The question addressed is timely and practically relevant: foundation models for pathology are increasingly used in clinical pipelines, and understanding their invariance to geometric transformations is important for robustness. If the central claim holds, the paper provides actionable guidance for training future pathology models and contributes to the broader understanding of learned invariances in vision transformers. The study is empirical, the two metrics are clearly defined, and the code is released, which supports reproducibility. However, the result is not yet firmly established because of underspecified rotation details, missing statistical reporting, and confounded model grouping. The paper is a useful benchmark but currently falls short of a definitive causal conclusion.

major comments (4)
  1. [Section 2.3] The rotation implementation is not specified: the paper does not state the interpolation method, the border-filling policy (e.g., zero-fill, reflection, or crop-to-valid), or whether the rotated patch is resized back to the original dimensions. This is load-bearing because Section 3 attributes the poorest alignment at 45°, 135°, 225°, and 315° to 'slight differences in the patch corners introduced by rotation,' implying that the rotated inputs contain non-tissue boundary content. Without specifying and controlling the border handling, the central t-tests may partly measure how models respond to padding artifacts rather than to rotation of the tissue content itself. Please describe the exact rotation pipeline and include a robustness check with at least two border-filling strategies (e.g., zero-padding vs. reflected padding) to show that the group comparison is not an artifact of the chosen implementation.
  2. [Section 3] The t-test statistics are reported as t = −8.88 and t = 6.91 with p < 0.0001, but the degrees of freedom and the unit of analysis are not given. If the tests are computed by pooling all patch-level measurements across the twelve models, the effective sample size is the number of models (12), not the number of patches, and the independence assumption is violated because patches from the same model are correlated. Please report the unit of analysis and either perform a model-level analysis (e.g., comparing the six augmented vs. six non-augmented models) or use a mixed-effects model with the model as a random effect. The current presentation does not allow the reader to assess whether the significance claim is statistically valid.
  3. [Section 3, Figure 2] Figure 2 shows only mean values without any measure of dispersion. Because invariance scores vary across patches and WSIs, the reader cannot determine whether the group differences are large relative to the within-group variance. Add error bars or confidence intervals, and report the standard deviation or interquartile range for each model. This is important for interpreting the practical magnitude of the reported effects.
  4. [Conclusion] The conclusion states that 'rotation augmentation is necessary to achieve learned invariance,' which is a causal claim. The evidence, however, is associative: the two groups of models differ not only in rotation augmentation but also in architecture details, pretraining data, objective functions, and training scale. A controlled comparison (e.g., the same architecture trained with and without rotation augmentation, or matching on other training factors) is required to attribute the observed differences specifically to rotation augmentation. The conclusion should be phrased as an association or hypothesis unless such a control is provided.
minor comments (5)
  1. [Section 2.3] The phrase 'rotations from 0° to 360° at 15° intervals' is ambiguous about whether 360° is included; clarify that 0° and 360° are the same rotation.
  2. [Section 2.1] The total number of patches extracted from the TCGA-KIRC WSIs is not reported; providing N would help the reader assess the statistical precision of the measurements.
  3. [Section 3, Figure 3] The text says 'Fig. 3 presents a heatmap of m-kNN and cosine distances,' but Figure 3 has two panels; specify in the caption which panel corresponds to which metric.
  4. [Equation (1)] The notation N_k(z'_i) is used in the equation but defined only in the preceding text; add a brief definition in or after the equation for readability.
  5. [General] The code repository link is provided, but the paper would benefit from stating the software versions used for rotation, feature extraction, and the m-kNN computation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the invariance metrics are defined independently of the conclusion, and the augmented-vs-non-augmented grouping comes from external training details rather than from the paper's own fit or self-citation.

full rationale

This paper performs an empirical comparison of twelve pre-trained pathology foundation models, computing cosine distance and mutual k-NN between latent representations of non-rotated and rotated WSI patches. No parameter is fitted to the outcome being predicted; the split into models that used rotation augmentation versus those that did not is based on the external training configurations of published models, and the two metrics in Eqs. (1) and (2) are defined directly from the model outputs and the rotation operator. The central t-test in Section 3 is a group comparison of measured quantities, not a prediction derived from a fitted parameter. There is also no self-citation chain used to force the choice of models, metrics, or conclusions. The paper's comment that alignment is poorest at 45, 135, 225, and 315 degrees 'likely due to slight differences in the patch corners introduced by rotation' identifies a possible measurement confound concerning border artifacts, but a potential confound in an empirical measurement is a validity or correctness concern rather than circularity: the measured quantities are not defined in terms of the conclusion. No equation is shown to reduce to another by construction, no fitted parameter is renamed as a prediction, and no uniqueness claim is imported from the authors' own prior work. The derivation chain is therefore self-contained with respect to the paper's claims, and the circularity score is 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

This is an empirical benchmark, so there are no derived constants and no invented entities. The main free choice is k for the m-kNN metric. The central assumptions are that the metrics measure what matters, that the rotation-augmentation grouping is correct, that one dataset supports generalization, and that the rotation transform is clean enough.

free parameters (1)
  • k in mutual k-nearest neighbours = 10
    Chosen by hand; no sensitivity analysis reported, so the m-kNN-based ranking and group comparison could depend on this choice.
assumptions (5)
  • domain assumption The two metrics, m-kNN and cosine distance, faithfully capture rotation invariance relevant for downstream pathology tasks.
    The paper motivates them with neighborhood and direction alignment, but does not validate against downstream task performance.
  • domain assumption The classification of each of the twelve models as using or not using rotation augmentation, taken from the cited model papers, is accurate.
    The central group comparison in Section 3 rests on this binary grouping; no verification is reported.
  • domain assumption TCGA-KIRC H&E patches are representative enough to support a general statement about pathology foundation models.
    Single dataset, single cancer type, single magnification; generalization is assumed.
  • ad hoc to paper Rotated patches preserve the same tissue content, with only minor corner artifacts.
    The paper's own observation about corner differences at non-cardinal angles indicates this assumption is only approximately true and is unquantified.
  • standard math Independent model-level observations and approximate normality for the t-test.
    The t-test in Section 3 assumes this; models sharing training data or architecture are not truly independent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are the Latent Representations of Foundation Models for Pathology Invariant to Rotation?." pith.science (2026). https://pith.science/paper/3JLWMUKV

@misc{pith2026241211938,
  author       = {Pith},
  title        = {Pith review of: Are the Latent Representations of Foundation Models for Pathology Invariant to Rotation?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JLWMUKV}},
  note         = {Machine review of arXiv:2412.11938}
}
abstract

Self-supervised foundation models for digital pathology encode small patches from H\&E whole slide images into latent representations used for downstream tasks. However, the invariance of these representations to patch rotation remains unexplored. This study investigates the rotational invariance of latent representations across twelve foundation models by quantifying the alignment between non-rotated and rotated patches using mutual $k$-nearest neighbours and cosine distance. Models that incorporated rotation augmentation during self-supervised training exhibited significantly greater invariance to rotations. We hypothesise that the absence of rotational inductive bias in the transformer architecture necessitates rotation augmentation during training to achieve learned invariance. Code: https://github.com/MatousE/rot-invariance-analysis.

Figures

Figures reproduced from arXiv: 2412.11938 by the authors.

Figure 1
Figure 1. (Left) Representations are compared based on simi￾larity of their k nearest neighbors, here k = 1. (Right) Distance is measured by the cosine of the angle between representations. To quantify the alignment between control and rotated latent repre￾sentations, we used two metrics: mutual k-nearest neighbours (m￾kNN) ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Graphs showing how model characteristics relate to representation invariance under rotation. (Left) Mean mutual k-nearest neighbour across rotations, with higher values indicating greater invariance. (Right) Mean cosine distance, where lower values indicate stronger alignment in latent space representations under rotation. 3. Results In this study, FM invariance to H&E WSI patch rotation is evaluated using m-kNN and… view at source ↗
Figure 3
Figure 3. (Top) Mutual k￾nearest neighbour across rotation angles, where higher values indicate greater invariance. (Bottom) Co￾sine distance across rotations, with lower values indicating greater la￾tent representation alignment. and rotated representations to quantify invariance [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Current Pathology Foundation Models are unrobust to Medical Center Differences

    cs.LG 2025-01 conditional novelty 6.0 of 10

    Ten pathology foundation models encode medical center identity more strongly than cancer type in their embeddings, according to a new Robustness Index.

  2. Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology

    eess.IV 2025-01 conditional novelty 5.0 of 10

    Deep sparse and binary Fisher vectors usually outperform pooling, Deep Sets, memory networks, and focal attention for single-vector WSI retrieval on four TCGA sites.

Reference graph

Works this paper leans on

12 extracted references · 7 linked inside Pith · cited by 2 Pith papers

  1. [2]

    arXiv:2311.08359 [cs]

    URL http://arxiv.org/abs/2311.08359. arXiv:2311.08359 [cs]. Richard J. Chen, Tong Ding, Ming Y. Lu, Drew F. K. Williamson, Guillaume Jaume, Andrew H. Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, Mane Williams, Lukas Oldenburg, Luca L. Weishaupt, Judy J. Wang, Anurag Vaidya, Long Phi Le, Georg Gerber, Sharifa Sahai, Walt Williams, and Fais...

    arXiv

  2. [3]

    doi: 10.1038/s41591-024-02857-3

    ISSN 1078-8956, 1546-170X. doi: 10.1038/s41591-024-02857-3. URL https://www.nature.com/articles/s41591-024-02857-3 . Alexandre Filiot, Ridouane Ghermi, Antoine Olivier, Paul Jacob, Lucas Fidon, Alice Mac Kain, Charlie Saillard, and Jean-Baptiste Schiratti. Scaling Self-Supervised Learn- ing for Histopathology with Masked Image Modeling, July

    doi:10.1038/s41591-024-02857-3

  3. [5]

    arXiv:2409.09173 [eess]

    URL http: //arxiv.org/abs/2409.09173. arXiv:2409.09173 [eess]. Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The Platonic Representa- tion Hypothesis, July

    arXiv

  4. [6]

    arXiv:2405.07987 [cs]

    URL http://arxiv.org/abs/2405.07987. arXiv:2405.07987 [cs]. Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similar- ity of Neural Network Models: A Survey of Functional and Representational Measures, August

    arXiv

  5. [7]

    arXiv:2305.06329 [cs]

    URL http://arxiv.org/abs/2305.06329. arXiv:2305.06329 [cs]. Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber, Anil V. Parwani, Andrew Zhang, and Faisal Mahmood. A visual-language foundation model for computa- tional pathology. Nature Medicine, 30(3):863–874, March

    arXiv

  6. [8]

    doi: 10.1038/s41591-024-02856-4

    ISSN 1078-8956, 1546- 170X. doi: 10.1038/s41591-024-02856-4. URL https://www.nature.com/articles/ s41591-024-02856-4 . 4 Are Latent Representations Invariant to Rotation? Dmitry Nechaev, Alexey Pchelnikov, and Ekaterina Ivanova. Hibou: A Family of Founda- tional Vision Transformers for Pathology, August

    doi:10.1038/s41591-024-02856-4

  7. [9]

    arXiv:2406.05074 [eess]

    URL http://arxiv.org/abs/ 2406.05074. arXiv:2406.05074 [eess]. Eugene Vorontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, Ellen Yang, Philippe Mathieu, Alexander Van Eck, Donghun Lee, Julian Viret, Eric Robert, Yi Kan Wang, Jeremy D. Kunz, Matthew C. H....

    arXiv

  8. [10]

    doi: 10.1038/s41591-024-03141-0

    ISSN 1078-8956, 1546-170X. doi: 10.1038/s41591-024-03141-0. URL https://www.nature.com/articles/s41591-024-03141-0 . Hanwen Xu, Naoto Usuyama, Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier Gonz´ alez, Yu Gu, Yanbo Xu, Mu Wei, Wenhui Wang, Shuming Ma, Furu Wei, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Jaylen Ros...

    doi:10.1038/s41591-024-03141-0

Show all 12 references
  1. [11]

    doi: 10.1038/s41586-024-07441-w

    ISSN 0028-0836, 1476-4687. doi: 10.1038/s41586-024-07441-w. URL https://www.nature. com/articles/s41586-024-07441-w . Eric Zimmermann, Eugene Vorontsov, Julian Viret, Adam Casson, Michal Zelechowski, George Shaikovski, Neil Tenenholtz, James Hall, David Klimstra, Razik Yousfi,...

    doi

  2. [12]

    arXiv:2408.00738 [cs]

    URL http://arxiv.org/abs/ 2408.00738. arXiv:2408.00738 [cs]. 5

    arXiv

  3. [2023]

    org/lookup/doi/10.1101/2023.07.21.23292757

    URL http://medrxiv. org/lookup/doi/10.1101/2023.07.21.23292757. Alexandre Filiot, Paul Jacob, Alice Mac Kain, and Charlie Saillard. Phikon-v2, A large and public feature extractor for biomarker prediction, September

    2023 doi

  4. [2024]

    arXiv:2404.15217 [cs]

    URL http://arxiv.org/ abs/2404.15217. arXiv:2404.15217 [cs]. Oguz Akin, Pierre Elnajjar, Matthew Heller, Rose Jarosz, Bradley J. Erickson, Shanah Kirk, Yueh Lee, Marston W. Linehan, Rabindra Gautam, Raghu Vikram, Kimberly M. Garcia, Charles Roche, Ermelinda Bonaccio, and Joe F...

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.