pith. machine review for the scientific record. sign in

arxiv: 2605.02660 · v1 · submitted 2026-05-04 · 📡 eess.IV · cs.CV

Recognition: unknown

Biological Spatial Priors Regularize Foundation Model Representations for Cross-Site MSI Generalization in Colorectal Cancer

Dasari Naga Raju
Authors on Pith no claims yet

Pith reviewed 2026-05-08 02:20 UTC · model grok-4.3

classification 📡 eess.IV cs.CV
keywords microsatellite instabilitycolorectal cancerspatial priorsfoundation modelscross-site generalizationH&E whole slide imagesperipheral distance encodingmultiple instance learning
0
0 comments X

The pith

Peripheral distance encoding as a spatial prior regularizes foundation models to achieve MSI AUC 0.959 internally and perfect MSS specificity on external slides without retraining.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to show that foundation model features for predicting microsatellite instability from H&E whole-slide images can be steered toward site-invariant biological signals by injecting tile-level spatial priors derived from MSI histology. Site-specific textures in the images cause poor generalization when models move from one institution's slides to another's, even with general-purpose foundation models. The key prior encodes peripheral distance to the tumor invasive margin to reflect the known Crohn's-like lymphocytic reaction, and is fed into a TransMIL aggregator ahead of self-attention so the transformer integrates this context across layers. When trained only on TCGA-COAD slides, this yields strong internal AUC and complete specificity on the held-out TCGA-READ set, outperforming plain foundation-model baselines. A reader would care because reliable image-based MSI detection could replace costly molecular testing and function across real-world clinical sites.

Core claim

The central claim is that biologically motivated spatial priors, especially peripheral distance encoding that captures margin proximity and associated immune reactions, regularize UNI2-h and Virchow2 representations inside a TransMIL aggregator so that MSI prediction generalizes from training on 137 TCGA-COAD slides to 50 TCGA-READ slides with AUC 0.959 plus or minus 0.012 and MSS specificity of 1.000, exceeding the strongest reference configuration (0.957 AUC and 0.939 specificity) while local immune neighborhood encoding matches internal performance but drops in cross-site specificity.

What carries the argument

Peripheral distance encoding: a tile-level spatial prior that measures distance from each tile to the tumor invasive margin to encode the peripheral lymphocytic reaction associated with MSI histology.

If this is right

  • Margin proximity encodes a more site-invariant signal than local lymphocyte-to-tumor density for MSI detection.
  • Spatial priors can be added to existing multiple-instance learning pipelines to reduce dependence on site-specific textures.
  • High cross-site specificity is attainable with modest training sets when biological context is explicitly supplied.
  • The approach works across two different foundation models without site-specific retraining or adaptation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar margin- or neighborhood-based priors could be derived for other digital pathology tasks that rely on spatial immune or stromal patterns.
  • If the priors prove robust on prospective multi-center cohorts, image-based MSI classifiers might be deployed without per-site fine-tuning.
  • The method invites testing whether additional priors drawn from other established MSI histological correlates further improve invariance.
  • The same regularization idea might apply to foundation models used for related tasks such as tumor subtyping or treatment-response prediction.

Load-bearing premise

The spatial priors encode conserved biological signals of MSI status rather than hospital-specific staining, scanning, or other unmeasured imaging artifacts, and the TCGA-READ external set is a fair test of generalization.

What would settle it

Performance collapsing to reference levels or below when the same models are tested on slides from a third independent site with different staining or scanning protocols, or when the distance values in the prior are replaced by random numbers.

Figures

Figures reproduced from arXiv: 2605.02660 by Dasari Naga Raju.

Figure 1
Figure 1. Figure 1: Pipeline for MSI prediction from whole slide images (H&E). Tile features are ex view at source ↗
Figure 2
Figure 2. Figure 2: TransMIL attention maps on representative TCGA-COAD slides. Left: no spatial view at source ↗
read the original abstract

Predicting microsatellite instability (MSI) status from routine hematoxylin and eosin (H&E) whole slide images (WSIs) offers a practical alternative to molecular testing, but models trained at one institution tend to generalize poorly to slides acquired at a different site. Foundation model representations, despite their generality, still encode site-specific texture alongside the conserved biological morphology underlying MSI. We investigate whether tile-level spatial priors derived from known MSI histology can guide these representations toward more site-invariant features. We introduce a biologically motivated spatial prior based on peripheral distance encoding, reflecting the Crohn's-like peripheral lymphocytic reaction at the tumor invasive margin, and evaluate a secondary local immune neighborhood encoding reflecting the lymphocyte-to-tumor ratio in each tile's immediate spatial neighborhood. Both priors are injected into a TransMIL aggregator before self-attention, allowing the transformer to integrate spatial biological context with UNI2-h or Virchow2 features across all attention layers. We evaluate six foundation model and MIL aggregator combinations as a reference, then assess the effect of each spatial prior. Training on TCGA-COAD (137 slides) and evaluating externally on TCGA-READ (50 slides) without retraining, peripheral distance encoding achieves MSI AUC 0.959 +/- 0.012 on COAD and MSS specificity 1.000 on READ, compared to 0.957 and 0.939 for the strongest reference configuration. Local immune neighborhood encoding achieves comparable internal AUC but lower cross-site specificity, suggesting margin proximity encodes a more site-invariant biological signal than local immune density. Results suggest biologically grounded spatial priors act as regularizers that reduce reliance on site-specific imaging patterns.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that biologically motivated spatial priors—peripheral distance encoding (reflecting Crohn's-like lymphocytic reaction at the tumor invasive margin) and local immune neighborhood encoding (lymphocyte-to-tumor ratio in local tiles)—can be injected into a TransMIL aggregator atop foundation model features (UNI2-h or Virchow2) to regularize representations and improve cross-site generalization for MSI prediction from H&E WSIs. Trained on 137 TCGA-COAD slides and tested on 50 TCGA-READ slides without retraining, the peripheral prior yields internal MSI AUC 0.959 ± 0.012 and external MSS specificity 1.000, outperforming the strongest reference configuration (0.957 AUC, 0.939 specificity); the local prior matches internal AUC but shows lower cross-site specificity.

Significance. If the results hold after addressing potential confounders, the work provides evidence that pathology-derived spatial priors can act as effective regularizers to reduce site-specific texture in foundation-model MIL pipelines, offering a practical route to more robust MSI classifiers. The external validation on TCGA-READ and the comparison across multiple foundation models and aggregators strengthen the empirical case for incorporating domain knowledge in computational pathology.

major comments (2)
  1. [Results and Methods] The headline cross-site specificity of 1.000 on the 50-slide READ set (with only a 0.002 internal AUC gain) is load-bearing for the claim that peripheral distance encoding captures conserved MSI biology rather than COAD/READ cohort differences; however, no ablation is reported that applies stain normalization, color jitter, or scanner metadata correction to the reference configuration, leaving open the possibility that the prior functions partly as a site-specific regularizer (see experimental results and methods on prior injection).
  2. [Experimental Evaluation] With only 137 training and 50 test slides, both from TCGA, the reported standard deviations and specificity values require explicit statistical testing (e.g., DeLong test for AUC differences or bootstrap confidence intervals for specificity) to establish that the observed gains are not attributable to small-sample variability or unmeasured fixation/staining shifts between cohorts.
minor comments (2)
  1. [Abstract] The abstract states that six foundation-model/MIL combinations were evaluated as reference but does not enumerate them; listing the exact combinations (e.g., UNI2-h + TransMIL, Virchow2 + ABMIL) in the main text or a table would improve reproducibility.
  2. [Methods] Clarify the precise mathematical definition and injection mechanism of the peripheral distance encoding (e.g., how boundary detection is performed and how the scalar distance is concatenated or embedded before the self-attention layers) to allow independent replication.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments, which highlight important considerations for strengthening the empirical claims regarding the role of spatial priors in cross-site generalization. We address each major comment below and will incorporate the suggested analyses and experiments in the revised manuscript.

read point-by-point responses

  1. Referee: [Results and Methods] The headline cross-site specificity of 1.000 on the 50-slide READ set (with only a 0.002 internal AUC gain) is load-bearing for the claim that peripheral distance encoding captures conserved MSI biology rather than COAD/READ cohort differences; however, no ablation is reported that applies stain normalization, color jitter, or scanner metadata correction to the reference configuration, leaving open the possibility that the prior functions partly as a site-specific regularizer (see experimental results and methods on prior injection).

    Authors: We appreciate the referee's point that the near-perfect external specificity requires careful isolation from potential cohort-specific imaging artifacts. While the peripheral distance encoding is derived directly from established MSI-associated histology (Crohn's-like reaction at the invasive margin) and is applied identically across sites, we agree that additional controls are warranted. In the revision, we will augment the reference (non-prior) TransMIL configurations with Macenko stain normalization and random color jitter during both training and inference, then re-evaluate cross-site specificity on TCGA-READ. This will clarify whether the observed gain stems from biological regularization or implicit site correction. revision: yes

  2. Referee: [Experimental Evaluation] With only 137 training and 50 test slides, both from TCGA, the reported standard deviations and specificity values require explicit statistical testing (e.g., DeLong test for AUC differences or bootstrap confidence intervals for specificity) to establish that the observed gains are not attributable to small-sample variability or unmeasured fixation/staining shifts between cohorts.

    Authors: We agree that formal statistical testing is essential to substantiate the reported differences given the sample sizes. In the revised manuscript, we will add DeLong tests for pairwise AUC comparisons between the prior-injected and reference models, along with bootstrap-derived 95% confidence intervals (1,000 iterations) for the external MSS specificity values. These results will be included in the main results section and supplementary tables. revision: yes

Circularity Check

0 steps flagged

No significant circularity: empirical priors and external evaluation

full rationale

The paper defines peripheral distance and local immune neighborhood encodings from external pathology knowledge of MSI histology, injects them as inputs into the existing TransMIL aggregator before self-attention, and reports AUC/specificity on held-out TCGA-READ slides without retraining. No equations reduce the performance metrics to fitted parameters or self-defined quantities by construction. No load-bearing self-citations, uniqueness theorems, or ansatzes imported from prior author work are present in the derivation. The central claim (priors act as regularizers for site-invariance) is tested via direct empirical comparison to reference configurations on external data, making the result self-contained against benchmarks rather than tautological.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on domain knowledge about MSI histology rather than new mathematical derivations. No free parameters are explicitly introduced in the abstract. The priors are motivated by established pathology observations rather than invented entities.

axioms (2)
  • domain assumption MSI-positive colorectal cancers exhibit a Crohn's-like peripheral lymphocytic reaction at the tumor invasive margin
    This observation directly motivates the peripheral distance encoding prior.
  • domain assumption Local lymphocyte-to-tumor ratio in a tile's immediate neighborhood reflects relevant immune context for MSI status
    This observation motivates the secondary local immune neighborhood encoding.

pith-pipeline@v0.9.0 · 5593 in / 1571 out tokens · 45727 ms · 2026-05-08T02:20:28.688503+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 1 canonical work pages

  1. [1]

    H. Sung, J. Ferlay, R.L. Siegel, M. Laversanne, I. Soerjomataram, A. Jemal, and F. Bray. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries.CA: A Cancer Journal for Clinicians, 71(3):209–249, 2021

    2020

  2. [2]

    Boland and A

    C.R. Boland and A. Goel. Microsatellite Instability in Colorectal Cancer.Gastroenterology, 138(6):2073–2087, 2010

    2073

  3. [3]

    D.T. Le, J.N. Durham, K.N. Smith, H. Wang, B.R. Bartlett, L.K. Aulakh, S. Lu, H. Kem- berling, C. Wilt, B.S. Luber, et al. Mismatch Repair Deficiency Predicts Response of Solid Tumors to PD-1 Blockade.Science, 357(6349):409–413, 2017

    2017

  4. [4]

    Kather, C.A

    J.N. Kather, C.A. Pearson, N. Halama, D. J¨ ager, J. Krause, S.H. Loosen, A. Marx, P. Boor, F. Tacke, U.P. Neumann, et al. Deep Learning Can Predict Microsatellite Instability Directly from Histology in Gastrointestinal Cancer.Nature Medicine, 25:1054–1056, 2019

    2019

  5. [5]

    Echle, N.T

    A. Echle, N.T. Rindtorff, T.J. Brinker, T. Luedde, A.T. Pearson, and J.N. Kather. Deep Learning in Cancer Pathology: A New Generation of Clinical Biomarkers.British Journal of Cancer, 124(4):686–696, 2021

    2021

  6. [6]

    R.J. Chen, T. Ding, M.Y. Lu, D.F.K. Williamson, G. Jaume, A.H. Song, B. Chen, S. Zhang, D. Shao, M. Shaban, et al. Towards a General-Purpose Foundation Model for Computational Pathology.Nature Medicine, 30:850–862, 2024

    2024

  7. [7]

    Vorontsov, A

    E. Vorontsov, A. Bozkurt, A. Casson, G. Shaikovski, M. Zelechowski, K. Severson, E. Zim- mermann, J. Hall, N. Tenenholtz, N. Fusi, et al. A Foundation Model for Clinical-Grade Computational Pathology and Rare Cancers Detection.Nature Medicine, 30:2924–2935, 2024. 12

    2024

  8. [8]

    Ilse, J.M

    M. Ilse, J.M. Tomczak, and M. Welling. Attention-Based Deep Multiple Instance Learning. InProceedings of ICML, pages 2127–2136, 2018

    2018

  9. [9]

    Lu, D.F.K

    M.Y. Lu, D.F.K. Williamson, T.Y. Chen, R.J. Chen, M. Barbieri, and F. Mahmood. Data- Efficient and Weakly Supervised Computational Pathology on Whole-Slide Images.Nature Biomedical Engineering, 5(6):555–570, 2021

    2021

  10. [10]

    J.R. Jass. Classification of Colorectal Cancer Based on Correlation of Clinical, Morpho- logical and Molecular Features.Histopathology, 50(1):113–130, 2007

    2007

  11. [11]

    Smyrk, P

    T.C. Smyrk, P. Watson, K. Kaul, and H.T. Lynch. Tumor-Infiltrating Lymphocytes Are a Marker for Microsatellite Instability in Colorectal Carcinoma.Cancer, 91(12):2417–2422, 2001

    2001

  12. [12]

    Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji, and Y. Zhang. TransMIL: Trans- former Based Correlated Multiple Instance Learning for Whole Slide Image Classification. In Advances in Neural Information Processing Systems, volume 34, 2021

    2021

  13. [13]

    Kather, J

    J.N. Kather, J. Krisam, P. Charoentong, T. Luedde, E. Herpel, C.A. Weis, T. Gaiser, A. Marx, N.A. Valous, D. Ferber, et al. Predicting Survival from Colorectal Cancer His- tology Slides Using Deep Learning: A Retrospective Multicenter Study.PLOS Medicine, 16(1):e1002730, 2019

    2019

  14. [14]

    Comprehensive Molecular Characterization of Human Colon and Rectal Cancer.Nature, 487:330–337, 2012

    The Cancer Genome Atlas Network. Comprehensive Molecular Characterization of Human Colon and Rectal Cancer.Nature, 487:330–337, 2012

    2012

  15. [15]

    Kingma and J

    D.P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. InProceedings of ICLR, 2015

    2015

  16. [16]

    R.J. Chen, W. Chen, H. Li, W. Pan, T. Ding, M.Y. Lu, M. Shaban, T.Y. Chen, M.M. Bilal, I. Siddiqui, et al. A General-Purpose Self-Supervised Model for Computational Pathology. Nature Medicine, 30:863–874, 2024

    2024

  17. [17]

    X. Wang, J. Yang, J. Zhang, Y. Guo, and Q. Hu. Transformer-Based Unsupervised Contrastive Learning for Histopathological Image Classification.Medical Image Analysis, 81:102559, 2022

    2022

  18. [18]

    M. Li, S. Tang, and M.C. Chan. DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification. InProceedings of CVPR, pages 18 771–18780, 2022

    2022

  19. [19]

    Zhang, Y

    H. Zhang, Y. Meng, Y. Zhao, Y. Qiao, X. Yang, S.E. Coupland, and Y. Zheng. DTFD-MIL: Double-Tier Feature Distillation for Whole Slide Image Analysis. InProceedings of CVPR, 2022

    2022

  20. [20]

    Gamper, N

    J. Gamper, N. Alemi Koohbanani, S. Graham, M. Jahanifar, S.A. Khurram, A. Azam, K. Hewitt, and N. Rajpoot. PanNuke Dataset Extension, Insights and Baselines.arXiv preprint arXiv:2003.10778, 2020

    work page arXiv 2003

  21. [21]

    Kather, L

    J.N. Kather, L. Heij, H.I. Grabsch, C. Loeffler, A. Echle, H.A. Muti, J. Krause, J. M. Niehues, K.S. Sommer, P. Bankhead, et al. Pan-Cancer Image-Based Detection of Clinically Actionable Genetic Alterations.Nature Cancer, 1:789–799, 2020

    2020

  22. [22]

    Srinidhi, O

    C.L. Srinidhi, O. Ciga, and A.L. Martel. Deep Neural Network Models for Computational Histopathology: A Survey.Medical Image Analysis, 67:101813, 2021. 13

    2021

  23. [23]

    Tellez, G

    D. Tellez, G. Litjens, P. Bandi, W. Bulten, J.M. Bokhorst, F. Ciompi, and J. van der Laak. Quantifying the Effects of Data Augmentation and Stain Color Normalization in Convolu- tional Neural Networks for Computational Pathology.Medical Image Analysis, 58:101544, 2019

    2019

  24. [24]

    Macenko, M

    M. Macenko, M. Niethammer, J.S. Marron, D. Borland, J.T. Woosley, X. Guan, C. Schmitt, and N.E. Thomas. A Method for Normalizing Histology Slides for Quantitative Analysis. In Proceedings of ISBI, pages 1107–1110, 2009

    2009

  25. [25]

    R.J. Chen, C. Chen, Y. Li, T.Y. Chen, A.D. Trister, R.G. Krishnan, and F. Mahmood. Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning. InProceedings of CVPR, pages 16144–16155, 2022

    2022

  26. [26]

    Saltz, R

    J. Saltz, R. Gupta, L. Hou, T. Kurc, P. Singh, V. Nguyen, D. Samaras, K.R. Shroyer, T. Zhao, R. Batiste, et al. Spatial Organization and Molecular Correlation of Tumor- Infiltrating Lymphocytes Using Deep Learning on Pathology Images.Cell Reports, 23(1):181– 193, 2018

    2018

  27. [27]

    Klauschen, K

    F. Klauschen, K. M¨ uller, B. Binder, A. Bockmayr, M. Haber, K. Wienert, B. Pieˇ skov´ a, S. Haeckel, and C. Sinn. Scoring of Tumor-Infiltrating Lymphocytes: From Visual Estimation to Machine Learning.Seminars in Cancer Biology, 52:151–157, 2018

    2018

  28. [28]

    Mlecnik, G

    B. Mlecnik, G. Bindea, A. Angell, P. Maby, M. Angelova, D. Tougeron, S.E. Church, L. Lafontaine, M. Fischer, T. Fredriksen, et al. Integrative Analyses of Colorectal Cancer Show Immunoscore Is a Stronger Predictor of Patient Survival Than Microsatellite Instability. Immunity, 44(3):698–711, 2016

    2016

  29. [29]

    Y. Fu, A. Jung, R.V. Torne, S. Gonzalez, H. V¨ okler, A. Bhatt, Y. Hao, C. Rivero-Hinojosa, and A. Bhatt. Pan-Cancer Computational Histopathology Reveals Mutations, Tumor Com- position and Prognosis.Nature Cancer, 1:800–810, 2020

    2020

  30. [30]

    Bandi, O

    P. Bandi, O. Geessink, Q. Manson, M. Van Dijk, M. Balkenhol, M. Hermsen, B.E. Bejnordi, B. Lee, K. Paeng, A. Zhong, et al. From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge.IEEE Transactions on Medical Imaging, 38(2):550–560, 2019

    2019

  31. [31]

    Bulten, H

    W. Bulten, H. Pinckaers, H. van Boven, R. Vink, T. de Bel, B. van Ginneken, J. van der Laak, C. Hulsbergen-van de Kaa, and G. Litjens. Automated Deep-Learning System for Gleason Grading of Prostate Cancer Using Biopsies: A Diagnostic Study.The Lancet Oncology, 21(2):233–241, 2020

    2020

  32. [32]

    Howard, J

    F.M. Howard, J. Kochanny, T. Kosovec, R. Nanda, and A.T. Pearson. Using Machine Learning to Predict the Tumor Microenvironment from Whole Slide Images.Communications Medicine, 1:33, 2021

    2021

  33. [33]

    Wulczyn, D.F.K

    E. Wulczyn, D.F.K. Williamson, J. Zou, J. Huang, B. Liu, M. Shulman, J. Doyle, T. Bhatt, R. Li, C. Srisuwananukorn, et al. Deep Learning-Based Survival Prediction for Multiple Cancer Types Using Histopathology Images.PLOS ONE, 15(6):e0233678, 2020

    2020

  34. [34]

    Diao, J.Y

    J.A. Diao, J.Y. Wang, W.F. Chui, V. Mountain, S.H. Gulka, B. Ramachandran, A.H. Beck, R.L. Camp, D.G. Rimm, and A. Choudhury. Human-Interpretable Image Features Derived from Densely Mapped Cancer Pathology Slides Predict Diverse Clinical Outcomes.Nature Biomedical Engineering, 5(12):1399–1413, 2021

    2021

  35. [35]

    Saldanha, P

    O.L. Saldanha, P. Quirke, N.P. West, J. James, J. Langer, G. Grabsch, A. Echle, R. Krause, T. Alber, J.N. Kather, and S. Foersch. Swarm Learning for Decentralized Artificial Intelli- gence in Cancer Histopathology.Nature Medicine, 28:1232–1239, 2022. 14

    2022