REVIEW 3 major objections 4 minor 3 cited by
EndoDINO: A Foundation Model for GI Endoscopy
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read EndoDINO shows that a self-supervised vision transformer trained on a large private corpus of endoscopy videos, then frozen, can beat task-specific models on anatomical landmark classification, polyp segmentation, and Mayo endoscopic…
desk verdict A useful large-scale pre-training study for GI endoscopy, but the SOTA claims are overstated and the LIMUC MES result is compromised by circular checkpoint selection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of the paper is DINOv2-style self-supervised pre-training applied to endoscopy-specific data. DINOv2 trains a model to make features agree across differently cropped views of the same image without any labels; here it is applied to curated frames sampled at 5 frames per second from 130,037 videos, reduced by near-duplicate removal to an index of 112 million frames, and then balanced-sampled from a four-level hierarchical k-means clustering (5M, 250K, 25K, and 5K clusters) to build pre-training sets of 100K to 10M images. The resulting frozen ViT encoders (ViT-B/14, ViT-L/14, ViT-g/14) are evaluated by fitting only simple heads: linear probing, a boosted linear head that concatenates patch tokens from the last four layers, and the DPT dense-prediction head. A notable procedural choice is that the DINOv2 training loss did not track downstream performance, so the authors selected checkpoints by evaluating them directly on the Mayo scoring task.
What would settle it
One decisive check would be to embed the HyperKvasir and LIMUC test images with the same DINOv2 encoder used for curation and search for near-duplicates in the 112-million-frame pre-training index; if a nontrivial fraction of test images have near-duplicate matches, or if the pre-training videos can be traced to the same clinical sites or patients as the evaluation data, the paper's generalization claim would be undermined. Publishing the pre-training video identifiers or an overlap report would settle the question.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that scaling self-supervised DINOv2-style pre-training to an endoscopy-specific corpus of up to 10 million curated images yields frozen features that outperform prior supervised and self-supervised models on multiple standard GI endoscopy benchmarks. The best EndoDINO ViT-g/14 model reaches a macro F1 of 0.748 on 3-class HyperKvasir Mayo scoring and 0.715 with an AUROC of 0.942 on 4-class LIMUC Mayo scoring, surpasses the prior best polyp segmentation mIoU of 0.834 with an mIoU of 0.864 using a ViT-L/14 frozen backbone, and with only 1% of the landmark training data achieves macro F1 of 0.995–0.997. The authors emphasize that these gains come with simple decoder heads and no end-to-end fine-tuning, which they attribute to the size and diversity of the pre-training data and to automated curation via deduplication and hierarchical clustering.
Load-bearing premise
The central claim that EndoDINO generalizes rather than memorizes rests on the assertion that the public evaluation datasets are entirely unrelated to the private pre-training videos, and because the pre-training corpus is closed and no leakage or provenance analysis is given, that premise is not currently checkable.
Editorial extensions
If this is right
- A single frozen EndoDINO pass can feed several downstream heads, so multiple real-time AI tasks such as polyp segmentation, classification, and severity scoring could run in parallel on one modest GPU during an endoscopy.
- Because the 1% landmark-training experiment still reaches macro F1 above 0.99, few-shot development of new endoscopy tasks becomes plausible with only a handful of labeled examples per class.
- Increasing curated pre-training data from 1M to 10M progressively improves the 4-class Mayo scoring task, suggesting that further scaling the video pool may keep helping.
- The simple linear and boosted-linear heads match or beat end-to-end trained segmentation models, indicating that a large part of the task knowledge lives in the frozen features, not in the decoder.
- Pre-training on a broad mix of equipment, indications, and upper/lower GI procedures, rather than on one disease's trial videos, appears to be what lets the model transfer to unrelated public datasets.
Reading between the lines
- Beyond the paper: the striking few-shot landmark numbers come from a 1% split that is more class-balanced than the full set, so part of the gain may reflect label balance rather than pure feature quality; the paper does not isolate these factors.
- Beyond the paper: if the private pre-training videos and the public evaluation datasets truly share no sites, equipment, or patients, EndoDINO would be a strong existence proof for domain-specific self-supervised learning in endoscopy; releasing the pre-training index or a leakage analysis would make that check possible.
- Beyond the paper: because the authors selected checkpoints by downstream performance rather than SSL loss, practitioners working in similar medical domains should probably also monitor target-task metrics and not assume SSL loss is a reliable guide.
- Beyond the paper: the same frozen features could plausibly support tasks the paper did not test, such as depth estimation, quality metrics, and localization, since those also rely on dense visual structure; that is an extension, not something the paper demonstrates.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces EndoDINO, a family of DINOv2-style vision transformers (ViT-B/14, ViT-L/14, ViT-g/14) pre-trained on a private dataset of up to 10M curated frames sampled from 130,037 VirgoCloud GI endoscopy videos. The authors evaluate EndoDINO as a frozen feature encoder with simple heads on four public benchmarks: HyperKvasir anatomical landmark classification (80/10/10 and 1/10/10 splits), KvasirSEG polyp segmentation, HyperKvasir 3-class Mayo endoscopic scoring, and LIMUC 4-class Mayo endoscopic scoring. The central claim, stated in the abstract, is that EndoDINO achieves state-of-the-art performance on these tasks.
Significance. If the stated results are valid, the paper would provide useful evidence that a large, domain-specific self-supervised pre-training corpus can yield a frozen backbone whose features match or exceed ImageNet/LVD-pretrained features on downstream GI endoscopy tasks with minimal task-specific adaptation. The scale of the pre-training video pool (130K videos) is substantially larger than prior public efforts, and the use of a curation pipeline based on deduplication and hierarchical k-means is methodologically interesting. However, the paper does not release the pre-training dataset, the pre-trained checkpoints, or evaluation code, and the central generalization claim currently rests on an unverifiable assumption that the private pre-training data are unrelated to the public evaluation sets. The checkpoint-selection procedure and the overstatement of the landmark-classification result are additional load-bearing concerns that need to be addressed before the main claims can be accepted.
major comments (3)
- [§3.1, Table 2 and Abstract] The abstract claims state-of-the-art performance in anatomical landmark classification, but Table 2 does not support this claim on the standard 80/10/10 split: EndoDINO ViT-B/14 achieves macro F1 0.833, which is equal to both the LVD-142M DINOv2 ViT-B/14 and ViT-g/14 baselines, not greater. The 1/10/10 split results (macro F1 up to 0.997) have no prior baseline reported on the same split, so they cannot be called state-of-the-art either. Please either revise the claim or add a same-split prior comparison.
- [§2.2, Figure 1, §3.4, Table 5] The checkpoint-selection procedure creates a circularity risk for the headline LIMUC 4-class MES result. Section 2.2 states that an evaluation pipeline on downstream tasks was used to select the best checkpoints, and Figure 1 plots performance on the LIMUC 4-class MES task as a function of training step. Section 3.4 then reports Table 5 on the LIMUC 15% holdout test set using the folds from Polat et al. If the selection curve in Figure 1 was computed on that same test set, or on any partition containing test images, then the reported AUROC of 0.942 and macro F1 of 0.715 are the result of test-set selection, and the comparison to prior work is invalid. The manuscript never states that checkpoint selection was restricted to training or validation partitions. Please specify the exact splits used for checkpoint selection and, if selection used test data, re-run the evaluation with a validation-only selection protocol.
- [§4 (Discussion) and §2 (Data)] The central generalization claim in Section 4 is that evaluation datasets are 'completely unrelated data capture efforts' to the pre-training data, so the reported performance reflects generalization rather than memorization. This is unverifiable as written because the pre-training videos are private and no data provenance or leakage analysis is provided. If any HyperKvasir or LIMUC images originated from the same source, patient population, or recording session as the VirgoCloud pre-training videos, the generalization claim would not follow. Please provide a concrete overlap/deduplication analysis between pre-training and evaluation data, or explicitly weaken the claim to state that no such analysis was performed.
minor comments (4)
- [§2.2] The phrase 'evaluation pipeline on an array of downstream tasks' is vague; the manuscript should list which downstream tasks and which splits were used for checkpoint selection, separately from the final evaluation protocol.
- [Table 2] The comparison across EndoDINO-1M, EndoDINO-5M, and EndoDINO-10M is confounded: model size and pre-training dataset size change simultaneously, so the table cannot support any scaling conclusion about dataset size alone.
- [Tables 4 and 5] The paper reports point estimates without standard deviations or confidence intervals, despite using 2-fold cross-validation (Table 4) and 10-fold cross-validation (Table 5). Reporting variability would help assess whether the observed gaps are meaningful.
- [§3.2, Table 3] The text says the ViT-g/14 linear head (1k to 6k parameters) is 'almost able to achieve similar performance' to the best end-to-end ViT-B/14 model; the gap in mDice is 0.021 (0.896 vs 0.875), which may be worth stating explicitly to avoid over-interpreting the qualitative phrase.
Circularity Check
Partial circularity: the headline LIMUC MES result is selected on the same benchmark, while the landmark and polyp results remain independent.
-
fitted input called prediction
[Section 2.2 / Figure 1 caption; Section 3.4 / Table 5]
"We noted that DINOv2 losses did not perfectly correspond to performance on various downstream tasks. As such, we created an evaluation pipeline on an array of downstream tasks to select our best-performing checkpoints. Figure 1 shows an overlay of DINOv2 loss with performance on our LIMUC 4 class MES task for our ViT-g/14 model. ... Peak performance on this downstream task occurs long after total DINOv2 loss is minimized, highlighting the importance of selecting model checkpoints based on downstream task performance."
The checkpoint used for the headline LIMUC MES result (Table 5: ViT-g/14 EndoDINO-10M-HKM, AUROC 0.942, macro F1 0.715) was chosen by evaluating checkpoints on the LIMUC 4-class MES task, as shown in Figure 1. Unless this selection curve was computed on a validation partition strictly disjoint from the 15% test set (which the paper never states), the reported result is the argmax of the selection curve for the same task: the model was selected on the target benchmark, so the final number is a selected maximum rather than an independent test. This makes the MES state-of-the-art claim substantially a product of the selection process rather than unbiased external evaluation. The landmark and polyp segmentation evaluations are not affected by this selection, so the circularity is partial.
full rationale
No self-citation chain is load-bearing: the paper cites external works (DINOv2, HyperKvasir, LIMUC, prior SSL endoscopy models) and reports comparisons against external baselines. The DINOv2 recipe is adopted from Meta AI, not from the authors' own prior work. The primary circularity risk is the checkpoint-selection protocol: Figure 1 explicitly selects the best checkpoint using the LIMUC 4-class MES task, and Section 3.4 reports LIMUC 4-class MES as a headline result. Because the manuscript never states that checkpoint selection was confined to a validation split disjoint from the 15% test holdout, the reported MES numbers may be the maximum of the selection curve, not independent evidence. This is a fitted-input-called-prediction pattern, but it is partial: the anatomical landmark 1/10/10 result and the KvasirSEG polyp segmentation results do not depend on LIMUC-based selection. The landmark SOTA claim is weakened by the absence of a prior baseline on the 1/10/10 split, and the 'completely unrelated data capture' generalization claim is a data-provenance/leakage risk rather than a circularity. No other circular steps were identified.
Assumptions & free parameters
free parameters (5)
- Pre-training dataset size =
100K, 1M, 5M, 10M images
- Hierarchical k-means cluster counts =
5M, 250K, 25K, 5K clusters
- Frame sampling rate =
5 fps
- Pre-training hyperparameters
- Checkpoint selection point =
Best checkpoint on LIMUC 4-class MES task
assumptions (4)
- domain assumption DINOv2 self-supervised pre-training on large curated in-domain data produces generalizable visual features
- domain assumption Evaluation datasets (HyperKvasir, KvasirSEG, LIMUC) are unrelated to the pre-training data
- domain assumption Hierarchical k-means curation improves pre-training data quality for GI endoscopy
- domain assumption Balanced sampling from clusters yields a representative distribution that preserves rare concepts
Cite this review
Pith. "Pith review of EndoDINO: A Foundation Model for GI Endoscopy." pith.science (2026). https://pith.science/paper/MCECVGM7
@misc{pith2026250105488,
author = {Pith},
title = {Pith review of: EndoDINO: A Foundation Model for GI Endoscopy},
year = {2026},
howpublished = {\url{https://pith.science/paper/MCECVGM7}},
note = {Machine review of arXiv:2501.05488}
}
read the original abstract
In this work, we present EndoDINO, a foundation model for GI endoscopy tasks that achieves strong generalizability by pre-training on a well-curated image dataset sampled from the largest known GI endoscopy video dataset in the literature. Specifically, we pre-trained ViT models with 1B, 307M, and 86M parameters using datasets ranging from 100K to 10M curated images. Using EndoDINO as a frozen feature encoder, we achieved state-of-the-art performance in anatomical landmark classification, polyp segmentation, and Mayo endoscopic scoring (MES) for ulcerative colitis with only simple decoder heads.
Figures
Forward citations
Cited by 3 Pith papers
-
EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment
EndoVLM pre-trains an endoscopy vision-language model on unordered image-set/report pairs using anatomy-guided sparse pooling, progressive semantic alignment, and a semantic-concentrated masked autoencoder, and it out...
-
Leaderless Collective Motion in Affine Formation Control over the Complex Plane
Modifying Laplacian weights yields leaderless affine collective motion of planar robot swarms, with closed-form eigenvectors/eigenvalues designed via complex-plane analysis.
-
Federated Foundation Model for GI Endoscopy Images
Federated self-supervised pretraining with masked autoencoders produces GI endoscopy representations that outperform single-site training and approach centralized training on classification, detection, and segmentation.
Reference graph
Works this paper leans on
-
[1]
Constance E. Ruhl and James E. Everhart. Indications and outcomes of gastrointestinal endoscopy, 2008
work page 2008
-
[2]
Acosta, Zahra Shakeri, John P.A
Ryan Han, Julián N. Acosta, Zahra Shakeri, John P.A. Ioannidis, Eric J. Topol, and Pranav Rajpurkar. Randomized controlled trials evaluating ai in clinical practice: A scoping evaluation. medRxiv, 2023
work page 2023
-
[3]
Artificial intelligence in endoscopy
Yutaka Okagawa, Seiichiro Abe, Masayoshi Yamada, Ichiro Oda, and Yutaka Saito. Artificial intelligence in endoscopy. Digestive Diseases and Sciences, 67(5):1553–1572, May 2022. Published: 2022/05/01
work page 2022
-
[4]
Carleton, Bharat Rao, and Aslam Syed
Shyam Thakkar, Neil M. Carleton, Bharat Rao, and Aslam Syed. Use of artificial intelligence-based analytics from live colonoscopies to optimize the quality of the colonoscopy examination in real time: Proof of concept. Gastroenterology, 158(5):1219–1221.e2, apr 2020
work page 2020
-
[5]
Kutsev Bengisu Ozyoruk, Guliz Irem Gokceler, Gulfize Coskun, Kagan Incetan, Yasin Almalioglu, Faisal Mahmood, Eva Curto, Luis Perdigoto, Marina Oliveira, Hasan Sahin, Helder Araujo, Henrique Alexandrino, Nicholas J. Durr, Hunter B. Gilbert, and Mehmet Turan. Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for e...
work page 2020
-
[6]
Dinov2: Learning robust visual features without supervision, 2024
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick La...
2024
-
[7]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009. 7 EndoDINO: A Foundation Model for GI Endoscopy
work page 2009
-
[8]
Advancing human-centric ai for robust x-ray analysis through holistic self-supervised learning, 2024
Théo Moutakanni, Piotr Bojanowski, Guillaume Chassagnon, Céline Hudelot, Armand Joulin, Yann LeCun, Matthew Muckley, Maxime Oquab, Marie-Pierre Revel, and Maria Vakalopoulou. Advancing human-centric ai for robust x-ray analysis through holistic self-supervised learning, 2024
work page 2024
Show all 23 references
-
[9]
Castro, Anton Schwaighofer, Matthew P
Fernando Pérez-García, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Matthew P. Lungren, Maria Wetscherek, Noel Codella, Stephanie L. Hyland, Javier Alvarez-Valle, and Ozan Oktay. Ra...
2024
-
[10]
Virchow2: Scaling self-supervised mixed magnification models in pathology, 2024
Eric Zimmermann, Eugene V orontsov, Julian Viret, Adam Casson, Michal Zelechowski, George Shaikovski, Neil Tenenholtz, James Hall, David Klimstra, Razik Yousfi, Thomas Fuchs, Nicolo Fusi, Siqi Liu, and Kristen Severson. Virchow2: Scaling self-supervised mixed magnification mod...
2024
-
[11]
Hipp, Darren Fahy, Benjamin Glass, Eric Walk, John Abel, Harsha Pokkalla, Andrew H
Dinkar Juyal, Harshith Padigela, Chintan Shah, Daniel Shenker, Natalia Harguindeguy, Yi Liu, Blake Martin, Yibo Zhang, Michael Nercessian, Miles Markey, Isaac Finberg, Kelsey Luu, Daniel Borders, Syed Ashar Javed, Emma Krause, Raymond Biju, Aashish Sood, Allen Ma, Jackson Nyma...
2024
-
[12]
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Hanna Borgli, Vajira Thambawita, Pia H Smedsrud, Steven Hicks, Debesh Jha, Sigrun L Eskeland, Kristin Ranheim Randel, Konstantin Pogorelov, Mathias Lux, Duc Tien Dang Nguyen, Dag Johansen, Carsten Griwodz, Håkon K Stensland, Enrique Garcia-Ceja, Peter T Schmidt, Hugo L Hammer,...
2020
-
[13]
Matuszewski
Edward Sanderson and Bogdan J. Matuszewski. A study on self-supervised pretraining for vision problems in gastrointestinal endoscopy, 2024
2024
-
[14]
Unsupervised segmentation of colonoscopy images, 2023
Heming Yao, Jérôme Lüscher, Benjamin Gutierrez Becker, Josep Arús-Pous, Tommaso Biancalani, Amelie Bigorgne, and David Richmond. Unsupervised segmentation of colonoscopy images, 2023
2023
-
[15]
Damasceno, Shreyas Fadnavis, Pooya Mobadersany, Chaitanya Parmar, Emily Scherer, Natalia Zemlianskaia, Lindsey Surace, Louis R
Krishna Chaitanya, Pablo F. Damasceno, Shreyas Fadnavis, Pooya Mobadersany, Chaitanya Parmar, Emily Scherer, Natalia Zemlianskaia, Lindsey Surace, Louis R. Ghanem, Oana Gabriela Cula, Tommaso Mansi, and Kristopher Standish. Arges: Spatio-temporal transformer for ulcerative col...
2024
-
[16]
Foundation model for endoscopy video analysis via large-scale self-supervised pre-train, 2024
Zhao Wang, Chang Liu, Shaoting Zhang, and Qi Dou. Foundation model for endoscopy video analysis via large-scale self-supervised pre-train, 2024
2024
-
[17]
An image is worth 16x16 words: Transformers for image recognition at scale, 2021
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...
2021
-
[18]
Scaling vision transformers, 2022
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers, 2022
2022
-
[19]
Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data
Benjamín Gutierrez Becker, Filippo Arcadu, Andreas Thalhammer, Citlalli Serna, Owen Feehan, Faye Drawnel, Young Oh, and Marco Prunotto. Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data. The...
2021
-
[20]
Huy V . V o, Vasil Khalidov, Timothée Darcet, Théo Moutakanni, Nikita Smetanin, Marc Szafraniec, Hugo Touvron, Camille Couprie, Maxime Oquab, Armand Joulin, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. Automatic data curation for self-supervised learning: A clustering-b...
2024
-
[21]
Improving the computer-aided estimation of ulcerative colitis severity according to mayo endoscopic score by using regression-based deep learning
Gorkem Polat, Haluk Tarik Kani, Ilkay Ergenc, Yesim Ozen Alahdab, Alptekin Temizel, and Ozlen Atug. Improving the computer-aided estimation of ulcerative colitis severity according to mayo endoscopic score by using regression-based deep learning. Inflammatory Bowel Diseases, 2...
2022
-
[22]
Vision transformers for dense prediction, 2021
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction, 2021
2021
-
[23]
Public imaging datasets of gastrointestinal endoscopy for artificial intelligence: a review
Shiqi Zhu, Jingwen Gao, Lu Liu, Minyue Yin, Jiaxi Lin, Chang Xu, Chunfang Xu, and Jinzhou Zhu. Public imaging datasets of gastrointestinal endoscopy for artificial intelligence: a review. Journal of Digital Imaging, 36(6):2578–2601, dec 2023. 8
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.