EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment

Bin Lv; Jianwei Xu; Liang Huang; Ling Zhang; Sijing Li; Yingda Xia; Yue Hu; Zhenyu Yi; Zhongwei Qiu

REVIEW 4 major objections 6 minor 12 references

EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that EndoVLM, pre-trained on 348K endoscopic image-set/report pairs without per-frame labels, outperforms existing general and endoscopy-specific foundation models across downstream tasks and supports zero-shot anatomy…

desk verdict EndoVLM is a genuinely useful endoscopy vision-language pretraining recipe with a convincing ablation, but its unvalidated LLM report-parsing step is a load-bearing risk that a serious referee should require auditing before the performance claims are trusted. read the letter →

arxiv 2608.04472 v1 pith:ZSCIZM34 submitted 2026-08-05 cs.CV cs.AIcs.CL

Zhenyu Yi , Jianwei Xu , Yue Hu , Zhongwei Qiu , Sijing Li , Liang Huang , Bin Lv , Ling Zhang

show 1 more author

Yingda Xia

This is my paper · ORCID

classification cs.CVcs.AIcs.CL

keywords endoscopyvision-languagepre-trainingfoundationmodelimage-setreportpairinganatomy-guidedsparsepoolingsemantic-awarealignmentmaskedautoencoderzero-shotanatomyrecognition

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EndoVLM tries to establish that routine endoscopic image-set/report pairs, rather than per-frame annotations, are enough supervision to build a reusable endoscopy foundation model. The paper pre-trains on 348K examinations, using each report's anatomical findings as queries that sparsely select the few relevant frames among dozens of redundant images, then aligns the pooled visual features to report semantics through anatomy- and pathology-aware soft targets, and adds masked reconstruction on those selected frames. If the claims hold, a single backbone can handle video polyp diagnosis, polyp segmentation, anatomy recognition, and zero-shot disease classification with simple downstream heads, outperforming existing foundation models and staying competitive with task-specific methods on several benchmarks. The practical significance is that endoscopy generates abundant unannotated image-report data at scale, so the method could turn routine clinical records into training signal.

What carries the argument

The machinery is a jointly trained three-part objective. First, Anatomy-Guided Sparse Pooling computes dot-product relevance between each text-encoded anatomical sentence embedding and every frame embedding, keeps the top K=3 frames per query, and re-normalizes attention over those frames to produce one visual vector per anatomy. Second, Progressive Semantic-Aware Alignment combines a symmetric InfoNCE loss between the image-set average and the whole-report embedding with a fine-grained contrastive loss over the pooled anatomy vectors, using soft targets that depend on matching anatomy and pathological status. Third, Semantic-Concentrated Masked Autoencoder masks 75% of patches in the union of AGSP-selected frames and reconstructs pixels by mean squared error. The text embeddings do double duty as queries for sparse pooling and as anchors for soft-target alignment; the taxonomy-derived soft targets are the piece that injects clinical structure directly into the contrastive distribution.

What would settle it

Take a random sample of the 348,000 reports, have expert gastroenterologists label each sentence's anatomical region and normal/abnormal status, and compare against the automatic parser. If agreement is low, especially on abnormal findings, the AGSP queries and PSAA soft targets are unreliable and the reported gains could be an artifact; alternatively, permuting the anatomy labels during pretraining should destroy the PSAA advantage if the alignment is genuinely anatomy-guided.

Watch

Extended reading notes

Core claim

The paper's central claim is that its three components close the modality gap between structured clinical reports and unordered, redundant image-sets. Anatomy-Guided Sparse Pooling treats each parsed sentence, a morphological finding tied to one of 17 gastrointestinal regions and a normal/abnormal status, as a query and uses top-K sparse attention to aggregate only the semantically relevant frames into an anatomy-specific visual vector. Progressive Semantic-Aware Alignment first runs patient-level global contrastive alignment, then a fine-grained contrastive loss whose soft targets encode the clinical taxonomy: identical healthy anatomies are not repelled, while abnormal findings within the same anatomy are softly pulled together by textual similarity. Semantic-Concentrated Masked Autoencoder applies pixel reconstruction only to the selected semantic-rich frames, adding low-level geometric detail back into the high-level semantic representation. The paper reports that this combination outperforms general and endoscopy-specific foundation models across downstream tasks, provides near-perfect zero-shot upper-GI anatomy recognition, and stays competitive with task-specific segmentation models using only a minimal head.

Load-bearing premise

The whole pretraining signal depends on the language model parsing each clinical report into correct finding-anatomy-status triplets and on the filtering step not discarding a biased subset of cases, and the paper reports no human-validation statistic for that parsing.

Editorial extensions

If this is right

Endoscopy foundation models can be pre-trained from routine clinical reports and unordered image packages, without per-frame labels, because AGSP selects the few salient frames for each anatomical query.
A single pretrained backbone covers both image-level tasks such as polyp diagnosis and anatomy recognition and dense prediction such as polyp segmentation, with a linear head or a single convolutional layer.
In the ablation, adding the taxonomy-aware fine-grained alignment to global alignment raises CVC-12k Dice by 19.4 points and LIMUC AUC by 8.9 points, so the anatomy/pathology soft targets, not the reconstruction loss, carry most of the gain.
Zero-shot transfer works for anatomy recognition and video disease diagnosis through the aligned visual-semantic space, meaning the model can be queried with text prompts instead of being fine-tuned.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the parser's accuracy on abnormal findings is high, the same image-set/report recipe should transfer to other multi-region imaging modalities with structured reports, such as dermoscopy body mapping or pathology whole-slide reports, but the fixed regional taxonomy would need to be rebuilt.
Editorial inference: the lower-ileum zero-shot weakness the paper attributes to procedural language suggests that reports describing the procedure rather than the mucosa need separate handling or a parser that distinguishes procedural sentences from finding sentences.
Editorial inference: because AGSP discards temporal order and most frames, adding a temporal-context branch over the retained frames could further improve video-level diagnosis; this is not tested in the paper.
Editorial inference: since the public benchmarks are disjoint from the private pre-training corpus, an external multi-center cohort study would be the natural next test of whether the generalization claim holds outside the two source centers.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about EndoVLM. First, it is a real engineering contribution: a vision-language pretraining framework that handles the messy reality of GI endoscopy, where one report covers dozens of unlabeled frames. The anatomy-guided sparse pooling (AGSP) is a sensible fix for redundant image-sets, the taxonomy-aware soft targets (PSAA) are a thoughtful way to use clinical structure, and the semantic-concentrated MAE adds a plausible low-level complement. The ablation is honest and each component earns its place: global alignment alone gets 85.2 AUC / 66.2 Dice on the two main tasks, adding fine-grained alignment jumps to 94.1/85.6, and the MAE gives a further small bump. Those numbers are coherent and the evaluation uses external public benchmarks, which is exactly the right way to make a claim like this.

Second, the soft spot is real and it sits at the center of the method. The Qwen3-based parsing of clinical reports into semantic triplets (finding, anatomy, pathology) has zero reported validation. No clinician agreement, no accuracy statistic, no failure analysis. Because AGSP uses those anatomy labels as queries and PSAA uses the pathology labels to build soft targets, every parsing error propagates directly into the pretraining signal. The same parser also decides which cases are 'ineligible' and filters them out, so systematic parser bias could silently shape the 348K corpus. This is not a minor detail; it is the load-bearing link between the raw data and the claimed representations. The stress-test note is right to flag it.

Some smaller issues: there are no error bars or significance tests, so the differences between EndoVLM and the closest baselines (e.g., 94.5 vs 94.0 AUC on LIMUC) may be noise. The zero-shot protocol is under-specified - prompt details, class selection, and the reported ~100% AUC in upper-GI anatomy all need more scrutiny. The fine-grained alignment has a mildly self-referential flavor: Eq. 1 pools frames by text-query attention, then Eq. 4 pulls those pooled vectors toward the same text embeddings. Global alignment and the MAE probably keep this from collapsing, but it deserves a comment or an experiment. Data and weights are not released, though the code link is a start.

Overall, the central claim is plausible and the paper is worth engaging with seriously. It is not a paradigm shift, but it gives the endoscopy community a reusable pretraining framework and a competitive backbone. Who is this for? Anyone building endoscopy foundation models or doing transfer learning on GI video/image tasks. My recommendation: send it to peer review, but make the Qwen3 parser audit (including clinician validation on a sample) a hard requirement, along with error bars or repeated-run statistics. Without that, the unmeasured label noise makes the exact numbers hard to trust.

Referee Report

4 major / 6 minor

Summary. The paper introduces EndoVLM, a vision-language pre-training model for gastrointestinal endoscopy. The model is trained on 348K examination-level pairs, where each pair is an unordered set of endoscopic images and a clinical report. Three main components are proposed: (1) Anatomy-Guided Sparse Pooling (AGSP) uses fine-grained text embeddings from the report as queries to select the top-K relevant frames per anatomical region; (2) Progressive Semantic-Aware Alignment (PSAA) combines a global image-set/report InfoNCE loss with a fine-grained contrastive loss over anatomy-pathology soft targets; (3) Semantic-Concentrated Masked Autoencoder (SC-MAE) reconstructs masked patches only on the semantically selected frames. The authors evaluate the model on polyp diagnosis, polyp segmentation generalization, ulcerative colitis severity grading, and zero-shot anatomy recognition and video polyp detection, reporting that it outperforms existing general and endoscopy-specific foundation models and is competitive with task-specific models. They also provide an ablation study showing each loss component contributes.

Significance. If the reported results hold, EndoVLM would be a valuable reusable backbone for endoscopic video and image-set analysis, reducing the need for frame-level annotations. The proposed use of LLM-parsed clinical reports to guide sparse frame selection is an interesting and practical direction, and the taxonomy-aware soft targets are a reasonable way to encode clinical prior knowledge. The paper provides a public code link and the ablations isolate the contribution of each component, which supports reproducibility. However, the strongest claims are conditional on the accuracy of the Qwen3-based parsing and on the fairness of the comparative protocol. The zero-shot results are presented with insufficient detail. As such, the significance is potentially high but currently not fully established.

major comments (4)

[Section 2.1 and Section 3] The entire pre-training pipeline relies on Qwen3 to parse each clinical report into semantic triplets (finding, anatomy, pathological status) and to filter ineligible examinations, yet the paper reports no validation of this parsing step—no accuracy, precision/recall against clinician-annotated reports, or failure analysis. Since the AGSP queries in Eq. (1) and the PSAA soft targets in Eq. (4) are built directly from these triplets, any systematic parsing error (e.g., mislabeled anatomy or flipped pathology status) directly corrupts the visual-text alignment, and the filtering step may silently bias the 348K-exam corpus by dropping complex cases. To support the central claim, the authors should provide a human-annotated parsing audit (e.g., 500 reports across both centers), report per-region and per-status precision/recall, and quantify the fraction of excluded examinations along with the reasons.
[Section 3, Tables 1 and 2] The empirical comparisons are reported as single numbers without error bars, multiple seeds, or statistical significance tests. This is concerning because several margins are small (e.g., PolypDiag F1: 97.3 vs 96.3 for EndoFM-LV; CVC-12k Dice: 86.4 vs 85.8 for DINOv3†) and for LIMUC EndoVLM actually trails GastroNet-5M (94.5/74.4 vs 95.5/–). The ablation in Table 2 also appears to have ambiguous checkmark placement: the third row seems to combine Lglo and LMAE, but the rendering is unclear. The paper should report mean and standard deviation over at least three random seeds, use paired significance tests against the strongest baselines, and clarify the exact configuration of each ablation row.
[Section 3, zero-shot evaluation] The zero-shot evaluation is under-specified. The text gives only the prompt 'This is an image of {cls}.' and refers to Fig. 2, but does not state the dataset split, the class vocabulary, the number of trials, or how video-level predictions are formed for PolypDiag (e.g., whether per-frame scores are mean-pooled). The near-perfect AUC values for upper-GI anatomy recognition and the 18% improvement over BiomedCLIP on video disease diagnosis cannot be verified without this protocol. Please provide the full zero-shot setup, per-class AUC/F1, and a confusion matrix; the acknowledged lower-ileum limitation should also be quantified with a per-class result.
[Section 2.3, Eqs. (1)-(5)] The fine-grained alignment loss has a self-referential structure: the visual representation z_u in Eq. (1) is constructed by attending to frames selected with the text query t_i,j, and Eq. (5) then pulls z_u toward that same text embedding. The soft target in Eq. (4) is also a function of text-text similarities. In principle, this could allow the model to minimize L_fg by matching text embeddings to themselves without acquiring new visual information. The ablation in Table 2 shows that adding L_fg improves downstream segmentation and grading, suggesting that the representation is not trivial; nevertheless, the paper should provide direct evidence that z_u contains visual information beyond the text selector, for example by probing the learned visual features on a frame-level anatomy classification task with ground-truth labels.

minor comments (6)

[Section 2.1] The 17 GI regions are referenced as 'aforementioned' but the mapping is only implicit from the Introduction; please list the eight gastroscopy and nine colonoscopy regions explicitly at the point of use.
[Table 2] The checkmarks do not visually align with the column headers in the current rendering; please use explicit checked/unchecked cells or a legend.
[Ablation paragraph] In the sentence 'boosting AUC by 8.9% and Dice by 19.4%. which highlights...', the period before 'which' should be a comma.
[Section 3] The repeated use of 'EndoFM-LV' (Wang et al., 2025b) versus 'EndoFM' (Wang et al., 2023) may confuse readers; please define the abbreviations on first use and make the distinction in the comparison table.
[Eq. (4)] The hard target y=1 for normal-normal pairs of the same anatomy is a design choice that differs from the soft-text-similarity used when either item is abnormal; please justify this asymmetry in the text.
[Section 3] The zero-shot figure (Fig. 2) should include axis labels, units, and error bars (if any) to be self-contained, since the main text does not provide the numeric values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EndoVLM's performance is validated on external public benchmarks, and the text-conditioned pooling and alignment mechanism is a training objective rather than a prediction that reduces to its inputs.

full rationale

The central claims of EndoVLM are evaluated on external datasets (PolypDiag, CVC-12k, Kvasir, ClinicDB, ColonDB, ETIS, Hyper-Kvasir, LIMUC) that are stated to be disjoint from the private pre-training corpus, so the superiority claim is not circular with respect to the pre-training data. The closest candidate for circularity is the fine-grained alignment: Eq. (1) constructs the visual representation z_{i,j} via text-conditioned sparse attention using query t_{i,j}, and Eqs. (4)-(5) then pull z toward textual targets derived from the same t embeddings. However, this is a self-supervised training loss, not a claimed prediction or first-principles result, and z is not forced by construction to equal t; the image content and learned encoder parameters remain independent variables in the optimization. The Qwen3-based parsing of reports into semantic triplets is load-bearing but unvalidated, which is a correctness and data-quality risk, not a circularity, because the downstream evaluation does not assume the parser is correct beyond its role in pre-training. There are no load-bearing self-citations, imported uniqueness theorems, or renamed known results. Accordingly, no circular step meeting the evidentiary bar is present.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-set hyperparameters (AGSP temperature, top-K, MAE mask ratio, loss weights), on unvalidated LLM-derived labels, and on standard contrastive-learning assumptions. No new physical or conceptual entity is introduced; the anatomy-specific visual representation z_i,j is a learned latent, not a postulated entity with independent falsifiable predictions.

free parameters (6)

AGSP temperature tau_a = 0.07
Hand-set in Eq. 1; controls sharpness of sparse attention over selected frames.
AGSP top-K frame count = 3
Hand-set in Eq. 1; determines how many frames are pooled per anatomical query.
Contrastive temperature tau = learned (not reported)
Learnable temperature in Eq. 2 and Eq. 3; fitted during pretraining.
Fine-grained loss weight lambda_1 = 1.0
Empirically set in Eq. 6; balances fine-grained alignment against global loss.
MAE loss weight lambda_2 = 1.0
Empirically set in Eq. 6; balances reconstruction against alignment.
MAE mask ratio = 75%
Inherited from MAE (He et al., 2022), applied to semantic-rich frames in Section 2.4.

assumptions (5)

ad hoc to paper Qwen3 report parsing is accurate enough to produce valid semantic triplets from clinical reports.
Section 2.1 uses Qwen3 with a fixed schema to derive semantic triplets; no parsing accuracy or human validation is reported.
domain assumption The 17-region taxonomy and binary pathological status are sufficient to represent clinically relevant content in reports.
Section 2.1 maps anatomy labels to 17 GI regions and pathological status to 0/1; this discards severity, qualifiers, and other report detail.
domain assumption Each report and its image-set are semantically consistent, and each image-set covers the anatomical regions described in the report.
The whole framework assumes many-to-many correspondence between the report and the unordered frames; Section 2.1 defines the dataset D = {(V_i, T_i)} and uses paired text to supervise image selection.
domain assumption Visual and text encoders initialized from pretrained ViT-B/16 and PubMedBERT are reasonable starting points.
Section 3 implementation details instantiate the encoders; the paper relies on their generic transferability.
domain assumption Contrastive alignment of image and text embeddings yields transferable representations for downstream tasks.
This is the standard CLIP-style assumption used throughout Section 2.3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment." pith.science (2026). https://pith.science/paper/ZSCIZM34

@misc{pith2026260804472,
  author       = {Pith},
  title        = {Pith review of: EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSCIZM34}},
  note         = {Machine review of arXiv:2608.04472}
}

read the original abstract

The development of foundation models (FMs) is crucial for advancing endoscopic image analysis. However, existing endoscopy FMs mainly rely on self-supervised learning from uni-modal images or videos, overlooking the rich semantic knowledge contained in clinical reports. Furthermore, effectively leveraging these records is hindered by a fundamental modality gap: structured anatomical descriptions are not naturally mapped to specific frames within the high-redundancy, uncurated visual streams. In this paper, we present EndoVLM, a novel vision-language FM pre-trained on over 348K endoscopic examinations, each pairing a clinical report with its corresponding image collection. An Anatomy-Guided Sparse Pooling mechanism utilizes textual descriptions as queries to drive sparse attention, efficiently aggregating semantically salient frames into anatomy-specific visual representations across redundant image-sets. Next, a Progressive Semantic-Aware Alignment strategy models clinical taxonomy (anatomy and pathological status) via structured soft targets, bridging the gap from global patient-level matching to fine-grained localized alignment. Finally, a Semantic-Concentrated Masked Autoencoder is applied exclusively to these semantic-rich frames, integrating low-level visual precision with robust high-level semantic representation. Extensive experiments across various downstream tasks demonstrate that EndoVLM outperforms existing foundation models and remains competitive with task-specific methods. Remarkably, EndoVLM also exhibits robust zero-shot generalization capabilities, highlighting its potential for broader clinical application.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 2 canonical work pages

[6]

Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

arXiv
[11]

Qwen3 technical report.arXiv preprint arXiv:2505.09388,

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

arXiv
[12]

Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.arXiv preprint arXiv:2303.00915, 2023

Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.arXiv preprint arXiv:2303.00915, 2023

arXiv 2023
[2014]

Dinov3.arXiv preprint arXiv:2508.10104,

Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104,

arXiv
[2015]

On the opportunities and risks of foundation models

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258,

arXiv
[2018]

Dinov2: Learning robust visual features without supervision

Maxime Oquab, Timothée Darcet, Théo Moutakanni, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

arXiv
[2020]

Endodino: A foundation model for gi endoscopy.arXiv preprint arXiv:2501.05488,

Patrick Dermyer, Angad Kalra, and Matt Schwartz. Endodino: A foundation model for gi endoscopy.arXiv preprint arXiv:2501.05488,

arXiv
[2021]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

arXiv 2010

Show all 12 references

[2022]

A benchmark for endoluminal scene segmentation of colonoscopy images

David Vázquez, Jorge Bernal, F Javier Sánchez, Gloria Fernández-Esparrach, Antonio M López, Adriana Romero, Michal Drozdzal, and Aaron Courville. A benchmark for endoluminal scene segmentation of colonoscopy images. Journal of healthcare engineering, 2017(1):4037190,

2017
[2023]

Tumorchain: Interleaved multimodal chain-of-thought reasoning for traceable clinical tumor analysis

Sijing Li, Zhongwei Qiu, Jiang Liu, Wenqiao Zhang, Tianwei Lin, Yihan Xie, Jianxiang An, Boxiang Yun, Chenglin Yang, Jun Xiao, et al. Tumorchain: Interleaved multimodal chain-of-thought reasoning for traceable clinical tumor analysis. InInternational Conference on Learning Rep...

2026
[2024]

Large-scale and fine-grained vision-language pre-training for enhanced ct image understanding

Zhongyi Shui, Jianpeng Zhang, Weiwei Cao, Sinuo Wang, Ruizhe Guo, Le Lu, Lin Yang, Xianghua Ye, Tingbo Liang, Qi Zhang, et al. Large-scale and fine-grained vision-language pre-training for enhanced ct image understanding. arXiv preprint arXiv:2501.14548,

arXiv
[2025]

Polyp-pvt: Polyp segmentation with pyramid vision transformers.arXiv preprint arXiv:2108.06932,

Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, and Ling Shao. Polyp-pvt: Polyp segmentation with pyramid vision transformers.arXiv preprint arXiv:2108.06932,

arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [6]

Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

arXiv

[2] [11]

Qwen3 technical report.arXiv preprint arXiv:2505.09388,

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

arXiv

[3] [12]

Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.arXiv preprint arXiv:2303.00915, 2023

Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.arXiv preprint arXiv:2303.00915, 2023

arXiv 2023

[4] [2014]

Dinov3.arXiv preprint arXiv:2508.10104,

Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104,

arXiv

[5] [2015]

On the opportunities and risks of foundation models

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258,

arXiv

[6] [2018]

Dinov2: Learning robust visual features without supervision

Maxime Oquab, Timothée Darcet, Théo Moutakanni, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

arXiv

[7] [2020]

Endodino: A foundation model for gi endoscopy.arXiv preprint arXiv:2501.05488,

Patrick Dermyer, Angad Kalra, and Matt Schwartz. Endodino: A foundation model for gi endoscopy.arXiv preprint arXiv:2501.05488,

arXiv

[8] [2021]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

arXiv 2010