Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Large-scale Self-supervised Video Foundation Model for Intelligent Surgery

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A video-level surgical pretraining model outperforms both natural-domain and surgical-domain baselines across 13 datasets.

desk verdict Large-scale surgical video pretraining with a real performance story, but the in-domain benchmark is confounded by training-split overlap. read the letter →

arxiv 2506.02692 v1 pith:FAELIIDA submitted 2025-06-03 cs.CV

Shu Yang , Fengtao Zhou , Leon Mayer , Fuxiang Huang , Yiliang Chen , Yihui Wang , Sunan He , Yuxiang Nie
show 11 more authors
Xi Wang Ömer Sümer Yueming Jin Huihui Sun Shuchang Xu Alex Qinyang Liu Zheng Li Jing Qin Jeremy YuenChun Teoh Lena Maier-Hein Hao Chen
This is my paper
classification cs.CV
keywords surgicalvideofoundationmodelself-supervisedlearningspatiotemporalpretrainingmaskedreconstructionknowledgedistillationsceneunderstandingworkflowrecognitionvideo-level
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that surgical AI models should be pretrained on video, not just still frames, because surgical understanding depends on motion, sequence, and tool-tissue interactions that static images cannot capture. To make that case, it introduces SurgVISTA, a video-level self-supervised framework trained on a new corpus of 3,650 surgical videos totaling about 3.55 million frames and spanning more than 20 procedures. It reports that SurgVISTA outperforms both natural-domain video models and existing surgical image-level foundation models on 13 datasets across four tasks, including phase recognition, action recognition, triplet recognition, and skill assessment. The central claim is that joint spatiotemporal pretraining on surgery-specific video supplies the temporal modeling that image-level surgical pretraining and general video pretraining both lack.

What carries the argument

The load-bearing mechanism is a dual-decoder masked video autoencoder. A video clip is tokenized with 2x16x16 cube embeddings, 85% of the tubes are masked, and a joint spatiotemporal ViT-B encoder processes the visible tokens; one decoder reconstructs masked pixels with an MSE loss to force learning of motion and global context, while a second decoder aligns encoder features to the spatial features of a surgery-specific image expert using a Smooth L1 distillation loss. The distillation branch compensates for the spatial detail lost in temporal downsampling. All components are trained from scratch on the surgical corpus, so temporal modeling is learned during pretraining rather than bolted on during fine-tuning.

What would settle it

Compare SurgVISTA against an identical architecture pretrained on a corpus that excludes every video from the same surgical procedure as each in-domain test set; if the margin over EndoSSL on Cholec80, M2CAI16-Workflow, and AutoLaparo collapses or reverses, the reported in-domain advantage is driven by exposure to same-procedure training videos rather than by video-level pretraining itself. The paper's own disclosure of five overlapping videos between M2CAI16-Workflow and the Cholec80 test set makes this check directly actionable.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that explicit spatiotemporal pretraining on surgical video transfers better to downstream surgical tasks than either natural-domain video pretraining or surgery-specific image-level pretraining. SurgVISTA achieves this by jointly reconstructing masked spacetime tubes and distilling spatial features from a surgery-specific image expert, so the encoder learns temporal dynamics and fine-grained anatomy in one pass. The paper reports consistent wins across all 13 evaluated datasets, with the largest margins on out-of-domain cataract and endoscopic submucosal dissection data, and shows that performance keeps rising as the pretraining corpus grows from 86,344 frames to 3.55 million frames.

Load-bearing premise

The central bet is that evaluating on test splits of benchmarks whose training videos were used in pretraining remains a fair test of generalization, as long as the test videos themselves were excluded.

Editorial extensions

If this is right

  • Image-level surgical foundation models can be improved by adding video-level pretraining without requiring new annotations.
  • End-to-end video pretraining removes the need for separate temporal modules at fine-tuning, simplifying downstream deployment.
  • Scaling surgical pretraining data helps even when newly added videos come from unrelated procedures, not just same-procedure footage.
  • Out-of-domain generalization to unseen procedures improves, suggesting the model learns transferable spatiotemporal primitives.
  • Evaluation integrity depends on excluding test videos from pretraining; the paper removes five overlapping Cholec80/M2CAI16 test videos to enforce this.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same dual-decoder recipe could transfer to other procedure-dense video domains such as endoscopy, interventional radiology, or robotic teleoperation, where temporal cues are central.
  • The reported in-domain gains might shrink if pretraining excluded all same-procedure training videos; a direct test would pretrain strictly on unrelated procedures for every in-domain benchmark.
  • The distillation weight of 0.05 and the 85% masking ratio are inherited from natural-domain practice, so ablating them on surgical data could yield further improvements.
  • The scaling results imply that noisy web-sourced surgical video is a valuable pretraining resource despite lacking expert annotations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces SurgVISTA, a video-level self-supervised pretraining framework for surgical video analysis. The method combines masked spatiotemporal reconstruction with image-level knowledge distillation from an existing surgical image model (EndoSSL), and is pretrained on a newly curated corpus of 3,650 videos (~3.55 million frames) spanning over 20 procedures. The authors evaluate SurgVISTA on 13 public and private datasets covering workflow recognition, action recognition, triplet recognition, and skill assessment, reporting consistent gains over natural-domain image- and video-level pretrained baselines and over existing surgical-domain pretrained models. Additional experiments examine data scaling laws and the contribution of knowledge distillation. The paper claims that SurgVISTA is the first video-level surgical pretraining framework and that it significantly outperforms all compared methods across all metrics and datasets.

Significance. If the central claim is established, the paper would provide a useful large-scale video-level pretraining recipe and benchmark for surgical scene understanding. The work has notable strengths: a broad evaluation spanning 13 datasets and four tasks, 95% confidence intervals and P-values, explicit data-scaling experiments, a knowledge-distillation ablation, and a plan to release code and the SurgPub collection. The out-of-domain evaluations on cataract and ESD data, where the pretraining corpus contains no corresponding procedure-specific videos, provide some independent evidence that larger and more diverse surgical video pretraining can improve transfer. However, the main in-domain superiority claim rests on an evaluation protocol in which the training splits of several downstream benchmarks are included in the pretraining corpus, so the reported gains over baselines that never saw those videos are confounded by direct exposure to the benchmark distribution. Because this confound affects the paper's headline claim, the current evidence is not sufficient for acceptance without substantial revision.

major comments (4)
  1. [Methods, Data Splitting Strategy; Extended Data Tables 2 and 3] The core in-domain evaluation is confounded by the inclusion of downstream training videos in the pretraining corpus. Specifically, the official training splits of Cholec80 (40 videos), M2CAI16-Workflow (27 videos), and AutoLaparo (10 videos) are used both for pretraining and for downstream fine-tuning; CholecT50's training set is derived from Cholec80 videos that were already included in pretraining; Cholec80-CVS training clips come from Cholec80; and Prostate21's training set includes all PSI-AVA videos that are also in the pretraining data. Excluding test splits from pretraining does not remove this overlap. Consequently, the large in-domain gains over natural-domain baselines and over EndoSSL (which was not pretrained on these exact videos) conflate the benefit of video-level pretraining with simple exposure to the benchmark's training frames. The Discussion paragraph on the trade-off between training scale and evaluation integrity acknowledges the issue but does not resolve it. To support the in-domain claim, the authors should either pretrain without any downstream training split or restrict in-domain claims to datasets whose training videos were entirely absent from pretraining.
  2. [Results, Fig. 2 and Extended Data Table 9] The statement that SurgVISTA 'significantly outperformed all comparative methods across all evaluation metrics and datasets' is contradicted by the authors' own table. On CATARACTS, SurgVISTA achieves video-level accuracy of 85.5%, identical to MVD's 85.5%, with P=0.478 for that metric. The blanket claim should be qualified to list the specific metrics and datasets where the improvement is statistically significant, and the non-significant comparisons should be acknowledged in the text.
  3. [Methods, Benchmark Datasets; Extended Data Table 3] The Cholec80 test set is modified by removing five videos that overlap with the M2CAI16-Workflow training split, leaving 35 instead of the official 40 test videos. Excluding the overlapping videos is a reasonable leakage-prevention step, but it means that Cholec80 numbers in this paper are not directly comparable to published results on the official split. The modified split should be clearly flagged in every table and figure involving Cholec80, and ideally the authors should report results both on the official 40-video test set and on the reduced 35-video set so that comparisons to prior work remain meaningful.
  4. [Methods, Experimental settings and Fig. 4] The central thesis that video-level surgical pretraining outperforms image-level surgical pretraining is not cleanly tested because there is no image-level baseline pretrained on the same SurgPub/SurgWeb data and evaluated under the identical fine-tuning protocol. The four surgical-domain baselines differ in pretraining corpus (Cholec80, Endo700k, 23.3M private frames, GenSurgery), backbone (ResNet-50, ViT-B, ViT-L, EfficientNet), and training objective, so their performance differences cannot be attributed specifically to the presence or absence of temporal pretraining. The no-KD ablation in Fig. 6 only removes the distillation branch while keeping video-level reconstruction intact. A necessary control is an image-level variant of SurgVISTA trained on the same frames with the same encoder and fine-tuning procedure; without this control, the claimed advantage of joint spatiotemporal pretraining over image-level pretraining remains underdetermined.
minor comments (7)
  1. [Extended Data Table 3] Prostate21 is listed under the surgical procedure 'Endoscopic Pituitary Surgery', but the text and the data provenance (PSI-AVA, radical prostatectomy) indicate that this should be 'Radical Prostatectomy'. Please correct the table.
  2. [Extended Data Table 2] For PSI-AVA, the 'Statistic' column reports 73,626 frames while the 'Available' column reports 72,318 frames; the discrepancy should be explained or corrected.
  3. [Extended Data Table 23] The rows for Setting B and Setting C are identical, and the Setting D row contains malformed entries such as '57.8950.10' and '41.44'; these appear to be copy/formatting errors and should be verified.
  4. [Extended Data Table 4] The P-value column contains '<0.00' instead of '<0.001'; please fix the typo.
  5. [Results, Data scaling laws] The section heading contains a duplicated word: 'surgical surgical pre-training data'; please correct.
  6. [Fig. 4 caption and text] The caption refers to subfigures (D)-(F) as out-of-domain, while the text says Fig. 4 (D)-(E); make the numbering consistent.
  7. [Introduction and Related work] The claim of being the 'first video-level surgical pre-training framework' should be qualified, since GSViT (ref. 19) is described in the same paper as a video-pretrained model using next-frame prediction; the novelty should be stated more precisely as joint masked spatiotemporal reconstruction with knowledge distillation.

Circularity Check

1 steps flagged · score 6.0 of 10

In-domain evaluation is partially circular because pretraining includes downstream training splits; out-of-domain results provide independent support.

  1. fitted input called prediction [Methods: Data Splitting Strategy; Pre-training Datasets; Benchmark Datasets; Results; Discussion]
    "First, no videos used during the pre-training phase were included in the test splits of any downstream datasets. ... For pre-training, we exclusively utilize the 40 videos from the training set [of Cholec80], amounting to a total of 86,344 frames. ... Following the official data split protocol, our pre-training stage includes videos from both the Cholec80 and M2CAI16-Workflow training sets. ... Overall, SurgVISTA significantly outperformed all comparative methods across all evaluation metrics and datasets."

    The pretraining corpus includes the official training splits of several downstream benchmarks (Cholec80, M2CAI16-Workflow, AutoLaparo, and PSI-AVA used for Prostate21 training), and the same training splits are later used for fine-tuning. Excluding only the test splits does not remove the confound: SurgVISTA has already learned representations from the exact videos and frames that constitute the downstream training distribution, whereas the natural-domain baselines and even EndoSSL have not. The reported in-domain superiority therefore partly reduces, by construction, to pretraining exposure to the benchmark training distribution rather than to a clean test of video-level surgical pretraining. The paper's own Discussion concedes this trade-off but does not resolve it.

full rationale

The core method—joint spatiotemporal masked reconstruction plus image-level knowledge distillation—is not circular in itself; the architecture and losses are defined independently of the evaluation targets. The main circularity is in the benchmark construction: the in-domain evaluation sets are defined as procedures included in the pretraining data, and the pretraining data includes the official training splits of Cholec80, M2CAI16-Workflow, AutoLaparo, and PSI-AVA (whose videos form Prostate21's training set), plus Cholec80-derived clips used for CholecT50 and Cholec80-CVS. Thus the claim that SurgVISTA 'significantly outperformed all comparative methods across all evaluation metrics and datasets' is not cleanly supported for the in-domain datasets, because the comparison partly measures exposure to the same training videos. The out-of-domain cataract and ESD results, along with the no-knowledge-distillation ablation, provide independent evidence that the video-level pretraining has some genuine benefit, but they do not rescue the blanket in-domain claim. No load-bearing self-citation chain, ansatz-smuggling via citation, or renaming of a known result was found.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two load-bearing assumptions that the paper treats as benign: pretraining on downstream training splits does not bias in-domain evaluation, and the EndoSSL distillation teacher does not leak benchmark information. Both assumptions are plausible but unverified, and both directly affect the interpretation of the reported gains. The listed free parameters are design choices that are not fitted to the downstream tasks but are also not ablated, so their influence on the central claim is unknown.

free parameters (4)
  • KD loss weight = 0.05
    Hand-chosen weight balancing reconstruction and EndoSSL distillation losses (Method, Implementation Details). No ablation over this value; it moderates the distillation branch that contributes to the claimed gains.
  • Masking ratio = 85%
    Tube masking ratio for masked video modeling, adopted from VideoMAE; not tuned for surgery. Affects the difficulty of the reconstruction pretext task and therefore the learned representations.
  • Pretraining clip schedule = 16 frames, interval 4, 1 fps
    Input clip length and sampling chosen in implementation details; assumes surgical dynamics are captured at 1 fps with 4-frame spacing, but no ablation justifies this temporal scale.
  • Pretraining epochs = 200
    Epoch count adjusted proportionally to dataset size; final checkpoint at end of training, not selected by a validation criterion. The claim of convergence is asserted, not shown with curves.
assumptions (3)
  • domain assumption Masked video reconstruction on 1-fps surgical clips induces transferable spatiotemporal representations.
    The entire method rests on this premise; it is validated only indirectly through downstream fine-tuning, never by measuring representation quality against a control on identical data.
  • ad hoc to paper Including downstream training videos in pretraining does not bias in-domain evaluation as long as test splits are excluded.
    Data Splitting Strategy states this policy. It is load-bearing for the in-domain comparisons, but pretraining on the training distribution of the same benchmarks can improve performance even without label leakage.
  • ad hoc to paper EndoSSL expert features provide clean spatial supervision without leaking downstream labels.
    The distillation branch aligns encoder features to EndoSSL, a model pretrained on 23.3M private laparoscopic frames. If that private corpus overlaps the benchmark videos, the distillation channel reintroduces information from the evaluation distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large-scale Self-supervised Video Foundation Model for Intelligent Surgery." pith.science (2026). https://pith.science/paper/FAELIIDA

@misc{pith2026250602692,
  author       = {Pith},
  title        = {Pith review of: Large-scale Self-supervised Video Foundation Model for Intelligent Surgery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FAELIIDA}},
  note         = {Machine review of arXiv:2506.02692}
}
read the original abstract

Computer-Assisted Intervention (CAI) has the potential to revolutionize modern surgery, with surgical scene understanding serving as a critical component in supporting decision-making, improving procedural efficacy, and ensuring intraoperative safety. While existing AI-driven approaches alleviate annotation burdens via self-supervised spatial representation learning, their lack of explicit temporal modeling during pre-training fundamentally restricts the capture of dynamic surgical contexts, resulting in incomplete spatiotemporal understanding. In this work, we introduce the first video-level surgical pre-training framework that enables joint spatiotemporal representation learning from large-scale surgical video data. To achieve this, we constructed a large-scale surgical video dataset comprising 3,650 videos and approximately 3.55 million frames, spanning more than 20 surgical procedures and over 10 anatomical structures. Building upon this dataset, we propose SurgVISTA (Surgical Video-level Spatial-Temporal Architecture), a reconstruction-based pre-training method that captures intricate spatial structures and temporal dynamics through joint spatiotemporal modeling. Additionally, SurgVISTA incorporates image-level knowledge distillation guided by a surgery-specific expert to enhance the learning of fine-grained anatomical and semantic features. To validate its effectiveness, we established a comprehensive benchmark comprising 13 video-level datasets spanning six surgical procedures across four tasks. Extensive experiments demonstrate that SurgVISTA consistently outperforms both natural- and surgical-domain pre-trained models, demonstrating strong potential to advance intelligent surgical systems in clinically meaningful scenarios.

Figures

Figures reproduced from arXiv: 2506.02692 by the authors.

Figure 1
Figure 1. Overview of the study. (A) Pre-training Dataset: Illustration of the primary anatomical structures and their associated surgical procedures, complemented by video-level statistics and detailed frame-level distribution analysis. (B) SurgVISTA Framework: An asymmetric encoder-decoder architecture featuring a unified encoder for comprehensive spatiotemporal modeling, and two decoders: one dedicated to video-level recon… view at source ↗
Figure 3
Figure 3. Experimental results of various natural-domain pre-trained methods and SurgVISTA on surgical tasks beyond phase recognition. Subfigure (A) presents the results for the surgical action recognition task, with evaluation metrics reported as accuracy and mAP. Subfigures (B)-(C) illustrate the surgical triplet recognition task, with performance reported in terms of triplet mAP and 2-tuple mAP. Subfigures (D)-(E) display … view at source ↗
Figure 4
Figure 4. Experimental results comparing SurgVISTA and various surgical-domain pre-trained methods on surgical phase recognition datasets. Subfigures (A)-(C) correspond to in-domain datasets, while (D)-(F) represent out-of-domain datasets. Evaluation metrics include image-level accuracy, video-level accuracy, phase-level precision, phase-level recall and phase-level Jaccard, providing a comprehensive assessment across multipl… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Experimental overview of how varying pre-training data scales affect the performance and generalization. Subfigure (A) illustrates the distribution of surgical procedures across progressively constructed sub-datasets. Subfigures (B)-(J) present performance variations a…
Figure 6
Figure 6. Figure 6: The effectiveness of knowledge distillation. Subfigures (A)-(B) present the performance difference between SurgVISTA with and without knowledge distillation. Significance testing was conducted using the Wilcoxon signed-rank one-sided test, demonstrating that expert kno…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On the Role of Depth in Surgical Vision Foundation Models: An Empirical Study of RGB-D Pre-training

    cs.CV 2026-01 conditional novelty 6.0 of 10

    RGB-D pre-training with explicit cross-modal objectives (MultiMAE) improves surgical detection, segmentation, pose, and depth estimation over RGB-only pre-training, with gains persisting when fine-tuned on 25% of labe...

Reference graph

Works this paper leans on

60 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    biomedical engineering7, 780–796 (2023)

    Kiyasseh, D.et al.A vision transformer for decoding surgeon activity from surgical videos.Nat. biomedical engineering7, 780–796 (2023)

    work page 2023

  2. [2]

    Medicine5, 187 (2022)

    Ma, R.et al.Surgical gestures as a method to quantify surgical performance and predict patient outcomes.NPJ Digit. Medicine5, 187 (2022)

    work page 2022

  3. [3]

    Medicine 6, 54 (2023)

    Kiyasseh, D.et al.Human visual explanations mitigate bias in ai-based assessment of surgeon skills.NPJ Digit. Medicine 6, 54 (2023)

    work page 2023

  4. [4]

    C.et al.Deep learning in surgical workflow analysis: a review of phase and step recognition.IEEE J

    Demir, K. C.et al.Deep learning in surgical workflow analysis: a review of phase and step recognition.IEEE J. Biomed. Heal. Informatics27, 5405–5417 (2023). 5.Maier-Hein, L.et al.Surgical data science for next-generation interventions.Nat. Biomed. Eng.1, 691–696 (2017)

    work page 2023

  5. [6]

    Surgical data science

    Surgical Data Science Initiative. Surgical data science. http://www.surgical-data-science.org/ (2025). Accessed: April 12, 2025

    work page 2025

  6. [7]

    image analysis76, 102306 (2022)

    Maier-Hein, L.et al.Surgical data science–from concepts toward clinical translation.Med. image analysis76, 102306 (2022)

    work page 2022

  7. [8]

    InProceedings of the IEEE/CVF international conference on computer vision, 9650–9660 (2021)

    Caron, M.et al.Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF international conference on computer vision, 9650–9660 (2021)

    2021

  8. [9]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000–16009 (2022)

    He, K.et al.Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000–16009 (2022). 10.Oquab, M.et al.Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193(2023)

    arXiv 2022

Show all 60 references
  1. [11]

    Azizi, S.et al.Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging.Nat. Biomed. Eng.7, 756–779 (2023)

    2023

  2. [12]

    Ma, J.et al.Towards a generalizable pathology foundation model via unified knowledge distillation.arXiv preprint arXiv:2407.18449(2024). 19/39

    2024 arXiv

  3. [13]

    Medicine6, 74 (2023)

    Huang, S.-C.et al.Self-supervised learning for medical image classification: a systematic review and implementation guidelines.NPJ Digit. Medicine6, 74 (2023)

    2023

  4. [14]

    & Topol, E

    Krishnan, R., Rajpurkar, P. & Topol, E. J. Self-supervised learning in medicine and healthcare.Nat. Biomed. Eng.6, 1346–1352 (2022)

    2022

  5. [15]

    J.et al.Towards a general-purpose foundation model for computational pathology.Nat

    Chen, R. J.et al.Towards a general-purpose foundation model for computational pathology.Nat. Medicine30, 850–862 (2024)

    2024

  6. [16]

    Image Analysis88, 102844 (2023)

    Ramesh, S.et al.Dissecting self-supervised learning methods for surgical computer vision.Med. Image Analysis88, 102844 (2023)

    2023

  7. [17]

    & Navab, N

    Bati´c, D., Holm, F., Özsoy, E., Czempiel, T. & Navab, N. Endovit: pretraining vision transformers on a large collection of endoscopic images.Int. J. Comput. Assist. Radiol. Surg.19, 1085–1091 (2024)

    2024

  8. [18]

    InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 569–578 (Springer, 2023)

    Hirsch, R.et al.Self-supervised learning for endoscopic video analysis. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 569–578 (Springer, 2023)

    2023

  9. [19]

    W., Jopling, J

    Schmidgall, S., Kim, J. W., Jopling, J. & Krieger, A. General surgery vision transformer: A video pre-trained foundation model for general surgery.arXiv preprint arXiv:2403.05949(2024)

    2024 arXiv

  10. [20]

    Czempiel, T.et al.Tecno: Surgical phase recognition with multi-stage temporal convolutional networks. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, 343–352 (Spri...

    2020

  11. [21]

    P.et al.Endonet: a deep architecture for recognition tasks on laparoscopic videos.IEEE transactions on medical imaging36, 86–97 (2016)

    Twinanda, A. P.et al.Endonet: a deep architecture for recognition tasks on laparoscopic videos.IEEE transactions on medical imaging36, 86–97 (2016)

    2016

  12. [22]

    Stauder, R.et al.The tum lapchole dataset for the m2cai 2016 workflow challenge.arXiv preprint arXiv:1610.09278 (2016)

    2016 arXiv

  13. [23]

    Image Analysis86, 102770 (2023)

    Wagner, M.et al.Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the heichole benchmark.Med. Image Analysis86, 102770 (2023)

    2023

  14. [24]

    Das, A.et al.Pitvis-2023 challenge: Workflow recognition in videos of endoscopic pituitary surgery.arXiv preprint arXiv:2409.01184(2024)

    2024 arXiv

  15. [25]

    InInternational conference on medical image computing and computer-assisted intervention, 442–452 (Springer, 2022)

    Valderrama, N.et al.Towards holistic surgical scene understanding. InInternational conference on medical image computing and computer-assisted intervention, 442–452 (Springer, 2022)

    2022

  16. [26]

    InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 486–496 (Springer, 2022)

    Wang, Z.et al.Autolaparo: A new dataset of integrated multi-tasks for image-guided surgical automation in laparoscopic hysterectomy. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 486–496 (Springer, 2022)

    2022

  17. [27]

    L.et al.Challenges in multi-centric generalization: Phase and step recognition in roux-en-y gastric bypass surgery

    Lavanchy, J. L.et al.Challenges in multi-centric generalization: Phase and step recognition in roux-en-y gastric bypass surgery. https://arxiv.org/abs/2312.11250 (2023). ArXiv:2312.11250

    2023 arXiv

  18. [28]

    Dosovitskiy, A.et al.An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929(2020)

    2020 arXiv

  19. [29]

    & Wang, L

    Tong, Z., Song, Y ., Wang, J. & Wang, L. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training.Adv. neural information processing systems35, 10078–10093 (2022)

    2022

  20. [30]

    Wang, Y .et al.Internvideo: General video foundation models via generative and discriminative learning.arXiv preprint arXiv:2212.03191(2022)

    2022 arXiv

  21. [31]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6312–6322 (2023)

    Wang, R.et al.Masked video distillation: Rethinking masked feature modeling for self-supervised video representation learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6312–6322 (2023)

    2023

  22. [32]

    InProceedings of the IEEE/CVF International Conference on Computer Vision, 19948–19960 (2023)

    Li, K.et al.Unmasked teacher: Towards training-efficient video foundation models. InProceedings of the IEEE/CVF International Conference on Computer Vision, 19948–19960 (2023)

    2023

  23. [33]

    White, C.et al.Livebench: A challenging, contamination-free llm benchmark.arXiv preprint arXiv:2406.193144(2024)

    2024 arXiv

  24. [34]

    & Heng, P.-A

    Guo, D., Si, W., Li, Z., Pei, J. & Heng, P.-A. Surgical workflow recognition and blocking effectiveness detection in laparoscopic liver resections with pringle maneuver. InProceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25)(2025)

    2025

  25. [35]

    image analysis52, 24–41 (2019)

    Al Hajj, H.et al.Cataracts: Challenge on automatic tool annotation for cataract surgery.Med. image analysis52, 24–41 (2019). 20/39

    2019

  26. [36]

    InProceedings of the 9th ACM multimedia systems conference, 421–425 (2018)

    Schoeffmann, K.et al.Cataract-101: video dataset of 101 cataract surgeries. InProceedings of the 9th ACM multimedia systems conference, 421–425 (2018)

    2018

  27. [37]

    J.et al.Frame-based classification of operation phases in cataract surgery videos

    Primus, M. J.et al.Frame-based classification of operation phases in cataract surgery videos. InMultiMedia Modeling: 24th International Conference, MMM 2018, Bangkok, Thailand, February 5-7, 2018, Proceedings, Part I 24, 241–253 (Springer, 2018)

    2018

  28. [38]

    Imaging40, 1911–1923 (2021)

    Jin, Y .et al.Temporal memory relation network for workflow recognition from surgical video.IEEE Transactions on Med. Imaging40, 1911–1923 (2021)

    2021

  29. [39]

    & Chen, H

    Yang, S., Luo, L., Wang, Q. & Chen, H. Surgformer: Surgical transformer with hierarchical temporal attention for surgical phase recognition. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 606–616 (Springer, 2024)

    2024

  30. [40]

    Yang, S.et al.Surgpetl: Parameter-efficient image-to-surgical-video transfer learning for surgical phase recognition.arXiv preprint arXiv:2409.20083(2024)

    2024 arXiv

  31. [41]

    Tools Appl.77, 16813–16832, DOI: 10.1007/s11042-017-5252-2 (2018)

    Schoeffmann, K.et al.Video retrieval in laparoscopic video recordings with dynamic content descriptors.Multim. Tools Appl.77, 16813–16832, DOI: 10.1007/s11042-017-5252-2 (2018)

    2018 doi

  32. [42]

    I.et al.Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos

    Nwoye, C. I.et al.Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Med. Image Analysis78, 102433 (2022)

    2022

  33. [43]

    I.et al.Recognition of instrument-tissue interactions in endoscopic videos via action triplets

    Nwoye, C. I.et al.Recognition of instrument-tissue interactions in endoscopic videos via action triplets. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, 364–374 (...

    2020

  34. [44]

    S.et al.Cholec80-cvs: An open dataset with an evaluation of strasberg’s critical view of safety for ai.Sci

    Ríos, M. S.et al.Cholec80-cvs: An open dataset with an evaluation of strasberg’s critical view of safety for ai.Sci. Data 10, 194 (2023)

    2023

  35. [45]

    Imaging (2023)

    Murali, A.et al.Latent graph representations for critical view of safety assessment.IEEE Transactions on Med. Imaging (2023)

    2023

  36. [46]

    In2009 IEEE conference on computer vision and pattern recognition, 248–255 (Ieee, 2009)

    Deng, J.et al.Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, 248–255 (Ieee, 2009). 47.Kay, W.et al.The kinetics human action video dataset.arXiv preprint arXiv:1705.06950(2017)

    2017 arXiv

  37. [48]

    Li, K.et al.Uniformerv2: Spatiotemporal learning by arming image vits with video uniformer.arXiv preprint arXiv:2211.09552(2022)

    2022 arXiv

  38. [49]

    something something

    Goyal, R.et al.The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, 5842–5850 (2017)

    2017

  39. [50]

    InProceedings of the IEEE conference on computer vision and pattern recognition, 6047–6056 (2018)

    Gu, C.et al.Ava: A video dataset of spatio-temporally localized atomic visual actions. InProceedings of the IEEE conference on computer vision and pattern recognition, 6047–6056 (2018)

    2018

  40. [51]

    & Zisserman, A

    Bain, M., Nagrani, A., Varol, G. & Zisserman, A. Frozen in time: A joint video and image encoder for end-to-end retrieval. InProceedings of the IEEE/CVF international conference on computer vision, 1728–1738 (2021)

    2021

  41. [52]

    & Zisserman, A

    Carreira, J., Noland, E., Banki-Horvath, A., Hillier, C. & Zisserman, A. A short note about kinetics-600.arXiv preprint arXiv:1808.01340(2018)

    2018 arXiv

  42. [53]

    & Zisserman, A

    Carreira, J., Noland, E., Hillier, C. & Zisserman, A. A short note on the kinetics-700 human action dataset.arXiv preprint arXiv:1907.06987(2019)

    2019 arXiv

  43. [54]

    InInternational conference on machine learning, 8748–8763 (PMLR, 2021)

    Radford, A.et al.Learning transferable visual models from natural language supervision. InInternational conference on machine learning, 8748–8763 (PMLR, 2021)

    2021

  44. [55]

    Chen, X., Fan, H., Girshick, R. & He, K. Improved baselines with momentum contrastive learning.arXiv preprint arXiv:2003.04297(2020)

    2020 arXiv

  45. [56]

    & Hinton, G

    Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. InInternational conference on machine learning, 1597–1607 (PMLR, 2020)

    2020

  46. [57]

    neural information processing systems33, 9912–9924 (2020)

    Caron, M.et al.Unsupervised learning of visual features by contrasting cluster assignments.Adv. neural information processing systems33, 9912–9924 (2020)

    2020

  47. [58]

    & Sun, J

    He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016). 21/39

    2016

  48. [59]

    InEuropean Conference on Computer Vision, 456–473 (Springer, 2022)

    Assran, M.et al.Masked siamese networks for label-efficient learning. InEuropean Conference on Computer Vision, 456–473 (Springer, 2022)

    2022

  49. [60]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14420–14430 (2023)

    Liu, X.et al.Efficientvit: Memory efficient vision transformer with cascaded group attention. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14420–14430 (2023)

    2023

  50. [61]

    Nwoye, C. I. & Padoy, N. Data splits and metrics for method benchmarking on surgical action triplet datasets.arXiv preprint arXiv:2204.05235(2022)

    2022 arXiv

  51. [62]

    S.et al.The saras endoscopic surgeon action detection (esad) dataset: Challenges and methods.arXiv preprint arXiv:2104.03178(2021)

    Bawa, V . S.et al.The saras endoscopic surgeon action detection (esad) dataset: Challenges and methods.arXiv preprint arXiv:2104.03178(2021)

    2021 arXiv

  52. [63]

    Statistic

    Murali, A.et al.The endoscapes dataset for surgical scene segmentation, object detection, and critical view of safety assessment: Official splits and benchmark.arXiv preprint arXiv:2312.12429(2023). 22/39 A Extended Data Table 1.Detailed distribution of the SurgPub and SurgWeb...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.