Pith. sign in

REVIEW 4 major objections 5 minor 47 references

SurgBench: A Unified Large-Scale Benchmark for Surgical Video Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pretraining on 53 million surgical video frames outperforms natural-video pretraining on surgical tasks.

desk verdict A genuinely useful surgical video benchmark, but the paper never rules out that SurgBench-P includes SurgBench-E test frames, which would dissolve the headline +7% claim. read the letter →

arxiv 2506.07603 v2 pith:RWV2LJQV submitted 2025-06-09 cs.CV cs.AI

Jianhui Wei , Zikai Xiao , Danyu Sun , Luqi Gong , Zongxin Yang , Zuozhu Liu , Jian Wu This is my paper · ORCID
classification cs.CVcs.AI
keywords surgicalvideounderstandingfoundationmodelsself-supervisedpretrainingMAEcontinualevaluationbenchmarkdomaingeneralizationphaseclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SurgBench, a combined pretraining dataset (SurgBench-P) and evaluation benchmark (SurgBench-E) for surgical video understanding. SurgBench-P pools about 53 million frames from 16 sources spanning 11 specialties and 22 procedures, and SurgBench-E turns 23,004 clips into 72 fine-grained tasks across six categories: phase classification, camera motion, tool recognition, disease diagnosis, action classification, and organ detection. The paper's central claim is that continually pretraining a VideoMAE model on SurgBench-P, starting from its Kinetics-400 weights, improves downstream accuracy by 7% top-1 and 7.9% top-3 relative to the Kinetics-400-pretrained model, and that the benefit carries over to an unseen gynecologic procedure. If the claim holds, diverse surgical-video pretraining provides a practical route to video models that transfer across unfamiliar surgical procedures and imaging modalities, which matters for automated decision support, skill assessment, and postoperative review.

What carries the argument

The load-bearing mechanism is a four-stage continual pretraining protocol applied to VideoMAE, a self-supervised masked-autoencoder video model that reconstructs randomly masked spatiotemporal tubes of patches (masking ratio 0.9). The stages move from all 225,250 collected clips, to filtering over-dominant sources such as A VOS, to upsampling underrepresented data, and finally to a 'precise IID-oriented stage with both upsampling and downsampling' that aligns the pretraining distribution with SurgBench-E. The evaluation side is the 6C-10S-72T taxonomy, which maps six task categories and ten sub-categories onto 72 finetuning tasks, allowing one backbone to be scored across phase, motion, tool, disease, action, and organ understanding.

What would settle it

A decisive test is to run the same four-stage continual pretraining twice, once with the final upsampling and downsampling decided before any SurgBench-E annotation is consulted and once informed by SurgBench-E label frequencies; if both reach the same accuracy on an unseen procedure such as LapGyn4, the generalization claim stands, and if only the SurgBench-E-informed run keeps the 7.9% top-3 gain, the central claim reduces to distribution matching.

Watch

Extended reading notes

Core claim

The paper claims that surgical video understanding is limited less by model architecture than by pretraining data: a model that has seen a broad mix of surgical video can keep learning transferable representations beyond what general video data provides. SurgBench operationalizes that mix by standardizing 16 heterogeneous sources into uniform clips and organizing an evaluation hierarchy of 6 categories, 10 sub-categories, and 72 tasks. With the same VideoMAE backbone, SurgMAE (continually pretrained on SurgBench-P) reaches 44.8% average top-1 and 73.1% average top-3 on SurgBench-E, against 37.8% and 65.2% for the Kinetics-400 baseline; the large variant reaches 48.7% and 78.8%. The paper also reports a 2.69% gain on LapGyn4, a gynecologic event-recognition dataset not seen in pretraining, as evidence that the learned representations generalize to unseen procedures and domains.

Load-bearing premise

The paper's load-bearing assumption is that the last pretraining stage, which adds and removes data to make the distribution look more balanced, is chosen using general data-quality rules rather than the evaluation benchmark's own labels or statistics; if that assumption fails, the reported gains over Kinetics-400 would mostly come from matching the test set instead of learning transferable surgical video knowledge.

Editorial extensions

If this is right

  • A single SurgBench-P-pretrained backbone can be fine-tuned into competitive models across six distinct surgical task families, reducing the need for task-specific pretraining pipelines.
  • The 72-task hierarchy gives researchers a common yardstick, so future surgical foundation models can be compared on the same sub-categories rather than on scattered single-task datasets.
  • The reported LapGyn4 gain indicates that continual pretraining on SurgBench-P transfers to procedures and video distributions not present in the pretraining corpus.
  • Model scale compounds the benefit: the large SurgMAE variant improves average top-1 by 3.9% and top-3 by 5.7% over the standard variant, so the benchmark can support scaling studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial caution: the paper does not state what information guides the final 'precise IID-oriented stage' of pretraining; if that stage rebalances toward SurgBench-E label or statistic frequencies, part of the reported gain over Kinetics-400 would be test-set distribution matching rather than transferable surgical knowledge.
  • Reproduction will be harder than the abstract suggests: the appendix notes that 330 A VOS video URLs have expired (leaving 296 annotated videos), three restricted-license sources used in pretraining are not redistributed, and one private hospital dataset appears only on request.
  • The paper gives different SurgBench-P frame counts in different sections (53M, 56M, and 74.4M), so the exact preprocessing definition needs pinning down before the headline numbers can be compared across future work.
  • The authors' own Section 7 limitations, long-tail class distribution, no language-supervision integration, and unoptimized architectures, suggest that the benchmark's hardest open problems are rare-class recognition and multimodal understanding, not the core pretraining pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SurgBench, a unified surgical video pretraining corpus (SurgBench-P) and evaluation benchmark (SurgBench-E) assembled from 16 sources, spanning 22 procedures, 11 specialties, and 72 fine-grained tasks in 6 categories. The authors continually pretrain VideoMAE models (Standard and Large) on SurgBench-P, fine-tune on SurgBench-E, and report that surgical-domain pretraining improves top-1 accuracy by 7% and top-3 accuracy by 7.9% over Kinetics-pretrained VideoMAE, with additional evidence from a LapGyn out-of-distribution experiment. The central claim is that self-supervised continual pretraining on a large, mixed-specialty surgical corpus yields transferable representations beyond those learned from natural video.

Significance. If the result holds, SurgBench would be a valuable community asset: it is substantially broader than existing surgical pretraining corpora (e.g., Endo-FM, GSViT, Surg-3M), and the 6C-10S-72T taxonomy provides a unified evaluation protocol. The paper also has methodological strengths: within SurgBench-E, clips from the same video are split so that they do not appear in both train and test, the backbone is frozen during fine-tuning to isolate representation quality, and the long-tail imbalance factor is capped. The LapGyn result, if taken at face value, provides independent evidence of cross-domain transfer. However, the headline comparison depends critically on whether SurgBench-E evaluation frames are excluded from the SurgBench-P pretraining corpus, a point the manuscript does not establish.

major comments (4)
  1. [Sections 3.3, 3.4, and Table 2] The central claim of Section 5.1 (+7% top-1, +7.9% top-3 over Kinetics) rests on SurgBench-P being a legitimate pretraining corpus that does not contain SurgBench-E test clips. The manuscript never states that evaluation frames are excluded from pretraining. Table 2 lists 'Pre-train Frames' and 'Evaluation Frames' for the same source IDs (e.g., S8 JIGSAWS: 569,048 vs. 537,645; S10 AutoLaparo: 2,155,843 vs. 160,221), and the two columns sum to 59,742,875 frames, exactly matching the total in Section 3.4. This arithmetic implies that essentially all frames from each source, including those later used for evaluation, were available to the pretraining stage. Section 3.3 guarantees only that, within SurgBench-E, clips from the same video do not appear in both the fine-tuning train and test sets; it is silent about pretraining. The paper must either state explicitly that SurgBench-E test clips were excluded from SurgBench-P, or the reported gains must be reinterpreted as test-content familiarity rather than transferable surgical representations.
  2. [Section 4, final IID-oriented stage] The four-stage continual pretraining procedure is underspecified at the load-bearing point. Stage 4 is described as 'a final precise IID-oriented stage with both upsampling and downsampling,' but the manuscript does not say what information guides this rebalancing. If the final stage uses label statistics or other statistics from the SurgBench-E sources to align the pretraining distribution with the evaluation tasks, then the comparison against Kinetics is biased by test-distribution matching. Please specify the exact criterion used for upsampling/downsampling in stage 4 and state whether any SurgBench-E labels, splits, or task definitions influenced the composition of SurgBench-P.
  3. [Sections 3.4, 4, and Abstract] The frame-count statements are internally inconsistent. The abstract and Section 3.4 state 53 million frames for SurgBench-P and 59.7 million total frames; Section 4 refers to 'SurgBench-P (74.4 million frames total)'; Figure 1 reports '60 million frames in total'; and Table 2 sums to 59,742,875 frames. This inconsistency matters because the pretraining corpus size is part of the experimental description, and the 74.4M figure is not reconcilable with the tabulated source counts. Please unify these numbers and clarify which count corresponds to the actual clips used in the final CPT stage (39,807 clips) versus the raw collected corpus.
  4. [Section 5.4, LapGyn OOD result] The LapGyn experiment is presented as evidence of cross-domain generalization to unseen procedures, but it is a single task with no error bars or significance testing. The reported 2.69% gain is plausible but could be within run-to-run variation. Since this result is the only independent support for the generalizability claim that does not depend on the pretraining/evaluation overlap question, it should be reported with multiple seeds, confidence intervals, or at least a statement of the number of runs.
minor comments (5)
  1. [Section 5.1] There is a dangling reference 'as shown in Table ??' in the paragraph discussing per-category performance fluctuations; this should point to a specific table or figure.
  2. [Section 3.1 and Appendix A] The manuscript states that A VOS, SimSurgSkill2021, and AIxSuture cannot be redistributed and are used only for pretraining, but Table 2 lists 'Pre-train Frames' and 'Evaluation Frames' for these sources as well. Please clarify whether the 'Evaluation Frames' column for these sources is used only for internal validation or is actually released as part of SurgBench-E.
  3. [Section 6 and Appendix C] The conclusion refers to 'SurgBench-P, a diverse pretraining dataset comprising 53 million frames,' but Table 6 has a typo in its caption ('SurBennch-P'). Please correct the typo.
  4. [Section 1] The phrase 'enrish task diversity' in the introduction appears to be a typo for 'enrich task diversity.'
  5. [Section 3.3 and Figure 2] The caption of Figure 2 says the accuracy curves 'validate the reliability and data quality of SurgBench-E,' but the figure shows fine-tuning accuracy on individual source datasets; it would be clearer to state that this is a sanity check of label quality rather than a validation of the benchmark's generalization properties.

Circularity Check

2 steps flagged · score 6.0 of 10

SurgBench-P and SurgBench-E are built from the same 16 sources, and the final CPT stage is explicitly aligned to the downstream distribution, so the headline +7%/+7.9% gains are partly in-distribution rather than evidence of transferable surgical representations.

  1. other [Section 3.3-3.4, Table 2]
    "S8 JIGSAWS ... 569,048 537,645; S9 CholecT50 ... 90,444 207,169; S10 AutoLaparo ... 2,155,843 160,221; ... Total 56,062,458 3,680,417"

    Every source that contributes SurgBench-E evaluation frames also contributes pre-train frames to SurgBench-P, and the pre-train/eval columns sum to the total SurgBench frame count (59,742,875). The paper only enforces non-overlap between fine-tuning train and test clips (§3.3: 'clips from the same video could only appear in either the training set or the test set'); it never states that evaluation test videos or clips are excluded from SurgBench-P. Thus the model used for continual pretraining has already observed frames from the same surgical video domain, and possibly the same videos, that later appear in SurgBench-E. The headline comparison against Kinetics therefore measures partly in-distribution familiarity, not clean cross-domain transfer.

  2. fitted input called prediction [Section 4, Continual pre-training]
    "The data used for pre-training, SurgBench-P (74.4 million frames total), underwent a four-stage refinement to ensure that it could learn general representations from large-scale data while aligning with downstream tasks. The training steps involved: ... (4) a final precise IID-oriented stage with both upsampling and downsampling."

    The final pretraining stage is explicitly designed to align the pretraining distribution with the downstream SurgBench-E tasks ('IID-oriented'). If the final upsampling/downsampling is guided by SurgBench-E labels or class statistics, then the pretraining distribution is matched to the evaluation tasks before the evaluation is run, so the reported +7% top-1 and +7.9% top-3 gains over Kinetics are at least partly a product of test-distribution matching. The paper does not specify what information guides that final rebalancing, so the transfer claim is underdetermined and the improvements cannot be cleanly attributed to generalizable surgical video representations.

full rationale

Most formal derivation-circularity patterns do not apply: the paper derives no equations, and it uses VideoMAE/Kinetics as an external architecture rather than relying on a self-supporting uniqueness theorem. The confound is benchmark construction, not arithmetic. Section 3.4 and Table 2 show that the pretraining and evaluation corpora are two partition columns of the same 16 source datasets, with no stated holdout of evaluation videos from pretraining. Section 3.3 only guarantees non-overlap between fine-tuning train and test clips. Thus the headline comparison is in-distribution: SurgMAE has already seen the surgical video domain during masked autoencoding. Section 4 compounds this by explicitly aligning the final pretraining stage to the downstream task distribution, making the gain partially a consequence of distribution matching. The LapGyn OOD experiment is genuinely external and provides some independent support, but it is a single task without error bars and cannot validate the headline SurgBench-E numbers. On balance, the central claim is partially circular-by-construction, warranting a score of 6 rather than a clean 0-2.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No mathematical derivation is attempted, so the ledger contains dataset-construction hyperparameters and assumptions about label validity rather than fitted physical constants or new entities. The main hidden costs are the hand-set imbalance cap, the clip-length and masking choices, and reliance on inherited annotations.

free parameters (4)
  • Imbalance factor cap = 10
    SurgBench-E construction controls label imbalance by removing scarce labels, down-sampling dominant labels, and adjusting clip duration so the imbalance factor stays within 10 (Section 3.3). This hand-set threshold changes benchmark difficulty and composition.
  • Pretraining clip length = 10 seconds
    All unlabeled videos are split into 10-second clips for SurgBench-P (Section 3.3). The choice affects temporal context and the number of pretraining clips.
  • VideoMAE masking ratio = 0.9
    Continual pretraining uses the VideoMAE masking ratio of 0.9 (Section 4), a standard but hand-chosen hyperparameter on which the pretraining result depends.
  • Temporal sampling rate = 4
    Inputs are 16 frames per clip sampled every 4 frames (Section 4), determining how much temporal information the model sees.
assumptions (3)
  • domain assumption Clip-level dominant-label assignment preserves task semantics
    SurgBench-E assigns the most frequent phase label to a whole clip and converts multi-label samples into multiple single-label samples (Section 3.3, Appendix E). If a clip contains multiple phases or tools, this conversion injects label noise that affects all compared models.
  • domain assumption Source dataset annotations are accurate ground truth
    All 72 tasks inherit labels from 10 public and private datasets with different annotation protocols and expertise (Table 2). The benchmark's validity assumes these labels are correct and mutually comparable.
  • domain assumption Inherited labels from pretraining sources do not leak into evaluation
    Clips from the same video are kept entirely in train or test (Section 3.3), but videos from the same dataset and procedure appear in both pretraining and evaluation. The benchmark assumes no information leakage through the shared source distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SurgBench: A Unified Large-Scale Benchmark for Surgical Video Analysis." pith.science (2026). https://pith.science/paper/RWV2LJQV

@misc{pith2026250607603,
  author       = {Pith},
  title        = {Pith review of: SurgBench: A Unified Large-Scale Benchmark for Surgical Video Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RWV2LJQV}},
  note         = {Machine review of arXiv:2506.07603}
}
read the original abstract

Surgical video understanding is pivotal for enabling automated intraoperative decision-making, skill assessment, and postoperative quality improvement. However, progress in developing surgical video foundation models (FMs) remains hindered by the scarcity of large-scale, diverse datasets for pretraining and systematic evaluation. In this paper, we introduce \textbf{SurgBench}, a unified surgical video benchmarking framework comprising a pretraining dataset, \textbf{SurgBench-P}, and an evaluation benchmark, \textbf{SurgBench-E}. SurgBench offers extensive coverage of diverse surgical scenarios, with SurgBench-P encompassing 53 million frames across 22 surgical procedures and 11 specialties, and SurgBench-E providing robust evaluation across six categories (phase classification, camera motion, tool recognition, disease diagnosis, action classification, and organ detection) spanning 72 fine-grained tasks. Extensive experiments reveal that existing video FMs struggle to generalize across varied surgical video analysis tasks, whereas pretraining on SurgBench-P yields substantial performance improvements and superior cross-domain generalization to unseen procedures and modalities. Our dataset and code are available upon request.

Figures

Figures reproduced from arXiv: 2506.07603 by the authors.

Figure 1
Figure 1. Data construction pipeline of SurgBench. We collect 16 datasets (including private datasets), [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Fine-tuning performance on different source datasets, showing accuracy gains as training progresses. The consistently increasing accuracy curves validate the reliability and data quality of SurgBench-E. the IF (Imbalance Factor) is controlled within 10 through dynamically controlling the clip dura￾tion, down-sampling the samples from dominant labels, and removing scarce labels. As for one video corresponding to mult… view at source ↗
Figure 3
Figure 3. Training and test label distribution of SurgBench-E. The pronounced long-tail pattern aligns [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Pie charts of task in SurgBench-E. We have six categories, with three category distributions [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The pie chart of SurgBench-E, contain￾ing 6 categories and 72 tasks. Following the VideoMAE methodology, we em￾ployed an asymmetric encoder-decoder architec￾ture. The pre-training task involved reconstruct￾ing randomly masked spatiotemporal "tubes" of patches using an …
Figure 6
Figure 6. Figure 6: Dynamics of pretraining and fine-tuning on SurgBench. For pretraining, we track 4 steps [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Performance on OOD dataset. We tested on the LapGyn task, which con￾tains out-of-distribution data not seen during SurgBench-P training. The pre-trained model demonstrates superior convergence speed and final accuracy when fine-tuned on this out-of￾domain downstream ta…
Figure 8
Figure 8. Figure 8: Comparison of fine-tuning performance between models pretrained on mixed surgical data [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Pie charts of task in SurgBench-E. We have six categories, with three category distributions [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 34 canonical work pages

  1. [1]

    B., Zappella, L., Khudanpur, S., Vidal, R., and Hager, G

    Ahmidi, N., Tao, L., Sefati, S., Gao, Y., Lea, C., Haro, B. B., Zappella, L., Khudanpur, S., Vidal, R., and Hager, G. D. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering , 64(9):2025--2041

    work page 2017

  2. [2]

    D., Girshick, R., Fried, G

    Bar, O., Neimark, D., Zohar, M., Hager, G. D., Girshick, R., Fried, G. M., Wolf, T., and Asselmann, D. (2020). Impact of data on generalization of ai for surgical intelligence applications. Scientific reports , 10(1):22208

    work page 2020

  3. [3]

    Bi, J., Wang, Y., Chen, H., Xiao, X., Hecker, A., Tresp, V., and Ma, Y. (2024). Visual instruction tuning with 500x fewer parameters through modality linear representation-steering. arXiv preprint arXiv:2412.12359

    arXiv 2024

  4. [4]

    Bi, J., Wang, Y., Yan, D., Xiao, X., Hecker, A., Tresp, V., and Ma, Y. (2025). Prism: Self-pruning intrinsic selection method for training-free multimodal data selection. arXiv preprint arXiv:2502.12119

    arXiv 2025

  5. [5]

    H., Hicks, S., Jha, D., Eskeland, S

    Borgli, H., Thambawita, V., Smedsrud, P. H., Hicks, S., Jha, D., Eskeland, S. L., Randel, K. R., Pogorelov, K., Lux, M., Nguyen, D. T. D., Johansen, D., Griwodz, C., Stensland, H. K., Garcia-Ceja, E., Schmidt, P. T., Hammer, H. L., Riegler, M. A., Halvorsen, P., and de Lange, T. (2020). HyperKvasir, a comprehensive multi-class image and video dataset for ...

    work page 2020

  6. [6]

    Che, C., Wang, C., Vercauteren, T., Tsoka, S., and Garcia-Peraza-Herrera, L. C. (2025). Surg-3m: A dataset and foundation model for perception in surgical settings. arXiv preprint arXiv:2503.19740

    arXiv 2025

  7. [7]

    Dimick, J. B. and Varban, O. A. (2015). Surgical video analysis: an emerging tool for improving surgeon performance

    work page 2015

  8. [8]

    Fujii, R., Hatano, M., Saito, H., and Kajita, H. (2024). Egosurgery-phase: a dataset of surgical phase recognition from egocentric open surgery videos. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 187--196. Springer

    work page 2024

Show all 47 references
  1. [9]

    D., Patel, K

    Goodman, E. D., Patel, K. K., Zhang, Y., Locke, W., Kennedy, C. J., Mehrotra, R., Ren, S., Guan, M., Zohar, O., Downing, M., et al. (2024). Analyzing surgical technique in diverse open surgical videos with multitask machine learning. JAMA surgery , 159(2):185--192

    2024

  2. [10]

    L., Suresh, V., Bittar, P., Ledbetter, L., Mithani, S

    Green, J. L., Suresh, V., Bittar, P., Ledbetter, L., Mithani, S. K., and Allori, A. (2019). The utilization of video technology in surgical education: a systematic review. journal of surgical research , 235:171--180

    2019

  3. [11]

    R., Pradarelli, J

    Grenda, T. R., Pradarelli, J. C., and Dimick, J. B. (2016). Using surgical video to improve technique and skill. Annals of surgery , 264(1):32--33

    2016

  4. [12]

    A., Van Lieshout, A

    Gr \"u ter, A. A., Van Lieshout, A. S., van Oostendorp, S. E., Henckens, S. P., Ket, J. C., Gisbertz, S. S., Toorenvliet, B. R., Tanis, P. J., Bonjer, H. J., and Tuynman, J. B. (2023). Video-based tools for surgical quality assessment of technical skills in laparoscopic proced...

    2023

  5. [13]

    Guo, H., Ma, Z., Zeng, Z., Luo, M., Zeng, W., Tang, J., and Zhao, X. (2025). Each fake news is fake in its own way: An attribution multi-granularity benchmark for multimodal fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, page...

    2025

  6. [14]

    o hrig, R., H \

    Hoffmann, H., Funke, I., Peters, P., Venkatesh, D. K., Egger, J., Rivoir, D., R \"o hrig, R., H \"o lzle, F., Bodenstedt, S., Willemer, M.-C., et al. (2024). Aixsuture: vision-based assessment of open suturing skills. International Journal of Computer Assisted Radiology and Su...

    2024

  7. [15]

    J., Kletz, S., M \" u nzer, B., Schoeffmann, K., and Keckstein, J

    Leibetseder, A., Petscharnig, S., Primus, M. J., Kletz, S., M \" u nzer, B., Schoeffmann, K., and Keckstein, J. (2018). Lapgyn4: a dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology. In Proceedings of the 9th ACM Multimedia Systems Confe...

    2018

  8. [16]

    P., and Goldenberg, M

    Levin, M., McKechnie, T., Khalid, S., Grantcharov, T. P., and Goldenberg, M. (2019). Automated methods of technical skill assessment in surgery: a systematic review. Journal of surgical education , 76(6):1629--1639

    2019

  9. [17]

    R., Schwaitzberg, S

    Li, J., Skinner, G., Yang, G., Quaranto, B. R., Schwaitzberg, S. D., Kim, P. C., and Xiong, J. (2024). Llava-surg: towards multimodal surgical assistant via structured surgical video learning. arXiv preprint arXiv:2408.07981

    2024 arXiv

  10. [18]

    Li, K., Wang, Y., Li, Y., Wang, Y., He, Y., Wang, L., and Qiao, Y. (2023). Unmasked teacher: Towards training-efficient video foundation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19948--19960

    2023

  11. [19]

    Li, Y., He, H., Cao, Y., Cheng, Q., Fu, X., and Tang, R. (2025a). M2iv: Towards efficient and fine-grained multimodal in-context learning in large vision-language models. arXiv preprint arXiv:2504.04633

    2025 arXiv

  12. [20]

    Li, Y., Yang, J., Li, B., and Tang, R. (2025b). Cama: Enhancing multimodal in-context learning with context-aware modulated attention. arXiv preprint arXiv:2505.17097

    2025

  13. [21]

    Li, Y., Yun, T., Yang, J., Feng, P., Huang, J., and Tang, R. (2025c). Taco: Enhancing multimodal in-context learning via task mapping-guided sequence configuration. arXiv preprint arXiv:2505.17098

    2025

  14. [22]

    J., Filiberto, A

    Loftus, T. J., Filiberto, A. C., Li, Y., Balch, J., Cook, A. C., Tighe, P. J., Efron, P. A., Upchurch Jr, G. R., Rashidi, P., Li, X., et al. (2020). Decision analysis and reinforcement learning in surgical decision-making. Surgery , 168(2):253--266

    2020

  15. [23]

    Ma, Y., Chen, X., Cheng, K., Li, Y., and Sun, B. (2021). Ldpolypvideo benchmark: a large-scale colonoscopy video dataset of diverse polyps. In Medical Image Computing and Computer Assisted Intervention--MICCAI 2021: 24th International Conference, Strasbourg, France, September ...

    2021

  16. [24]

    Ma, Z., Luo, M., Guo, H., Zeng, Z., Hao, Y., and Zhao, X. (2024). Event-radar: Event-driven multi-view learning for multimodal fake news detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 5809--5821

    2024

  17. [25]

    S., and Moeslund, T

    Madan, N., M gelmose, A., Modi, R., Rawat, Y. S., and Moeslund, T. B. (2024). Foundation models for video understanding: A survey. Authorea Preprints

    2024

  18. [26]

    Mesejo, P., Pizarro, D., Abergel, A., Rouquette, O., Beorchia, S., Poincloux, L., and Bartoli, A. (2016). Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE transactions on medical imaging , 35(9):2051--2063

    2016

  19. [27]

    Misawa, M., Kudo, S.-e., Mori, Y., Hotta, K., Ohtsuka, K., Matsuda, T., Saito, S., Kudo, T., Baba, T., Ishida, F., et al. (2021). Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastroint...

    2021

  20. [28]

    Nasirihaghighi, S., Ghamsarian, N., Husslein, H., and Schoeffmann, K. (2024). Event recognition in laparoscopic gynecology videos with hybrid transformers. In MultiMedia Modeling (MMM 2024) , pages 82--95. Springer

    2024

  21. [29]

    I., Yu, T., Gonzalez, C., Seeliger, B., Mascagni, P., Mutter, D., Marescaux, J., and Padoy, N

    Nwoye, C. I., Yu, T., Gonzalez, C., Seeliger, B., Mascagni, P., Mutter, D., Marescaux, J., and Padoy, N. (2022). Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Medical Image Analysis , 78:102433

    2022

  22. [30]

    J., Peabody, J

    Prebay, Z. J., Peabody, J. O., Miller, D. C., and Ghani, K. R. (2019). Video review for measuring and improving skill in urological surgery. Nature Reviews Urology , 16(4):261--267

    2019

  23. [31]

    W., Jopling, J., and Krieger, A

    Schmidgall, S., Kim, J. W., Jopling, J., and Krieger, A. (2024). General surgery vision transformer: A video pre-trained foundation model for general surgery. arXiv preprint arXiv:2403.05949

    2024 arXiv

  24. [32]

    H., Thambawita, V., Hicks, S

    Smedsrud, P. H., Thambawita, V., Hicks, S. A., Gjestang, H., Nedrejord, O. O., N ss, E., Borgli, H., Jha, D., Berstad, T. J. D., Eskeland, S. L., Lux, M., Espeland, H., Petlund, A., Nguyen, D. T. D., Garcia-Ceja, E., Johansen, D., Schmidt, P. T., Toth, E., Hammer, H. L., de La...

    2021

  25. [33]

    R., and Shah, M

    Soomro, K., Zamir, A. R., and Shah, M. (2012). Ucf101: A dataset of 101 human actions classes from videos in the wild

    2012

  26. [34]

    Tong, Z., Song, Y., Wang, J., and Wang, L. (2022). Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems , 35:10078--10093

    2022

  27. [35]

    P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., and Padoy, N

    Twinanda, A. P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., and Padoy, N. (2016). Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging , 36(1):86--97

    2016

  28. [36]

    M., Müller, B., Davitashvili, T., Capek, M., Reinke, A., Yu, T., Vardazaryan, A., Nwoye, C

    Wagner, M., Müller-Stich, B.-P., Kisilenko, A., Tran, D., Heger, P., Mündermann, L., Lubotsky, D. M., Müller, B., Davitashvili, T., Capek, M., Reinke, A., Yu, T., Vardazaryan, A., Nwoye, C. I., Padoy, N., Liu, X., Lee, E.-J., Disch, C., Meine, H., Xia, T., Jia, F., Kondo, S., ...

    2021

  29. [37]

    Wang, G., Bai, L., Wang, J., Yuan, K., Li, Z., Jiang, T., He, X., Wu, J., Chen, Z., Lei, Z., et al. (2025). Endochat: Grounded multimodal large language model for endoscopic surgery. arXiv preprint arXiv:2501.11347

    2025 arXiv

  30. [38]

    Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., and Qiao, Y. (2023a). Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14549--14560

    2023

  31. [39]

    Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al. (2024). Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision , pages 396--416. Springer

    2024

  32. [40]

    Wang, Z., Liu, C., Zhang, S., and Dou, Q. (2023b). Foundation model for endoscopy video analysis via large-scale self-supervised pre-train. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 101--111. Springer

    2023

  33. [41]

    Wang, Z., Lu, B., Long, Y., Zhong, F., Cheung, T.-H., Dou, Q., and Liu, Y. (2022). Autolaparo: A new dataset of integrated multi-tasks for image-guided surgical automation in laparoscopic hysterectomy. In International Conference on Medical Image Computing and Computer-Assiste...

    2022

  34. [42]

    Yu, T., Mutter, D., Marescaux, J., and Padoy, N. (2018). Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition. arXiv preprint arXiv:1812.00033

    2018 arXiv

  35. [43]

    Zeng, Z., Luo, M., Kong, X., Liu, H., Guo, H., Yang, H., Ma, Z., and Zhao, X. (2024). Mitigating world biases: A multimodal multi-view debiasing framework for fake news video detection. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 6492--6500

    2024

  36. [44]

    Zhang, B., Li, K., Cheng, Z., Hu, Z., Yuan, Y., Chen, G., Leng, S., Jiang, Y., Zhang, H., Li, X., et al. (2025). Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106

    2025 arXiv

  37. [45]

    B., Yuan, L., Zhou, H., Yan, S., Sun, J

    Zhao, L., Gundavarapu, N. B., Yuan, L., Zhou, H., Yan, S., Sun, J. J., Friedman, L., Qian, R., Weyand, T., Zhao, Y., et al. (2024). Videoprism: A foundational visual encoder for video understanding. arXiv preprint arXiv:2402.13217

    2024 arXiv

  38. [46]

    Zhou, C., Jiang, R., Luan, F., Meng, S., Wang, Z., Dong, Y., Zhou, Y., and He, B. (2025a). Dual-arm robotic fabric manipulation with quasi-static and dynamic primitives for rapid garment flattening. IEEE/ASME Transactions on Mechatronics

    2025

  39. [47]

    Zhou, C., Xu, H., Hu, J., Luan, F., Wang, Z., Dong, Y., Zhou, Y., and He, B. (2025b). Ssfold: Learning to fold arbitrary crumpled cloth using graph dynamics from human demonstration. IEEE Transactions on Automation Science and Engineering

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.