Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces EgoExOR, a surgical dataset that synchronizes egocentric wearable streams with exocentric room cameras, and reports a dual-branch scene-graph model reaching 0.72 macro F1.

desk verdict A genuinely new OR dataset worth engaging; the benchmark numbers need sync validation and error bars before they can be trusted. read the letter →

arxiv 2505.24287 v1 pith:LT2V54XP submitted 2025-05-30 cs.CV

Ege Özsoy , Arda Mamur , Felix Tristram , Chantal Pellegrini , Magdalena Wysocki , Benjamin Busam , Nassir Navab This is my paper
classification cs.CV
keywords egocentricvisionexocentricoperatingroomdatasetscenegraphgenerationsurgicalactivityunderstandinggazetrackinghandposeestimationmultimodalfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EgoExOR is presented as the first operating-room dataset to synchronize multi-view first-person and third-person recordings: three wearable glasses (RGB, gaze, hand pose, audio), five overhead RGB-D cameras, and an ultrasound screen capture, recorded over 94 minutes of two emulated spine procedures. The paper's central claim is that this combination matters: a dual-branch model that keeps egocentric and exocentric cues separate reaches a macro F1 of 0.72 on surgical scene graph generation, outperforming the adapted MM2SG (0.67) and ORacle (0.63) baselines. An ablation shows exocentric-only input falls to 0.42 and egocentric-only to 0.68, supporting the claim that the two perspectives carry complementary information. The dataset, with 36 entity classes, 22 relation classes, and 568,235 triplets across 84,553 frames, is offered as a benchmark resource for OR perception.

What carries the argument

The carrier of the argument is the synchronized EgoExOR capture itself together with the dual-branch fusion model. The dataset's annotations, covering 36 entity classes and 22 relation classes, give the benchmark its target, while the model's dual-branch design is the mechanism that tests the dataset's value: each branch applies a 2-layer transformer to its modalities, pools to N embeddings, concatenates them, and lets a large vision-language model predict triplets, so fine-grained egocentric cues such as gaze and hand pose are kept separate from global exocentric cues instead of being mixed in a single encoder.

What would settle it

Open the released HDF5 files and measure the inter-stream delay for each take: detect the clapper event in each camera's audio and video, then track a visible event across the full take to see whether stream offsets drift by more than one frame, about 67 ms at 15 FPS. Any such drift would break the frame-level correspondence between scene graph labels and audiovisual content, and the reported F1 scores would need to be recomputed on correctly aligned frames.

Watch

Extended reading notes

Core claim

The paper establishes EgoExOR as a dataset that unites perspectives no prior OR dataset combined: egocentric RGB, gaze, hand pose, and audio from wearable glasses, plus exocentric RGB-D, ultrasound screen capture, and a fused point cloud, all synchronized at 15 FPS across 41 takes of two simulated spine procedures. On top of this multimodal capture, the authors propose a dual-branch baseline for surgical scene graph generation: an egocentric branch encodes first-person RGB, hand pose, and gaze, an exocentric branch encodes RGB-D, ultrasound, audio, and point clouds, and the fused embeddings are fed to a vision-language model that autoregressively emits (subject, predicate, object) triplets. The reported result is a macro F1 of 0.72 overall (0.79 on ultrasound-guided injection, 0.68 on minimally invasive spine surgery), beating adapted MM2SG (0.67) and ORacle (0.63), with the ablation showing that egocentric-only input reaches 0.68 while exocentric-only input drops to 0.42.

Load-bearing premise

Everything rests on the assumption that the clapper-based manual synchronization keeps all streams aligned at frame level for all 41 takes; the paper states no drift was observed but reports no quantitative synchronization error.

Editorial extensions

If this is right

  • Adding egocentric signals to exocentric OR models is not incremental: in the reported benchmark, the full dual-branch model outperforms both single-perspective variants, and exocentric-only performance is far lower.
  • Fine-grained tool-hand predicates such as injecting, aspirating, and controlling score well when gaze and hand pose are available, indicating that wearable sensing carries a substantial part of the interaction signal.
  • Low-frequency predicates remain the failure mode: cutting and anaesthetising score far below the macro average, pointing to class imbalance as the next bottleneck for OR scene graph generation.
  • The 84,553 annotated timepoints give a new training and evaluation resource for multi-perspective scene graph generation and, more broadly, for human-centric OR perception tasks.
  • More modalities lead to better scene graph prediction in the reported experiments, supporting the design decision to include ultrasound, audio, point clouds, gaze, and hand pose in a synchronized benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the egocentric branch carries most of the fine-grained predicate signal, a wearable-only model might support real-time assistance in constrained settings, trading global context for lower sensor overhead; the paper does not test this.
  • Because synchronization relies on a clapper with no reported error bound, a quantitative synchronization-error audit would make the reported F1 numbers reproducible and comparable with future ego-exo OR datasets.
  • The simulated, scripted procedures mean the reported F1 may not transfer directly to live operating rooms; measuring the domain gap on real surgical recordings would be a natural extension.
  • The class-imbalance pattern suggests that dataset growth should target rare predicates such as cutting and anaesthetising rather than simply adding more frames of common interactions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper introduces EgoExOR, a new operating-room dataset recorded at 15 FPS over 94 minutes (84,553 frames) from two emulated spine procedures: Ultrasound-Guided Injection (UI) and Minimally Invasive Spine Surgery (MISS). The dataset combines egocentric RGB, gaze, hand tracking, and audio from Project Aria glasses (worn by different roles, plus microscope and OR-light viewpoints) with exocentric RGB-D from five ceiling-mounted Azure Kinects, ultrasound screen captures, and per-frame colored point clouds. It provides dense scene graph annotations with 36 entity classes, 22 relation classes, and 568,235 triplets. The paper also defines a surgical scene graph generation benchmark, adapts two existing models (ORacle and MM2SG), and proposes a new dual-branch baseline that fuses egocentric and exocentric modalities. Reported macro F1 scores are 0.63 for ORacle, 0.67 for MM2SG, and 0.72 for the proposed model, with an ablation in the appendix showing egocentric-only (0.68) and exocentric-only (0.42) variants.

Significance. If the synchronization and annotation claims hold, EgoExOR is a potentially valuable and genuinely novel resource: it appears to be the first OR dataset to combine synchronized multi-view egocentric and exocentric recordings with gaze, hand pose, screen capture, and dense scene graph annotations, and it is publicly released with a documented HDF5 structure, missing-data conventions, and pre-defined train/val/test splits. The benchmark is a reasonable starting point for surgical scene graph generation, and the per-predicate results in the appendix give useful diagnostic information. The machine-checkable release format, clearly stated sensor setup, and explicit ablation of ego vs exo inputs are strengths. The proposed dual-branch model is a modest baseline contribution whose main interest is in showing that the new modalities can be consumed; the dataset itself, not the model, is the primary contribution.

major comments (5)
  1. [Section 3.2, Section 4.1, Appendix A.2] The synchronization of the Aria glasses, ultrasound HDMI capture, and Azure Kinects is asserted but not quantitatively demonstrated. The text states that a clapper at the beginning of each take was used and that 'We did not observe any drift in the streams during the duration recordings,' but no synchronization error is reported. At 15 FPS one frame is 66.7 ms, and even a modest relative clock drift of 100 ppm accumulates to about 0.56 s (roughly 8 frames) over a 94-minute session. Because every scene graph annotation is attached to a frame index and the benchmark F1 scores in Table 3 presuppose that all modalities are temporally coincident, the core value proposition of a synchronized ego-exo dataset is not yet established. In addition, Appendix A.2 says acquisition was continuous over nine raw sessions and takes were segmented post hoc, which is hard to reconcile with a clapper at the start of each take. Please provide per-take or per-session quantitative synchronization validation (e.g., maximum frame offset, drift rate, or cross-modal event alignment error) and clarify the actual calibration protocol.
  2. [Table 3 and Table 4] All macro F1 scores are reported from a single training run with no error bars, confidence intervals, or significance tests. The headline gap between the proposed model (0.72) and MM2SG (0.67) is only 0.05, and the ablation in Table 4 shows a 0.04 gap between the egocentric-only variant (0.68) and the full model. Without multiple seeds or statistical testing, these differences could be within training noise. Please report mean and standard deviation over at least three independent training runs (or bootstrap confidence intervals) for all rows in Tables 3 and 4, and state whether the observed gaps are statistically significant.
  3. [Section 4.3] The annotation process is described as one trained annotator labeling each frame and a second annotator verifying the output, but no inter-annotator agreement metric is reported. Given that the dataset's primary supervision signal is 568,235 scene graph triplets, a sample-based agreement measure (e.g., triplet-level Cohen's kappa or pairwise F1 between two independent annotators) is necessary to establish annotation reliability and to calibrate the achievable performance ceiling for the benchmark.
  4. [Appendix A.2 (Missing-data handling)] The supplementary material defines conventions for absent signals (zero-filled RGB and audio, NaN hand keypoints, invalid gaze as (-1,-1), empty point clouds) but never quantifies how often each modality is missing across the 41 takes. If gaze, hand tracking, point clouds, or ultrasound are unavailable for a substantial fraction of timepoints, the comparison in Table 3 between models that consume those modalities and those that do not is confounded by the imputation strategy. Please report per-modality coverage statistics (e.g., the percentage of frames with valid data per take and overall).
  5. [Section 5, Figure 2, Table 3] The contribution of the dual-branch architecture is not isolated from the contribution of the additional modalities. MM2SG does not use gaze or hand data, so the 0.05 F1 gain achieved by the proposed model could be entirely due to these extra input signals rather than to the perspective-specific branch structure. To support the architectural claim, add an ablation that feeds all seven modalities through a single shared encoder (e.g., an ORacle/MM2SG-style unified stream with the same token budget) and compare it with the dual-branch variant.
minor comments (7)
  1. [Figure 1 caption] The caption uses 'EgoExoR' but the dataset is called 'EgoExOR'; please make the spelling consistent.
  2. [Section 5, Implementation Details] 'LLaV A-7B' contains an erroneous space; it should read 'LLaVA-7B'.
  3. [Section 6, Limitations] 'ressource' should be 'resource'.
  4. [Section 3.2] The phrase 'sub-millimeter accuracy (2D pixel + depth)' for the Aria eye-tracking system is not accompanied by a reference or a specification of the measurement conditions; please cite the Project Aria documentation or state the accuracy more cautiously.
  5. [Table 4] The checkmark alignment in the table is ambiguous, especially for the 'Ego Only' row, which appears to show five checkmarks but does not clearly indicate which columns are selected; please render the table so that each checked modality is unambiguously visible.
  6. [Appendix A.1] The sentence 'While using only egocentric input performs much better' is grammatically incomplete and should be rephrased, for example, 'While using only egocentric input performs much better than exocentric-only input, the best performance is achieved when all inputs are used together.'
  7. [References] Reference [5] is listed as 'Egosurgery-tool' but the text refers to 'EgoSurgery'; please standardize the capitalization and verify the reference formatting.

Circularity Check

0 steps flagged · score 0.0 of 10

No structural circularity: EgoExOR is an empirical dataset and benchmark paper whose headline results are measured on a held-out test split; the unvalidated synchronization assumption is a validity risk, not a circular derivation.

full rationale

The paper's central contribution is data collection and a comparative benchmark rather than a formal derivation. The model comparisons in Table 3 and the ablations in Table 4 are evaluated on a take-level held-out test split (Section 4.2) with the standard macro F1 metric, so the reported numbers are measured outcomes, not quantities forced by construction. The adapted baselines ORacle and MM2SG are prior works by overlapping authors, but their lower performance is not derived from any citation or definition; they are trained under the same protocol, and the ablation provides independent evidence that modality composition affects performance. The 'first to combine' statement is a novelty claim, not a derivation, and the paper does not define EgoExOR's success metric in terms of its own inputs. The synchronization procedure in Section 3.2 rests on a clapper and manual alignment with no quantitative drift error, but this is an empirical quality risk that would degrade all compared methods equally rather than make any prediction equivalent to its input. No equation or fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation. The correct classification is a non-circular empirical study with a data-quality caveat.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on data collection and annotation assumptions rather than mathematical axioms. The main ones are temporal synchronization accuracy, annotation quality, and the validity of the simulated OR as a proxy for real clinical environments. There are no free parameters in the scientific sense and no invented theoretical entities.

assumptions (3)
  • domain assumption Clapper-based manual synchronization aligns all streams to frame level without drift across all takes
    Section 3.2 states a clapper was used and no drift was observed, but no quantitative sync error is reported. The benchmark assumes temporal alignment.
  • domain assumption Manual scene graph annotations are accurate ground truth; one annotator labels and a second verifies, with no reported inter-annotator agreement
    Section 4.3 describes the annotation protocol but does not quantify annotation quality. All F1 scores are measured against these labels.
  • domain assumption Two emulated procedures by biomedical engineers are sufficiently representative of real OR workflows to support conclusions about OR perception
    Section 3.4 and Limitations acknowledge the simulation setup; the paper's significance claims rely on this representativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding." pith.science (2026). https://pith.science/paper/LT2V54XP

@misc{pith2026250524287,
  author       = {Pith},
  title        = {Pith review of: EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LT2V54XP}},
  note         = {Machine review of arXiv:2505.24287}
}
read the original abstract

Operating rooms (ORs) demand precise coordination among surgeons, nurses, and equipment in a fast-paced, occlusion-heavy environment, necessitating advanced perception models to enhance safety and efficiency. Existing datasets either provide partial egocentric views or sparse exocentric multi-view context, but do not explore the comprehensive combination of both. We introduce EgoExOR, the first OR dataset and accompanying benchmark to fuse first-person and third-person perspectives. Spanning 94 minutes (84,553 frames at 15 FPS) of two emulated spine procedures, Ultrasound-Guided Needle Insertion and Minimally Invasive Spine Surgery, EgoExOR integrates egocentric data (RGB, gaze, hand tracking, audio) from wearable glasses, exocentric RGB and depth from RGB-D cameras, and ultrasound imagery. Its detailed scene graph annotations, covering 36 entities and 22 relations (568,235 triplets), enable robust modeling of clinical interactions, supporting tasks like action recognition and human-centric perception. We evaluate the surgical scene graph generation performance of two adapted state-of-the-art models and offer a new baseline that explicitly leverages EgoExOR's multimodal and multi-perspective signals. This new dataset and benchmark set a new foundation for OR perception, offering a rich, multimodal resource for next-generation clinical perception.

Figures

Figures reproduced from arXiv: 2505.24287 by the authors.

Figure 1
Figure 1. Overview of one timepoint from the EgoExoR dataset, showcasing synchronized multi-view [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed EgoExOR model for surgical scene graph generation. The model [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Qualitative examples from EgoExOR. Correctly predicted entities and predicates are [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Additional qualitative examples from the EgoExOR model. Correctly predicted entities [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Representative frames illustrating the key procedural steps captured in our two surgical [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EgoExo-Con: Exploring View-Invariant Video Temporal Understanding

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Most Video-LLMs answer temporal questions far less consistently when the same event is shown from ego and exo views, and a GRPO variant with a reasoning-similarity reward partially closes the gap.

Reference graph

Works this paper leans on

50 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Mvor: A multi-view rgb-d operating room dataset for 2d and 3d human pose estimation

    Vinkle Srivastav, Thibaut Issenhuth, Abdolrahim Kadkhodamohammadi, Michel de Mathelin, Afshin Gangi, and Nicolas Padoy. Mvor: A multi-view rgb-d operating room dataset for 2d and 3d human pose estimation. arXiv preprint arXiv:1808.08180, 2018

    arXiv 2018

  2. [2]

    4d-or: Semantic scene graphs for or domain modeling

    Ege Özsoy, Evin Pınar Örnek, Ulrich Eck, Tobias Czempiel, Federico Tombari, and Nassir Navab. 4d-or: Semantic scene graphs for or domain modeling. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VII. Springer, 2022

    work page 2022

  3. [3]

    Mm-or: A large multimodal operating room dataset for semantic understanding of high-intensity surgical environments

    Ege Özsoy, Chantal Pellegrini, Tobias Czempiel, Felix Tristram, Kun Yuan, David Bani-Harouni, Ulrich Eck, Benjamin Busam, Matthias Keicher, and Nassir Navab. Mm-or: A large multimodal operating room dataset for semantic understanding of high-intensity surgical environments. In CVPR, 2025

    work page 2025

  4. [4]

    Egosurgery-phase: A dataset of surgical phase recognition from egocentric open surgery videos

    Ryo Fujii, Masashi Hatano, Hideo Saito, and Hiroki Kajita. Egosurgery-phase: A dataset of surgical phase recognition from egocentric open surgery videos. In MICCAI, 2024

    work page 2024

  5. [5]

    Egosurgery-tool: A dataset of surgical tool and hand detection from egocentric open surgery videos

    Ryo Fujii, Hideo Saito, and Hiroki Kajita. Egosurgery-tool: A dataset of surgical tool and hand detection from egocentric open surgery videos. arXiv preprint arXiv:2406.03095, 2024

    arXiv 2024

  6. [6]

    Surgical process modelling: a review

    Florent Lalys and Pierre Jannin. Surgical process modelling: a review. International Journal of Computer Assisted Radiology and Surgery, Springer V erlag, 9:495–511, 2014

    work page 2014

  7. [7]

    Lena Maier-Hein, Swaroop S. Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, Makoto Hashizume, Darko Katic, Hannes Kenngott, Michael Kranzfelder, Anand Malpani, Keno März, Thomas Neumuth, Nicolas Padoy, Carla Pugh, Nicolai Schoch, Danail Stoyanov, Russell Taylor...

    work page 2017

  8. [8]

    The mnist database of handwritten digit images for machine learning research [best of the web]

    Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141–142, 2012

    2012

Show all 50 references
  1. [9]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115(3):211–252, 2015

    2015

  2. [10]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research , 32(11):1231–1237, 2013

    2013

  3. [11]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern reco...

    2016

  4. [12]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vi...

    2020

  5. [13]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Scaling egocentric vision: The epic-kitchens dataset. In Proceedings of the European conference on comput...

    2018

  6. [14]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision ...

    2022

  7. [15]

    Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyl- los Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedin...

    2024

  8. [16]

    Endonet: a deep architecture for recognition tasks on laparoscopic videos

    Andru P Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel De Mathelin, and Nicolas Padoy. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging , 36(1):86–97, 2016

    2016

  9. [17]

    Arthrophase: A novel dataset and method for phase recognition in arthroscopic video

    Ali Bahari Malayeri, Matthias Seibold, Nicola Cavalcanti, Jonas Hein, Sascha Jecklin, Lazaros Vlachopoulos, Sandro Fucentese, Sandro Hodel, and Philipp Furnstahl. Arthrophase: A novel dataset and method for phase recognition in arthroscopic video. arXiv preprint arXiv:2502.07431, 2025

    2025 arXiv

  10. [18]

    Recognition of instrument-tissue interactions in endoscopic videos via action triplets

    Chinedu Innocent Nwoye, Cristians Gonzalez, Tong Yu, Pietro Mascagni, Didier Mutter, Jacques Marescaux, and Nicolas Padoy. Recognition of instrument-tissue interactions in endoscopic videos via action triplets. In Medical Image Computing and Computer Assisted Intervention– MIC...

    2020

  11. [19]

    Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos

    Chinedu Innocent Nwoye, Tong Yu, Cristians Gonzalez, Barbara Seeliger, Pietro Mascagni, Didier Mutter, Jacques Marescaux, and Nicolas Padoy. Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Medical Image Analysis, 78, 2022

    2022

  12. [20]

    Opera: Attention-regularized transformers for surgical phase recognition

    Tobias Czempiel, Magdalini Paschali, Daniel Ostler, Seong Tae Kim, Benjamin Busam, and Nassir Navab. Opera: Attention-regularized transformers for surgical phase recognition. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th Interna- tional Confer...

    2021

  13. [21]

    Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks

    Amy Jin, Serena Yeung, Jeffrey Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, and Li Fei-Fei. Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In 2018 IEEE winter conference on applications of computer ...

    2018

  14. [22]

    Epic fields: Marrying 3d geometry and video understanding

    Vadim Tschernezki, Ahmad Darkhalil, Zhifan Zhu, David Fouhey, Iro Laina, Diane Larlus, Dima Damen, and Andrea Vedaldi. Epic fields: Marrying 3d geometry and video understanding. Advances in Neural Information Processing Systems , 36:26485–26500, 2023

    2023

  15. [23]

    Hd-epic: A highly-detailed egocentric video dataset

    Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, Omar Emara, Sam Pollard, Kranti Parida, Kaiting Liu, Prajwal Gatti, Siddhant Bansal, Kevin Flanagan, et al. Hd-epic: A highly-detailed egocentric video dataset. arXiv preprint arXiv:2502.04144, 2025

    2025 arXiv

  16. [24]

    Hoi4d: A 4d egocentric dataset for category-level human-object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

    2022

  17. [25]

    Aria digital twin: A new benchmark dataset for egocentric 3d machine perception

    Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Peters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe, and Yuheng Carl Ren. Aria digital twin: A new benchmark dataset for egocentric 3d machine perception. In Proceedings of the IEEE/CVF International Conference o...

    2023

  18. [26]

    Assembly101: A large-scale multi-view video dataset for understanding procedural activities

    Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov, Kun He, Dipika Singhania, Robert Wang, and Angela Yao. Assembly101: A large-scale multi-view video dataset for understanding procedural activities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2022

  19. [27]

    Automatic operating room surgical activity recognition for robot-assisted surgery

    Aidean Sharghi, Helene Haugerud, Daniel Oh, and Omid Mohareri. Automatic operating room surgical activity recognition for robot-assisted surgery. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, ...

    2020

  20. [28]

    When do they stop?: A first step towards automatically identifying team communication in the operating room

    Keqi Chen, Lilien Schewski, Vinkle Srivastav, Joël Lavanchy, Didier Mutter, Guido Beldi, Sandra Keller, and Nicolas Padoy. When do they stop?: A first step towards automatically identifying team communication in the operating room. arXiv preprint arXiv:2502.08299, 2025

    2025 arXiv

  21. [29]

    Project aria: A new tool for egocentric multi-modal ai research, 2023

    Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Meredith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Duncan...

    2023

  22. [30]

    Database analysis comparing incidence and complication rates between inpatient and outpatient laminotomies for lumbar disc herniation

    Kevin Y Heo, Janice M Bonsu, Sameer Khawaja, Anthony Karzon, Prashant V Rajan, Lauren A Barber, and Sangwook Tim Yoon. Database analysis comparing incidence and complication rates between inpatient and outpatient laminotomies for lumbar disc herniation. North American Spine So...

    2024

  23. [31]

    Racoosin, Sally M

    Judith A. Racoosin, Sally M. Seymour, Laurelle Cascio, and Rajdeep Gill. Serious neurologic events after epidural glucocorticoid injection—the fda’s risk assessment. New England Journal of Medicine, 373(24):2299–2301, 2015

    2015

  24. [32]

    Daly, Kai Zheong Lim, Jennifer Lewis, Kelly Saber, Mohammed Molla, Naor Bar-Zeev, and Tony Goldschlager

    Chris D. Daly, Kai Zheong Lim, Jennifer Lewis, Kelly Saber, Mohammed Molla, Naor Bar-Zeev, and Tony Goldschlager. Lumbar microdiscectomy and post-operative activity restrictions: A protocol for a single blinded randomised controlled trial. BMC Musculoskeletal Disorders , 18:312, 2017

    2017

  25. [33]

    M. P. Steinmetz and E. C. Benzel, editors. Benzel’s Spine Surgery: Techniques, Complication Avoidance, and Management. Elsevier, 5th edition, 2021

    2021

  26. [34]

    Spine ultrasound injection course in action (888) 998-6343

    R3 Medical Training. Spine ultrasound injection course in action (888) 998-6343. YouTube,

  27. [35]

    Ultrasound guided lumbar disc herniation injection

    Murat Karkucak. Ultrasound guided lumbar disc herniation injection. YouTube, 2022. Available at https://www.youtube.com/watch?v=Vm6BTPd0w5A, accessed May 11, 2025

    2022

  28. [36]

    Jon Kimball, Andrew Yew, and Daniel C. Lu. Minimally invasive surgery for lumbar microdiscectomy. YouTube, 2013. Available at https://www.youtube.com/watch?v= aXyZ2FJMh2s, accessed May 11, 2025

    2013

  29. [37]

    Learning and reasoning with the graph structure representation in robotic surgery

    Mobarakol Islam, Lalithkumar Seenivasan, Lim Chwee Ming, and Hongliang Ren. Learning and reasoning with the graph structure representation in robotic surgery. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, ...

    2020

  30. [38]

    Advancing surgical vqa with scene graph knowledge

    Kun Yuan, Manasi Kattel, Joël L Lavanchy, Nassir Navab, Vinkle Srivastav, and Nicolas Padoy. Advancing surgical vqa with scene graph knowledge. International Journal of Computer Assisted Radiology and Surgery, 19(7):1409–1417, 2024

    2024

  31. [39]

    Dynamic scene graph representation for surgical video

    Felix Holm, Ghazal Ghazaei, Tobias Czempiel, Ege Özsoy, Stefan Saur, and Nassir Navab. Dynamic scene graph representation for surgical video. In Proceedings of the IEEE/CVF international conference on computer vision , pages 81–87, 2023

    2023

  32. [40]

    Oracle: Large vision- language models for knowledge-guided holistic or domain modeling

    Ege Özsoy, Chantal Pellegrini, Matthias Keicher, and Nassir Navab. Oracle: Large vision- language models for knowledge-guided holistic or domain modeling. In International Con- ference on Medical Image Computing and Computer-Assisted Intervention , pages 455–465. Springer, 2024. 12

    2024

  33. [41]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.08485, 2023

    2023 arXiv

  34. [42]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023

    2023

  35. [43]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  36. [44]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  37. [45]

    Clap: Learning audio concepts from natural language supervision, 2022

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap: Learning audio concepts from natural language supervision, 2022

    2022

  38. [46]

    Point transformer v3: Simpler faster stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4840–4851, 2024. 13 ...

    2024

  39. [48]

    Target Identification

    Medication2. Target Identification

  40. [49]

    Patient Disinfection

  41. [50]

    Each pair of images shows one synchronized egocentric view (left) alongside one exocentric room camera view (right)

    Roll Out Preparation Figure 5: Representative frames illustrating the key procedural steps captured in our two surgical procedures. Each pair of images shows one synchronized egocentric view (left) alongside one exocentric room camera view (right). Missing-data handling. Senso...

  42. [2025]

    Available at https://www.youtube.com/watch?v=icPj6_OTpEQ, accessed May 11, 2025

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.