Pith. sign in

REVIEW 4 major objections 5 minor 72 references

Efficient Frame Extraction: A Novel Approach Through Frame Similarity and Surgical Tool Tracking for Video Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Surgical phase segmentation can be trained on one-tenth of the frames and still gain accuracy when frame selection is driven by tracked surgical tool motion.

desk verdict The core claim is confounded by a two-stream ensemble; the RGB-only KAFR result is below baseline, so the paper needs a controlled comparison before the headline can be believed. read the letter →

arxiv 2501.11153 v3 pith:J6NGPTPT submitted 2025-01-19 cs.CV

Huu Phong Nguyen , Shekhar Madhav Khairnar , Sofia Garces Palacios , Amr Al-Abbas , Melissa E. Hogg , Amer H. Zureikat , Patricio M. Polanco , Herbert Zeh III
show 1 more author
Ganesh Sankaranarayanan
This is my paper · ORCID
classification cs.CV
keywords KinematicsAdaptiveFrameRecognitionSurgicalphasesegmentationTooltrackingKeyextractionConvolutionalNeuralNetworksDeeplearningvideoanalysisYOLOv8
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that surgical phase segmentation can be made cheaper and more accurate by deleting frames that look redundant from the standpoint of instrument motion. It introduces KAFR, a frame-selection rule that keeps only frames where the tracked centroids of surgical tool parts move enough, using accumulated displacement in Adaptive 1 or accumulated velocity change in Adaptive 2, and trains X3D CNNs on that subset. On a newly annotated Gastrojejunostomy dataset, the method reports a tenfold reduction in training frames with a 4.32% relative accuracy gain (0.749 to 0.7814); on a Pancreaticojejunostomy dataset, it reports a fivefold reduction with a 2.05% relative accuracy gain (0.8801 to 0.8982). A sympathetic reader would care because, if true, frame reduction is not a trade-off against accuracy but a route to better accuracy with lower storage and computation costs.

What carries the argument

The machinery is the KAFR selection rule and its two variants. Adaptive 1 defines key-frame pairs as $K(d) = \{(x_i, x_j) \mid D(x_i, x_j) \leq d\}$ with $D(x_i,x_j) = f\left(\sum_{s \in S} \sum_{k=i+1}^{j} \|s(x_i)-s(x_k)\|\right)$ and $f(z_d)=1/(z_d+\epsilon)^{\beta_d}$, so pairs whose cumulative centroid displacement stays within a threshold are treated as key frames. Adaptive 2 replaces displacement with accumulated absolute velocity differences $\sum_{s \in S} \sum_{k=i+1}^{j} |V_s(x_i)-V_s(x_k)|$ with $f(z_v)=1/(z_v+\epsilon)^{\beta_v}$. These signals are computed from centroids supplied by a fine-tuned YOLOv8 detector, and the threshold is set indirectly by choosing a target percentage of training frames. The retained frames feed two X3D CNN classifiers, one on RGB and one on optical flow, whose probability outputs are ensembled for the final phase label.

What would settle it

Run the same two-stream X3D setup on a surgical video where the phase-defining action is performed by an untracked tool or by camera motion, with the Enterotomy example in Figure 13 as a natural candidate; if KAFR selects fewer frames in those segments than a uniform sampler at equal retention and its accuracy drops to or below the uniform-sampler level, the assumption that low centroid motion equals redundancy is falsified.

Watch

Extended reading notes

Core claim

The central claim is that in robotic anastomosis videos, semantic redundancy tracks kinematic redundancy at the instrument centroids. KAFR computes a distance or velocity-variation signal over tracked tool parts, jaw, wrist, and shaft, and marks frame pairs whose cumulative signal stays below a threshold as similar, keeping only the bounding key frames. Training two X3D CNN streams, one on RGB and one on Farneback optical flow, on those key frames outperforms training on the full 6 fps sequence, with relative accuracy gains of 4.32% on the GJ dataset and 2.05% on the PJ dataset at roughly 10% and 20% of the frames respectively. The authors interpret this as removing frames that inject confusion into a frame-by-frame learner rather than simply compressing the video.

Load-bearing premise

The load-bearing premise is that consecutive frames in which the tracked tool centroids barely move contain no information needed to classify the surgical phase, so throwing those frames away cannot cost accuracy.

Editorial extensions

If this is right

  • GJ phase segmentation reaches 0.7814 accuracy and 0.7141 F1 with a tenfold frame reduction, so practitioners can train on a small fraction of frames without sacrificing performance.
  • KAFR makes frame count a tunable resource: at 5% or 1% retention accuracy falls, so the method identifies both a floor for redundant frames and a critical-mass requirement for training.
  • The two-stream ensemble of RGB and optical flow contributes the largest gains, meaning the selection rule and the multi-stream classifier are complementary components of the reported result.
  • On the PJ dataset the same procedure transfers with a fivefold reduction, indicating the method is not overfit to one anastomosis type.
  • Because KAFR removes duplicated and near-static frames, it cuts memory and per-epoch runtime while retaining phase-defining moments, which the paper reports as a practical resource-efficiency benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is online inference: if tool kinematics are computed in real time, KAFR could decide at capture time which frames to store, lowering the storage cost of surgical video archives.
  • The method's usefulness is bounded by what it tracks: in a phase like Enterotomy, where the untracked tool does the critical cutting, KAFR drops informative frames, so procedures with frequent camera motion or tool occlusion are a risk zone.
  • Because KAFR selects frames before classification, it could be bolted onto temporal models such as transformers or TCNs rather than only the X3D CNN used here, turning frame selection into a preprocessing step for long-video learning generally.
  • The reported gains compare against training on all frames at 6 fps; a more direct baseline for the redundancy claim would be an equally reduced uniform sample, which the paper only partially addresses through its MSE comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Kinematics Adaptive Frame Recognition (KAFR), a frame-subsampling method for surgical phase segmentation. Frames are selected by tracking surgical tool centroids with YOLOv8 and thresholding a decreasing function of cumulative centroid displacement (Adaptive 1) or variation of velocity (Adaptive 2). Selected key frames are used to train X3D CNNs; in the final configuration two X3D models (RGB and optical flow) are ensembled. The authors introduce a newly annotated Gastrojejunostomy (GJ) dataset (42 videos, six phases plus idle) and reuse a Pancreaticojejunostomy (PJ) dataset, reporting a 4.32% relative accuracy gain with a claimed tenfold frame reduction on GJ and a 2.05% gain with fivefold reduction on PJ.

Significance. If the reported gains were attributable to KAFR, the paper would offer a practical way to reduce the computational cost of training on long surgical videos, and the new GJ dataset plus public code would be useful community resources. However, the experiments as reported do not isolate KAFR's effect: the headline result uses a two-stream ensemble whose individual RGB stream underperforms the single-stream baseline, and the 'tenfold' claim corresponds to a different configuration than the one producing the headline accuracy. The central claim is therefore currently unsupported, even though the underlying hypothesis is interesting and the dataset/code contributions are real.

major comments (4)
  1. [V-D, Table 5; Section VI] The headline GJ result of 0.7814 in Table 5 is produced by ensembling two X3D models (RGB and optical flow) trained on a 15% KAFR subset, while the baseline of 0.749 is a single X3D model. The RGB-only KAFR stream in the same configuration reaches only 0.7438, below the baseline, and the optical-flow-only stream reaches 0.7291. The reported 4.32% gain is therefore confounded by the addition of a second model and input modality and cannot be attributed to KAFR. The same confound applies to PJ: the 0.8982 result in Section VI is explicitly obtained 'with the ensembling method,' whereas the baseline 0.8801 is a single-stream X3D model from prior work.
  2. [Abstract/Conclusion; Tables 4 and 5] The abstract and conclusion claim a 'tenfold reduction' in frames accompanies the accuracy 0.7814, but the configuration producing 0.7814 uses 15% of the frames (Table 5), which is roughly a 6.7x reduction, not tenfold. The only 10% configuration in Table 4 (Four Objects, Velocity) gives 0.7684, a 2.53% gain, not the headline 4.32% gain. The paper appears to combine the best accuracy from one configuration with the largest reduction from another.
  3. [IV-C, V-A, V-C] The evaluation selects the reported result from a large grid of configurations. The threshold is 'determined based on the desired number of extracted frames' (Section IV-C), so the retained frame fraction is a free parameter; the object count (one/two/four), the kinematic signal (velocity vs acceleration), and the stream combination are all varied, and the best accuracy is reported without error bars or repeated-seed variance. With 21 configurations in Table 4 alone, plus the two-stream configurations in Table 5 and the PJ results in Section VI, selection effects are a serious concern. In addition, the GJ baseline of 0.749 is not fully specified; the text refers to 'settings described in prior work' rather than reporting a GJ-specific run with the same protocol.
  4. [V-D, Figure 13] The authors themselves document a case where KAFR removes informative frames: in the Enterotomy phase, the untracked right-hand tool performed the critical cutting, so the tracked left-hand tool showed little motion and KAFR produced a large gap in the selected frames. This concession directly undermines the paper's core assumption that frames with little tracked-tool motion are redundant, and the paper provides no phase-level or per-case analysis quantifying how often this failure occurs. At minimum, robustness to this failure mode needs to be demonstrated before the method can be recommended as generally applicable.
minor comments (5)
  1. [Equations (2) and (5)] The notation in Equations (2) and (5) is inconsistent: D is defined with a sum over s in S, but the term ||s(x_i) - s(x_k)|| treats s as a point index, and the subset s mentioned in the text plays no role. Please rewrite these definitions with clear per-tool centroid notation.
  2. [Table 6] The entry for Phase 2 of GJ05262020PH reads '1 4.80%', presumably '14.80%'.
  3. [IV-D] Section IV-D defines accuracy and F1 changes as relative changes, but the abstract's '4.32%' and '0.16%' could easily be read as absolute percentage points; please state this explicitly wherever such gains are reported.
  4. [VII, Table 7] The state-of-the-art comparison in Table 7 lists results on Cholec80, Cholec51, CATARACTS, and M2cai16, which are different datasets and evaluation protocols; this is not a controlled comparison with the proposed method and should be relabeled as context rather than a competitive evaluation.
  5. [Section VII, runtime discussion] The runtime example in Section VII (approximately 19'15'' per epoch on full data versus 5'24'' on 10% of the data) is not a tenfold speedup and is closer to 3.5x; please report measured wall-clock times instead of extrapolating linearly from the data fraction.

Circularity Check

2 steps flagged · score 5.0 of 10

The tenfold/fivefold data-reduction ratio is set by construction (the KAFR threshold is tuned to a target percentage), and the headline accuracy gain is a grid-selected validation maximum that also confounds KAFR with two-stream ensembling; the kinematic-selection idea itself retains empirical content.

  1. fitted input called prediction [Section IV-C (Hyper-parameter), Table 3; abstract and conclusion 'tenfold reduction' claims]
    "Please note that the threshold values for KAFR are determined based on the desired number of extracted frames. Instead of selecting a value between 0 and 1 as in the Equation 1, we choose a number of frames by a percentage of the total number of training samples and adjust the threshold accordingly."

    Because the threshold is adjusted to hit a user-chosen percentage, the retained-frame count is an input, not an output. The abstract's 'tenfold reduction' and conclusion's 'fivefold reduction' therefore restate the chosen percentages by construction: Eq. (1)/(4) is not predicting how many frames are redundant; it is forced to keep exactly the requested fraction. Only which frames survive is determined by kinematics; the amount of reduction is defined by the experimenter's target.

  2. fitted input called prediction [Sections V-A to V-D, Tables 4-5]
    "Nevertheless, we attained the best accuracy of 0.7684 by utilizing four objects with a 10% frame allocation. ... Table 5 shows the results achieved using the two best candidates from the previous section, namely Acceleration with Two Objects using 15% of the data and Velocity with Four Objects using 10% of the data. The outputs of the two channels (Optical Flow and RGB) are ensembled. As we can see, the accuracy is further improved to 0.7814 (4.32%) ..."

    The reported gains are selected maxima, not predictions. Object count, kinematic signal, and percentage are all chosen by scanning the validation set (1/2/4 objects x velocity/acceleration x 1-50% frames), and the same validation set is used for the headline numbers; no held-out test set is described. Moreover, the winning 0.7814 is an ensemble of two X3D models, while the 0.749 baseline is a single X3D model; Table 5 shows the RGB-only KAFR stream at 0.7438, below the baseline. Thus the 4.32% improvement is statistically forced (best of a grid) and architecturally confounded, so it cannot be attributed to the KAFR frame-selection mechanism.

full rationale

The mathematical definition of KAFR (Eqs. 1-6) is not circular in itself: a frame score is computed from tracked tool displacement/velocity, and a threshold picks frames. No derivational step makes the accuracy identity equal to the input. However, two parts of the claimed contribution are problematic. First, the data-reduction ratio is an input: the threshold is calibrated to a selected percentage, so 'tenfold/fivefold reduction' is true by construction rather than discovered. Second, the accuracy improvement is reported from a grid search over object counts, kinematic types, and percentages on the validation set, and the best result additionally uses a two-stream ensemble compared against a single-stream baseline; the RGB-only KAFR number in Table 5 is below baseline. These are evaluation/attribution problems rather than a self-referential derivation, so the paper is not an 8-10 case; but the central headline contains a by-construction reduction and a statistically/architecturally confounded accuracy claim, warranting a partial-circularity score of 5. Self-citations to [3] are not load-bearing for the method's derivation: the prior source code is reused and the baseline is externally checkable.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unstated or loosely specified choices: the exact form of the similarity function, the threshold tuning procedure, the number of tracked objects, and the retained frame fraction. None of these are derived or independently validated; they are parameters swept in the experiments. The most fragile domain assumption is that tool motion alone marks phase-informative frames, which the authors themselves partially refute in the Enterotomy example.

free parameters (5)
  • KAFR threshold d = Adjusted per target frame percentage (1%, 5%, 10%, 15%, 20%, 30%, 50%); exact d values not reported
    The threshold in Eqs. (1) and (4) is chosen to retain a desired fraction of frames, and the fraction itself is a selection parameter swept in the experiments.
  • Exponents beta_d and beta_v = Not reported
    Constants in Eqs. (3) and (6) define the decreasing similarity function f; no values or sensitivity analysis are given.
  • Epsilon in denominators = Not reported, described only as small
    Introduced to avoid division by zero in Eqs. (3) and (6); its value affects the similarity scores.
  • Number of tracked tool objects/parts = 1, 2, or 4 objects, with best selected per data fraction
    The paper reports results for one, two, and four tracked tool parts and selects the best configuration for headline comparisons (Section V-B).
  • Retained frame fraction = 10% (claimed tenfold reduction) but best GJ ensemble uses 15%; PJ best uses 20%
    The fraction of frames retained is a free choice; the claimed tenfold reduction in the abstract conflicts with the 15% used for the best GJ result in Table 5.
assumptions (5)
  • domain assumption YOLOv8 detections of surgical tool parts are accurate enough that centroid trajectories reflect true tool motion.
    The whole KAFR pipeline depends on the tracking phase; Section III-C and the Discussion in Section VIII acknowledge tracking loss and ID reassignment.
  • ad hoc to paper Phase-relevant actions are reflected in the motion of the tracked tools, and frames with little motion are redundant.
    This is the core heuristic of Adaptive 1 and Adaptive 2 (Section III-B); the paper's own Enterotomy example in Section V-D contradicts it.
  • domain assumption The endoscopic camera is approximately stationary.
    Explicitly stated as a limitation in Section VIII: camera movement affects distance computation and reduces accuracy.
  • domain assumption The prior X3D baseline settings transfer to the GJ dataset.
    The GJ baseline 0.749 is described as using research settings from prior work (Section V-A), but no GJ-specific baseline training details are reported.
  • ad hoc to paper The decreasing function f(z) = 1/(z + epsilon)^beta is a suitable similarity measure.
    Eqs. (3) and (6) assume this functional form without justification or comparison to other similarity functions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Frame Extraction: A Novel Approach Through Frame Similarity and Surgical Tool Tracking for Video Segmentation." pith.science (2026). https://pith.science/paper/J6NGPTPT

@misc{pith2026250111153,
  author       = {Pith},
  title        = {Pith review of: Efficient Frame Extraction: A Novel Approach Through Frame Similarity and Surgical Tool Tracking for Video Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6NGPTPT}},
  note         = {Machine review of arXiv:2501.11153}
}
abstract

The interest in leveraging Artificial Intelligence (AI) for surgical procedures to automate analysis has witnessed a significant surge in recent years. One of the primary tools for recording surgical procedures and conducting subsequent analyses, such as performance assessment, is through videos. However, these operative videos tend to be notably lengthy compared to other fields, spanning from thirty minutes to several hours, which poses a challenge for AI models to effectively learn from them. Despite this challenge, the foreseeable increase in the volume of such videos in the near future necessitates the development and implementation of innovative techniques to tackle this issue effectively. In this article, we propose a novel technique called Kinematics Adaptive Frame Recognition (KAFR) that can efficiently eliminate redundant frames to reduce dataset size and computation time while retaining useful frames to improve accuracy. Specifically, we compute the similarity between consecutive frames by tracking the movement of surgical tools. Our approach follows these steps: $i)$ Tracking phase: a YOLOv8 model is utilized to detect tools presented in the scene, $ii)$ Similarity phase: Similarities between consecutive frames are computed by estimating variation in the spatial positions and velocities of the tools, $iii$) Classification phase: An X3D CNN is trained to classify segmentation. We evaluate the effectiveness of our approach by analyzing datasets obtained through retrospective reviews of cases at two referral centers. The newly annotated Gastrojejunostomy (GJ) dataset covers procedures performed between 2017 and 2021, while the previously annotated Pancreaticojejunostomy (PJ) dataset spans from 2011 to 2022 at the same centers.

Figures

Figures reproduced from arXiv: 2501.11153 by the authors.

Figure 2
Figure 2. FIGURE 2 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 1
Figure 1. FIGURE 1 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. FIGURE 3 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: FIGURE 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: FIGURE 5 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: FIGURE 6 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: FIGURE 7 [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: displays the distribution of the number of frames for each class ID corresponding to the sixteen objects listed in [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: FIGURE 9 [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: FIGURE 10 [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: FIGURE 11 [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: FIGURE 12 [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: FIGURE 13 [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: FIGURE 14 [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 68 canonical work pages

  1. [1]

    Pso- convolutional neural networks with heterogeneous learning rate

    Huu Phong Nguyen, Augusto Santos, and Bernardete Ribeiro. Pso- convolutional neural networks with heterogeneous learning rate. IEEE Access, 10:89970–89988, 2022

    work page 2022

  2. [2]

    Video action recognition col- laborative learning with dynamics via pso-convnet transformer

    Huu Phong Nguyen and Bernardete Ribeiro. Video action recognition col- laborative learning with dynamics via pso-convnet transformer. Scientific Reports, 13(1):14624, 2023

    work page 2023

  3. [3]

    The development of a deep learning model for automated segmentation of the robotic pancreaticojejunostomy

    Amr I Al Abbas, Babak Namazi, Imad Radi, Rodrigo Alterio, Andres A Abreu, Benjamin Rail, Patricio M Polanco, Herbert J Zeh III, Melissa E Hogg, Amer H Zureikat, et al. The development of a deep learning model for automated segmentation of the robotic pancreaticojejunostomy. Surgical Endoscopy, pages 1–9, 2024

    work page 2024

  4. [4]

    Y olo- extract: Improved yolov5 for aircraft object detection in remote sensing images

    Zhiguo Liu, Y uan Gao, Qianqian Du, Meng Chen, and Wenqiang Lv. Y olo- extract: Improved yolov5 for aircraft object detection in remote sensing images. IEEE Access, 11:1742–1751, 2023

    work page 2023

  5. [5]

    Recurrent convolutional neural network for object recognition

    Ming Liang and Xiaolin Hu. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3367–3375, 2015

    work page 2015

  6. [6]

    Object detection with discriminatively trained part-based mod- els

    Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. Object detection with discriminatively trained part-based mod- els. IEEE transactions on pattern analysis and machine intelligence , 32(9):1627–1645, 2009

    work page 2009

  7. [7]

    k-nn attention-based video vision transformer for action recognition

    Weirong Sun, Y ujun Ma, and Ruili Wang. k-nn attention-based video vision transformer for action recognition. Neurocomputing, 574:127256, 2024

    work page 2024

  8. [8]

    Pattern Recognition: Contributions and Applications to Image Classification and Video Recognition

    Huu Phong Nguyen. Pattern Recognition: Contributions and Applications to Image Classification and Video Recognition . PhD thesis, Universidade de Coimbra, 2023

    work page 2023

Show all 72 references
  1. [9]

    Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures

    Shruti Garg, Sudhir Sharma, Sumit Dhariwal, W Deva Priya, Mangal Singh, and S Ramesh. Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures. Multimedia Tools and Applications , pages 1–23, 2024

    2024

  2. [10]

    A spatio-temporal network for video semantic segmentation in surgical videos

    Maria Grammatikopoulou, Ricardo Sanchez-Matilla, Felix Bragman, David Owen, Lucy Culshaw, Karen Kerr, Danail Stoyanov, and Imanol Luengo. A spatio-temporal network for video semantic segmentation in surgical videos. International Journal of Computer Assisted Radiology and Surg...

    2024

  3. [11]

    Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning.Pattern Recognition, 108:107355, 2020

    Ang Li, Zhenjiang Miao, Yigang Cen, Xiao-Ping Zhang, Linna Zhang, and Shiming Chen. Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning.Pattern Recognition, 108:107355, 2020

    2020

  4. [12]

    Human- like mechanism deep learning model for longitudinal motion control of autonomous vehicles

    Zhenhai Gao, Tong Y u, Fei Gao, Rui Zhao, and Tianjun Sun. Human- like mechanism deep learning model for longitudinal motion control of autonomous vehicles. Engineering Applications of Artificial Intelligence , 133:108060, 2024

    2024

  5. [13]

    AI applications in healthcare a comprehensive review of advancements and challenges

    Balaram Y adav Kasula. AI applications in healthcare a comprehensive review of advancements and challenges. International Journal of Manag- ment Education for Sustainable Development , 6(6), 2023

    2023

  6. [14]

    Aklilu, Min Woo Sun, Shelly Goel, Sebastiano Bartoletti, Anita Rau, Griffin Olsen, Kay S

    Josiah G. Aklilu, Min Woo Sun, Shelly Goel, Sebastiano Bartoletti, Anita Rau, Griffin Olsen, Kay S. Hung, Sophie L. Mintz, Vicki Luong, Arnold Milstein, Mark J. Ott, Robert Tibshirani, Jeffrey K. Jopling, Eric C. Soren- son, Dan E. Azagury, and Serena Y eung-Levy. Artificial i...

    2024

  7. [15]

    Clinical applications of artificial intelligence in robotic surgery

    J Everett Knudsen, Umar Ghaffar, Runzhuo Ma, and Andrew J Hung. Clinical applications of artificial intelligence in robotic surgery. Journal of Robotic Surgery, 18(1):102, 2024

    2024

  8. [16]

    Preserving privacy in surgical video analysis using a deep learning classifier to identify out-of-body scenes in endoscopic videos

    Joël L Lavanchy, Armine V ardazaryan, Pietro Mascagni, Didier Mutter, and Nicolas Padoy. Preserving privacy in surgical video analysis using a deep learning classifier to identify out-of-body scenes in endoscopic videos. Scientific Reports, 13(1):9235, 2023

    2023

  9. [17]

    Automated segmentation of phases, steps, and tasks in laparoscopic cholecystectomy using deep learning

    Shruti R Hegde, Babak Namazi, Niyenth Iyengar, Sarah Cao, Alexis Desir, Carolina Marques, Heidi Mahnken, Ryan P Dumas, and Ganesh Sankaranarayanan. Automated segmentation of phases, steps, and tasks in laparoscopic cholecystectomy using deep learning. Surgical Endoscopy, 38(1)...

    2024

  10. [18]

    Deep learning in surgical workflow analysis: a review of phase and step recognition

    Kubilay Can Demir, Hannah Schieber, Tobias Weise, Daniel Roth, Matthias May, Andreas Maier, and Seung Hee Y ang. Deep learning in surgical workflow analysis: a review of phase and step recognition. IEEE Journal of Biomedical and Health Informatics , 27(11):5405–5417, 2023

    2023

  11. [19]

    Real-time segmentation and recognition of surgical tasks in cataract surgery videos

    Gwénolé Quellec, Mathieu Lamard, Béatrice Cochener, and Guy Cazuguel. Real-time segmentation and recognition of surgical tasks in cataract surgery videos. IEEE transactions on medical imaging, 33(12):2352–2360, 2014

    2014

  12. [20]

    On-line recognition of surgical activity for monitoring in the operating room

    Nicolas Padoy, Tobias Blum, Hubertus Feussner, Marie-Odile Berger, and Nassir Navab. On-line recognition of surgical activity for monitoring in the operating room. In AAAI, pages 1718–1724, 2008

    2008

  13. [21]

    Sv-rcnet: workflow recognition from surgical videos using recurrent convolutional network

    Y ueming Jin, Qi Dou, Hao Chen, Lequan Y u, Jing Qin, Chi-Wing Fu, and Pheng-Ann Heng. Sv-rcnet: workflow recognition from surgical videos using recurrent convolutional network. IEEE transactions on medical imaging, 37(5):1114–1126, 2017. 14

    2017

  14. [22]

    Recurrent fully convolutional networks for video segmentation

    Sepehr V alipour, Mennatullah Siam, Martin Jagersand, and Nilanjan Ray. Recurrent fully convolutional networks for video segmentation. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages 29–36. IEEE, 2017

    2017

  15. [23]

    A deep learning framework for recognising surgical phases in laparoscopic videos

    Nour Aldeen Jalal, Tamer Abdulbaki Alshirbaji, Paul D Docherty, Thomas Neumuth, and Knut Moeller. A deep learning framework for recognising surgical phases in laparoscopic videos. IF AC-PapersOnLine, 54(15):334– 339, 2021

    2021

  16. [24]

    Db-lstm: Densely-connected bi-directional lstm for human action recognition

    Jun-Y an He, Xiao Wu, Zhi-Qi Cheng, Zhaoquan Y uan, and Y u-Gang Jiang. Db-lstm: Densely-connected bi-directional lstm for human action recognition. Neurocomputing, 444:319–331, 2021

    2021

  17. [25]

    3dfcnn: Real-time action recognition using 3d deep neural networks with raw depth information

    Adrian Sanchez-Caballero, Sergio de López-Diz, David Fuentes-Jimenez, Cristina Losada-Gutiérrez, Marta Marrón-Romera, David Casillas-Perez, and Mohammad Ibrahim Sarker. 3dfcnn: Real-time action recognition using 3d deep neural networks with raw depth information. Multimedia To...

    2022

  18. [26]

    Alzheimer’s disease diagnostics by a deeply supervised adaptable 3d convolutional network

    Ehsan Hosseini-Asl, Georgy Gimel’farb, and Ayman El-Baz. Alzheimer’s disease diagnostics by a deeply supervised adaptable 3d convolutional network. arXiv preprint arXiv:1607.00556, 2016

    2016 arXiv

  19. [27]

    Human action recognition based on temporal pose cnn and multi-dimensional fusion

    Yi Huang, Shang-Hong Lai, and Shao-Heng Tai. Human action recognition based on temporal pose cnn and multi-dimensional fusion. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0–0, 2018

    2018

  20. [28]

    Flatten transformer: Vision transformer using focused linear attention

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. Flatten transformer: Vision transformer using focused linear attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5961–5971, 2023

    2023

  21. [29]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Y utong Lin, Y ue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012–10022, 2021

    2021

  22. [30]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF international conference on computer vision , pages 6836– 6846, 2021

    2021

  23. [31]

    Endonet: a deep architecture for recognition tasks on laparoscopic videos

    Andru P Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel De Mathelin, and Nicolas Padoy. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging, 36(1):86–97, 2016

    2016

  24. [32]

    Cataracts: Challenge on automatic tool annotation for cataract surgery

    Hassan Al Hajj, Mathieu Lamard, Pierre-Henri Conze, Soumali Roy- chowdhury, and Xiaowei Hu et al. Cataracts: Challenge on automatic tool annotation for cataract surgery. Medical Image Analysis, 52:24–41, 2019

    2019

  25. [33]

    M2cai workflow challenge: Convolutional neural networks with time smoothing and hidden markov model for video frames classification

    Remi Cadene, Thomas Robert, Nicolas Thome, and Matthieu Cord. M2cai workflow challenge: Convolutional neural networks with time smoothing and hidden markov model for video frames classification. arXiv preprint arXiv:1610.05541, 2016

    arXiv

  26. [34]

    Y . Gao, S. S. V edula, C. E. Reiley, N. Ahmidi, B. V aradarajan, H. Liu, L. Tao, L. Zappella, B. Bejar, D. Y uh, C. C. G. Chen, R. Vidal, S. Khu- danpur, and G. Hager. Jhu-isi gesture and skill assessment working set (jigsaws): A surgical activity dataset for human motion mod...

    2014

  27. [35]

    Comparison of video shot boundary detection techniques

    John S Boreczky and Lawrence A Rowe. Comparison of video shot boundary detection techniques. Journal of Electronic Imaging , 5(2):122– 128, 1996

    1996

  28. [36]

    Long-term recurrent convolutional networks for visual recognition and description

    Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini V enugopalan, Kate Saenko, and Trevor Darrell. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE conference on computer vision and p...

    2015

  29. [37]

    A novel keyframe extraction method for video classification using deep neural networks

    Rukiye Savran Kızıltepe, John Q Gan, and Juan José Escobar. A novel keyframe extraction method for video classification using deep neural networks. Neural Computing and Applications , 35(34):24513–24524, 2023

    2023

  30. [38]

    Fast and robust key frame extraction method for gesture video based on high-level feature representation

    Huimin Y ang, Qiuhong Tian, Qiaoli Zhuang, Linye Li, and Qinglong Liang. Fast and robust key frame extraction method for gesture video based on high-level feature representation. Signal, Image and Video Processing, 15:617–626, 2021

    2021

  31. [39]

    Key frame extraction algorithm of motion video based on priori

    Qi Zhong, Y uan Zhang, Jinguo Zhang, Kaixuan Shi, Y ang Y u, and Chang Liu. Key frame extraction algorithm of motion video based on priori. IEEE Access, 8:174424–174436, 2020

    2020

  32. [40]

    Detecting moving objects, ghosts, and shadows in video streams

    Rita Cucchiara, Costantino Grana, Massimo Piccardi, and Andrea Prati. Detecting moving objects, ghosts, and shadows in video streams. IEEE transactions on pattern analysis and machine intelligence , 25(10):1337– 1342, 2003

    2003

  33. [41]

    Y ou only look once: Unified, real-time object detection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. Y ou only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 779– 788, 2016

    2016

  34. [42]

    Rich fea- ture hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich fea- ture hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014

    2014

  35. [43]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th Euro- pean Conference, Zurich, Switzerland, September 6-12, 2014, Proceedin...

    2014

  36. [44]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei- Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee, 2009

    2009

  37. [45]

    Labelme: a database and web-based tool for image annotation

    Bryan C Russell, Antonio Torralba, Kevin P Murphy, and William T Freeman. Labelme: a database and web-based tool for image annotation. International journal of computer vision , 77:157–173, 2008

    2008

  38. [46]

    Scott Daniel, and Sankaranarayanan Ganesh

    Madhav Khairnar Shekhar, Nguyen Huu Phong, Desir Alexis, Holcomb Carla, J. Scott Daniel, and Sankaranarayanan Ganesh. Machine learning- based automated assessment of intracorporeal suturing in laparoscopic fundoplication. arXiv preprint arXiv:2412.16195, 2024

    2024 arXiv

  39. [47]

    X3d: Expanding architectures for efficient video recognition

    Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 203–213, 2020

    2020

  40. [48]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016

    2016

  41. [49]

    Metrics matter in surgical phase recognition

    Isabel Funke, Dominik Rivoir, and Stefanie Speidel. Metrics matter in surgical phase recognition. arXiv preprint arXiv:2305.13961, 2023

    2023 arXiv

  42. [50]

    Deep learning for surgical phase recognition using endoscopic videos

    Annetje CP Guédon, Senna EP Meij, Karim NMMH Osman, Helena A Kloosterman, Karlijn J van Stralen, Matthijs CM Grimbergen, Quirijn AJ Eijsbouts, John J van den Dobbelsteen, and Andru P Twinanda. Deep learning for surgical phase recognition using endoscopic videos. Surgical endos...

    2021

  43. [51]

    Development of a code- free machine learning model for the classification of cataract surgery phases

    Samir Touma, Fares Antaki, and Renaud Duval. Development of a code- free machine learning model for the classification of cataract surgery phases. Scientific Reports, 12(1):2398, 2022

    2022

  44. [52]

    Sages consensus recommendations on an annotation framework for surgical video

    Ozanan R Meireles, Guy Rosman, Maria S Altieri, Lawrence Carin, Gregory Hager, Amin Madani, Nicolas Padoy, Carla M Pugh, Patricia Sylla, Thomas M Ward, et al. Sages consensus recommendations on an annotation framework for surgical video. Surgical endoscopy, 35(9):4918– 4929, 2021

    2021

  45. [53]

    Artificial intelligence for phase recognition in complex laparoscopic cholecystectomy

    Tomer Golany, Amit Aides, Daniel Freedman, Nadav Rabani, Y un Liu, Ehud Rivlin, Greg S Corrado, Y ossi Matias, Wisam Khoury, Hanoch Kashtan, et al. Artificial intelligence for phase recognition in complex laparoscopic cholecystectomy. Surgical Endoscopy , 36(12):9215–9223, 2022

    2022

  46. [54]

    Machine learning for surgical phase recognition: a systematic review

    Carly R Garrow, Karl-Friedrich Kowalewski, Linhong Li, Martin Wagner, Mona W Schmidt, Sandy Engelhardt, Daniel A Hashimoto, Hannes G Kenngott, Sebastian Bodenstedt, Stefanie Speidel, et al. Machine learning for surgical phase recognition: a systematic review. Annals of surgery...

    2021

  47. [55]

    Phase segmentation methods for an automatic surgical workflow analysis

    Dinh Tuan Tran, Ryuhei Sakurai, Hirotake Y amazoe, Joo-Ho Lee, et al. Phase segmentation methods for an automatic surgical workflow analysis. International journal of biomedical imaging , 2017, 2017

    2017

  48. [56]

    Evaluating model performance with hard-swish activation function adjustments

    Sai Abhinav Pydimarry, Shekhar Madhav Khairnar, Sofia Garces Palacios, Ganesh Sankaranarayanan, Darian Hoagland, Dmitry Nepomnayshy, and Huu Phong Nguyen. Evaluating model performance with hard-swish activation function adjustments. RECPAD, 2024

    2024

  49. [57]

    Two-frame motion estimation based on polynomial expansion

    Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. In Image Analysis: 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29–July 2, 2003 Proceedings 13 , pages 363–370. Springer, 2003

    2003

  50. [58]

    Robotic pancreaticoduodenectomy decreases the risk of clinically relevant post-operative pancreatic fistula: a propensity score matched nsqip analy- sis

    Charles C Vining, Kristine Kuchta, Y aniv Berger, Pierce Paterakos, Darryl Schuitevoerder, Kevin K Roggin, Mark S Talamonti, and Melissa E Hogg. Robotic pancreaticoduodenectomy decreases the risk of clinically relevant post-operative pancreatic fistula: a propensity score matc...

    2021

  51. [59]

    Finks, Nicholas H

    John F. Finks, Nicholas H. Osborne, and John D. Birkmeyer. Trends in hospital volume and operative mortality for high-risk surgery. New England Journal of Medicine , 364(22):2128–2137, 2011. 15

    2011

  52. [60]

    Improved hospital morbidity, mortality, and survival after the whipple procedure

    DA VID W Crist, JAMES V Sitzmann, and JOHN L Cameron. Improved hospital morbidity, mortality, and survival after the whipple procedure. Annals of surgery, 206(3):358, 1987

    1987

  53. [61]

    Surgical ges- tures can be used to assess surgical competence in robot-assisted surgery: A validity investigating study of simulated rarp

    Rikke Groth Olsen, Morten Bo Søndergaard Svendsen, Martin G Tols- gaard, Lars Konge, Andreas Røder, and Flemming Bjerrum. Surgical ges- tures can be used to assess surgical competence in robot-assisted surgery: A validity investigating study of simulated rarp. Journal of Robot...

    2024

  54. [62]

    Tobias Czempiel, Magdalini Paschali, and Matthias et al. Keicher. Tecno: Surgical phase recognition with multi-stage temporal convolutional net- works. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 , pages 343–352, Cham, 2020. Springer Internation...

    2020

  55. [63]

    Lovit: Long video transformer for surgical phase recognition

    Y ang Liu, Maxence Boels, Luis C Garcia-Peraza-Herrera, Tom V er- cauteren, Prokar Dasgupta, Alejandro Granados, and Sebastien Ourselin. Lovit: Long video transformer for surgical phase recognition. Medical Image Analysis, 99:103366, 2025

    2025

  56. [64]

    Multi-task recurrent convolutional network with correlation loss for surgical video analysis

    Y ueming Jin, Huaxia Li, Qi Dou, Hao Chen, Jing Qin, Chi-Wing Fu, and Pheng-Ann Heng. Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Medical image analysis , 59:101572, 2020

    2020

  57. [65]

    Surgplan: Surgical phase localization network for phase recognition

    Xingjian Luo, Y ou Pang, Zhen Chen, Jinlin Wu, Zongmin Zhang, Zhen Lei, and Hongbin Liu. Surgplan: Surgical phase localization network for phase recognition. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2024

    2024

  58. [66]

    Dynamic scene graph representation for surgical video

    Felix Holm, Ghazal Ghazaei, Tobias Czempiel, Ege Özsoy, Stefan Saur, and Nassir Navab. Dynamic scene graph representation for surgical video. In Proceedings of the IEEE/CVF international conference on computer vision, pages 81–87, 2023

    2023

  59. [67]

    Last: Latent space- constrained transformers for automatic surgical phase recognition and tool presence detection

    Rong Tao, Xiaoyang Zou, and Guoyan Zheng. Last: Latent space- constrained transformers for automatic surgical phase recognition and tool presence detection. IEEE Transactions on Medical Imaging, 42(11):3256– 3268, 2023

    2023

  60. [68]

    Keyframe extraction from laparoscopic videos based on visual saliency detection

    Constantinos Loukas, Christos V arytimidis, Konstantinos Rapantzikos, and Meletios A Kanakis. Keyframe extraction from laparoscopic videos based on visual saliency detection. Computer methods and programs in biomedicine, 165:13–23, 2018

    2018

  61. [69]

    Mpeg: A video compression standard for multimedia applications

    Didier Le Gall. Mpeg: A video compression standard for multimedia applications. Communications of the ACM , 34(4):46–58, 1991

    1991

  62. [70]

    Advanced capsule net- works via context awareness

    Huu Phong Nguyen and Bernardete Ribeiro. Advanced capsule net- works via context awareness. In Artificial Neural Networks and Ma- chine Learning–ICANN 2019: Theoretical Neural Computation: 28th In- ternational Conference on Artificial Neural Networks, Munich, Germany, Septembe...

    2019

  63. [71]

    An improvement for capsule networks using depthwise separable convolution

    Huu Phong Nguyen and Bernardete Ribeiro. An improvement for capsule networks using depthwise separable convolution. In Iberian conference on pattern recognition and image analysis , pages 521–530. Springer, 2019

    2019

  64. [72]

    Action recognition for american sign language

    Huu Phong Nguyen and Bernardete Ribeiro. Action recognition for american sign language. RECPAD, 2018. 16 -1 --4 ; I n, t { A4 J t G N t/ i D # HUU PHONG NGUYEN received a BSc in Physics from Vietnam National University, Hanoi, an MSc in Information Technology from Shinawa- tra...

    2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.