REVIEW 6 cited by
Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Surgical video segmentation is a critical task in computer-assisted surgery and is vital for enhancing surgical quality and patient outcomes. Recently, the Segment Anything Model 2 (SAM2) framework has shown superior advancements in image and video segmentation. However, SAM2 struggles with efficiency due to the high computational demands of processing high-resolution images and complex and long-range temporal dynamics in surgical videos. To address these challenges, we introduce Surgical SAM 2 (SurgSAM2), an advanced model to utilize SAM2 with an Efficient Frame Pruning (EFP) mechanism, to facilitate real-time surgical video segmentation. The EFP mechanism dynamically manages the memory bank by selectively retaining only the most informative frames, reducing memory usage and computational cost while maintaining high segmentation accuracy. Our extensive experiments demonstrate that SurgSAM2 significantly improves both efficiency and segmentation accuracy compared to the vanilla SAM2. Remarkably, SurgSAM2 achieves a 3$\times$ FPS compared with SAM2, while also delivering state-of-the-art performance after fine-tuning with lower-resolution data. These advancements establish SurgSAM2 as a leading model for surgical video analysis, making real-time surgical video segmentation in resource-constrained environments a reality. Our source code is available at https://github.com/jinlab-imvr/Surgical-SAM-2.
Forward citations
Cited by 6 Pith papers
-
HRVVS: A High-resolution Video Vasculature Segmentation Network via Hierarchical Autoregressive Residual Priors
A new dataset and network for segmenting hepatic vasculature in high-resolution hepatectomy videos, reporting the best scores on the new benchmark.
-
Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study
A new surgical VQA benchmark with 167,384 questions shows generalist VLMs handle basic surgical perception but fall to near-random on medical-knowledge questions, and medical VLMs underperform generalist models.
-
SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
SurgVLM, a family of surgical vision-language models trained on 1.81M frames and 7.79M conversations, outperforms 14 commercial VLMs on a six-dataset surgical benchmark.
-
SAM-I2V: Upgrading SAM to Support Promptable Video Segmentation with Less than 0.2% Training Cost
SAM-I2V upgrades SAM to video segmentation with three lightweight modules (temporal integrator, selective memory, memory prompts), reaching about 90% of SAM 2.1's average J&F at 0.2% of its training cost.
-
Memory-Augmented SAM2 for Training-Free Surgical Video Segmentation
MA-SAM2 adds context-aware and occlusion-resilient memory to SAM2 and reports Challenge IoU of 62.49 percent on EndoVis2017 and 64.40 percent on EndoVis2018, beating SAM2 by 6.10 and 4.36 points.
-
SASVi -- Segment Any Surgical Video
SASVi uses a Mask2Former overseer to automatically re-prompt SAM2 during surgical videos, improving temporal consistency of segmentations from scarce annotations on Cholec80, CATARACTS, and Cataract1k.
Discussion (0). Continue with ORCID to comment.