Pith. sign in

REVIEW 9 cited by

SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.16268 v3 pith:JIKD4656 submitted 2024-10-21 cs.CV

Shuangrui Ding , Rui Qian , Xiaoyi Dong , Pan Zhang , Yuhang Zang , Yuhang Cao , Yuwei Guo , Dahua Lin
show 1 more author
Jiaqi Wang
This is my paper · ORCID
classification cs.CV
keywords segmentationvideoframesam2longobjectpathwaysdesignlong-term
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The Segment Anything Model 2 (SAM 2) has emerged as a powerful foundation model for object segmentation in both images and videos, paving the way for various downstream video applications. The crucial design of SAM 2 for video segmentation is its memory module, which prompts object-aware memories from previous frames for current frame prediction. However, its greedy-selection memory design suffers from the "error accumulation" problem, where an errored or missed mask will cascade and influence the segmentation of the subsequent frames, which limits the performance of SAM 2 toward complex long-term videos. To this end, we introduce SAM2Long, an improved training-free video object segmentation strategy, which considers the segmentation uncertainty within each frame and chooses the video-level optimal results from multiple segmentation pathways in a constrained tree search manner. In practice, we maintain a fixed number of segmentation pathways throughout the video. For each frame, multiple masks are proposed based on the existing pathways, creating various candidate branches. We then select the same fixed number of branches with higher cumulative scores as the new pathways for the next frame. After processing the final frame, the pathway with the highest cumulative score is chosen as the final segmentation result. Benefiting from its heuristic search design, SAM2Long is robust toward occlusions and object reappearances, and can effectively segment and track objects for complex long-term videos. Notably, SAM2Long achieves an average improvement of 3.0 points across all 24 head-to-head comparisons, with gains of up to 5.3 points in J&F on long-term video object segmentation benchmarks such as SA-V and LVOS. The code is released at https://github.com/Mark12Ding/SAM2Long.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Efficient Tracking and Understanding Object Transformations

    cs.CV 2026-07 conditional novelty 6.0 of 10

    FluxGraph detects object transformations reactively via SAM2's multi-mask disagreement, cutting TubeletGraph's inference cost by 3.3–10.7x with comparable tracking and state-graph quality.

  2. REMIND: RE-Identification with Memory for INDoor Navigation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    REMIND combines frozen DINOv3 features, dual-bank multi-prototype memory, part/background descriptors, neighbour co-occurrence, and joint Hungarian assignment to re-identify generic indoor objects after long gaps, rea...

  3. SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Tracking

    cs.CV 2025-07 conditional novelty 6.0 of 10

    SAMITE improves zero-shot visual object tracking by selecting trustworthy past frames via prototype similarity and adding positional mask prompts, outperforming prior SAM2-based trackers on most benchmarks.

  4. SAM2RL: Towards Reinforcement Learning Memory Control in Segment Anything Model 2

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A reinforcement learning agent that controls SAM 2 memory bank updates achieves a +4.91% tracking quality gain over SAM 2 when overfitted per video, indicating untapped potential in memory control.

  5. Zero-Shot Multi-Animal Tracking in the Wild

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A zero-shot multi-animal tracker combining Grounding DINO + SAM 2 with three hand-designed heuristics beats prior methods on four animal-tracking benchmarks with fixed hyperparameters.

  6. FreeVPS: Repurposing Training-Free SAM2 for Generalizable Video Polyp Segmentation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    FreeVPS pairs a per-frame polyp segmenter with frozen SAM2 tracking and two filtering modules to reduce error accumulation, improving in-domain and out-of-domain video polyp segmentation.

  7. Memory-Augmented SAM2 for Training-Free Surgical Video Segmentation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    MA-SAM2 adds context-aware and occlusion-resilient memory to SAM2 and reports Challenge IoU of 62.49 percent on EndoVis2017 and 64.40 percent on EndoVis2018, beating SAM2 by 6.10 and 4.36 points.

  8. HiM2SAM: Enhancing SAM2 with Hierarchical Motion Estimation and Memory Optimization towards Long-term Tracking

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Training-free upgrades to SAM2, a two-level motion refiner and a long/short memory bank, raise long-term tracking AUC on LaSOT and LaSOText while adding only a few milliseconds per frame.

  9. THU-Warwick Submission for EPIC-KITCHEN Challenge 2025: Semi-Supervised Video Object Segmentation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    On the EPIC-KITCHENS VISOR test set, the proposed Cutie-based egocentric video object segmentation method with SAM2-pretrained Hiera-L and Depth Anything V2 fusion reports a J&F score of 90.1%.

Pith tools