REVIEW 15 cited by
SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
The Segment Anything Model 2 (SAM 2) has demonstrated strong performance in object segmentation tasks but faces challenges in visual object tracking, particularly when managing crowded scenes with fast-moving or self-occluding objects. Furthermore, the fixed-window memory approach in the original model does not consider the quality of memories selected to condition the image features for the next frame, leading to error propagation in videos. This paper introduces SAMURAI, an enhanced adaptation of SAM 2 specifically designed for visual object tracking. By incorporating temporal motion cues with the proposed motion-aware memory selection mechanism, SAMURAI effectively predicts object motion and refines mask selection, achieving robust, accurate tracking without the need for retraining or fine-tuning. SAMURAI operates in real-time and demonstrates strong zero-shot performance across diverse benchmark datasets, showcasing its ability to generalize without fine-tuning. In evaluations, SAMURAI achieves significant improvements in success rate and precision over existing trackers, with a 7.1% AUC gain on LaSOT$_{\text{ext}}$ and a 3.5% AO gain on GOT-10k. Moreover, it achieves competitive results compared to fully supervised methods on LaSOT, underscoring its robustness in complex tracking scenarios and its potential for real-world applications in dynamic environments.
Forward citations
Cited by 15 Pith papers
-
Is Tracking really more challenging in First Person Egocentric Vision?
A synchronized FPV/TPV tracking benchmark reveals that the view-specific performance drop is smaller than previously claimed and that many 'egocentric' difficulties actually stem from the human-object activity domain.
-
Efficient Tracking and Understanding Object Transformations
FluxGraph detects object transformations reactively via SAM2's multi-mask disagreement, cutting TubeletGraph's inference cost by 3.3–10.7x with comparable tracking and state-graph quality.
-
SAM-MT: Real-Time Interactive Multi-Target Video Segmentation
A modified video segmentation architecture decouples processing latency from target count, enabling real-time (>36 FPS) tracking of 10+ objects simultaneously while preserving individual identities.
-
HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization
HERO-VQL combines top-down attention guidance with egocentric augmentations and consistency training, improving visual query localization accuracy on VQ2D over prior state-of-the-art methods.
-
SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Tracking
SAMITE improves zero-shot visual object tracking by selecting trustworthy past frames via prototype similarity and adding positional mask prompts, outperforming prior SAM2-based trackers on most benchmarks.
-
SAM2RL: Towards Reinforcement Learning Memory Control in Segment Anything Model 2
A reinforcement learning agent that controls SAM 2 memory bank updates achieves a +4.91% tracking quality gain over SAM 2 when overfitted per video, indicating untapped potential in memory control.
-
DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
A diffusion model fuses per-person masks with pose keypoints to generate identity-preserving, two-person interactive videos from a single reference image, outperforming prior single-person-animation pipelines.
-
Lean-SAM2: Target-Anchored Memory and Encoder Acceleration for SAM2
Lean-SAM2 combines target-anchored memory pruning, condensed insurance memory, and risk-aware window routing to accelerate SAM2.1 inference ~1.4× with better accuracy than Efficient-SAM2.
-
SurgSLOT: Segment Anything in Surgical Videos via Semantic Long-term Tracking
SAM2S, a SAM2 variant trained on the new 61k-frame SA-SV surgical benchmark, improves average J&F to 80.42 at 68 FPS for interactive surgical-video object segmentation.
-
Zero-Shot Multi-Animal Tracking in the Wild
A zero-shot multi-animal tracker combining Grounding DINO + SAM 2 with three hand-designed heuristics beats prior methods on four animal-tracking benchmarks with fixed hyperparameters.
-
FreeVPS: Repurposing Training-Free SAM2 for Generalizable Video Polyp Segmentation
FreeVPS pairs a per-frame polyp segmenter with frozen SAM2 tracking and two filtering modules to reduce error accumulation, improving in-domain and out-of-domain video polyp segmentation.
-
BleedOrigin: Dynamic Bleeding Source Localization in Endoscopic Submucosal Dissection via Dual-Stage Detection and Tracking
A new ESD bleeding-source dataset and a dual-stage detection-tracking framework report 96.85% onset, 70.24% source, and 96.11% tracking accuracy within defined tolerances.
-
HiM2SAM: Enhancing SAM2 with Hierarchical Motion Estimation and Memory Optimization towards Long-term Tracking
Training-free upgrades to SAM2, a two-level motion refiner and a long/short memory bank, raise long-term tracking AUC on LaSOT and LaSOText while adding only a few milliseconds per frame.
-
THU-Warwick Submission for EPIC-KITCHEN Challenge 2025: Semi-Supervised Video Object Segmentation
On the EPIC-KITCHENS VISOR test set, the proposed Cutie-based egocentric video object segmentation method with SAM2-pretrained Hiera-L and Depth Anything V2 fusion reports a J&F score of 90.1%.
-
Adapting SAM 2 for Visual Object Tracking: 1st Place Solution for MMVPR Challenge Multi-Modal Tracking
A SAM 2 based tracker with backward tracking and tracklet interpolation ranked first on the 2024 ICPR multi-modal tracking challenge, reaching 89.4 AUC.
Discussion (0). Continue with ORCID to comment.