Pith. sign in

REVIEW 11 cited by

Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.09626 v1 pith:CJPOKJUD submitted 2024-03-14 cs.CV

Guo Chen , Yifei Huang , Jilan Xu , Baoqi Pei , Zhe Chen , Zhiqi Li , Jiahao Wang , Kunchang Li
show 2 more authors
Tong Lu Limin Wang
This is my paper
classification cs.CV
keywords mambavideounderstandingmodelingtasksvideosalternativeextensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Understanding videos is one of the fundamental directions in computer vision research, with extensive efforts dedicated to exploring various architectures such as RNN, 3D CNN, and Transformers. The newly proposed architecture of state space model, e.g., Mamba, shows promising traits to extend its success in long sequence modeling to video modeling. To assess whether Mamba can be a viable alternative to Transformers in the video understanding domain, in this work, we conduct a comprehensive set of studies, probing different roles Mamba can play in modeling videos, while investigating diverse tasks where Mamba could exhibit superiority. We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks. Our extensive experiments reveal the strong potential of Mamba on both video-only and video-language tasks while showing promising efficiency-performance trade-offs. We hope this work could provide valuable data points and insights for future research on video understanding. Code is public: https://github.com/OpenGVLab/video-mamba-suite.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SSM enrichment before hierarchical adaptive compression cuts video tokens 85% and yields 1.8× speedup while matching or exceeding uncompressed baselines on zero-shot reasoning segmentation.

  2. AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understanding

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A 2B-parameter video-language model using an RWKV linear-RNN backbone and sorted token merging achieves competitive long-video QA accuracy with far lower memory cost than transformer-based models.

  3. M4V: Multimodal Mamba for Efficient Text-to-Video Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    M4V shows a Mamba-based text-to-video model can roughly match attention-based PyramidFlow on VBench while cutting mixer-layer FLOPs by 45% at 768x1280.

  4. LADY: Linear Attention for Autonomous Driving Efficiency without Transformers

    cs.AI 2025-12 conditional novelty 5.0 of 10

    LADY shows that an end-to-end driving model using only linear attention can match transformer-based planners on NAVSIM/Bench2Drive while fusing arbitrary-length historical sensor frames at constant per-frame cost.

  5. Animate-X++: Universal Character Image Animation with Dynamic Backgrounds

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Animate-X++ turns cartoon images into pose-driven animations with text-controlled moving backgrounds, claiming state-of-the-art results on a new synthetic anthropomorphic benchmark.

  6. Mamba-OTR: a Mamba-based Solution for Online Take and Release Detection from Untrimmed Egocentric Video

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A Mamba-based model with focal loss and ground-truth-anchored window regularization improves online take/release endpoint detection from 25.16 to 45.48 mp-mAP on EPIC-KITCHENS-100.

  7. MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MoMa adapts frozen CLIP to video by injecting Mamba-computed scale and bias into each layer, improving accuracy and efficiency on multiple action recognition benchmarks.

  8. DySS: Dynamic Queries and State-Space Learning for Efficient 3D Object Detection from Multi-Camera Videos

    cs.CV 2025-06 conditional novelty 5.0 of 10

    DySS combines state-space feature learning with dynamic query merging and pruning to improve both accuracy and speed for camera-based 3D detection on nuScenes.

  9. Exploring Non-Local Spatial-Angular Correlations with a Hybrid Mamba-Transformer Framework for Light Field Super-Resolution

    cs.CV 2025-09 conditional novelty 4.0 of 10

    LFMT, a hybrid Mamba-Transformer network with unidirectional subspace scanning, reports new state-of-the-art light field super-resolution results on five benchmarks.

  10. Time-Scaling State-Space Models for Dense Video Captioning

    cs.CV 2025-09 conditional novelty 4.0 of 10

    A state-space model with transfer state processes videos chunk by chunk, carries the hidden state forward, and performs online dense video captioning with the same state as full-sequence processing.

  11. Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision

    cs.CV 2025-06 accept novelty 3.0 of 10

    A comprehensive review of cross-view video understanding that uses both first-person and third-person cameras, organized into a three-direction taxonomy with a dataset catalog and future research gaps.

Pith tools