REVIEW 5 cited by
VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in large-scale video-language models have shown significant potential for real-time planning and detailed interactions. However, their high computational demands and the scarcity of annotated datasets limit their practicality for academic researchers. In this work, we introduce VideoLLaMB, a novel and efficient framework for long video understanding that leverages recurrent memory bridges and temporal memory tokens to enable seamless encoding of entire video sequences with preserved semantic continuity. Central to our approach is a SceneTiling algorithm that segments videos into coherent semantic units, facilitating robust understanding across tasks without requiring additional training. VideoLLaMB achieves state-of-the-art performance, surpassing existing models by 4.2 points on four VideoQA benchmarks and by 2.06 points on egocentric planning tasks. Notably, it maintains strong performance under extreme video length scaling (up to 8 times) and excels at fine-grained frame retrieval on our proposed Needle in a Video Haystack (NIAVH) benchmark. With linear GPU memory scaling, VideoLLaMB processes up to 320 frames using a single Nvidia A100 GPU, despite being trained on only 16 frames-offering an unprecedented balance of accuracy, scalability, and cost-effectiveness. This makes it highly accessible and practical for the academic community.
Forward citations
Cited by 5 Pith papers
-
FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding
A training-free frame-selection method that weights frame embeddings by query relevance and maximizes the selected subspace's volume improves keyframe recall and VQA accuracy across eight MLLMs on Video-MME and LongVi...
-
Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning
Question-guided dual geometric memories with relevance-novelty utility reportedly reach state-of-the-art video spatial reasoning on two in-domain and five out-of-distribution benchmarks.
-
Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding
A hierarchical event memory with segment-tree proposals and a future prediction branch achieves state-of-the-art online video temporal grounding on TACoS, ActivityNet Captions, and MAD.
-
Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMs
STTM is a training-free token merging method that reduces video LLM token counts by 50-70% while keeping accuracy nearly unchanged, yielding 2-3x speedups on six video QA benchmarks.
-
LongAnimation: Long Animation Generation with Dynamic Global-Local Memory
LongAnimation uses a dynamic global-local memory, built from a long-video-understanding model's KV cache, to colorize animation sequences of about 500 frames with stable color consistency.
Discussion (0). Sign in to comment.