REVIEW 16 cited by
VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Long-form video understanding is complicated by the high redundancy of video data and the abundance of query-irrelevant information. To tackle these challenges, we propose VideoTree, a training-free framework which builds a query-adaptive and hierarchical video representation for LLM reasoning over long-form videos. First, VideoTree extracts query-relevant information from the input video through an iterative process, progressively refining the selection of keyframes based on their relevance to the query. Furthermore, VideoTree leverages the inherent hierarchical structure of long video data, which is often overlooked by existing LLM-based methods. Specifically, we incorporate multi-granularity information into a tree-based representation, allowing VideoTree to extract query-relevant details from long videos in a coarse-to-fine manner. This enables the model to effectively handle a wide range of video queries with varying levels of detail. Finally, VideoTree aggregates the hierarchical query-relevant information within the tree structure and feeds it into an LLM reasoning model to answer the query. Our experiments show that our method improves both reasoning accuracy and efficiency. Specifically, VideoTree outperforms existing training-free approaches on EgoSchema and NExT-QA with less inference time, achieving 61.1% and 75.6% accuracy on the test set without additional video-specific training. Moreover, on the long split of Video-MME (average 44 minutes), VideoTree achieves better performance than GPT-4V and many other MLLMs that were extensively trained on video data.
Forward citations
Cited by 16 Pith papers
-
Movie Facts and Fibs (MF$^2$): A Benchmark for Long Movie Understanding
MF2 evaluates long-movie understanding by asking models to classify fact/fib claim pairs; the best model trails humans by 23.5 points in pairwise accuracy.
-
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Structured physical memories (captions, event graphs, rule cues) retrieved and verified at inference time consistently improve frozen VLMs on physical-plausibility benchmarks without fine-tuning.
-
Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
AVP, an iterative plan-observe-reflect agent that samples query-relevant video segments directly, reports state-of-the-art accuracy on five long-video QA benchmarks with roughly 5.4x less inference time than the leadi...
-
CAViAR: Critic-Augmented Video Agentic Reasoning
CAViAR, an agent-plus-critic system for long video reasoning, improves on direct video LLM inference across LVBench, Neptune, and ActivityNet-RTL.
-
ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval
A hierarchical prompt pyramid over CLIP with ancestor-descendant attention improves partially relevant video retrieval.
-
Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames
TCoT uses a single VLM to select question-relevant video frames from segments, then answers from that curated context, improving video QA accuracy across four benchmarks and three VLMs.
-
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
A 3B agent trained with supervised tool-use traces and reinforcement learning answers week-long egocentric video questions by dynamically selecting hierarchical retrieval, video-LLM, and VLM tools.
-
TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervision
Weakly supervised vision-language model jointly generating open-ended video QA answers with temporal groundings, reporting SOTA on NExT-GQA, MSVD-QA, and ActivityNet-QA.
-
MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks
AVHaystacks is a new 3100-question benchmark for audio-visual QA across 500 videos, and the MAGNET multi-agent pipeline beats current baselines on it.
-
CLaMR: Contextualized Late-Interaction for Multimodal Content Retrieval
CLaMR jointly encodes four video modalities in a vision-language model and uses token-level, per-modality matching to retrieve the right video and the right modality for a query.
-
Towards Sparse Video Understanding and Reasoning
A video-QA agent that carries only a structured text summary between rounds beats dense-frame baselines on accuracy while using a handful of frames per video.
-
LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering
LeAdQA improves video question answering by using LLM-rewritten causal queries to drive temporal grounding that selects relevant video segments for the answering model.
-
DIVE: Deep-search Iterative Video Exploration A Technical Report for the CVRR Challenge at CVPR 2025
DIVE, an iterative question-decomposition system with intent estimation and object-centric video summarization, achieves 81.44% on CVRR-ES.
-
Uneven Event Modeling for Partially Relevant Video Retrieval
UEM retrieves partially relevant videos by adaptively segmenting frames into uneven events and refining the best-matching event with text-conditioned attention.
-
TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representations
TriPSS fuses CIELAB color, ResNet-50, and LLaMA caption embeddings through PCA and HDBSCAN to extract keyframes, reporting F1 of 0.6104 on TVSum20 and 0.5902 on SumMe.
-
Infinite Video Understanding
The paper argues that video understanding research should aim at processing streams of arbitrary, unbounded duration and outlines the challenges, directions, and metrics needed.
Discussion (0). Sign in to comment.