{"work":{"id":"79edc1c3-c8fc-42ee-b2b8-e2f477b88399","openalex_id":null,"doi":null,"arxiv_id":"2604.05015","raw_key":null,"title":"Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding","authors":null,"authors_text":null,"year":2026,"venue":"cs.CV","abstract":"With the rapid advancement of video understanding, existing benchmarks are becoming increasingly saturated, exposing a critical discrepancy between inflated leaderboard scores and real-world model capabilities. To address this widening gap, we introduce Video-MME-v2, a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding. To systematically evaluate model capabilities, we design a \\textbf{progressive tri-level hierarchy} that incrementally increases the complexity of video comprehension, ranging from multi-point visual information aggregation, to temporal dynamics modeling, and ultimately to complex multimodal reasoning. Besides, in contrast to conventional per-question accuracy, we propose a \\textbf{group-based non-linear evaluation} strategy that enforces both consistency across related queries and coherence in multi-step reasoning. It penalizes fragmented or guess-based correctness and assigns credit only to answers supported by valid reasoning. To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline, involving 12 annotators and 50 independent reviewers. Backed by \\textbf{3,300 human-hours} and up to \\textbf{5 rounds} of quality assurance, Video-MME-v2 aims to serve as one of the most authoritative video benchmarks. Extensive experiments reveal a substantial gap between current best model Gemini-3-Pro and human experts, and uncover a clear hierarchical bottleneck where errors in visual information aggregation and temporal modeling propagate to limit high-level reasoning. We further find that thinking-based reasoning is highly dependent on textual cues, improving performance with subtitles but sometimes degrading it in purely visual settings. By exposing these limitations, Video-MME-v2 establishes a demanding new testbed for the development of next-generation video MLLMs.","external_url":"https://arxiv.org/abs/2604.05015","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-02T11:56:55.466179+00:00","pith_arxiv_id":"2604.05015","created_at":"2026-05-10T07:42:06.732889+00:00","updated_at":"2026-07-02T11:56:55.466179+00:00","title_quality_ok":true,"display_title":"Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding","render_title":"Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding"},"hub":{"state":{"work_id":"79edc1c3-c8fc-42ee-b2b8-e2f477b88399","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":14,"external_cited_by_count":null,"distinct_field_count":2,"first_pith_cited_at":"2026-04-18T07:56:32+00:00","last_pith_cited_at":"2026-06-04T06:53:19+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-07-02T15:52:41.045931+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":3},{"context_role":"dataset","n":1}],"polarity_counts":[{"context_polarity":"background","n":3},{"context_polarity":"use_dataset","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}