Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs

Chunfeng Wang; Hao Li; Kun Zhan; Wenzhang Sun; Xiangchen Yin; Yujia Chen

REVIEW 2 major objections 3 minor 18 references

Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs

T0 review · 2 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that aggregate scaling curves for Video LLMs conceal large, opposing item-level changes as visual budgets grow, and provides a per-item trajectory protocol that exposes the hidden churn.

desk verdict A careful, high-effort measurement paper that convincingly shows aggregate Video-LLM scaling curves hide item-level churn, but the headline harmful-transition rates need a nested-grid check before being taken at face value. read the letter →

arxiv 2608.07014 v1 pith:JSSFJVFH submitted 2026-08-07 cs.CV

Wenzhang Sun , Chunfeng Wang , Xiangchen Yin , Yujia Chen , Hao Li , Kun Zhan This is my paper · ORCID

classification cs.CV

keywords videoLLMsscalingcurvesitem-levelchurnoracleheadroomvisualconfusiontextoverwriteconfidencecascadeevaluationprotocol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that aggregate scaling curves overstate how safely video-language models improve with larger visual budgets. Tracking the same frozen model on the same item under controlled frame counts and resolutions shows large opposing movements: an item that is wrong at 16 frames can be right at 128, while another flips in the opposite direction. Across five open Video LLMs, matched multiple-choice grids show item-level oracle headroom of 8.8–18.9 accuracy points, and 12.5–25.5% of items are correct at a lower budget but wrong at a higher one. The pattern persists in open-ended QA, summarization, and dialogue generation, and across sampling, resolution, and raw-video/cache pipelines. The paper's contribution is an evaluation protocol: per-item response trajectories plus matched-grid metrics that make cancellation visible, along with a released artifact and a confidence cascade that matches fixed-128f accuracy at 31.7% lower average shared frame cost.

What carries the argument

The central object is the paired budget trajectory τ_i(C)=(y_{i,c})_{c∈C}: the correctness (or task score) of one frozen model on one item across an ordered configuration grid. Two functionals are built from it: best fixed accuracy A_fixed = max_c (1/n)Σ_i y_{i,c}, and item oracle A_oracle = (1/n)Σ_i max_c y_{i,c}; their difference is oracle headroom. The paper's key identity is A_{c′}−A_c = R_{c,c′}−H_{c,c′}, with R and H the rescued and harmed fractions, while pairwise churn is R+H, so a small net change implies nothing about either directional flow. Visual confusion is defined as an item correct at a lower-cost configuration but wrong at a higher one, with text overwrite as the special case where the lower-cost correct configuration is text only. This construction turns the response matrix into an audit object and a routing surface for confidence cascades.

What would settle it

Run a large video LLM benchmark under a matched frame grid and check whether every item's correctness trajectory is monotone: if no item is correct at a lower budget and wrong at a higher one, and the item oracle equals best fixed accuracy, the central claim would be refuted for that model–benchmark pair. A cheaper check uses the released per-item response matrix to compute the fraction of items with rescued or harmed transitions; the claim predicts a substantial fraction (12.5–25.5% confusion on matched MCQA), so a result near zero would falsify it.

Watch

Extended reading notes

Core claim

The central discovery is that the mean scaling curve is a difference of two large opposing flows, not a summary of stable behavior. For any ordered budget pair c<c′, the net accuracy change equals rescued minus harmed items, R−H, while the true movement is their sum R+H; the paper shows empirically that R and H are both substantial on Video-MME, MLVU, and AVSD across five frozen models. Consequently no single visual budget—frame count, resolution, or temporal–spatial allocation—serves all items; best-fixed accuracy is systematically below an item-level oracle by 8.8–18.9 points on the matched MCQA grid, and text overwrite, where an item is correct from text alone but wrong once video is added, affects 5.0–7.3% of items on V1 short. The pattern persists in continuous generation metrics, where Token-F1 oracle gaps are 2.7–4.8 points even when mean quality improves with budget.

Load-bearing premise

The load-bearing premise is that configurations can be ordered by visual cost monotonically—frame count at fixed resolution, resolution at fixed frame count, and total pixels for mixed comparisons—so that "higher budget" means higher cost; the paper notes that if true operational cost (wall-clock, energy, or price) orders configurations differently, the reported confusion and harmful-transition rates could change, although frame-only grids show the same churn.

Editorial extensions

If this is right

Aggregate accuracy curves should be reported together with rescued and harmed flows: a near-zero net change can mask 5–15% of items changing state between adjacent budgets.
Cross-model comparisons require matched grids, because different context ceilings (for example LLaVA capped at 32f versus Qwen reaching 256f) conflate architectural limits with scaling behavior.
Any fixed-budget evaluation underestimates achievable performance: the item-level oracle exceeds the best fixed accuracy by up to 18.9 points on the matched MCQA grid.
Adaptive inference can exploit item-level budget preferences: the released confidence cascade matches fixed-128f accuracy at 31.7% lower average shared-frame cost, and an untuned agreement rule transfers to three other models within 0.4–1.0 accuracy points.
Protocol provenance is essential: raw-video and cached-frame execution replicate the aggregate headroom (both +8.78 points) but disagree on about 20% of trajectory classes, so labels from different pipelines must not be pooled.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If per-item budget preferences are as heterogeneous as reported, test-time compute allocation should be reframed as per-item routing rather than a single budget choice; the paper's oracle bounds quantify the ceiling any router can reach.
The same paired-trajectory construction could be applied along other axes—prompt wording, context length, or sampling temperature—to expose analogous cancellation in static multimodal LLM evaluation.
Because most confused items are model-exclusive while cross-model overlap still exceeds a matched-size random baseline, benchmark design may need item-level diagnostics that separate task-level difficulty from model-specific sensitivity rather than a single difficulty score.
A testable extension is to use the released per-item outputs to train an early-budget trajectory predictor; the paper's early-to-late slope correlation of r=−0.16 suggests a low ceiling for such predictors, which would bound achievable cascade savings.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about arXiv:2608.07014. First, it is a serious measurement paper: the authors actually did the matched-grid, per-item work, shipped a large artifact, and were unusually transparent about re-audits. Second, the headline numbers carry a caveat the paper does not fully resolve: the frame-budget trajectories are not shown to be nested, so "correct at 16f, wrong at 32f" may partly be a resampling artifact, not a budget-size effect.

What is new: the paired item-level trajectory audit, the rescued/harmed flow decomposition, and the matched-grid comparison across five models with protocol provenance. The identity A(c') - A(c) = R - H is a genuinely useful way to reframe scaling evaluation. The empirical work is careful: 5,000-pair bootstrap CIs, McNemar and permutation tests, raw/cache replication with exactly matching +8.78 point headroom, and disclosed re-audits for Qwen3 and InternVL3.5. That is real evidence, and the release of 0.13M per-item records is the right kind of contribution. The citation pattern is unremarkable; the relevant scaling and frame-selection literature is covered.

The soft spots, in order of importance. First, the nesting issue. Appendix C.2/C.4 say cached cells "uniformly subsample the requested budget" and raw cells decode uniformly, but nowhere does the paper state that the 16f/32f/64f/128f/256f grids are nested or aligned. If they are independent evenly spaced subsets, then a budget transition changes both the number of frames and their timestamps. The paper itself shows at fixed 64f that uniform vs. random sampling flips 4.7% of items, and confused items are 3.83x enriched for sampling sensitivity. So a meaningful slice of the 12.5-25.5% harmful-transition rate could be a frame-selection artifact. I do not think this kills the central claim—the fixed-resolution oracle gaps, the iso-compute results, and the persistent regressions in the sampling intervention all survive—but the headline rates need a nested-grid replication or at minimum an explicit statement and sensitivity analysis.

Second, the cascade threshold is tuned on the same split it is evaluated on. That is fine as an in-sample illustration, but it is presented as an operational result; it needs held-out validation or softer framing. Third, Table 2 lacks confidence intervals, and the artifact has no direct URL or commit hash in the text. Minor, but for a reproducibility-first paper, the missing link is awkward.

Overall, this is a solid paper with one load-bearing assumption that needs checking. The authors are clearly thinking clearly and honestly. Send it to peer review; make the nesting question a required revision, then the numbers can be trusted.

Referee Report

2 major / 3 minor

Summary. The paper proposes an item-level paired-trajectory audit for Video LLM scaling: for each frozen model–item pair, the authors record correctness or a continuous score under several visual budgets and then compute oracle headroom, visual confusion, harmful transitions, and text overwrite. Across five open Video LLMs, four MCQA splits, open-ended QA and summarization, and controlled dialogue generation, they report substantial configuration complementarity (8.8–18.9 accuracy points of oracle headroom and 12.5–25.5% visual confusion on the matched MCQA grid) even where aggregate scaling curves look stable or saturating. They also document protocol dependence, a sampling intervention that recovers 29.0% of terminal regressions, and a confidence cascade that matches fixed-128f accuracy at 31.7% lower equivalent frame cost.

Significance. The paper is methodologically strong in several respects: the central quantities are direct counts from released trajectories, the identity A(c')−A(c)=R−H is an exact algebraic decomposition, confidence intervals use paired item bootstraps, policy contrasts use McNemar tests, and the raw-video/cache replication gives the same +8.78-point headroom in both pipelines. The full disclosure of the Qwen3-VL-8B re-audit and the release of approximately 0.13M per-item records with provenance tags are exemplary transparency practices. If the central measurement concern below is resolved, the paper makes a significant contribution: it provides a reusable audit protocol that turns aggregate scaling curves into per-item response matrices and shows that mean improvements can conceal large opposing flows. The main caveat is the frame-nesting issue, which affects the causal interpretation of 'higher budget harms'; this is addressable and does not require rejecting the paper's empirical program.

major comments (2)

[§3, Appendix C.4] The frame-budget grids are not stated to be nested. Appendix C.4 describes cached cells as 'uniformly subsample the requested budget' from a 1-fps JPEG cache and raw cells as 'uniform decoding' from raw video, with no condition that the frame set at budget c is a subset of the frame set at budget c′>c. Under independent uniform draws, the flow counts Rc,c′ and Hc,c′ in Eq. (1) mix an increase in budget size with a change in which timestamps are visible. This is not hypothetical: the paper's own fixed-budget sampling comparison at 64f (Appendix E.2, §6.2) flips 4.7% of items between uniform and random sampling, and visually confused items are 3.83× enriched for sampling sensitivity. Because the reported 5–15% adjacent churn (§4, Figure 2) and 12.5–25.5% visual confusion (Table 2) are of the same order as the sampling-induced flip rate, the headline interpretation 'correct at a lower budget but wrong at a higher one' is not yet identified with the budget amount. Please either state and verify a nested sampling rule (for example floor(i×T/N) on a common timeline) for every reported cell, or repeat the matched-grid analysis with explicitly nested frame sets or with fixed frame positions and report whether the headroom and confusion rates survive. This is load-bearing for the central claim and should be addressed before publication.
[§7, Tables 7 and 18, Appendix H] The reference cascade is selected in-sample. The margin threshold θ=0.91 and the 16f→32f→128f design are chosen from a full sweep on the same Video-MME v1-short split (Table 18), and the headline 31.7% cost reduction is measured on that split. The paper is careful to call it a 'reference' cascade and to separate deployable from oracle-gated policies, but as written §7 and the Abstract present the cascade as an operational demonstration. Please add a held-out or cross-validated evaluation, or explicitly relabel the numbers as a retrospective illustration for the demonstrated policy class. This does not affect the item-level heterogeneity measurement, but it does affect one of the claimed applications.

minor comments (3)

[Abstract and Appendix H] The Abstract's '31.7% lower shared frame cost' refers to equivalent-frame cost; Appendix H shows that the wall-clock savings are only realized when frame sharing or KV-cache reuse is available, and that the no-share cascade is 13.7% slower than fixed-128f. Consider stating 'equivalent-frame cost' in the Abstract to avoid a misreading as wall-clock savings.
[Appendix C.5] The cost ordering for mixed frame-resolution comparisons (total pixel count) is a definitional assumption. The main text should state explicitly that the headline matched-grid results are frame-only at fixed resolution, so they do not depend on the total-pixel ordering.
[Appendix D.3] Since Qwen3-VL-8B required a re-audit and InternVL3-8B showed a +1.78-point run-to-run difference at 64f, the paper should state whether all final Table 2 numbers come from a single audited runner or from a protocol that permits such reruns; this would help readers interpret the exact-count gates.

Circularity Check

1 steps flagged · score 2.0 of 10

No central circularity; direct-count audit with one minor in-sample cascade benchmark.

fitted input called prediction [Section 7 and Appendix G.1/H, Tables 7 and 18]
"A confidence cascade matches fixed-128f accuracy while reducing average shared frame cost by 31.7%, illustrating one operational use of the response matrix. ... The best deployable point across all designs remains the frame-only 16f→32f→128f cascade. ... Cascade 16f→32f→128f, θ=0.91"

The cascade's margin threshold θ=0.91 is selected from a sweep over the same V1-short data, and Table 18 reports several deployable designs at exactly the fixed-128f accuracy of 76.5%. The statement that the cascade 'matches fixed-128f accuracy' is therefore an in-sample equality produced by threshold selection on the evaluation set, not an independent prediction or first-principles result. This is a minor, non-load-bearing circularity: the paper labels the cascade as a reference illustration, and it separately reports an untuned cross-model agreement rule (Table 8) that is not fitted, providing an external check. The main scaling claims do not depend on this step.

full rationale

The central derivation chain is self-contained: Eq. (1) is an algebraic decomposition A_c' − A_c = R − H, and oracle headroom, visual confusion, text overwrite, churn, and score-oracle gaps are direct counts over the released per-item trajectories, not fitted relations or predicted quantities. The headline numbers (8.8–18.9 pt headroom, 12.5–25.5% confusion) are computed from those counts and are externally reproducible from the released pipeline. Self-citations (e.g., Chen et al. ICML, Sun et al. 2026) appear only in generic related-work lists and are not load-bearing for any derivation. Appendix C.1 and C.5 explicitly disclose protocol-conditioned labels and ordinal cost assumptions; these are validity limitations, not circular reductions. The only mild circular element is the reference cascade, where the accuracy match is in-sample due to threshold selection on the same data; because it is presented as an operational illustration rather than a central prediction, and because an untuned cross-model rule provides an independent check, the overall circularity score remains low.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim is a measurement result, so the ledger is dominated by protocol choices rather than fitted physical constants. The only tuned number is the cascade threshold. No new entities are postulated.

free parameters (2)

delta (score regression threshold) = 0.02 score points (primary); 0.01 and 0.05 checked
Defines a later-budget regression for continuous Token-F1 trajectories (Section 3). The qualitative pattern is reported at all three thresholds, so it is a choice of analysis threshold rather than a fitted constant.
cascade margin threshold theta = 0.91 (best deployable cascade)
Selected from a sweep on Video-MME short (Appendix G.1, Table 18) and evaluated on the same split; the cross-model rule in Table 8 is untuned. This is a fitted operational parameter, not part of the central measurement claim.

assumptions (5)

domain assumption Total pixel count (frames times resolution) is the cost measure used to order configurations in mixed frame/resolution comparisons.
Appendix C.5 defines the cost ordering that underlies all lower-cost/higher-cost comparisons (visual confusion, harmful transitions). If wall-clock or energy orders configurations differently, the computed rates could shift.
domain assumption Greedy deterministic decoding is representative of model behavior for trajectory measurement.
Appendix C.2 states all official cells use greedy decoding. Sampling-based decoding could produce different trajectories; the paper itself documents protocol dependence of item labels.
domain assumption The shared matched grid {text, 16f, 32f, 64f} is a fair cross-model comparison despite different architectural context ceilings.
Section 3 defines matched grids; LLaVA and InternVL3.5 are capped at lower budgets. This avoids pooling but may understate headroom for models that support higher budgets.
domain assumption Token-F1 and ROUGE-L are adequate continuous quality metrics for open-ended generation trajectories.
Section 5 uses Token-F1 as primary and ROUGE-L as robustness; Appendix C.1 notes continuous scores are metric-dependent.
domain assumption A text-only correct answer that becomes wrong after adding video indicates that visual evidence overwrote the text answer.
Section 3 defines text overwrite this way; assumes the text-only configuration is a lower-cost baseline and that visual frames introduce competing information rather than that the text answer was itself spurious.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs." pith.science (2026). https://pith.science/paper/JSSFJVFH

@misc{pith2026260807014,
  author       = {Pith},
  title        = {Pith review of: Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JSSFJVFH}},
  note         = {Machine review of arXiv:2608.07014}
}

abstract

Aggregate scaling curves suggest that Video LLMs improve smoothly or saturate as visual budgets grow. We show that this view can conceal large, opposing changes at the item level. We represent each frozen model--item pair by its response trajectory under controlled visual budgets and derive matched-grid measures of configuration complementarity, harmful transitions, and text overwrite. Across five open Video LLMs from three architecture families, four multiple-choice benchmark splits, open-ended QA and summarization, and fixed-history dialogue generation, no single budget serves all items. On the four-model matched MCQA grid, item-level oracle headroom spans $8.8$--$18.9$ accuracy points and $12.5$--$25.5\%$ of items are correct at a lower budget but wrong at a higher one. Task-appropriate continuous metrics show the same complementarity beyond multiple choice: Token-F1 oracle gaps are $2.7$--$3.7$ score points on MLVU generation and $3.8$--$4.8$ points on AVSD current-turn generation, even when mean quality improves with budget. The effect persists across frame count, spatial resolution, sampling policy, temporal--spatial allocation, and independently executed raw-video and cached pipelines, with per-item rates and membership tracking protocol choices. A controlled sampling intervention recovers $29.0\%$ of terminal regressions, and a structured frame audit identifies several recurring evidence pathways. We release per-item trajectories, protocol provenance, derived annotations, and reproducible analysis code as an auditing artifact. A confidence cascade matches fixed-$128f$ accuracy while reducing average shared frame cost by $31.7\%$, illustrating one operational use of the response matrix.

Figures

Figures reproduced from arXiv: 2608.07014 by the authors.

**Figure 1.** A smooth aggregate curve can hide opposing item-level transitions. Qwen2.5-VL-7B on Video-MME v1 short (n=786). Aggregate accuracy is the net result of correct-to-wrong and wrong-to-correct flows; paired trajectories expose the cancellation, configuration complementarity, and text-overwrite subsets hidden by the mean. 12.5–25.5%. Full-item LLaVA replications add a third architecture family. On MLVU open-ended QA and… view at source ↗

**Figure 2.** Directional flows remain substantial even as the aggregate curve saturates. Adjacentbudget transition matrices for the Qwen2.5 V1-short raw grid. Green and red cells are rescued and harmed items; their difference gives the net accuracy change, while their sum gives churn. At 128f →256f, a −0.6-point net change hides 41 state changes. text 16 32 64 128 256 Frame budget 40 50 60 70 80 Accuracy (%) Video-MME v1 short … view at source ↗

**Figure 3.** Aggregate scaling curves across models and benchmark splits. Curves mostly improve and saturate, but the paired item-level analysis in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Text overwrite across four models on Video-MME v1 short. Top: transitions from textonly to 128f for Qwen models and to the standard-input 64f ceiling for InternVL models. Bottom: overwrite rate among text-correct items for the six largest task types. The effect is cro…

**Figure 5.** Figure 5: Iso-compute heterogeneity at approximately 9.7M total input pixels. 128f×76K wins on average, but 32f×302K, 64f×151K, and 128f×76K are the per-item optimum for 247, 254, and 285 items. The item oracle is 81.6%, +6.2 points above the best fixed allocation. 6 WHAT CHANGE…

**Figure 6.** Figure 6: Feature importance for the oracle GBM selector (Qwen2.5-VL-7B on short). Trajectory [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

**Figure 7.** Figure 7: Aggregate accuracy on the partially-filled frame [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 3 canonical work pages

[6]

LLaV A-OneVision: Easy visual task transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaV A-OneVision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024a. KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. VideoChat: Chat-centric video under...

arXiv
[7]

Stepaudio 2.5 technical report.arXiv preprint arXiv:2605.23463,

Bin Lin, Bo Zhao, Boyong Wu, Chao Yan, Chen Wu, Cheng Yi, Chengyuan Yao, Daijiao Liu, Fei Tian, Feng Tian, et al. Stepaudio 2.5 technical report.arXiv preprint arXiv:2605.23463,

arXiv
[8]

11 Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao

URL https://arxiv.org/ abs/2405.19261. 11 Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-XL: Extra-long vision language model for hour-scale video understanding. arXiv preprint arXiv:2409.14485,

arXiv
[9]

Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration

Wenzhang Sun, Zhenyu Wang, Zhangchi Hu, Chunfeng Wang, Hao Li, and Wei Chen. Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration. arXiv preprint arXiv:2602.03028,

arXiv
[11]

Less is more: Token-efficient video-qa via adaptive frame-pruning and semantic graph integration.arXiv preprint arXiv:2508.03337, 2025a

Shaoguang Wang, Weiyu Guo, Ziyang Chen, Yijie Xu, Xuming Hu, and Hui Xiong. Less is more: Token-efficient video-qa via adaptive frame-pruning and semantic graph integration.arXiv preprint arXiv:2508.03337, 2025a. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, et al. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning...

arXiv
[13]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. InInternational Conference on Learning Representations (ICLR), 2024a. Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. DeeR-VLA: Dynamic inference o...

arXiv
[15]

Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu

URLhttps://arxiv.org/abs/2412.03248. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. MLVU: Benchmarking multi-task long video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13691–13701,

arXiv
[16]

InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025a

Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025a. Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar, Zhenheng Yang, and Yang You. FOCUS: Effi...

arXiv

Show all 18 references

[18]

vs. 128f

The full sweep includes one additional Qwen3 variant:agree( 16f,32f)→128f , which reaches 77.0% at 49.1 average frames with an 82.2% stop rate. This trades slightly more compute for essentially the same accuracy as the →64f variant. We use the same untuned agreement rule for e...

2019
[285]

is remarkably balanced (Figure 5), confirming that the temporal-vs-spatial trade-off is genuinely item-dependent. Independent iso-pixel sweep at fixed 64f.To isolate the resolution dimension under a clean full-benchmark protocol, we evaluate Qwen2.5-VL-7B on V1 short at fixed ...

2024
[2019]

Qwen3-VL technical report, 2025a

Shuai Bai, Yuxuan Cai, Ruizhe Chen, et al. Qwen3-VL technical report, 2025a. URL https: //arxiv.org/abs/2511.21631. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaoha...

arXiv
[2020]

Cascade-aware training of language models.arXiv preprint arXiv:2406.00060,

Congchao Wang, Sean Augenstein, Keith Rush, Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat, Aditya Krishna Menon, and Alec Go. Cascade-aware training of language models.arXiv preprint arXiv:2406.00060,

arXiv
[2021]

El Yacoubi

Marija Brkic, Anas Filali Razzouki, Yannis Tevissen, Khalil Guetari, and Mounim A. El Yacoubi. Frame sampling strategies matter: A benchmark for small vision-language models.arXiv preprint arXiv:2509.14769,
[2022]

mPLUG-Owl3: Towards long image-sequence understanding in multi-modal large language models.arXiv preprint arXiv:2408.04840,

Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mPLUG-Owl3: Towards long image-sequence understanding in multi-modal large language models.arXiv preprint arXiv:2408.04840,

arXiv
[2023]

LLaV A-NeXT: A strong zero-shot video understanding model.LLaVA Blog, 2024a

12 Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. LLaV A-NeXT: A strong zero-shot video understanding model.LLaVA Blog, 2024a. URLhttps://llava-vl.github.io/blog/2024-04-30-llava-next-video/. Yuanhan Zhang, Jinming...

2024 arXiv
[2024]

Beyond logits: Coherent hallucination mitigation via attention contrastive decoding

Yujia Chen, Rui Sun, Huayu Mai, Wangkai Li, Zhangyu He, Bingzhou Wang, Aibing Li, Wen- zhang SUN, and Tianzhu Zhang. Beyond logits: Coherent hallucination mitigation via attention contrastive decoding. InForty-third International Conference on Machine Learning. 10 Hyungjin Chu...

arXiv
[2025]

Video-MME-v2: Towards the next stage in benchmarks for comprehensive video understanding.arXiv preprint arXiv:2604.05015,

Chaoyou Fu et al. Video-MME-v2: Towards the next stage in benchmarks for comprehensive video understanding.arXiv preprint arXiv:2604.05015,

arXiv
[2026]

Bo Feng, Zhengfeng Lai, Shiyu Li, Zizhen Wang, Simon Wang, Ping Huang, and Meng Cao

URL https: //arxiv.org/abs/2602.08024. Bo Feng, Zhengfeng Lai, Shiyu Li, Zizhen Wang, Simon Wang, Ping Huang, and Meng Cao. Breaking down video LLM benchmarks: Knowledge, spatial perception, or true temporal understanding? arXiv preprint arXiv:2505.14321,

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [6]

LLaV A-OneVision: Easy visual task transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaV A-OneVision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024a. KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. VideoChat: Chat-centric video under...

arXiv

[2] [7]

Stepaudio 2.5 technical report.arXiv preprint arXiv:2605.23463,

Bin Lin, Bo Zhao, Boyong Wu, Chao Yan, Chen Wu, Cheng Yi, Chengyuan Yao, Daijiao Liu, Fei Tian, Feng Tian, et al. Stepaudio 2.5 technical report.arXiv preprint arXiv:2605.23463,

arXiv

[3] [8]

11 Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao

URL https://arxiv.org/ abs/2405.19261. 11 Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-XL: Extra-long vision language model for hour-scale video understanding. arXiv preprint arXiv:2409.14485,

arXiv

[4] [9]

Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration

Wenzhang Sun, Zhenyu Wang, Zhangchi Hu, Chunfeng Wang, Hao Li, and Wei Chen. Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration. arXiv preprint arXiv:2602.03028,

arXiv

[5] [11]

Less is more: Token-efficient video-qa via adaptive frame-pruning and semantic graph integration.arXiv preprint arXiv:2508.03337, 2025a

Shaoguang Wang, Weiyu Guo, Ziyang Chen, Yijie Xu, Xuming Hu, and Hui Xiong. Less is more: Token-efficient video-qa via adaptive frame-pruning and semantic graph integration.arXiv preprint arXiv:2508.03337, 2025a. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, et al. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning...

arXiv

[6] [13]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. InInternational Conference on Learning Representations (ICLR), 2024a. Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. DeeR-VLA: Dynamic inference o...

arXiv

[7] [15]

Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu

URLhttps://arxiv.org/abs/2412.03248. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. MLVU: Benchmarking multi-task long video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13691–13701,

arXiv

[8] [16]

InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025a

Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025a. Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar, Zhenheng Yang, and Yang You. FOCUS: Effi...

arXiv