Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

FlexSelect: Flexible Token Selection for Efficient Long Video Understanding

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Keeping just 6.25 percent of visual tokens, ranked by one reference attention layer, improves long-video QA and speeds inference up to ninefold.

desk verdict A sensible token-pruning idea with a real efficiency story, but the evaluation is missing the one control that would show the attention ranking is what drives the gains. read the letter →

arxiv 2506.00993 v1 pith:PXGUKE73 submitted 2025-06-01 cs.CV

Yunzhu Zhang , Yu Lu , Tianyi Wang , Fengyun Rao , Yi Yang , Linchao Zhu This is my paper
classification cs.CV
keywords long-videounderstandingvisualtokenselectioncross-modalattentionreferencelayerrank-supervisedlearningVideoLLMinferenceaccelerationneedle-in-haystackevaluationpruning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlexSelect addresses the cost of long-video understanding by asking which visual tokens a VideoLLM actually needs. The paper's claim is that one intermediate transformer layer holds a faithful query-to-video relevance signal: ranking visual tokens by their mean cross-modal attention at that layer isolates the content that matters. Selecting only 6.25% of tokens this way improves accuracy on VideoMME, MLVU, LongVideoBench, and LVBench while cutting response time up to 9x. A 0.5B selector trained to copy the reference layer's ranking delivers most of the gain without an extra pass through the large model. If correct, this gives a training-free, plug-and-play way to extend the effective context length of any VideoLLM.

What carries the argument

The central object is the reference layer, a specific transformer depth (layer 19 for LLaVA-Video-7B) chosen by a needle-in-a-haystack test in which five synthetic 'needle' image-query pairs are inserted into 128 real videos and the layer that best recovers the needle tokens in its top-K attention-ranked set is selected. That layer's per-token mean cross-modal attention is the relevance score driving token selection; the lightweight selector is a small transformer that is trained to output the same ranking, with differentiable sorting making the Spearman rank loss trainable. All later accuracy and speed results follow from ranking tokens with this score and retaining the top 6.25%.

What would settle it

On a held-out set of real VideoMME questions with manually annotated relevant frames, measure the fixed reference layer's Recall@K; if it is at or below the random baseline K/N, the central claim that this layer's attention faithfully identifies relevant tokens is falsified.

Watch

Extended reading notes

Core claim

The paper discovers that in a pretrained VideoLLM, cross-modal attention scores are unreliable in early layers, most faithful at an intermediate 'reference layer', and weakened again in the deepest layers, where information condenses into the final token. FlexSelect therefore scores every visual token as the mean attention it receives from the text query across heads at that reference layer, keeps only the top-6.25% of tokens, and feeds them to the unchanged VideoLLM decoder. The same ranking signal is reproduced by a 0.5B lightweight transformer trained with a Spearman rank loss and differentiable sorting, so the expensive forward pass through the large model can be skipped during selection.

Load-bearing premise

The load-bearing premise is that a single reference layer, chosen on synthetic needle queries, gives attention scores that rank query-relevant tokens faithfully for all real long-video questions and benchmarks.

Editorial extensions

If this is right

  • At a budget of 6,720 selected visual tokens out of 107,520, FlexSelect outperforms feeding all tokens to the VideoLLM, and both smaller (1,680) and larger (13,440) budgets reduce accuracy, showing the method works as a filter rather than a naive compressor.
  • Raising the sampled frame count from 64 to 512 improves accuracy with selection active, while the unmodified model's accuracy collapses past 64 frames, so selection effectively extends the model's usable context length.
  • FlexSelect-Lite's 0.5B selector retains most of the gains of direct reference-layer scoring (for LLaVA-Video-7B, +3.9 average vs +5.5) while dropping response time at 512 frames from 38.2s to 4.2s.
  • On 72B models, FlexSelect lifts LLaVA-Video-72B to 66.9 on LongVideoBench and Qwen2.5-VL-72B to 56.6 on LVBench, new state-of-the-art scores among the open-source methods compared in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reference layer is chosen with five synthetic needle-query pairs on 128 videos; an untested extension is per-video or per-question reference-layer selection, which could push accuracy higher if the optimal layer shifts with content or task.
  • Because the ranking signal is learnable by a 0.5B model, the paper's evidence suggests the cross-modal relevance structure is highly compressible, so an even smaller ranker or a selector trained directly on downstream QA reward is a natural next step.
  • The accuracy jumps could partly reflect that irrelevant tokens actively distract the VideoLLM rather than just costing compute; a controlled lesion experiment that swaps kept tokens for random ones at matched budget would separate the signal effect from the noise-removal effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FlexSelect, a token-selection framework for long-video VideoLLMs. In its training-free mode, the method partitions a long video into stride-sampled frame sets, computes cross-modal attention scores from a chosen 'reference' transformer layer, and keeps the top-ranked visual tokens within each set before feeding the aggregated tokens to the VideoLLM. In its 'FlexSelect-Lite' mode, a small transformer initialized from a 0.5B model is trained with a Spearman rank loss to imitate the reference layer's ranking, avoiding partial forward passes through the large model. The method is evaluated on VideoMME, MLVU, LongVB, and LVBench with LLaVA-Video (7B/72B), InternVL2.5-8B, and Qwen2.5-VL (7B/72B), reporting consistent accuracy gains over the base models and up to 9x speedup on LLaVA-Video-7B.

Significance. If the central mechanism is confirmed, FlexSelect would be a practical, architecture-agnostic efficiency module for long-video understanding, requiring no retraining of the host VideoLLM. The paper's layer-wise Recall@K analysis is a useful diagnostic, and the rank-supervised distillation of attention rankings into a lightweight selector is a sensible design that the authors evaluate across model scales. However, the current evidence does not yet isolate the attention-ranking mechanism from temporal coverage, and the reference layer is chosen using the VideoMME test set. The significance is therefore conditional on additional control experiments and a clean evaluation protocol.

major comments (3)
  1. [Appendix A.1, Table 1] The central claim is that reference-layer cross-modal attention rankings identify the tokens that matter for long-video QA, but the paper never compares FlexSelect against a token-budget-matched uniform or random selector. With N=512 frames and K=8 frame sets, a cheap control is to keep every 16th frame (32 frames total) or to sample 6,720 visual tokens uniformly at random from the full frame set sequence. The current comparisons in Table 2 and Figure 5 vary input frames and selection budgets but always use the attention-based selection, so the reported gains conflate 'more temporal coverage' with 'better token ranking.' If uniform/random selection at the same token budget matches FlexSelect on VideoMME, MLVU, LongVB, and LVBench, the load-bearing ingredient is the sampling/partitioning, not the attention ranking. I request this control experiment for at least the main configuration (LLaVA-Video-7B, 512 frames, 6,720 selected tokens).
  2. [Related Work, Table 1] The reference layer is selected by Recall@K computed on 128 videos sampled from the VideoMME test set (Appendix A.1), and the same VideoMME test split is later used for the main evaluation in Table 1. This is test-set tuning of a key hyperparameter: the choice of layer 19 for LLaVA-Video-7B, and the corresponding layers for the other models, are informed by labels from the benchmark on which the method is then measured. The paper should either select the reference layer on a validation set or a separate set of videos, or demonstrate that the reported results are insensitive to the layer choice within a broad plateau. As it stands, the VideoMME improvements in particular may be partly an artifact of this selection procedure.
  3. [Section 3.2] The Related Work section claims that existing attention-based token pruning methods (FastV, PyramidDrop, SparseVLM, FrameFusion, Dycoke) 'suffer performance degradation' or are 'suboptimal,' and the abstract positions FlexSelect as superior to these approaches. However, Table 1 does not include any of these closest baselines under the same evaluation protocol. A direct comparison with FrameFusion and Dycoke, which are video-specific attention-based pruning methods, is necessary to support the claim that the reference-layer mechanism improves over existing token pruning rather than merely matching them. Without such comparisons, the 'SoTA' statement in the conclusion is not supported for the pruning-method family.
minor comments (5)
  1. [Appendix A.1] The definition of the j-th frame set, 'Fj = {f_i | i ≡ j mod N},' appears to contain a typo: with K frame sets the congruence should be modulo K (or 'i mod K = j'), otherwise the sets are not well-defined for j ranging over 1..K.
  2. [Throughout] The sentence 'layer 60 for LLaVA-Video-7B' conflicts with the main text and Implementation Details, which both identify layer 19 as the reference layer for LLaVA-Video-7B; this is presumably a typo and should be corrected to LLaVA-Video-72B.
  3. [Equation (3)] There are several typographical errors that should be fixed: 'Morever', 'Spearnman', 'sematic', 'acuracy', 'relavant', 'infromation', and 'destributions' in the appendix. The notation 'arg max L' in Appendix A.3 should be written 'arg max_L Recall@K(L)'.
  4. [Figure 6] The displayed Spearman rank correlation formula is missing parentheses in the numerator and denominator, making it difficult to parse; please reformat the equation so the sums are clearly grouped.
  5. [Figure 6] The x-axis label 'Selection Layer' is ambiguous; it should be 'Layer index used for token selection' for clarity.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild VideoMME test-set loop in reference-layer selection; no derivation-level circularity; central claim independently supported on three other benchmarks.

  1. fitted input called prediction [Appendix A.1 and Appendix A.3; §3.1 Eq. (2); Table 1 (VideoMME column).]
    "We randomly sample 128 videos from the VideoMME [11] test set and insert each needle-query pair into them, resulting in a total of 640 test samples. We compute Recall@K on these samples. ... The reference layer L for token selection are set to Layer 15 for InternVL2.5-8B, Layer 19 for LLaVA-Video-7B, Layer 20 for Qwen2.5VL-7B, Layer 60 for LLaVA-Video-72B and Qwen2.5VL-72B determined by arg max_L Recall@K(L)."

    The central hyperparameter of FlexSelect, the reference layer, is selected by maximizing Recall@K on 128 videos sampled from the VideoMME test set. The same benchmark's test set is then used in Table 1 to report the headline accuracy gains. Thus the VideoMME improvement is not a fully independent prediction of the reference-layer choice: part of the gain can reflect fitting the layer to the test distribution rather than a general property of cross-modal attention. The other three benchmarks (MLVU, LongVB, LVBench) were not used in this selection, so the core token-ranking claim retains independent support despite this mild self-referential loop.

full rationale

FlexSelect's derivation is not circular in the strict sense: the token ranking is a training-free heuristic validated by Recall@K on synthetic needle frames, the lightweight selector is a distilled proxy supervised by the reference layer's own rankings, and final claims are evaluated as end-task accuracy against unmodified base models on held-out benchmarks. There is no load-bearing self-citation chain: the cited LLaVA-Video, LongVA, LMMS-Eval, and VideoMME references are external bases, benchmarks, and tools, not prior results of the present authors. The one mild circular element is the reference-layer hyperparameter: it is chosen by maximizing Recall@K on 128 videos drawn from the VideoMME test set, and the same benchmark's test set is later used in Table 1 to report accuracy gains. This makes the VideoMME component partly self-referential, but it is a single scalar hyperparameter and the MLVU, LongVB, and LVBench results were not used in selection, so the central claim retains independent empirical content. The absence of a token-budget-matched uniform/random baseline is a control concern for the mechanism attribution, but it is not a circularity of the derivation itself.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a few tuned hyperparameters (reference layer, token budget, frames) and on the assumption that reference-layer attention faithfully encodes query relevance. No new physical or conceptual entities are introduced.

free parameters (3)
  • reference layer index per model = LLaVA-Video-7B: 19, InternVL2.5-8B: 15, Qwen2.5VL-7B: 20, LLaVA-Video-72B: 60, Qwen2.5VL-72B: 60
    Chosen by argmax Recall@K on 640 needle-in-haystack samples built from 128 VideoMME test videos (Appendix A.1). This is a hyperparameter fitted to data, and the data overlaps with the VideoMME evaluation benchmark.
  • max selected tokens = 6,720 (LLaVA-Video-7B), 8,256 (InternVL2.5-8B), 7,010 (Qwen2.5VL-7B), analogous for 72B
    Set to 6.25% of original input length based on ablations (Table 2: 6,720 gives best VideoMME). Free parameter tuned on the benchmark.
  • input frames N and max subset frames S = N=1024 for Qwen2.5VL, N=512 for LLaVA-Video and InternVL; S=64 for all
    Chosen in implementation details; input frames affect recall and accuracy (Table 2). These are free parameters.
assumptions (4)
  • domain assumption Cross-modal attention scores in pretrained VideoLLMs encode query-relevant semantic alignment at some intermediate layer
    This is the motivating premise of the paper (Section 3.1), supported only by the needle-in-haystack experiment, not by a proof.
  • domain assumption Needle-in-haystack Recall@K on five synthetic needle-query pairs generalizes to real video QA relevance
    Used to pick the reference layer (Section 3.1, Appendix A.1); no evidence that the five pairs cover temporal, counting, or multi-object queries.
  • standard math The differentiable sorting approximation to Spearman rank correlation provides valid gradients for ranking distillation
    Adopted from Blondel et al. [2]; relied on in Section 3.3 for training.
  • ad hoc to paper Frame sets defined by stride sampling provide comprehensive temporal coverage
    Design choice in Section 3.2, with a typo in the stated formula (i ≡ j mod N); assumes uniform coverage is sufficient.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlexSelect: Flexible Token Selection for Efficient Long Video Understanding." pith.science (2026). https://pith.science/paper/PXGUKE73

@misc{pith2026250600993,
  author       = {Pith},
  title        = {Pith review of: FlexSelect: Flexible Token Selection for Efficient Long Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXGUKE73}},
  note         = {Machine review of arXiv:2506.00993}
}
read the original abstract

Long-form video understanding poses a significant challenge for video large language models (VideoLLMs) due to prohibitively high computational and memory demands. In this paper, we propose FlexSelect, a flexible and efficient token selection strategy for processing long videos. FlexSelect identifies and retains the most semantically relevant content by leveraging cross-modal attention patterns from a reference transformer layer. It comprises two key components: (1) a training-free token ranking pipeline that leverages faithful cross-modal attention weights to estimate each video token's importance, and (2) a rank-supervised lightweight selector that is trained to replicate these rankings and filter redundant tokens. This generic approach can be seamlessly integrated into various VideoLLM architectures, such as LLaVA-Video, InternVL and Qwen-VL, serving as a plug-and-play module to extend their temporal context length. Empirically, FlexSelect delivers strong gains across multiple long-video benchmarks including VideoMME, MLVU, LongVB, and LVBench. Moreover, it achieves significant speed-ups (for example, up to 9 times on a LLaVA-Video-7B model), highlighting FlexSelect's promise for efficient long-form video understanding. Project page available at: https://yunzhuzhang0918.github.io/flex_select

Figures

Figures reproduced from arXiv: 2506.00993 by the authors.

Figure 1
Figure 1. (a) Visualization of cross-modal attention maps of LLaVA-Video-7B across layers (user [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Recall@K values across different lay￾ers in LLaVA-Video-7B. Recall@K metric is the recall ratio of ground-truth relevant tokens (e.g., needle-frame tokens) among the top-K tokens ranked by a layer’s cross-modal attention scores. A higher Recall@K indicates the attention scores of that layer can more accurately identify the se￾mantically related visual tokens. We choose the optimal layer with the highest Recall@K as … view at source ↗
Figure 3
Figure 3. Overview of FlexSelect token selection pipeline. Given a long video and a query, FlexSelect first partitions the video into frame sets and encodes each into visual tokens. For each set, a token selector identifies semantically relevant tokens by ranking cross-modal attention scores from a reference layer in a pre-trained VideoLLM or a lightweight selector network trained to approximate it. In this process, the proje… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Illustration of our rank-supervised training. We align lightweight model’s pre￾dicted scores ˆr with the reference layer’s sematic relevance scores r ref by optimizing the spear￾man rank correlation coefficient between them. Once trained, the ranking derived from these…
Figure 5
Figure 5. Figure 5: Response time when sampling differ￾ent number of frames. FlexSelect accelerates inference by selecting semantically relevant vi￾sual tokens faithfully. 0 7 14 19(Lref) 27 Selection Layer 35 40 45 50 55 60 65 70 75 VideoQA Acc (%) VideoMME MLVU LongVB LVBench [PITH_FUL…
Figure 7
Figure 7. Figure 7: Recall@K across layers of different VideoLLMs. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: PCA visualization of query tokens, needle visual tokens(i.e. semantically related tokens) [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Some token selection examples of our FlexSelect on LLaVA-Video-7B. We choose 8 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding

    cs.AI 2026-06 conditional novelty 6.0 of 10

    ReMem improves zero-shot long-video QA by combining LLM-based temporal granularity parsing, CLIP-based dual-semantic frame scoring, and structure-aware dynamic frame routing.

  2. Stateful Token Reduction for Long-Video Hybrid VLMs

    cs.CV 2026-02 conditional novelty 6.0 of 10

    For hybrid Mamba–Transformer video models, keeping 25% of visual tokens with a query-based progressive schedule gives 3.8–4.2x prefilling speedups with near-baseline accuracy; the paper attributes this to stateful com...

  3. HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A hierarchical benchmark for multimodal models on human-centric visual understanding finds frontier models average under 60% and miss question-uncued visual evidence, with test-time scaling helping only marginally.

Reference graph

Works this paper leans on

52 extracted references · 9 canonical work pages · cited by 3 Pith papers

  1. [1]

    Qwen2.5-vl technical report,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report,

  2. [2]

    Fast differentiable sorting and ranking, 2020

    Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking, 2020. URLhttps://arxiv.org/abs/2002.08871

    arXiv 2020

  3. [3]

    Principal component analysis.Analytical methods, 6(9): 2812–2831, 2014

    Rasmus Bro and Age K Smilde. Principal component analysis.Analytical methods, 6(9): 2812–2831, 2014

    work page 2014

  4. [4]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models, 2024

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models, 2024. URLhttps://arxiv.org/abs/2403.06764

    arXiv 2024

  5. [5]

    Longvila: Scaling long-context visual language models for long videos, 2024

    Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, and Song Han. Longvila: Scaling long-context visual language models for long videos, 2024. URLhttps://arxiv.org/abs/2408.10188

    arXiv 2024

  6. [6]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, 2025

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Jia...

    arXiv 2025

  7. [7]

    Scaling video-language models to 10k frames via hierarchical differential distillation, 2025

    Chuanqi Cheng, Jian Guan, Wei Wu, and Rui Yan. Scaling video-language models to 10k frames via hierarchical differential distillation, 2025. URL https://arxiv.org/abs/2504.02438

    arXiv 2025

  8. [8]

    Rewind: Understanding long videos with instructed learnable memory, 2025

    Anxhelo Diko, Tinghuai Wang, Wassim Swaileh, Shiyan Sun, and Ioannis Patras. Rewind: Understanding long videos with instructed learnable memory, 2025. URL https://arxiv. org/abs/2411.15556

    arXiv 2025

Show all 52 references
  1. [9]

    Feather the throttle: Revisiting visual token pruning for vision-language model acceleration, 2024

    Mark Endo, Xiaohan Wang, and Serena Yeung-Levy. Feather the throttle: Revisiting visual token pruning for vision-language model acceleration, 2024. URL https://arxiv.org/abs/ 2412.13180

    2024 arXiv

  2. [10]

    Video-ccam: Enhancing video-language understanding with causal cross-attention masks for short and long videos, 2024

    Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang, Gang Liu, and Hui Wang. Video-ccam: Enhancing video-language understanding with causal cross-attention masks for short and long videos, 2024. URLhttps://arxiv.org/abs/2408.14023

    2024 arXiv

  3. [11]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis, 2024

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. Video-mme: The first-ever compre...

    2024 arXiv

  4. [12]

    Framefusion: Combining similarity and importance for video token reduction on large visual language models, 2024

    Tianyu Fu, Tengxuan Liu, Qinghao Han, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, and Yu Wang. Framefusion: Combining similarity and importance for video token reduction on large visual language models, 2024. URL https://arxiv.org/abs/2501. 01986

    2024

  5. [13]

    Linvt: Empower your image-level large language model to understand videos, 2024

    Lishuai Gao, Yujie Zhong, Yingsen Zeng, Haoxian Tan, Dengjie Li, and Zheng Zhao. Linvt: Empower your image-level large language model to understand videos, 2024. URL https: //arxiv.org/abs/2412.05185

    2024 arXiv

  6. [14]

    Llava-onevision: Easy visual task transfer,

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer,

  7. [15]

    Aria: An open multimodal native mixture-of-experts model, 2025

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Fan Zhou, Chengen Huang, Yanpeng Li, Chongyan Zhu, Xiaoyi Ren, Chao Li, Yifan Ye, Peng Liu, Lihuan Zhang, Hanshu Yan, Guoyin Wang, Bei Chen, and Junnan Li. Aria: An open multimodal native mixture-of...

    2025 arXiv

  8. [16]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  9. [17]

    Temporal preference optimization for long-form video understanding, 2025

    Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy. Temporal preference optimization for long-form video understanding, 2025. URLhttps://arxiv.org/abs/2501. 13919

    2025

  10. [18]

    Llama-vid: An image is worth 2 tokens in large language models, 2023

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models, 2023. URLhttps://arxiv.org/abs/2311.17043

    2023 arXiv

  11. [19]

    Kangaroo: A powerful video-language model supporting long-context video input, 2024

    Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input, 2024. URLhttps://arxiv.org/abs/2408.15542

    2024 arXiv

  12. [20]

    Video-xl-pro: Recon- structive token compression for extremely long video understanding, 2025

    Xiangrui Liu, Yan Shu, Zheng Liu, Ao Li, Yang Tian, and Bo Zhao. Video-xl-pro: Recon- structive token compression for extremely long video understanding, 2025. URL https: //arxiv.org/abs/2503.18478

    2025 arXiv

  13. [21]

    Nvila: Efficient frontier visual language models, 2025

    Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, Xiuyu Li, Yunhao Fang, Yukang Chen, Cheng-Yu Hsieh, De-An Huang, An-Chieh Cheng, Vishwesh Nath, Jinyi Hu, Sifei Liu, Ranjay Krishna, Daguang Xu, Xiaolon...

    2025 arXiv

  14. [22]

    Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution, 2025

    Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution, 2025. URL https: //arxiv.org/abs/2409.12961

    2025 arXiv

  15. [23]

    Quota: Query-oriented token assignment via cot query decouple for long video comprehension, 2025

    Yongdong Luo, Wang Chen, Xiawu Zheng, Weizhong Huang, Shukang Yin, Haojia Lin, Chaoyou Fu, Jinfa Huang, Jiayi Ji, Jiebo Luo, and Rongrong Ji. Quota: Query-oriented token assignment via cot query decouple for long video comprehension, 2025. URL https: //arxiv.org/abs/2503.08689

    2025 arXiv

  16. [24]

    Hello GPT-4o, 5 2024

    OpenAI. Hello GPT-4o, 5 2024. URL https://openai.com/index/hello-gpt-4o/ . Accessed: 2024-05-20

    2024

  17. [25]

    Yarn: Efficient context window extension of large language models, 2023

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models, 2023. URL https://arxiv.org/abs/2309. 00071

    2023

  18. [26]

    Ui-tars: Pioneering automated gui interaction with native agents, 2025

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei 11 Zheng...

    2025 arXiv

  19. [27]

    Video-xl: Extra-long vision language model for hour-scale video understanding,

    Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding,

  20. [28]

    The proof and measurement of association between two things

    Charles Spearman. The proof and measurement of association between two things. In James J. Jenkins and Donald G. Paterson, editors,Studies in Individual Differences: The Search for Intelligence, pages 45–58. Appleton-Century-Crofts, 1961. doi: 10.1037/11491-005. URL https://do...

    1961 doi

  21. [29]

    Dycoke: Dynamic compression of tokens for fast video large language models, 2025

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Dycoke: Dynamic compression of tokens for fast video large language models, 2025. URL https://arxiv.org/abs/2411. 15024

    2025

  22. [30]

    URLhttps://arxiv.org/abs/2409.14485

    arXiv

  23. [31]

    Qwen2.5 technical report, 2025

    Qwen Team, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang...

    2025 arXiv

  24. [32]

    Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model’s p...

    2024 arXiv

  25. [33]

    Dai, Anja Hauth, and et al

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, and et al. Gemini: A family of highly capable multimodal models, 2024. URLhttps://arxiv.org/abs/2312.11805

    2024 arXiv

  26. [34]

    Adaretake: Adaptive redundancy reduction to perceive longer for video-language understanding, 2025

    Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Adaretake: Adaptive redundancy reduction to perceive longer for video-language understanding, 2025. URL https: //arxiv.org/abs/2503.12559

    2025 arXiv

  27. [35]

    Stop looking for important tokens in multimodal language models: Duplication matters more, 2025

    Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, and Linfeng Zhang. Stop looking for important tokens in multimodal language models: Duplication matters more, 2025. URLhttps://arxiv.org/abs/2502.11494

    2025 arXiv

  28. [36]

    Lvbench: An extreme long video understanding benchmark, 2024

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Lvbench: An extreme long video understanding benchmark, 2024. URLhttps://arxiv.org/abs/2406.08035

    2024 arXiv

  29. [37]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction, 2025

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction, 2025. URLhttps://arxiv. org/abs/2410.17247

    2025 arXiv

  30. [38]

    Slowfast-llava-1.5: A family of token-efficient video large language models for long-form video understanding, 2025

    Mingze Xu, Mingfei Gao, Shiyu Li, Jiasen Lu, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, and Afshin Dehghan. Slowfast-llava-1.5: A family of token-efficient video large language models for long-form video understanding, 2025. URL https://arxiv.org/abs/ 2503.18943

    2025 arXiv

  31. [39]

    Longvideobench: A benchmark for long- context interleaved video-language understanding, 2024

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long- context interleaved video-language understanding, 2024. URL https://arxiv.org/abs/ 2407.15754

    2024 arXiv

  32. [41]

    Videollama 3: Frontier multimodal foundation models for image and video understanding, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. Videollama 3: Frontier multimodal foundation models for image and video understanding, 20...

    2025 arXiv

  33. [42]

    mplug-owl3: Towards long image-sequence understanding in multi-modal large language models, 2024

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models, 2024. URLhttps://arxiv.org/abs/2408.04840. 12

    2024 arXiv

  34. [43]

    Long context transfer from language to vision,

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision,

  35. [44]

    [cls] attention is all you need for training-free visual token pruning: Make vlm inference faster, 2024

    Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Minqi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. [cls] attention is all you need for training-free visual token pruning: Make vlm inference faster, 2024. URLhttps://arxiv.org/abs/2412.01818

    2024 arXiv

  36. [45]

    Lmms-eval: Reality check on the evaluation of large multimodal models, 2024

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024. URL https://arxiv.org/abs/2407.12772

    2024 arXiv

  37. [46]

    Sparsevlm: Visual token sparsification for efficient vision-language model inference, 2025

    Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, and Shanghang Zhang. Sparsevlm: Visual token sparsification for efficient vision-language model inference, 2025. URL https: //arxiv.org/abs/...

    2025 arXiv

  38. [47]

    URLhttps://arxiv.org/abs/2406.16852

    arXiv

  39. [48]

    Mlvu: Benchmarking multi-task long video understanding, 2025

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: Benchmarking multi-task long video understanding, 2025. URLhttps://arxiv.org/abs/2406.04264

    2025 arXiv

  40. [49]

    Llava-mini: Efficient image and video large multimodal models with one vision token.arXiv preprint arXiv:2501.03895, 2025

    Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. Llava-mini: Efficient image and video large multimodal models with one vision token.arXiv preprint arXiv:2501.03895, 2025

    2025 arXiv

  41. [51]

    Video instruction tuning with synthetic data, 2024

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data, 2024. URLhttps://arxiv.org/abs/2410.02713

    2024 arXiv

  42. [53]

    Apollo: An exploration of video understanding in large multimodal models, 2024

    Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, Serena Yeung-Levy, and Xide Xia. Apollo: An exploration of video understanding in large multimodal models, 2024. URL https://arxiv.or...

    2024 arXiv

  43. [2024]

    URLhttps://arxiv.org/abs/2408.03326

    arXiv

  44. [2025]

    URLhttps://arxiv.org/abs/2502.13923

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.