Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DynTok compresses a video's visual tokens to 44.4% of their original count while keeping accuracy, and spends the savings on more frames to lift long-video scores.

desk verdict Modest but useful token compression; zero-shot evidence is solid, mechanism attribution needs a fixed-grouping control. read the letter →

arxiv 2506.03990 v1 pith:FVZZXBKP submitted 2025-06-04 cs.CL cs.CV

Hongzhi Zhang , Jingyuan Zhang , Xingguang Ji , Qi Wang , Fuzheng Zhang This is my paper
classification cs.CLcs.CV
keywords videounderstandingvisualtokencompressionmergingcosinesimilaritylongvision-languagemodelDynTokefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DynTok is a compression method that needs no learned parameters, and it cuts the visual tokens that a video language model sends to the large language model (LLM) backbone. It examines each row of image patches, measures the cosine similarity between neighboring patch embeddings from the vision encoder, and merges consecutive patches whose similarity exceeds a threshold into a single averaged token. The paper reports that this reduces the token count to 44.4% of the baseline, a 2.2x reduction, while keeping accuracy essentially unchanged. Because each frame uses fewer tokens, the model can afford to take in more frames per video, reaching 65.3% on Video-MME and 72.5% on MLVU. The practical claim is that much of a video's visual signal is redundant at the patch level, and exploiting that redundancy locally preserves spatial layout while costing only linear similarity computation.

What carries the argument

The central object is the row-wise dynamic token group. For each horizontal line of image patches, DynTok computes the cosine similarity $s = \frac{x_{k-1} \cdot x_k}{\|x_{k-1}\| \|x_k\|}$ between the CLIP/SigLIP representations of neighboring patches and compares it with threshold $S_{\mathrm{th}}$. Consecutive patches above threshold are fused by averaging into a single visual token; a patch below threshold starts a new group, so high-detail regions naturally keep more tokens. The row-local scope preserves spatial order, and the appended grid marker maintains row boundary information for the LLM. Because only left-neighbor similarities are needed, the compression cost is linear in the number of tokens.

What would settle it

On a video benchmark dominated by dense text or small moving objects, run DynTok zero-shot at $S_{\mathrm{th}}=0.6$ and compare per-task accuracy with the uncompressed baseline; if OCR or action accuracy collapses while the token savings stay near 2.2x, the similarity-is-redundancy assumption fails exactly where fine-grained local detail matters.

Watch

Extended reading notes

Core claim

The paper's central claim is that a purely local, threshold-driven merging of adjacent visual tokens can remove more than half of a video's visual tokens without hurting, and in some settings improving, video understanding. DynTok operates on each row of the patch grid produced by a vision encoder such as SigLIP: a patch starts a new group whenever its cosine similarity to the patch on its left falls below a threshold $S_{\mathrm{th}}$; otherwise it joins its predecessor's group, and every group is averaged into one token. A grid marker appended to each row tells the LLM where rows end. The authors show this works zero-shot on an already trained model, keeping at least 96% of baseline accuracy on MVBench and above 98% on most other benchmarks while using under half the tokens, and that training with the compressed tokens closes most of the remaining gap. The same token budget can then be spent on more frames, which is why the method's long-video numbers, 65.3% on Video-MME and 72.5% on MLVU, exceed the baseline that uses fewer frames.

Load-bearing premise

The entire compression rests on cosine similarity between neighboring patches being a faithful measure of redundancy: the paper assumes that when two adjacent patches look alike to the vision encoder, averaging them loses nothing the task needs.

Editorial extensions

If this is right

  • A 2.2x token reduction means the LLM's attention cost over the visual prefix drops by roughly 4.8x, since attention scales quadratically with sequence length.
  • Within a fixed token budget, DynTok can process roughly twice as many frames as the baseline, which is exactly how it converts compression into better long-video accuracy.
  • The threshold $S_{\mathrm{th}}$ is a continuous dial: $S_{\mathrm{th}}=0.4$ gives 5x compression with near-baseline accuracy, $S_{\mathrm{th}}=0.5$ gives 3x with above-baseline accuracy, and $S_{\mathrm{th}}=0.6$ gives 2.2x.
  • Because the compression is parameter-free, any existing video LLM can adopt it without retraining for an immediate token saving, and retraining on merged tokens recovers most of the zero-shot losses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same row-wise merging should transfer to still images with the same threshold, giving image models a content-adaptive token budget; the paper explicitly leaves image understanding for future work.
  • Editorial inference: since the mechanism compresses before the LLM and the paper's own numbers show almost no zero-shot loss on temporal tasks, pairing DynTok with per-layer token dropping inside the LLM, which the paper lists as unexplored, should compound the savings without much additional accuracy cost.
  • Editorial inference: the zero-shot drops on OCR and action tasks suggest a cheap extension: make $S_{\mathrm{th}}$ task- or layer-dependent rather than fixed, which could recover those losses while keeping compression on smooth regions.
  • Editorial inference: the reported scaling behavior implies a token-budget allocation rule, spend saved tokens on more frames when the video is long and temporal reasoning dominates, and raise $S_{\mathrm{th}}$ when spatial detail dominates, which the paper sketches but does not state as a policy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DynTok, a dynamic visual-token compression method for video LLMs. For each row of image-patch tokens, DynTok computes the cosine similarity between adjacent SigLIP/CLIP features; whenever the similarity exceeds a threshold Sth, the token is merged into the current group, and each group is averaged into one token, with a grid marker appended at each row end. The method is parameter-free at inference and can be used zero-shot or combined with fine-tuning. Experiments on MVBench, PerceptionTest, NextQA, LongVideoBench, MLVU, and Video-MME show that a DynTok-trained model at Sth=0.6 uses 44.4% of the baseline tokens while matching or slightly exceeding baseline accuracy (e.g., 64.0 vs. 63.6 on Video-MME), and that using more input frames yields 65.3% on Video-MME and 72.5% on MLVU. The paper also reports a per-task breakdown showing that zero-shot DynTok loses most on OCR and action tasks, while training with DynTok recovers much of this gap.

Significance. If the central claim is validated, DynTok is a practically attractive method: it is simple, training-compatible, and offers a 2.2x token reduction with no aggregate accuracy loss, while also enabling more frames under a fixed token budget. The zero-shot variant and the per-task analysis are useful contributions, and the method is clearly distinct from fixed-pooling and learned-compressor baselines. The main significance hinges on whether the adaptive cosine-similarity grouping is actually the operative mechanism, and whether the long-video gains survive a proper evaluation protocol.

major comments (3)
  1. [Section 4.2, Table 1 and Figure 3] The central 2.2x-reduction-without-loss claim is established only against the uncompressed baseline. There is no control that removes the similarity signal while keeping the same token budget, the same group-averaging operation, and the same row grid markers. A random grouping per row, a fixed block-merging scheme, or a fixed-stride pooling at matched token counts should be evaluated in both the zero-shot and trained settings. If such a control matches DynTok's accuracy, then the adaptive similarity threshold is not the source of the result, and the claim that DynTok 'preserves essential content' via similarity is unsupported. This is a required ablation before the mechanism-specific claims can be accepted.
  2. [Section 4.2, Figures 3-4, Table 1 last row] The 'DynTok-moreframes' results (65.3% on Video-MME and 72.5% on MLVU) appear to be obtained after sweeping Sth and the number of frames on the test benchmarks. The paper does not state whether these configurations were selected on a validation split or pre-registered before evaluation. If the reported numbers are the best over the test-set sweeps in Figures 3 and 4, they are optimistically biased and should not be compared directly with baseline numbers from a fixed protocol. Please specify the selection procedure or report a fixed configuration (for example, Sth=0.6 and a fixed frame count) for all benchmarks.
  3. [Section 3.2, Eq. (1)] Similarity is computed on the CLIP/SigLIP representation X, while the averaging is applied to the post-MLP tokens H. Because the MLP connector is not guaranteed to preserve cosine neighborhoods, the paper should justify this choice or provide a small ablation that computes similarity directly on H, or otherwise show that cosine similarity in X correlates with redundancy in H. This is not a fatal issue, but it is load-bearing for the claimed information-preservation mechanism, especially given the zero-shot drops on fine-grained tasks in Table 2.
minor comments (6)
  1. [Section 1, Contributions] The text reads 'adaptively splitting and fussing the similar adjacent tokens'; this should be 'fusing'.
  2. [Section 4.1] There is a typo 'trianing' in the description of the single-image stage, and 'avaliable' in Section 4.2; both should be corrected.
  3. [Section 3.2] The notation H' ∈ R^{l,demb} should clarify whether the grid markers are included in the count l, since the markers are part of the concatenated token sequence.
  4. [Section 4.2] The claim that 160 frames with DynTok produce 'roughly equivalent' token consumption to 64 baseline frames would benefit from an explicit token-count calculation, including grid-marker tokens, because the current numbers (15k vs. 12.5k) are not exactly matched.
  5. [Section 4.1 and Table 1] The paper first says the baseline is trained on '10 million samples' and later states the video dataset contains 1.79M training samples; please clarify whether the 10M figure includes the single-image stage and whether all video samples are used for the video stage.
  6. [Figure 4] The rightmost subplot plots accuracy against token count, but the legend labels only frame counts; please make the mapping between frame counts and token counts explicit, since it is central to the token-budget comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DynTok's similarity-threshold merging is an empirical modeling assumption, and its compression-ratio and accuracy claims are benchmark outcomes, not inputs encoded by construction.

full rationale

The paper's derivation chain is self-contained: DynTok defines a row-wise cosine-similarity threshold Sth (Section 3.2, Eq. 1), merges adjacent tokens whose similarity exceeds Sth, averages each group, and adds a grid marker to preserve row structure. The reported 44.4% token ratio is the observed outcome of applying Sth = 0.6 to the evaluated videos, and the accuracy numbers are measured on external benchmarks (Video-MME, MLVU, MVBench, etc.) against a baseline trained with identical settings except for the DynTok mechanism. No equation in the paper defines the target accuracy in terms of the compression ratio, and no fitted parameter is renamed as a prediction. The threshold Sth is a hyperparameter swept in Figure 3, not a value chosen by inverting the benchmark result, so the 'comparable performance at 44.4% tokens' claim does not reduce by construction to the method's inputs. The paper also does not rely on self-citation as load-bearing evidence: the cited prior works are external systems and benchmarks, and no referenced result is used to forbid alternative interpretations or to assert uniqueness. The assumption that high cosine similarity indicates redundancy is a substantive modeling choice rather than a circular definition; the paper itself reports where this proxy degrades (OCR and action tasks in Table 2), which further confirms that the mechanism is empirically testable rather than tautologically valid. The absence of a control that merges tokens by fixed or random grouping at the same token budget is an ablation gap and a correctness risk, not a circularity: it affects whether the similarity signal is the operative cause, but it does not make any equation or claim equivalent to its own inputs. Overall, no self-definitional, fitted-input, self-citation, uniqueness-importation, ansatz-smuggling, or renaming circularity is present, so the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new entities and its only free parameters are the similarity threshold Sth and the inference frame count, both chosen by the user and in the experiments tuned on test benchmarks. The core assumptions are that adjacent-token cosine similarity indicates redundancy, averaging preserves meaning, and row-marker tokens convey spatial layout.

free parameters (2)
  • Similarity threshold Sth = 0.6 for main results; 0.5 for low-budget regime; sampled from {0.4,0.45,0.5,0.55,0.6} during training
    Controls the trade-off between compression ratio and accuracy; selected based on Video-MME and benchmark sweeps (Figures 3-4).
  • Inference frame count = 160 for long-video headline results (Video-MME 65.3%, MLVU 72.5%)
    Chosen from the frame sweep in Figure 4; baseline uses 64-96 frames, so DynTok's reported gains combine compression with more temporal context.
assumptions (4)
  • domain assumption Cosine similarity between adjacent CLIP token embeddings is a valid proxy for visual information redundancy.
    DynTok's grouping decision depends entirely on this; used in Section 3.2 similarity measurement.
  • domain assumption Averaging the embeddings of merged tokens preserves the information needed for downstream video QA.
    The fusing step averages all tokens in a group, assuming no critical information is lost.
  • domain assumption A single marker token at the end of each row is sufficient to convey spatial layout to the LLM.
    DynTok adds one grid marker per row and relies on it to preserve spatial information.
  • domain assumption Training with random thresholds sampled from a small set generalizes to inference thresholds.
    The training procedure varies Sth to make the model robust, but the inference threshold is still a free choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding." pith.science (2026). https://pith.science/paper/FVZZXBKP

@misc{pith2026250603990,
  author       = {Pith},
  title        = {Pith review of: DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FVZZXBKP}},
  note         = {Machine review of arXiv:2506.03990}
}
read the original abstract

Typical video modeling methods, such as LLava, represent videos as sequences of visual tokens, which are then processed by the LLM backbone for effective video understanding. However, this approach leads to a massive number of visual tokens, especially for long videos. A practical solution is to first extract relevant visual information from the large visual context before feeding it into the LLM backbone, thereby reducing computational overhead. In this work, we introduce DynTok, a novel \textbf{Dyn}amic video \textbf{Tok}en compression strategy. DynTok adaptively splits visual tokens into groups and merges them within each group, achieving high compression in regions with low information density while preserving essential content. Our method reduces the number of tokens to 44.4% of the original size while maintaining comparable performance. It further benefits from increasing the number of video frames and achieves 65.3% on Video-MME and 72.5% on MLVU. By applying this simple yet effective compression method, we expose the redundancy in video token representations and offer insights for designing more efficient video modeling techniques.

Figures

Figures reproduced from arXiv: 2506.03990 by the authors.

Figure 1
Figure 1. Illustration of two test video frames with their [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed token compression method, DynTok. For each row of image patches, every [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of varying the token merging threshold [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Model performance on Video-MME by varying numbers of video frames used. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The impact of varying the token merging threshold on token grouping results for the same test image. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The token grouping results on different test images. The token merging threshold is set as 0.6. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    CRAFT recursively merges video tokens with training-free similarity selection plus learnable gated fusion, retaining ~97% of average accuracy at 8x compression across six benchmarks.

  2. Stateful Token Reduction for Long-Video Hybrid VLMs

    cs.CV 2026-02 conditional novelty 6.0 of 10

    For hybrid Mamba–Transformer video models, keeping 25% of visual tokens with a query-based progressive schedule gives 3.8–4.2x prefilling speedups with near-baseline accuracy; the paper attributes this to stateful com...

Reference graph

Works this paper leans on

47 extracted references · 13 canonical work pages · cited by 2 Pith papers

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, and Malcolm Reynolds. 2022. Flamingo: a visual language model for few-shot learning. In NeurIPS

    work page 2022

  2. [2]

    Anthropic. 2024. Claude-3.5. https://www.anthropic.com/news/claude-3-5-sonnet

    work page 2024

  3. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. https://api.semanticscholar.org/CorpusID:261101015 Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    work page 2023

  4. [4]

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. 2023. Token merging: Your ViT but faster. In International Conference on Learning Representations

    work page 2023

  5. [5]

    Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Ma \ n as, Zhiqiu Lin ..., Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, and Vikas Chandra. 2024. ht...

    arXiv 2024

  6. [6]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    arXiv 2024

  7. [7]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. 2024. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476

    arXiv 2024

  8. [8]

    Xiangxiang Chu, Limeng Qiao, Xinyang Lin, Shuang Xu, Yang Yang, Yiming Hu, Fei Wei, Xinyu Zhang, Bo Zhang, Xiaolin Wei, et al. 2023. Mobilevlm: A fast, reproducible and strong vision language assistant for mobile devices. arXiv preprint arXiv:2312.16886

    arXiv 2023

Show all 47 references
  1. [9]

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024 a . Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075

    2024 arXiv

  2. [10]

    Tianyu Fu, Tengxuan Liu, Qinghao Han, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, and Yu Wang. 2024 b . https://arxiv.org/abs/2501.01986 Framefusion: Combining similarity and importance for video token reduction on large visual language models . Preprint, arXiv:2501.01986

    2024 arXiv

  3. [11]

    Wenbo Hu, Zi-Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, and Kai-Wei Chang. 2024. Matryoshka query transformer for large vision-language models. arXiv preprint arXiv:2405.19315

    2024 arXiv

  4. [12]

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. 2025. Miradata: A large-scale video dataset with long durations and structured captions. Advances in Neural Information Processing Systems, 37:48955--48970

    2025

  5. [13]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2025. https://openreview.net/forum?id=zKv8qULV6n LL a VA -onevision: Easy visual task transfer . Transactions on Machine Learning Research

    2025

  6. [14]

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li. 2024 a . Aria: An open multimodal native mixture-of-experts model. arXiv preprint arXiv:2410.05993

    2024 arXiv

  7. [15]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023 a . https://api.semanticscholar.org/CorpusID:256390509 Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models . In International Conference on Machine Learning

    2023

  8. [16]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023 b . Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    2023 arXiv

  9. [17]

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. 2024 b . Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2024

  10. [18]

    Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, and Lei Zhang. 2024 c . Tokenpacker: Efficient visual projector for multimodal llm. arXiv preprint arXiv:2407.02392

    2024 arXiv

  11. [19]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  12. [20]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. 2024 a . Videogpt+: Integrating image and video encoders for enhanced video understanding. arXiv preprint arXiv:2406.09418

    2024 arXiv

  13. [21]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. 2024 b . Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    2024

  14. [22]

    OpenAI. 2023. Gpt-4v. https://openai.com/index/gpt-4v-system-card/

    2023

  15. [23]

    OpenAI. 2024. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/

    2024

  16. [24]

    Viorica Pătrăucean, Lucas Smaira, Ankush Gupta, Adrià Recasens Continente, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, Antoine Miech, Alex Frechette, Hanna Klimczak, Raphael Koste...

    2023

  17. [25]

    Michael S Ryoo, Honglu Zhou, Shrikant Kendre, Can Qin, Le Xue, Manli Shu, Silvio Savarese, Ran Xu, Caiming Xiong, and Juan Carlos Niebles. 2024. xgen-mm-vid (blip-3-video): You only need 32 tokens to represent a video even in vlms. arXiv preprint arXiv:2410.16267

    2024 arXiv

  18. [26]

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. 2024. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint arXiv:2403.15388

    2024

  19. [27]

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. 2024. https://api.semanticscholar.org/CorpusID:274192345 Dycoke: Dynamic compression of tokens for fast video large language models . ArXiv, abs/2411.15024

    2024 arXiv

  20. [28]

    Gemini Team. 2024. https://arxiv.org/abs/2403.05530 Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context . Preprint, arXiv:2403.05530

    2024 arXiv

  21. [29]

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. 2024. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406.16860

    2024 arXiv

  22. [30]

    Han Wang, Yuxiang Nie, Yongjie Ye, Deng GuanYu, Yanjie Wang, Shuai Li, Haiyang Yu, Jinghui Lu, and Can Huang. 2024 a . https://arxiv.org/abs/2412.09530 Dynamic-vlm: Simple dynamic visual token compression for videollm . Preprint, arXiv:2412.09530

    2024 arXiv

  23. [31]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 b . https://arxiv.org/abs/2409.12191 Qwen...

    2024 arXiv

  24. [32]

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. 2024 c . Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pages 396--416. Springer

    2024

  25. [33]

    Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. 2024. Longvlm: Efficient long video understanding via large language models. In European Conference on Computer Vision, pages 453--470. Springer

    2024

  26. [34]

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. 2024. Longvideobench: A benchmark for long-context interleaved video-language understanding. arXiv preprint arXiv:2407.15754

    2024 arXiv

  27. [35]

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. Next-qa: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9777--9786

    2021

  28. [36]

    Fuzhao Xue, Yukang Chen, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al. 2024. Longvila: Scaling long-context visual language models for long videos. arXiv preprint arXiv:2408.10188

    2024 arXiv

  29. [37]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  30. [38]

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. 2024 b . Visionzip: Longer is better but not necessary in vision language models. arXiv preprint arXiv:2412.04467

    2024

  31. [39]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2...

    2024 arXiv

  32. [40]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. https://arxiv.org/abs/2303.15343 Sigmoid loss for language image pre-training . Preprint, arXiv:2303.15343

    2023 arXiv

  33. [41]

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2024 a . https://arxiv.org/abs/2407.12772 Lmms-eval: Reality check on the evaluation of large multimodal models . Preprint, arX...

    2024 arXiv

  34. [42]

    Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. 2025. Llava-mini: Efficient image and video large multimodal models with one vision token. arXiv preprint arXiv:2501.03895

    2025 arXiv

  35. [44]

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2024 c . Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713

    2024 arXiv

  36. [45]

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264

    2024 arXiv

  37. [46]

    Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, and Dan Jurafsky. 2022. https://doi.org/10.18653/v1/2022.acl-short.45 Problems with cosine as a measure of embedding similarity for high frequency words . In Proceedings of the 60th Annual Meeting of the Association for Computationa...

    2022 doi

  38. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  39. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.