Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GreedyPrune claims that visual token pruning should be treated as selecting a critical token set that maximizes semantic saliency under a pairwise diversity constraint, and that a greedy algorithm can do this well enough to preserve…

desk verdict The greedy saliency-plus-diversity idea is reasonable, but the reported SOTA margins don't survive arithmetic on the tables and the tau tuning looks per-benchmark. read the letter →

arxiv 2506.13166 v1 pith:73ZQD33P submitted 2025-06-16 cs.CV

Ruiguang Pei , Weiqing Sun , Zhihui Fu , Jun Wang This is my paper
classification cs.CV
keywords visualtokenpruninglargevision-languagemodelsgreedyalgorithmsemanticsaliencydiversitycosinesimilaritytraining-freeinferenceaccelerationredundancy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GreedyPrune is a training-free visual token pruning method for large vision-language models (LVLMs). The paper argues that token pruning should be reformulated as selecting a 'critical visual token set' that keeps both semantic importance and visual diversity, and that a greedy algorithm can approximate this combinatorial problem well enough to preserve accuracy. The paper claims that its cosine-similarity saliency metric beats attention-score-based selection and that greedy redundancy removal keeps models accurate even at an 88.9% token reduction. On nine image benchmarks, GreedyPrune reports 94.22% average accuracy for LLaVA-1.5-7B and 94.98% for LLaVA-1.5-13B at that compression, ahead of the compared baselines. This matters because visual token processing dominates LVLM inference cost, especially on resource-constrained devices.

What carries the argument

The central machinery is a two-step greedy selection procedure. First, each visual token is scored by its cosine similarity to the last text token of the prompt, a semantic saliency metric the paper argues is less biased than cross-attention scores. Second, tokens are processed in descending saliency order: the top remaining token becomes a pivot and is added to the critical set, and all remaining tokens whose pairwise cosine similarity with the pivot exceeds a threshold $\tau$ are discarded as redundant. The paper formulates the underlying objective as an unconstrained binary quadratic program, notes that exact solution is NP-hard, and uses the greedy rule as a tractable approximation that satisfies the constraint step by step. The threshold $\tau$ controls the saliency-diversity trade-off, with lower $\tau$ demanding stronger diversity and higher $\tau$ allowing more high-saliency picks.

What would settle it

Run GreedyPrune on a dataset where the prompt contains no image-relevant words, such as a generic 'Describe this image' instruction; if removing 88.9% of tokens on that prompt drops accuracy to or below random token selection, then the cosine-similarity saliency ranking, rather than the greedy diversity step, is not doing the claimed work.

Watch

Extended reading notes

Core claim

The paper's central claim is that the key to training-free visual token pruning is to solve a critical-set selection problem: pick a fixed-size subset of visual tokens that maximizes total semantic saliency while keeping pairwise cosine similarity below a threshold, so the retained tokens are not redundant. The authors propose a specific saliency metric, the cosine similarity between the last text token and each visual token, arguing it captures semantics without the positional-attention bias of cross-attention scores and is compatible with FlashAttention. They then select tokens greedily: sort visual tokens by this saliency, take the highest-scoring remaining token as a pivot, remove all remaining tokens whose cosine similarity to the pivot exceeds the threshold tau, and repeat until the target subset size is reached. The paper demonstrates through ablation that both components matter, and reports that high pruning ratios, such as retaining only 64 of 576 tokens, keep near-full accuracy: 94.22% average on LLaVA-1.5-7B and 94.98% on LLaVA-1.5-13B across nine datasets, outperforming the second-best compared method by 3.82 and 4.08 points respectively.

Load-bearing premise

The load-bearing assumption is that the cosine similarity between the prompt's last text token and a visual token is a trustworthy measure of that token's semantic importance, and that deleting tokens similar to already-selected tokens never removes information the model still needs.

Editorial extensions

If this is right

  • If GreedyPrune works as claimed, LVLMs can run with about one ninth of their visual tokens while retaining most benchmark accuracy, making large models more practical on memory- and latency-limited hardware.
  • Attention-based pruning can be replaced by a cosine-similarity metric that avoids the bottom-of-image attention bias and remains compatible with FlashAttention.
  • The same plug-and-play procedure transfers across model families, including LLaVA-1.5, LLaVA-NeXT, and Qwen2-VL, without fine-tuning or calibration.
  • At equal pruning ratios and equal TFLOPS budgets, the reported results place GreedyPrune ahead of FastV, DART, and DivPrune on the nine evaluated benchmarks.
  • The optimal threshold $\tau$ varies by dataset, so tuning it per deployment or per task is a direct way to trade diversity against semantic saliency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open whether the cosine-similarity saliency metric remains reliable when the prompt is very short or does not name the image content; a natural extension would be to blend it with other signals, such as CLIP score or attention, only when the text is uninformative.
  • The greedy procedure requires pairwise comparisons between the pivot and all remaining tokens, an overhead that might matter at low pruning ratios; a batched or approximative similarity computation could extend the method to real-time settings.
  • The same critical-set formulation could be applied to other redundant sequences, such as KV caches or long context windows, by swapping the saliency metric while keeping the greedy diversity constraint.
  • Because $\tau$ needs tuning per dataset, an adaptive threshold chosen per input or per image, rather than a global constant, is a testable improvement implied by the paper's own ablation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes GreedyPrune, a training-free visual token pruning method for large vision-language models (LVLMs). It formulates critical visual token selection as a combinatorial optimization problem: maximize total semantic saliency (measured by cosine similarity between the last text token and each visual token) subject to pairwise cosine-similarity constraints among selected tokens, and solves it with a greedy pivot-token selection. Experiments on LLaVA-1.5-7B/13B, LLaVA-1.6-7B, and Qwen2-VL across nine benchmarks claim state-of-the-art accuracy at high pruning ratios (e.g., 94.98% average at 88.9% pruning on LLaVA-1.5-13B, 4.08% above the second-best method) with reduced end-to-end latency.

Significance. If the empirical claims hold, GreedyPrune would be a practically useful training-free, plug-and-play acceleration method for LVLMs, with a simple and interpretable selection criterion that is compatible with Flash Attention. The paper's strengths include a clear problem formulation, extensive evaluation across multiple model families and nine benchmarks, ablations isolating the saliency metric and the redundancy-removal component, and qualitative visualizations. However, the central quantitative claims are currently undermined by internal inconsistencies in the reported tables and by an unspecified protocol for the threshold τ, so the significance cannot be fully assessed until these issues are resolved.

major comments (5)
  1. [Tables 1, 2, and 4] The main result for LLaVA-1.5-7B with 64 retained tokens is internally inconsistent. Table 1 reports Ours(τ=0.94) with Avg 94.22% and per-dataset scores (MME 1442, POPE 84.4, MMB 61.3, Ocrbench 286, TextVQA 42, OK-VQA 49.2, Nocaps 98.4, Flickr30K 69.5, GQA 60.4). Table 4 reports Ours(τ=0.78) for the same retained-token count with the same Avg 94.22% but different per-dataset scores (MME 1427, POPE 84.5, MMB 61.0, Ocrbench 284, TextVQA 41.9, OK-VQA 49.6, Nocaps 98.4, Flickr30K 69.7, GQA 60.2). Table 2's GreedyPrune row matches the Table 4 numbers, not the Table 1 numbers. This means the reported Avg cannot be reproduced from the displayed values, and the reader cannot tell which τ is the actual configuration for the headline result.
  2. [Table 1, LLaVA-1.5-13B row] The claimed Avg of 94.98% is not recoverable from the per-dataset values. Using the same normalization implied by the Original row (each per-dataset score divided by the Original score, then averaged), the Ours(τ=0.78) row yields about 95.9%, and the ratio-of-averages method yields 98.1%; neither matches 94.98%. The stated 4.08% margin over DivPrune therefore cannot be verified from the published table.
  3. [Section 3.3 and Figure 5] The threshold τ is a free parameter with strong effect on accuracy (the ablation shows POPE peaking at 0.78 and MME still rising at 0.93), yet the paper does not specify how τ is selected for each setting. This is compounded by the fact that identical settings use different τ values in different tables (τ=0.94 in Table 1 vs τ=0.78 in Table 4 for LLaVA-1.5-7B at 64 tokens). Without a defined selection protocol (e.g., a validation split) or a sensitivity analysis, the state-of-the-art comparisons may reflect per-benchmark tuning rather than a single plug-and-play method.
  4. [Section 3.2 and contributions] The paper claims that greedy algorithms 'can efficiently achieve near-optimal solutions' to the formulated NP-hard UBQP problem, but no approximation bound, analysis, or empirical optimality-gap study is provided. The text only states that heuristics provide approximate solutions. This is an overstatement of the theoretical support and should be revised or substantiated with a concrete bound for the greedy policy.
  5. [Equation (1)] The diversity constraint is written as cos(v_i, v_j) ≤ τ for all v_i, v_j ∈ V, but it should apply to the selected subset S; as written, the constraint is independent of the choice of S. The correct constraint appears only implicitly in Eq. (2) via the z_i z_j product. This is a formal error in the problem statement that should be corrected.
minor comments (5)
  1. [Title, Section 1] The title contains a typo: 'Retenting' should be 'Retaining'; similarly, 'tranning-free' in Section 1 and 'Reasearch' in the affiliations should be corrected.
  2. [Section 4.3] The text refers to 'Figure 3' for the τ ablation, but the ablation is shown in Figure 5; the algorithm overview is Figure 3. Please fix the cross-references.
  3. [Section 3.3] The notation for the critical token set alternates between V_critic and Vcritic; please unify the notation throughout the paper.
  4. [Tables 1–4] The paper would benefit from reporting variance or multiple runs for the key tables; no error bars or seeds are provided for the benchmark results.
  5. [Section 4.2 and Table 3] The terminology for pruning ratios is inconsistent: 88.9% pruning corresponds to 11.1% retained, while Table 3 lists 'Retained token ratio' values of 33.3%, 22.2%, and 11.1%. Please ensure consistent language throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the saliency metric is validated by an independent removal-impact diagnostic and the greedy algorithm is a standard approximate solver for the stated objective; reported accuracies are external benchmark measurements.

full rationale

The paper's claimed derivation chain is self-contained and does not reduce to its inputs by construction. The cosine-similarity saliency metric is proposed as an alternative to attention scores and is validated by an independent diagnostic (Figure 2a) in which removing the top-20% highest-saliency tokens causes larger performance drops than removing attention-selected tokens; downstream accuracies are external benchmark measurements, not fit quantities. The greedy selection procedure directly implements the stated combinatorial objective (maximize saliency subject to pairwise cosine similarity below tau), and tau is disclosed as a hyperparameter rather than derived from target accuracy. No load-bearing self-citation or imported uniqueness theorem appears. The main concerns are verifiability/correctness issues rather than circularity: Table 1 and Table 4 report different tau values (0.94 vs 0.78) for the same LLaVA-1.5-7B 64-token setting while listing the same 94.22% average with different per-dataset scores, and the averaging formula for 'Avg' rows is not specified; these inconsistencies undermine the SOTA claim but do not constitute a circular reduction in the paper's derivation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on two similarity-based proxies (saliency and redundancy) that are empirically motivated, plus an unproven greedy optimality assumption; tau is a tunable hyperparameter.

free parameters (1)
  • tau (cosine similarity threshold) = 0.78 to 0.94 depending on pruning ratio
    The threshold controls how aggressively similar tokens are removed; different values are reported for each pruning ratio and model (Table 1, Table 3) with no stated selection rule.
assumptions (4)
  • domain assumption Cosine similarity between the last text token and visual tokens is an effective semantic saliency metric.
    Introduced in Section 3.1; supported only by a removal-impact experiment, not by theory.
  • domain assumption Pairwise cosine similarity above tau correctly identifies redundant visual tokens.
    Used in Section 3.3 to delete similar tokens; no independent validation of this redundancy proxy.
  • ad hoc to paper The greedy algorithm yields a near-optimal solution to the subset selection problem.
    Section 3.2 states heuristic methods including greedy provide approximate solutions but gives no approximation bound.
  • standard math UBQP is NP-hard and Lagrange relaxation is an appropriate analysis tool.
    Section 3.2 cites Kochenberger et al. 2014; this justifies using a heuristic but is not used in the final algorithm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models." pith.science (2026). https://pith.science/paper/73ZQD33P

@misc{pith2026250613166,
  author       = {Pith},
  title        = {Pith review of: GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73ZQD33P}},
  note         = {Machine review of arXiv:2506.13166}
}
read the original abstract

Although Large Vision Language Models (LVLMs) have demonstrated remarkable performance in image understanding tasks, their computational efficiency remains a significant challenge, particularly on resource-constrained devices due to the high cost of processing large numbers of visual tokens. Recently, training-free visual token pruning methods have gained popularity as a low-cost solution to this issue. However, existing approaches suffer from two key limitations: semantic saliency-based strategies primarily focus on high cross-attention visual tokens, often neglecting visual diversity, whereas visual diversity-based methods risk inadvertently discarding semantically important tokens, especially under high compression ratios. In this paper, we introduce GreedyPrune, a training-free plug-and-play visual token pruning algorithm designed to jointly optimize semantic saliency and visual diversity. We formalize the token pruning process as a combinatorial optimization problem and demonstrate that greedy algorithms effectively balance computational efficiency with model accuracy. Extensive experiments validate the effectiveness of our approach, showing that GreedyPrune achieves state-of-the-art accuracy across various multimodal tasks and models while significantly reducing end-to-end inference latency.

Figures

Figures reproduced from arXiv: 2506.13166 by the authors.

Figure 1
Figure 1. Comparison between baselines and GreedyPrune [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Rethinking semantic saliancy of visual token [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of the architecture of LVLM with GreedyPrune [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance-Latency trade-off comparisons across different datasets on LLaVA-1.5-13B. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: ablation study of similarity value τ tinues to rise. These findings suggest that different datasets exhibit varying preferences regarding the redundancy level of the visual token set, which some favor lower redundancy, whereas others prioritize higher semantic saliency…
Figure 6
Figure 6. Figure 6: The columns in the figure are arranged from left to right as follows: input image, FastV, DART, Divprune, and the [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

Reference graph

Works this paper leans on

53 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Agrawal, H.; Desai, K.; Wang, Y.; Chen, X.; Jain, R.; Johnson, M.; Batra, D.; Parikh, D.; Lee, S.; and Anderson, P. 2019. nocaps: novel object captioning at scale. In Proceedings of the IEEE International Conference on Computer Vision, 8948--8957

    work page 2019

  2. [2]

    R.; Singh, G.; Akbari, M.; and Zhang, Y

    Alvar, S. R.; Singh, G.; Akbari, M.; and Zhang, Y. 2025. DivPrune : Diversity -based Visual Token Pruning for Large Multimodal Models . ArXiv:2503.02175 [cs]

    arXiv 2025

  3. [3]

    Arif, K. H. I.; Yoon, J.; Nikolopoulos, D. S.; Vandierendonck, H.; John, D.; and Ji, B. 2024. HiRED : Attention - Guided Token Dropping for Efficient Inference of High - Resolution Vision - Language Models . ArXiv:2408.10945 [cs]

    arXiv 2024

  4. [4]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5- VL Technical Report . ArXiv:2502.13923 [cs]

    arXiv 2025

  5. [5]

    Bo Li*, K. Z. F. P. X. D. Y. D. H. L. Y. Z. G. Z. C. L., Peiyuan Zhang*; and Liu, Z. 2024. LMMs-Eval: Accelerating the Development of Large Multimoal Models

    work page 2024

  6. [6]

    Bolya, D.; Fu, C.-Y.; Dai, X.; Zhang, P.; Feichtenhofer, C.; and Hoffman, J. 2023. Token Merging : Your ViT But Faster . ArXiv:2210.09461 [cs]

    arXiv 2023

  7. [7]

    Chen, J.; Ye, L.; He, J.; Wang, Z.-Y.; Khashabi, D.; and Yuille, A. 2024 a . Efficient Large Multi -modal Models via Visual Context Compression . ArXiv:2406.20092 [cs]

    arXiv 2024

  8. [8]

    Chen, L.; Li, J.; Dong, X.; Zhang, P.; He, C.; Wang, J.; Zhao, F.; and Lin, D. 2023. ShareGPT4V: Improving Large Multi-Modal Models with Better Captions. arXiv preprint arXiv:2311.12793

    arXiv 2023

Show all 53 references
  1. [9]

    Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B. 2024 b . An Image is Worth 1/2 Tokens After Layer 2: Plug -and- Play Inference Acceleration for Large Vision - Language Models . ArXiv:2403.06764 [cs]

    2024 arXiv

  2. [10]

    Chen, Z.; Wang, W.; Cao, Y.; Liu, Y.; Gao, Z.; Cui, E.; Zhu, J.; Ye, S.; Tian, H.; Liu, Z.; Gu, L.; Wang, X.; Li, Q.; Ren, Y.; Chen, Z.; Luo, J.; Wang, J.; Jiang, T.; Wang, B.; He, C.; Shi, B.; Zhang, X.; Lv, H.; Wang, Y.; Shao, W.; Chu, P.; Tu, Z.; He, T.; Wu, Z.; Deng, H.; G...

    2025 arXiv

  3. [11]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P.; and Hoi, S. 2023. InstructBLIP : Towards General -purpose Vision - Language Models with Instruction Tuning . ArXiv:2305.06500 [cs]

    2023 arXiv

  4. [12]

    Dao, T. 2024. Flash A ttention-2: Faster Attention with Better Parallelism and Work Partitioning. In International Conference on Learning Representations (ICLR)

    2024

  5. [13]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale . ArXiv:2010.11929 [cs]

    2021 arXiv

  6. [14]

    Endo, M.; Wang, X.; and Yeung-Levy, S. 2024. Feather the Throttle : Revisiting Visual Token Pruning for Vision - Language Model Acceleration . ArXiv:2412.13180 [cs]

    2024 arXiv

  7. [15]

    Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; et al. 2023. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. arXiv preprint arXiv:2306.13394

    2023 arXiv

  8. [16]

    Gagrani, M.; Goel, R.; Jeon, W.; Park, J.; Lee, M.; and Lott, C. 2024. On Speculative Decoding for Multimodal Large Language Models. arXiv:2404.08856

    2024 arXiv

  9. [17]

    Han, J.; Du, L.; Wu, Y.; Zhou, X.; Du, H.; and Zheng, W. 2025. AdaFV : Rethinking of Visual - Language alignment for VLM acceleration. ArXiv:2501.09532 [cs]

    2025 arXiv

  10. [18]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  11. [19]

    Jin, Y.; Li, J.; Liu, Y.; Gu, T.; Wu, K.; Jiang, Z.; He, M.; Zhao, B.; Tan, X.; Gan, Z.; Wang, Y.; Wang, C.; and Ma, L. 2024. Efficient Multimodal Large Language Models: A Survey. arXiv:2405.10739

    2024

  12. [20]

    Kochenberger, G.; Hao, J.-K.; Glover, F.; Lewis, M.; L \"u , Z.; Wang, H.; and Wang, Y. 2014. The unconstrained binary quadratic programming problem: a survey. Journal of combinatorial optimization, 28: 58--81

    2014

  13. [21]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 a . BLIP -2: Bootstrapping Language - Image Pre -training with Frozen Image Encoders and Large Language Models . ArXiv:2301.12597 [cs]

    2023 arXiv

  14. [22]

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, X.; and Wen, J.-R. 2023 b . Evaluating Object Hallucination in Large Vision-Language Models. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 292--305...

    2023

  15. [23]

    Liang, X.; Guan, C.; Lu, J.; Chen, H.; Wang, H.; and Hu, H. 2025. Dynamic Token Reduction during Generation for Vision Language Models . ArXiv:2501.14204 [cs]

    2025 arXiv

  16. [24]

    Lin, B.; Ye, Y.; Zhu, B.; Cui, J.; Ning, M.; Jin, P.; and Yuan, L. 2024. Video- LLaVA : Learning United Visual Representation by Alignment Before Projection . ArXiv:2311.10122 [cs]

    2024 arXiv

  17. [25]

    Lin, Z.; Lin, M.; Lin, L.; and Ji, R. 2025. Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference . ArXiv:2405.05803 [cs]

    2025 arXiv

  18. [26]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved Baselines with Visual Instruction Tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 26296--26306

    2024

  19. [27]

    Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024 b . LLaVA-NeXT: Improved reasoning, OCR, and world knowledge

    2024

  20. [28]

    Liu, T.; Shi, L.; Hong, R.; Hu, Y.; Yin, Q.; and Zhang, L. 2024 c . Multi- Stage Vision Token Dropping : Towards Efficient Multimodal Large Language Model . ArXiv:2411.10803 [cs]

    2024 arXiv

  21. [29]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; Chen, K.; and Lin, D. 2024 d . MMBench: Is Your Multi-modal Model an All-around Player? arXiv:2307.06281

    2024 arXiv

  22. [30]

    Liu, Y.; Li, Z.; Huang, M.; Yang, B.; Yu, W.; Li, C.; Yin, X.-C.; Liu, C.-L.; Jin, L.; and Bai, X. 2024 e . OCRBench: on the hidden mystery of OCR in large multimodal models. Science China Information Sciences, 67(12)

    2024

  23. [31]

    Marino, K.; Rastegari, M.; Farhadi, A.; and Mottaghi, R. 2019. OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge. In Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  24. [32]

    Meng, Y.; Li, K.; Huang, C.; Gao, C.; Chen, X.; Li, Y.; and Zhang, X. 2025. PLPHP : Per - Layer Per - Head Vision Token Pruning for Efficient Large Vision - Language Models . ArXiv:2502.14504 [cs]

    2025 arXiv

  25. [33]

    Pei, X.; Huang, T.; and Xu, C. 2024. Cross- Self KV Cache Pruning for Efficient Vision - Language Inference . ArXiv:2412.04652 [cs]

    2024 arXiv

  26. [34]

    A.; Wang, L.; Cervantes, C

    Plummer, B. A.; Wang, L.; Cervantes, C. M.; Caicedo, J. C.; Hockenmaier, J.; and Lazebnik, S. 2016. Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. arXiv:1505.04870

    2016 arXiv

  27. [35]

    Qiao, Y.; Yu, Z.; Zhao, Z.; Chen, S.; Sun, M.; Guo, L.; Wu, Q.; and Liu, J. 2024. VL-Mamba : Exploring State Space Models for Multimodal Learning. In Rezagholizadeh, M.; Passban, P.; Samiee, S.; Partovi Nia, V.; Cheng, Y.; Deng, Y.; Liu, Q.; and Chen, B., eds., Proceedings of ...

    2024

  28. [36]

    J.; and Yan, Y

    Shang, Y.; Cai, M.; Xu, B.; Lee, Y. J.; and Yan, Y. 2024. LLaVA - PruMerge : Adaptive Token Reduction for Efficient Large Multimodal Models . ArXiv:2403.15388 [cs]

    2024

  29. [37]

    Singh, A.; Natarajan, V.; Shah, M.; Jiang, Y.; Chen, X.; Batra, D.; Parikh, D.; and Rohrbach, M. 2019. Towards VQA Models That Can Read. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 , 8317--8326. Computer Visio...

    2019

  30. [38]

    Su, J.; Lu, Y.; Pan, S.; Murtadha, A.; Wen, B.; and Liu, Y. 2023. RoFormer: Enhanced Transformer with Rotary Position Embedding. arXiv:2104.09864

    2023 arXiv

  31. [39]

    N.; Kaiser, L.; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2023. Attention Is All You Need . ArXiv:1706.03762 [cs]

    2023 arXiv

  32. [40]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Fan, Y.; Dang, K.; Du, M.; Ren, X.; Men, R.; Liu, D.; Zhou, C.; Zhou, J.; and Lin, J. 2024. Qwen2- VL : Enhancing Vision - Language Model 's Perception of the World at Any Resolution ....

    2024 arXiv

  33. [41]

    Wen, Z.; Gao, Y.; Li, W.; He, C.; and Zhang, L. 2025 a . Token Pruning in Multimodal Large Language Models : Are We Solving the Right Problem ? ArXiv:2502.11501 [cs]

    2025 arXiv

  34. [42]

    Wen, Z.; Gao, Y.; Wang, S.; Zhang, J.; Zhang, Q.; Li, W.; He, C.; and Zhang, L. 2025 b . Stop Looking for Important Tokens in Multimodal Language Models : Duplication Matters More . ArXiv:2502.11494 [cs]

    2025 arXiv

  35. [43]

    Wu, X.; Zeng, F.; Wang, X.; and Chen, X. 2024. PPT : Token Pruning and Pooling for Efficient Vision Transformers . ArXiv:2310.01812 [cs]

    2024 arXiv

  36. [44]

    Xing, L.; Huang, Q.; Dong, X.; Lu, J.; Zhang, P.; Zang, Y.; Cao, Y.; He, C.; Wang, J.; Wu, F.; and Lin, D. 2025. PyramidDrop : Accelerating Your Large Vision - Language Models via Pyramid Visual Redundancy Reduction . ArXiv:2410.17247 [cs]

    2025 arXiv

  37. [45]

    Xu, S.; Li, X.; Yuan, H.; Qi, L.; Tong, Y.; and Yang, M.-H. 2024. LLAVADI: What Matters For Multimodal Large Language Models Distillation. arXiv:2407.19409

    2024 arXiv

  38. [46]

    Yang, S.; Chen, Y.; Tian, Z.; Wang, C.; Li, J.; Yu, B.; and Jia, J. 2024. VisionZip: Longer is Better but Not Necessary in Vision Language Models. arXiv preprint arXiv:2412.04467

    2024

  39. [47]

    A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; and Liu, Z

    Zhang, K.; Li, B.; Zhang, P.; Pu, F.; Cahyono, J. A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; and Liu, Z. 2024 a . LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models. arXiv:2407.12772

    2024 arXiv

  40. [48]

    Zhang, Q.; Cheng, A.; Lu, M.; Zhuo, Z.; Wang, M.; Cao, J.; Guo, S.; She, Q.; and Zhang, S. 2024 b . [ CLS ] Attention is All You Need for Training - Free Visual Token Pruning : Make VLM Inference Faster . ArXiv:2412.01818 [cs]

    2024 arXiv

  41. [49]

    Zhang, Z.; Yadav, S.; Han, F.; and Shutova, E. 2024 c . Cross-modal Information Flow in Multimodal Large Language Models. arXiv preprint arXiv:2411.18620

    2024 arXiv

  42. [50]

    N.; and Yu, L

    Zhao, S.; Wang, Z.; Juefei-Xu, F.; Xia, X.; Liu, M.; Wang, X.; Liang, M.; Zhang, N.; Metaxas, D. N.; and Yu, L. 2024. Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction . ArXiv:2412.00556 [cs]

    2024 arXiv

  43. [51]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. MiniGPT -4: Enhancing Vision - Language Understanding with Advanced Large Language Models . ArXiv:2304.10592 [cs]

    2023 arXiv

  44. [52]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.