Pith. sign in

REVIEW 4 major objections 6 minor 48 references

SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SlimVLM filters low-attention visual tokens first, then prunes attention heads and MLP channels, keeping the average multimodal benchmark score within 6.2% at 20% parameter removal.

desk verdict Solid VLM pruning paper with a genuinely useful token-selection trick, but the headline numbers lean on a 24-sample MMBench calibration set and test-set-tuned hyperparameters. read the letter →

arxiv 2608.03580 v1 pith:J5HAPHMF submitted 2026-08-04 cs.CV

Yaozhi Wen , Jialong Guo , Zhenliang Ni , Han Shu , Xinghao Chen This is my paper
classification cs.CV
keywords structuredpruningvision-languagemodelsvisualtokenselectionattention-basedimportancesensitivity-awarePearsoncorrelationmodelcompressionGrouped-QueryAttention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Structured pruning of a vision-language model's language backbone tends to destroy accuracy because the hundreds of visual tokens entering the decoder distort which modules look important. SlimVLM's claim is that this interference can be removed before pruning: by reading the decoder's own text-to-visual attention maps, discarding low-attention image tokens, and only then scoring attention heads and MLP channels for removal. The paper further claims that modules differ in how much sparsity they tolerate, so each module's pruning ratio should be set by the Pearson correlation between its pruned and unpruned outputs rather than by a global ratio. On 7-billion- and 13-billion-parameter open VLMs, the method reports a 6.2% average score drop at 20% pruning, above prior structured-pruning baselines, and a 25% inference speedup. That would make 20% pruning practical for phone- and edge-deployed VLMs without retraining the full model.

What carries the argument

Two mechanisms carry the argument. The first is a per-layer visual-token mask built from average text-to-visual attention scores: for each decoder layer, the paper extracts the cross-attention matrix between text queries and visual keys, averages over text queries to get a score per visual token, and discards any token with score below the threshold α = mean(p)·β, with β = 0.2. This mask is used only during module-importance estimation, not at inference. The second is a sensitivity-aware dynamic ratio schedule r = r0·exp(γ(L/T − 1)), where L is the Pearson correlation between a module's pruned and unpruned output; modules whose outputs remain linearly reconstructible get larger pruning ratio

What would settle it

Re-run the 20% pruning pipeline twice on the same 7B VLM: once with the 24-sample MMBench calibration set, once with 24 samples drawn from a text-heavy benchmark such as TextVQA, then evaluate both on TextVQA and VizWiz. A performance swing larger than the roughly 3% spread the paper observes across gamma and T settings would show the reported average depends on calibration-set composition.

Watch

Extended reading notes

Core claim

Structured pruning of a VLM's language backbone fails, the paper claims, because the many visual tokens entering it distort module-importance estimates. Ablations show that using all or random visual tokens during importance scoring lowers post-pruning accuracy, while retaining high-attention tokens restores most of it. The remedy couples this token mask to a sensitivity-aware ratio: each attention-head group and MLP block is pruned according to the Pearson correlation between pruned and unpruned outputs, plus a least-squares correction. With both components, 20% of parameters come off at a reported 6.2% average relative-score cost on the standard 7B model; the pipeline also handles high-res

Load-bearing premise

The central claim assumes that 24 sampled MMBench instances stand in for all eight evaluation benchmarks; if those calibration samples are not representative, the computed module ratios and the reported averages are optimistic.

Editorial extensions

If this is right

  • At 20% pruning, the paper reports an average relative score drop of 6.2% on the standard 7B model and 6.5% on the high-resolution 7B model across eight multimodal benchmarks; the 13B variant drops only 3.58%.
  • At 40% pruning the average drop is about 24.8 to 25.4 percent, and the gap to the two structured-pruning baselines widens, which the paper attributes to module-wise ratio allocation rather than a uniform rate.
  • The token-selection mask is not applied at inference, only during importance estimation, so the pruned model retains its full visual context; the paper also shows the mask can be combined with a token-compression method for an additional 1% average drop.
  • Post-pruning LoRA fine-tuning on a small instruction dataset recovers about 3 percentage points of average score at 20% pruning and 8.85 points at 40%.
  • At 20% pruning, inference throughput on the 7B model rises by 24.94%, from 85.39 to 106.69 tokens per second.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same attention-derived token mask could be reused at runtime as a token-pruning policy, since the paper's own visualizations show discarded tokens are mostly background and the mask is computed from attention already available in the decoder; this would compound the speedup without changing the pruning ratios.
  • Beyond the paper: the reported distribution of module ratios, where deep MLP layers and middle attention heads retain more capacity, is effectively a per-layer sensitivity map and could guide layer-skipping or early-exit placement in VLMs, which the paper does not discuss.
  • Beyond the paper: because the dynamic schedule is calibrated on a 24-sample MMBench set, a natural test is to recalibrate on task-mixed samples; if the assigned ratios shift materially, the method would need per-deployment calibration rather than a single default.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SlimVLM, a structured pruning framework for vision-language models. It has two main components: (i) an adaptive visual-token selection strategy that uses average text-to-visual attention scores to discard low-attention visual tokens before module-importance estimation, and (ii) a sensitivity-aware dynamic pruning-ratio mechanism that adjusts per-module pruning rates based on the Pearson correlation between pruned and unpruned module outputs. The method prunes attention heads and MLP channels, and is evaluated on LLaVA-1.5-7B, LLaVA-Next-7B, and Qwen2.5-VL on eight multimodal benchmarks. The authors report that at 20% pruning, SlimVLM retains 93.8% average relative performance on LLaVA-1.5 and outperforms FLAP and Wanda-sp; they also include ablations for the token-selection strategy, dynamic pruning ratios, hyperparameters, and development-set size.

Significance. If the empirical results hold, SlimVLM would be a practically useful contribution to VLM deployment, combining token-level redundancy with structured weight pruning. The paper's strengths are the clear ablation of the token-selection mechanism (Table 4), the inclusion of multiple architectures including GQA-based Qwen2.5-VL, and the extension to 13B models in the appendix. The central idea that visual-token interference corrupts importance estimation is plausible and well motivated. However, the headline quantitative claims rest on a small calibration set that overlaps with an evaluation benchmark and on hyperparameters tuned on the same benchmarks; these issues need to be addressed before the state-of-the-art claim can be accepted.

major comments (4)
  1. [§4.1, Table 6, Appendix D] The pruning configuration—visual-token masks (§3.1) and per-module ratios via Eq. 6 (§3.2)—is computed on a development set of 24 samples from MMBench, and MMBench/MMB_CN are also evaluation targets. The reported average retention (93.8% at 20% pruning, Table 1) is therefore partly circular for those benchmarks, and for TextVQA/VizWiz it assumes that 24 MMBench images represent very different task distributions. Appendix D concedes this risk. I would like to see pruning performed with calibration sets drawn from held-out distributions, or at least multiple random draws with mean/std of the eight scores, to substantiate the 'near-original accuracy' claim.
  2. [§4.3, Tables 8 and 13] β, γ, and T are tuned on the evaluation benchmarks: β=0.2 is chosen from Table 13, and γ=5, T=1 from Table 8, with average normalized to 100%. This makes the reported 93.8% average a selected maximum rather than a prediction for a fixed method. Report results with default values chosen without peeking at these benchmarks, or use nested validation, and show that the advantage over FLAP/Wanda-sp is not an artifact of this selection.
  3. [§4.2, Table 1] The abstract and §4.2 claim 'near-original accuracy' under 20% pruning. Averaged over benchmarks this is 93.8%, but Table 1 shows a 13.5% relative drop on TextVQA (48.07→41.56) and 8.7% on MMB_CN (50.26→45.88) for LLaVA-1.5. The average is also helped by relative gains on VizWiz and POPE. The claim should be qualified as 'near-original average accuracy across the selected benchmarks,' and the per-benchmark variability should be discussed.
  4. [§3.3, Eq. (10)] The channel-importance formula is not reproducibly specified: C=Sigmoid(E/Ē) is a vector, W' is D×D, and W_down_i is a column of the down projection, but the product 'C_0 W'_0 W_down_i, ..., C_D W'_D W_down_i' is not defined as a matrix expression. It is unclear whether C is applied element-wise to rows/columns of W' and how the norm is taken. Please rewrite with explicit indexing/dimensions; this is central to the channel-pruning contribution.
minor comments (6)
  1. [§3.2, Eq. (6)] L and T are used without definition; L is presumably the Pearson correlation from Eq. (5). Define both before Eq. (6).
  2. [§4.1] The phrase '24 randomly sampled instances from both the English and Chinese versions of MMBench' is ambiguous; state whether this is 24 total or 24 per language version.
  3. [Table 2] No baseline comparison is provided for Qwen2.5-VL. If baselines are not available or too costly, state this explicitly.
  4. [§4.2, first paragraph] 'All methods use the same visual token selection strategy' is important context; move it to the setup section so the reader knows Table 1 compares the dynamic-ratio component while the token-selection contribution is isolated in Table 4.
  5. [Table 7] The speed measurements are single estimates. Report the measurement conditions (batch size, GPU, input resolution) and, if possible, variance over repeats.
  6. [§B.3] The phrase 'the same setup described in Section "Experiment"' should refer to a specific section number, e.g., §4.1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SlimVLM's reported results are empirical benchmark measurements, not quantities forced by construction from the calibration set.

full rationale

Walking the derivation chain, the pruning masks and per-module ratios are obtained from 24 MMBench calibration samples (Sec. 4.1) and from Pearson-correlation output comparisons (Eqs. 5-8); the paper's headline numbers are measured evaluation scores on eight external benchmarks (Table 1) and are not equal by construction to those calibration inputs. The self-citation to SlimLLM [23] is used as a baseline (Table 10) and as related work, not as the justification for the proposed method, so it is not load-bearing. The Appendix D limitation ('the quality and diversity of the development samples may still affect the reliability of importance estimation') is a data-representativeness and generalization concern, not a circular derivation; it does not identify an equation or fitted parameter that makes the output identical to the input. Hyperparameter analyses (Tables 8, 13) are empirical robustness checks, not renamed predictions. No circular step is present.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

All calibration choices are empirical, not derived: beta, gamma, T, r0, development set size, and skipped-layer ranges are external inputs. The method introduces no new physical entities or mediators.

free parameters (6)
  • beta threshold in Eq. 4 = 0.2
    Threshold alpha = mean(p) * beta for visual-token retention; tuned by ablation on evaluation benchmarks (Table 13).
  • gamma in Eq. 6 = 5 at 20% pruning, 1 at 40%
    Sensitivity of the dynamic pruning ratio; selected via Table 8 on the main benchmark set.
  • T in Eq. 6 = 1 at 20%, implicit at 40%
    Expected linear-correlation threshold; treated as a tunable hyperparameter (Table 8).
  • r0 initial pruning ratio = 0.20 and 0.40
    Global target pruning ratio is an input, not derived; it sets all module ratios.
  • development set size = 24 samples from MMBench EN and CN
    Module importance is computed on this set; Table 6 tests 12/24/36 samples.
  • skipped layer ranges = LLaVA layers 4-30 pruned; Qwen layers 2-27 pruned
    The paper prunes only certain layers to preserve function; the choice is ad hoc (Section 4.1).
assumptions (6)
  • domain assumption Attention scores from text-to-visual cross-attention reflect visual token informativeness for pruning.
    Section 3.1 uses average text-to-visual attention p_i as the selection metric; if low-attention tokens are still needed for importance estimation, the mask biases pruning.
  • domain assumption Pearson correlation between pruned and unpruned module outputs is a valid proxy for end-task accuracy loss.
    Equations 5-6 set pruning ratios from linear correlation; the paper does not justify this proxy with end-task gradients.
  • ad hoc to paper The exponential relation r = r0 * exp(gamma * (L/T - 1)) is a reasonable sensitivity schedule.
    Equation 6 is introduced heuristically; no derivation from a loss or error model is provided.
  • domain assumption 24 sampled MMBench instances represent all evaluation distributions.
    Section 4.1 computes all importance scores on this set; Appendix D acknowledges possible unreliability.
  • standard math PCA of MLP outputs identifies channel importance in the output feature space.
    Section 3.3 uses eigenvalues and eigenvectors of MLP outputs; PCA is standard linear algebra, but its adequacy as an importance metric is assumed.
  • domain assumption Skipping early and late layers preserves function while pruning middle layers.
    Section 4.1 specifies pruned layers 4-30 for LLaVA and 2-27 for Qwen; no criterion is given for this split.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models." pith.science (2026). https://pith.science/paper/J5HAPHMF

@misc{pith2026260803580,
  author       = {Pith},
  title        = {Pith review of: SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J5HAPHMF}},
  note         = {Machine review of arXiv:2608.03580}
}
read the original abstract

While Vision-Language Models (VLMs) have demonstrated remarkable performance in processing and understanding both text and images, their large parameter sizes lead to significant computational overhead, limiting their deployment on resource-constrained devices. While pruning has been effective for compressing Large Language Models (LLMs), directly applying it to VLMs leads to significant performance drops, largely due to redundant visual tokens interfering with importance estimation. To this end, we propose SlimVLM, a structured pruning framework designed to compress VLMs while preserving their task performance. We introduce an adaptive visual token selection strategy for VLMs that leverages average text-to-visual attention scores to assess the importance of visual tokens, removing redundant ones during pruning based on a set threshold, thereby optimizing the importance calculation. Recognizing the varying tolerance to sparsity across different modules, we also propose a Sensitivity-aware dynamic pruning mechanism that determines the appropriate pruning ratio for each module by calculating the linear reconstruction error between the outputs of the pruned and unpruned modules, ensuring overall performance stability. Experimental results show that SlimVLM outperforms existing methods across multiple multimodal benchmarks, achieving state-of-the-art performance.

Figures

Figures reproduced from arXiv: 2608.03580 by the authors.

Figure 1
Figure 1. The trade-off between accuracy and inference [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of our proposed SlimVLM. Our approach consists of two main components: (1) Adaptive Visual Token Selection, which reuses decoder attention maps and text queries to compute importance scores for filtering out less informative visual tokens; and (2) Structured Pruning, which removes redundant attention heads and MLP channels based on their importance scores Si and Ii , respectively. As the model … view at source ↗
Figure 3
Figure 3. Performance of SlimVLM on the LLaVA-1.5 across three multimodal benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The distribution of dynamic pruning rates in LLaVA-1.5: Attention Heads and MLP Channels under 20% pruning ratio. Adaptive visual token selection strategy. From [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparative analysis of model performance. Model Size Larger Baseline (14.89GB) Prune Ratio:20% (12.29GB) Prune Ratio:30% (10.92GB) Prune Ratio:40% (10.01GB) Prune Ratio:50% (9.76GB) [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 7
Figure 7. Figure 7: Performance of SlimVLM on the LLaVA-1.5 across six multimodal benchmarks. The [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: This figure illustrates the retained visual tokens at layers [3, 16, 29] for a sample from the [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Visual examples from LLaVA-1.5-7B under 20% pruning ratio. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 6 canonical work pages

  1. [1]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URLhttps://arxiv.org/abs/2304.08485

    arXiv 2023

  2. [2]

    Qwen2.5-vl technical report,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report,

  3. [3]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou,...

    arXiv 2025

  4. [4]

    Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven C. H. Hoi. From images to textual prompts: Zero-shot vqa with frozen large language models, 2023. URLhttps://arxiv.org/abs/2212.10846

    arXiv 2023

  5. [5]

    Zhou Yu, Xuecheng Ouyang, Zhenwei Shao, Meng Wang, and Jun Yu. Prophet: Prompting large language models with complementary answer heuristics for knowledge-based visual question answering.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47 (8):6797–6808, August 2025. ISSN 1939-3539. doi: 10.1109/tpami.2025.3562422. URL http://dx.doi.org/10.1...

    work page arXiv 2025

  6. [6]

    Embodied Lifelong Learning for Task and Motion Planning

    Jorge Mendez-Mendez, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Embodied lifelong learning for task and motion planning, 2023. URLhttps://arxiv.org/abs/2307.06870

    work page Pith review arXiv 2023

  7. [7]

    End-to-end task- completion neural dialogue systems, 2018

    Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. End-to-end task- completion neural dialogue systems, 2018. URLhttps://arxiv.org/abs/1703.01008

    arXiv 2018

  8. [8]

    Lawrence Zitnick, Dhruv Batra, and Devi Parikh

    Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Dhruv Batra, and Devi Parikh. Vqa: Visual question answering, 2016. URL https://arxiv. org/abs/1505.00468

    arXiv 2016

Show all 48 references
  1. [9]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi, 2024

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A m...

    2024 arXiv

  2. [10]

    Visionzip: Longer is better but not necessary in vision language models, 2024

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models, 2024. URL https://arxiv.org/abs/2412.04467

    2024

  3. [11]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models, 2024

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models, 2024. URLhttps://arxiv.org/abs/2403.06764

    2024 arXiv

  4. [12]

    Hiprune: Training- free visual token pruning via hierarchical attention in vision-language models, 2025

    Jizhihui Liu, Feiyi Du, Guangdao Zhu, Niu Lian, Jun Li, and Bin Chen. Hiprune: Training- free visual token pruning via hierarchical attention in vision-language models, 2025. URL https://arxiv.org/abs/2508.00553

    2025 arXiv

  5. [13]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation langua...

    2023 arXiv

  6. [14]

    Qwen technical report, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Si...

    2023 arXiv

  7. [15]

    A survey on multimodal large language models.National Science Review, 11(12), November 2024

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models.National Science Review, 11(12), November 2024. ISSN 2053-714X. doi: 10.1093/nsr/nwae403. URL http://dx.doi.org/10.1093/nsr/nwae403

    2024 doi

  8. [16]

    Zipvl: Efficient large vision-language models with dynamic token sparsification, 2024

    Yefei He, Feng Chen, Jing Liu, Wenqi Shao, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Zipvl: Efficient large vision-language models with dynamic token sparsification, 2024. URL https://arxiv.org/abs/2410.08584

    2024 arXiv

  9. [17]

    Divprune: Diversity-based visual token pruning for large multimodal models, 2025

    Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang. Divprune: Diversity-based visual token pruning for large multimodal models, 2025. URL https:// arxiv.org/abs/2503.02175

    2025 arXiv

  10. [18]

    Puzzle: Distillation-based nas for inference-optimized llms,

    Akhiad Bercovich, Tomer Ronen, Talor Abramovich, Nir Ailon, Nave Assaf, Mohammad Dabbah, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Netanel Haber, Ehud Karpas, Roi Koren, Itay Levy, Pavlo Molchanov, Shahar Mor, Zach Moshe, Najeeb Nabwani, Omri Puny, Ran Rubin, Ita...

  11. [19]

    What matters in transformers? not all attention is needed, 2024

    Shwai He, Guoheng Sun, Zheyu Shen, and Ang Li. What matters in transformers? not all attention is needed, 2024. URLhttps://arxiv.org/abs/2406.15786

    2024 arXiv

  12. [20]

    URLhttps://arxiv.org/abs/2411.19146

    arXiv

  13. [21]

    Dlp: Dynamic layerwise pruning in large language models, 2025

    Yuli Chen, Bo Cheng, Jiale Han, Yingying Zhang, Yingting Li, and Shuhao Zhang. Dlp: Dynamic layerwise pruning in large language models, 2025. URL https://arxiv.org/abs/ 2505.23807

    2025 arXiv

  14. [22]

    You only prune once: Designing calibration-free model compression with policy learning, 2025

    Ayan Sengupta, Siddhant Chaudhary, and Tanmoy Chakraborty. You only prune once: Designing calibration-free model compression with policy learning, 2025. URL https://arxiv.org/ abs/2501.15296

    2025 arXiv

  15. [23]

    Slimllm: Accurate structured pruning for large language models, 2025

    Jialong Guo, Xinghao Chen, Yehui Tang, and Yunhe Wang. Slimllm: Accurate structured pruning for large language models, 2025. URLhttps://arxiv.org/abs/2505.22689. 11

    2025 arXiv

  16. [24]

    Shortgpt: Layers in large language models are more redundant than you expect, 2024

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redundant than you expect, 2024. URLhttps://arxiv.org/abs/2403.03853

    2024 arXiv

  17. [25]

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models, 2024

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models, 2024. URL https://arxiv.org/abs/ 2403.15388

    2024

  18. [26]

    Visionthink: Smart and efficient vision language model via reinforcement learning, 2025

    Senqiao Yang, Junyi Li, Xin Lai, Bei Yu, Hengshuang Zhao, and Jiaya Jia. Visionthink: Smart and efficient vision language model via reinforcement learning, 2025. URL https: //arxiv.org/abs/2507.13348

    2025 arXiv

  19. [27]

    Multi- stage vision token dropping: Towards efficient multimodal large language model, 2024

    Ting Liu, Liangtao Shi, Richang Hong, Yue Hu, Quanjun Yin, and Linfeng Zhang. Multi- stage vision token dropping: Towards efficient multimodal large language model, 2024. URL https://arxiv.org/abs/2411.10803

    2024 arXiv

  20. [28]

    Vscan: Rethinking visual token reduction for efficient large vision-language models, 2025

    Ce Zhang, Kaixin Ma, Tianqing Fang, Wenhao Yu, Hongming Zhang, Zhisong Zhang, Yaqi Xie, Katia Sycara, Haitao Mi, and Dong Yu. Vscan: Rethinking visual token reduction for efficient large vision-language models, 2025. URLhttps://arxiv.org/abs/2505.22654

    2025

  21. [29]

    Boosting multimodal large language models with visual tokens withdrawal for rapid inference, 2025

    Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. Boosting multimodal large language models with visual tokens withdrawal for rapid inference, 2025. URL https://arxiv.org/ abs/2405.05803

    2025 arXiv

  22. [30]

    Fit and prune: Fast and training-free visual token pruning for multi-modal large language models, 2024

    Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. Fit and prune: Fast and training-free visual token pruning for multi-modal large language models, 2024. URL https://arxiv.org/abs/ 2409.10197

    2024 arXiv

  23. [31]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction, 2025

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction, 2025. URLhttps://arxiv. org/abs/2410.17247

    2025 arXiv

  24. [32]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv. org/abs/1706.03762

    2023 arXiv

  25. [33]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering, 2019. URL https://arxiv.org/abs/ 1902.09506

    2019 arXiv

  26. [34]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https: //llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  27. [35]

    Mme: A comprehensive evaluation benchmark for multimodal large language models, 2025

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2025. URL https://arxiv.org/...

    2025 arXiv

  28. [36]

    Mmbench: Is your multi-modal model an all-around player?, 2024

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player?, 2024. URLhttps://arxiv.org/abs/2307.06281

    2024 arXiv

  29. [37]

    Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022. URLhttps://arxiv.org/abs/2209.09513

    2022 arXiv

  30. [38]

    Evaluating object hallucination in large vision-language models, 2023

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models, 2023. URL https://arxiv.org/abs/ 2305.10355

    2023 arXiv

  31. [39]

    Towards vqa models that can read, 2019

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read, 2019. URLhttps://arxiv. org/abs/1904.08920

    2019 arXiv

  32. [40]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering, 2017

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering, 2017. URLhttps://arxiv.org/abs/1612.00837. 12

    2017 arXiv

  33. [41]

    Fluctuation-based adaptive structured pruning for large language models, 2023

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models, 2023. URL https://arxiv.org/abs/2312. 11983

    2023

  34. [42]

    Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P

    Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P. Bigham. Vizwiz grand challenge: Answering visual questions from blind people,

  35. [43]

    Lmms-eval: Accelerating the development of large multimodal models, March 2024

    Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du, Yuhao Dong, Haotian Liu, Yuan- han Zhang, Ge Zhang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Accelerating the development of large multimodal models, March 2024. URL https://github.com/EvolvingLMMs-Lab/ lmms-eval

    2024

  36. [44]

    Mobilevlm v2: Faster and stronger baseline for vision language model, 2024

    Xiangxiang Chu, Limeng Qiao, Xinyu Zhang, Shuang Xu, Fei Wei, Yang Yang, Xiaofei Sun, Yiming Hu, Xinyang Lin, Bo Zhang, and Chunhua Shen. Mobilevlm v2: Faster and stronger baseline for vision language model, 2024. URLhttps://arxiv.org/abs/2402.03766

    2024 arXiv

  37. [45]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models, 2024. URLhttps://arxiv.org/abs/2306.11695

    2024 arXiv

  38. [48]

    Experiment,

    P. Langley. Crafting papers on machine learning. In Pat Langley, editor,Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pages 1207–1216, Stanford, CA, 2000. Morgan Kaufmann. 13 6 Appendix GQA MMB MMB_CN MME POPE SQA_IMG VQA-v2 T extVQA VizWiz ...

  39. [2018]

    URLhttps://arxiv.org/abs/1802.08218

    arXiv

  40. [2025]

    URLhttps://arxiv.org/abs/2502.13923

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.