Pith. sign in

REVIEW 3 major objections 5 minor 40 references

Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free calibration step lets VLMs drop 94.4% of visual tokens while retaining 96.4% of full-token accuracy, by treating token reduction as calibration rather than selection.

desk verdict Well-executed token-reduction paper with a genuinely new calibration idea; consistent gains, but missing code and error bars, and PRCA's question-blindness is a real but non-fatal concern. read the letter →

arxiv 2607.27700 v1 pith:IW3VRSM3 submitted 2026-07-30 cs.CV

Jiasheng Li , Zhong Ji , Yan Zhang , Huihui Li This is my paper · ORCID
classification cs.CV
keywords visualtokenreductionsemanticdriftvision-languagemodelscalibrationperturbationrobustnessconfidencegatingtraining-freeinferenceacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that standard visual token pruning and compression degrade VLM answers through 'semantic drift': critical cues are lost, diluted, or misassigned. To counter this, it introduces CaRe, a training-free method that first selects a small set of robust 'calibration anchors' — tokens whose influence on the vision projector is stable under random perturbations — and then injects reliable information from discarded tokens into those anchors before the LLM reasons. The central claim is that calibration, not selection, is the right frame for aggressive token reduction: on nine benchmarks across three VLM families, CaRe outperforms existing pruning and merging methods, retaining 96.4% of full-token performance while discarding 94.4% of tokens and delivering up to 2.30x end-to-end speedup.

What carries the argument

The central mechanism is the two-stage calibration pipeline. Stage I, Perturbation-Robust Calibration Anchoring (PRCA), measures each token's 'model-side influence' by perturbing it along m random symmetric directions before the projector and computing a normalized finite-difference response, then penalizes directional instability (score ψ_i = μ_i − λσ_i) and enforces diversity through iterative marginal-cosine-distance selection. Stage II, Confidence-Gated Token Calibration (CGTC), computes semantic-spatial affinity between residual and anchor tokens, keeps only candidates that pass affinity and concentration thresholds, and fuses them into anchors via a normalized weighted average with nor

What would settle it

Take a dataset where each question requires reading a small, non-salient detail (e.g., a street number or a short text) and compare the anchor set chosen by PRCA with the ground-truth region for that detail. If, for a nontrivial fraction of examples, the detail is not covered by any anchor or any admitted calibration signal, and the model still answers correctly, then the success is not due to recovering that cue; if it fails, the assumption that stable model-side influence identifies question-critical tokens is falsified. A simpler variant: replace PRCA anchors with randomly chosen anchors (s

Watch

Extended reading notes

Core claim

The paper's core claim is that token reduction is a calibration problem, not a selection problem. Specifically, the authors show that a compact visual representation can be built by (1) choosing K tokens that have high and directionally consistent influence on the frozen multimodal projector — using a symmetric finite-difference response with a directional-risk penalty and a diversity constraint — and (2) gating the remaining unselected tokens by semantic-spatial affinity and assignment concentration, then softly adding the qualified signals into their matched anchors while preserving the original anchor norm. The resulting K calibrated tokens are claimed to preserve the semantics of the ful

Load-bearing premise

The method hinges on the assumption that a token's local response to random perturbations of the frozen projector measures how semantically critical it is for the question being asked — if the projector is flat along the directions that matter for a particular question, the chosen anchors can miss the very cues the answer depends on, because anchor selection is not question-conditioned.

Editorial extensions

If this is right

  • At 192 retained tokens on LLaVA-1.5-7B (66.7% pruning), CaRe reports 99.9% of full-token normalized performance; at 160 tokens on LLaVA-NeXT (94.4% pruning) it reports 96.4%.
  • Across three VLM families and nine benchmarks, CaRe outperforms both selection baselines (FastV, SparseVLM, DivPrune, ZOO-Prune, V2Drop) and compression baselines (VisionZip, FiCoCO-V) at matched token budgets.
  • The accuracy gain is complementary: adding either PRCA or CGTC alone improves the selection baseline, and combining them gives the largest gains, with confidence gating contributing up to 0.4 points over ungated calibration.
  • Speedup comes with no training: because selection and calibration operate on the frozen projector, CaRe adds negligible compute and achieves up to 2.30x end-to-end latency reduction over the unpruned model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test of the framing is to compare CaRe against a question-conditioned oracle that selects anchors based on the actual question tokens; if CaRe approaches that oracle, the perturbation-based influence measure is a valid proxy for task-relevant semantics.
  • The same calibration logic could be applied to other modalities or to the vision encoder itself — the authors note calibration cannot recover information already lost at encoding, so pushing calibration into earlier stages is a plausible next step.
  • Because the confidence gate rejects ambiguous residual tokens, the method may also reduce hallucination in low-information settings; the reported POPE gains are consistent with that, but it is not directly shown.
  • The representation-level 'semantic retention' metric (CSR) is auxiliary, not causal; a causal version would need to ablate which specific residual tokens are responsible for the answer change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CaRe, a training-free visual-token reduction method for VLMs. CaRe first selects K 'calibration anchors' using a perturbation-robust influence score computed from symmetric perturbations through the frozen multimodal projector (PRCA, Eqs. 1–3), then injects selected residual-token information into those anchors under a semantic-spatial confidence gate (CGTC, Eqs. 4–9). The method is evaluated on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B over nine benchmarks, with reported gains over three baselines at multiple token budgets; a headline result is retaining 96.4% of full-token performance while pruning 94.4% of visual tokens. The paper also defines a calibrated semantic retention (CSR) diagnostic and includes module ablations, hyperparameter sensitivity studies, efficiency comparisons, and qualitative failure cases.

Significance. If the empirical claims withstand scrutiny, this is a useful contribution: CaRe is training-free, backbone-agnostic, and consistently improves over six recent token-reduction baselines on three model families with fixed token budgets. The evaluation breadth, the explicit module ablations, the efficiency measurements, and the honest failure-case discussion are strengths. However, the paper's stronger conceptual claim—that CaRe preserves semantic fidelity by calibrating before reasoning—is not directly established. The perturbation score in Eq. (1) is not question-conditioned, the CSR diagnostic is explicitly non-causal, and no uncertainty quantification or held-out hyperparameter selection is provided. These gaps matter because the reported margins over the best baseline are small, and the mechanism is claimed to be the source of the gains.

major comments (3)
  1. [Experiments / Implementation details; Tables 1–3, Figs. 4, 6, 7] All main results appear to come from a single run (random seed 42, Supp. Sec. C), and no error bars, confidence intervals, or repeated-seed statistics are reported. The gains over the strongest baseline are often small: e.g., in Table 1 the normalized averages are 99.9% vs 98.0%, 98.3% vs 97.2%, and 96.5% vs 94.8% at the three budgets, and on individual benchmarks several entries differ by less than 1 point. In addition, the fixed hyperparameters (h, m, lambda, eta, theta_s, theta_c, alpha) are selected using ablations on POPE (Figs. 4 and 6) and on a six-benchmark average (Fig. 7, Tables S6–S8) from the same set of benchmarks that appears in the final comparison tables. This makes it difficult to distinguish real gains from tuning noise. I request repeated-seed results with error bars or significance tests, and either a held-out validation split for hyperparameter choice or a sensitivit
  2. [Sec. 3.1, Eqs. (1)–(3); Sec. 3.2, Eq. (6)] The core anchoring score is not question-conditioned. Eq. (1) measures the response of the frozen projector M to symmetric Gaussian perturbations of a token; for an affine or locally linear M, M(v+hu)-M(v-hu) is essentially independent of v, so the score reflects the local Jacobian norm rather than the token's semantic contribution to the answer. Anchor selection in Eq. (3) uses only the influence score and cosine diversity, with no reference to the question. At 94.4% pruning (160/2880 tokens in Table 3), a question-critical token can be absent from the anchor set, and if its affinity to the selected anchors is below theta_c in Eq. (6), it is suppressed and permanently lost. The paper's own Limitations paragraph and Fig. S3 admit that weakly encoded or OCR-sensitive cues cannot be reconstructed, and VQAT/TextVQA is consistently among the larger degradations in Tables 1, 3, and S4—exactly
  3. [Table 4 and Supp. Sec. D, Table S5] The module ablation does not isolate the specific signal in Eq. (1). When PRCA is removed, the paper adopts ZOO-Prune's sensitivity selector as the baseline; when PRCA is added, the comparison is against ZOO-Prune. This shows the full system is better than that particular selector, but it does not show that the perturbation-robust influence score is necessary. A diversity-only selector (e.g., maximal-spread random anchors with the same number of tokens) combined with the same CGTC would be a stronger ablation, as would a score-free compression baseline. Without such controls, the empirical results are also consistent with the weaker explanation that any reasonable anchor set plus residual-injection calibration improves over pure pruning.
minor comments (5)
  1. [Figure order] Figure 5 is discussed after Figures 6–8; renumber to match citation order.
  2. [Table 4] The '-' baseline row is ZOO-Prune, but this is stated only in the text. Mark the row explicitly as 'ZOO-Prune' so the table is self-contained.
  3. [Eq. (21)] The normalized average is a ratio of absolute scores, and for MME the score is a sum of Perception+Cognition. State clearly that the average is a normalized retention ratio, not an absolute accuracy, and consider reporting absolute MME values separately in the average or weighting them differently.
  4. [Supp. Sec. B, Eq. (15)] The symbols d_v and H are used in Eq. (15) before being defined; move the definitions before the equation.
  5. [Abstract / Efficiency] The 'up to 2.30x end-to-end speedup' is not tied to a specific benchmark or decoding condition. Specify where this number comes from (which model, pruning ratio, and measurement protocol) in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CaRe's headline results are empirical benchmark outcomes, not derivations that reduce to their own fitted inputs.

full rationale

CaRe is an empirical, training-free token-reduction method. Its claimed outcome—retaining 96.4% of full-token performance at 94.4% pruning—is a measured benchmark result, not a quantity derived from the method's own parameters. The core equations (1)–(9) define anchor selection and calibration entirely from visual tokens, a frozen projector, and user-set hyperparameters; none of these equations contains the downstream benchmark answers, the full-token accuracy values, or the competing methods' outputs. The perturbation response in Eq. (1) is an adopted saliency assumption, but assumptions are not circularity. The CSR diagnostic in Eqs. (10)–(13) is explicitly declared auxiliary ('CSR is used only as an auxiliary representation-level diagnostic rather than a causal measure of answer preservation') and is not used to set hyperparameters or as the paper's headline evidence. The hyperparameters h, m, λ, η, θ_s, θ_c, and α are selected via ablations, partly on POPE with LLaVA-1.5-7B, and the same model/benchmark subsequently appears in the final tables; this is a potential test-set-overfitting or reporting concern, but not definitional circularity under the stated rules. The Limitations paragraph and Figure S3 openly acknowledge that weakly encoded or OCR-sensitive evidence cannot be reconstructed, which is a genuine empirical limitation rather than a hidden circular step. Self-citations in the related work and the use of ZOO-Prune as a comparison/ablation baseline are not load-bearing justifications for CaRe's central mechanism; the perturbation principle is also attributed to external prior work (Sogi, Shibata, and Terao 2024). No step reduces, by the paper's own equations or by self-citation, to the quantity it claims to predict.

Assumptions & free parameters 11 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a set of hand-set hyperparameters, a perturbation-based proxy for semantic importance, and a self-defined semantic-retention diagnostic. No new physical entities are introduced. The method is empirically tuned rather than derived, so the main burden is parameter fitting and proxy validity rather than invented ontology.

free parameters (11)
  • h = 5e-4
    Perturbation step size in Eq. 1; chosen from POPE ablation in Fig. 4a.
  • m = 64
    Number of perturbation directions in Eq. 1; chosen as accuracy/efficiency trade-off in Fig. 4b.
  • lambda = 0.5
    Directional-risk penalty in psi_i = mu_i - lambda sigma_i; selected via ablation in Table S6.
  • eta = 0.45
    Spatial affinity weight in Eq. 4; selected via ablation in Table S7.
  • theta_s = 0.4
    Affinity threshold for best-anchor matching confidence; tuned in Fig. 6.
  • theta_c = 0.9
    Confidence threshold in Eq. 6; tuned in Fig. 6.
  • alpha = 0.15
    Calibration strength in anchor update v_j' = v_j + alpha g_j; selected via Table S8.
  • sigma_p = 0.2
    Spatial decay in Eq. 4; set without a dedicated ablation.
  • tau_g = 0.07
    Temperature for assignment distribution pi_r; set without a dedicated ablation.
  • tau_c = 0.1
    Temperature for sigmoid confidence c_sim; set without a dedicated ablation.
  • tau_s = 0.07
    Temperature for soft assignment in Eq. 7; set without a dedicated ablation.
assumptions (5)
  • domain assumption The perturbation response of the frozen projector M is a faithful proxy for a token's model-side influence and semantic value as an anchor.
    Introduced in Eq. 1 and used throughout PRCA; no independent evidence is given that local sensitivity correlates with question-relevant semantics.
  • domain assumption Cosine similarity in projected visual feature space plus 2D patch distance captures semantic-spatial affinity well enough for reliable token-to-anchor assignment.
    Appears in Eq. 4; the paper does not test alternative affinity functions or show that this metric preserves task-relevant groupings.
  • domain assumption Anchors selected without conditioning on the question, then calibrated with residual tokens, preserve task-relevant semantics under aggressive pruning.
    PRCA is question-independent; the paper relies on diversity and calibration to recover cues needed for specific questions.
  • domain assumption m=64 random Gaussian perturbation directions with median/MAD normalization give stable anchor scores across images and backbones.
    The stability of the perturbation estimator is only tested indirectly through final benchmark accuracy, not through statistical analysis of the scores.
  • standard math The central difference quotient in Eq. 1 approximates a directional derivative of the projector.
    Standard finite-difference estimation; unproblematic but requires h small enough relative to the projector's local curvature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs." pith.science (2026). https://pith.science/paper/IW3VRSM3

@misc{pith2026260727700,
  author       = {Pith},
  title        = {Pith review of: Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IW3VRSM3}},
  note         = {Machine review of arXiv:2607.27700}
}
read the original abstract

Large Vision-Language Models (VLMs) suffer from prohibitive inference overhead due to long sequences of visual tokens. However, existing visual token reduction methods mainly improve efficiency by pruning or compressing redundant tokens without examining whether the resulting representation remains semantically consistent with the original representation. Mapping the original N-token visual sequence to K tokens may discard, dilute, or misassign critical visual cues, triggering severe semantic drift that deviates the VLM's understanding. In this paper, we first introduce the principle of 'Calibrate Before Reason' to visual token reduction and propose CaRe, a training-free robust framework that calibrates compact visual representations before reasoning to preserve semantic fidelity in VLMs. CaRe consists of two mutually complementary modules: 1) Perturbation-Robust Calibration Anchoring, which identifies calibration anchors with stable model-side influence under multi-directional perturbations; 2) Confidence-Gated Token Calibration, which extracts reliable calibration signals from unselected tokens and injects them into anchors. Extensive evaluations across diverse VLM architectures and benchmarks verify that CaRe outperforms state-of-the-art token reduction baselines. While pruning 94.4% of visual tokens, our method retains 96.4% of the original full-token performance, delivering up to 2.30 times faster end-to-end inference speed relative to unpruned vanilla models.

Figures

Figures reproduced from arXiv: 2607.27700 by the authors.

Figure 1
Figure 1. Illustration and comparison of semantic drift in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. From Visual Token Reduction to Visual Token Calibration. Conventional token reduction methods rely on (a) token [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the proposed CaRe framework. (a) The overall architecture illustrates the complete token calibration [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Ablation with hyperparameters “h” and “m”. Comparison on Diverse Tasks Results on LLaVA-1.5-7B. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative visualizations. (a) Baseline; (b) sensitivity-based token selection; (c) similarity-based token selection; (d) [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: Inference efficiency comparison between CaRe and [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 7
Figure 7. Figure 7: Ablation study of λ, η and α on the normalized average over six benchmarks with LLaVA-1.5-7B. Inference Efficiency. To evaluate efficiency, [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 5 linked inside Pith

  1. [1]

    R.; Singh, G.; Akbari, M.; and Zhang, Y

    Alvar, S. R.; Singh, G.; Akbari, M.; and Zhang, Y. 2025. DivPrune: Diversity-based visual token pruning for large multimodal models. In Proceedings of the Computer Vision and Pattern Recognition Conference, 9392--9401

    2025

  2. [2]

    Arif, K. H. I.; Yoon, J.; Nikolopoulos, D. S.; Vandierendonck, H.; John, D.; and Ji, B. 2025. HiRED: Attention-guided token dropping for efficient inference of high-resolution vision-language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 1773--1781

    2025

  3. [3]

    Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  4. [4]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    arXiv 2025

  5. [5]

    Cao, J.; Ye, P.; Li, S.; Yu, C.; Tang, Y.; Lu, J.; and Chen, T. 2024. MADTP: Multimodal alignment-guided dynamic token pruning for accelerating vision-language transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15710--15719

    2024

  6. [6]

    Chen, J.; Liu, X.; Wen, Z.; Wang, Y.; Huang, S.; and Chen, H. 2026. Variation-aware vision token dropping for faster large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3489--3499

    2026

  7. [7]

    Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B. 2024 a . An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, 19--35. Springer

    2024

  8. [8]

    Chen, Z.; Wu, J.; Wang, W.; Su, W.; Chen, G.; Xing, S.; Zhong, M.; Zhang, Q.; Zhu, X.; Lu, L.; et al. 2024 b . InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24185--24198

    2024

Show all 40 references
  1. [9]

    N.; and Hoi, S

    Dai, W.; Li, J.; Li, D.; Tiong, A.; Zhao, J.; Wang, W.; Li, B.; Fung, P. N.; and Hoi, S. 2023. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36: 49250--49267

    2023

  2. [10]

    Fu, T.; Liu, T.; Han, Q.; Dai, G.; Yan, S.; Yang, H.; Ning, X.; and Wang, Y. 2025. FrameFusion: Combining similarity and importance for video token reduction on large vision language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22654--22663

    2025

  3. [11]

    Gao, T.; Zhao, S.; Fang, S.; Zhu, S.; and Ma, C. 2026. QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3553--3562

    2026

  4. [12]

    Han, Y.; Liu, X.; Zhang, Z.; Ding, P.; Chen, J.; Chen, H.; Wang, D.; Yan, Q.; and Huang, S. 2026. Filter, correlate, compress: Training-free token reduction for MLLM acceleration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 4601--4609

    2026

  5. [13]

    Ji, Z.; Li, Z.; Zhang, Y.; Pang, Y.; and Li, X. 2025. Visual Semantic Contextualization Network for Multi-Query Image Retrieval. IEEE Transactions on Multimedia

    2025

  6. [14]

    Kim, Y.; Zhang, Y.; Liu, H.; Jung, A.; Lee, S.; and Hong, S. 2026. ZOO-Prune: Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 39572--39582

    2026

  7. [15]

    Lee, D.; Hur, J.; Choi, J.; Yu, J.; and Kim, J. 2026. Frequency-Aware Token Reduction for Efficient Vision Transformer. Advances in Neural Information Processing Systems, 38: 34058--34083

    2026

  8. [16]

    Li, W.; Yuan, Y.; Liu, J.; Tang, D.; Wang, S.; Qin, J.; Zhu, J.; and Zhang, L. 2025. TokenPacker: Efficient visual projector for multimodal LLM. International Journal of Computer Vision, 133(10): 6794--6812

    2025

  9. [17]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296--26306

    2024

  10. [18]

    Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024 b . LLaVA-NeXT: Improved reasoning, OCR, and world knowledge

    2024

  11. [19]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in Neural Information Processing Systems, 36: 34892--34916

    2023

  12. [20]

    J.; and Yan, Y

    Shang, Y.; Cai, M.; Xu, B.; Lee, Y. J.; and Yan, Y. 2025. LLaVA-PruMerge: Adaptive token reduction for efficient large multimodal models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22857--22867

    2025

  13. [21]

    Sogi, N.; Shibata, T.; and Terao, M. 2024. Object-aware query perturbation for cross-modal image-text retrieval. In European Conference on Computer Vision, 447--464. Springer

    2024

  14. [22]

    Tong, B.; Lai, B.; Zhou, Y.; Luo, G.; Shen, Y.; Li, K.; Sun, X.; and Ji, R. 2025. FlashSloth: Lightning multimodal large language models via embedded visual compression. In Proceedings of the Computer Vision and Pattern Recognition Conference, 14570--14581

    2025

  15. [23]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; et al. 2024. Qwen2-VL: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  16. [24]

    Wang, R.; Zhao, D.; Yuan, Z.; Shao, T.; Chen, G.; Kao, D.; Hong, S.; and Min, B.-C. 2026 a . PRIMT: Preference-based reinforcement learning with multimodal feedback and trajectory synthesis from foundation models. Advances in Neural Information Processing Systems, 38: 153095--153135

    2026

  17. [25]

    Wang, Z.; Li, M.; Yin, H.; Liu, W.; and Wang, Z. 2026 b . PosPrune: Visual Token Pruning with Positional Bias Correction for Efficient Large Vision-Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 10485--10493

    2026

  18. [26]

    Wu, Z.; Wang, S.; and Yu, X. 2026. MeToM: Metadata-Guided Token Merging for Efficient Video LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10441--10450

    2026

  19. [27]

    Xing, L.; Huang, Q.; Dong, X.; Lu, J.; Zhang, P.; Zang, Y.; Cao, Y.; He, C.; Wang, J.; Wu, F.; et al. 2025. PyramidDrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2025

  20. [28]

    Yang, C.; Sui, Y.; Xiao, J.; Huang, L.; Gong, Y.; Li, C.; Yan, J.; Bai, Y.; Sadayappan, P.; Hu, X.; et al. 2025 a . TopV: Compatible token pruning with inference time optimization for fast and low-memory multimodal vision language model. In Proceedings of the Computer Vision a...

    2025

  21. [29]

    Yang, S.; Chen, Y.; Tian, Z.; Wang, C.; Li, J.; Yu, B.; and Jia, J. 2025 b . VisionZip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19792--19802

    2025

  22. [30]

    Zhang, E.; Yu, F.; Wu, A.; Wen, Z.; Yan, K.; Ding, S.; Qi, B.; and Zhang, L. 2026 a . D ^2 Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 12412--12420

    2026

  23. [31]

    Zhang, J.; Guo, X.; Cai, K.; Lv, Q.; Fan, Y.; Chai, W.; Wang, J.; and Wang, K. 2026 b . Hybrid Token Compression for Vision-Language Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 31889--31899

    2026

  24. [32]

    A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; et al

    Zhang, K.; Li, B.; Zhang, P.; Pu, F.; Cahyono, J. A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; et al. 2025 a . LMMs-Eval: Reality check on the evaluation of large multimodal models. In Findings of the Association for Computational Linguistics: NAACL 2025, 881--916

    2025

  25. [33]

    Zhang, Q.; Cheng, A.; Lu, M.; Zhang, R.; Zhuo, Z.; Cao, J.; Guo, S.; She, Q.; and Zhang, S. 2025 b . [CLS] Attention is All You Need for Training-Free Visual Token Pruning: Make VLM Inference Faster. arXiv:2412.01818

    2025 arXiv

  26. [34]

    Zhang, S.; Fang, Q.; Yang, Y.; and Feng, Y. 2025 c . LLaVA-Mini: Efficient image and video large multimodal models with one vision token. In International Conference on Learning Representations, volume 2025, 53285--53310

    2025

  27. [35]

    A.; Okuno, T.; Nakata, Y.; Keutzer, K.; et al

    Zhang, Y.; Fan, C.-K.; Ma, J.; Zheng, W.; Huang, T.; Cheng, K.; Gudovskiy, D. A.; Okuno, T.; Nakata, Y.; Keutzer, K.; et al. 2025 d . SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference. In International Conference on Machine Learning, 74840--7...

    2025

  28. [36]

    Zhang, Y.; Ji, Z.; Meng, C.; and Pang, Y. 2025 e . iEBAKER: Improved remote sensing image-text retrieval framework via eliminate before align and keyword explicit reasoning. Expert Systems with Applications, 128968

    2025

  29. [37]

    Zhang, Y.; Ji, Z.; Pang, Y.; Han, J.; and Li, X. 2024 a . Modality-experts coordinated adaptation for large multimodal models. Science China Information Sciences, 67(12): 220107

    2024

  30. [38]

    Zhang, Y.; Ji, Z.; Wang, D.; Pang, Y.; and Li, X. 2024 b . USER: Unified semantic enhancement with momentum contrast for image-text retrieval. IEEE Transactions on Image Processing, 33: 595--609

    2024

  31. [39]

    Zhang, Y.; Zhijin, Q.; Feng, X.; Guiguang, D.; and Jungong, H. 2026 c . Cofact GVR : Counterfactual Intervention for Grounded Visual Reasoning. In Forty-third International Conference on Machine Learning

    2026

  32. [40]

    Zhu, J.; Wang, W.; Chen, Z.; Liu, Z.; Ye, S.; Gu, L.; Tian, H.; Duan, Y.; Su, W.; Shao, J.; et al. 2025. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479

    2025 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.