REVIEW 12 cited by
Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Vision tokens in multimodal large language models often dominate huge computational overhead due to their excessive length compared to linguistic modality. Abundant recent methods aim to solve this problem with token pruning, which first defines an importance criterion for tokens and then prunes the unimportant vision tokens during inference. However, in this paper, we show that the importance is not an ideal indicator to decide whether a token should be pruned. Surprisingly, it usually results in inferior performance than random token pruning and leading to incompatibility to efficient attention computation operators.Instead, we propose DART (Duplication-Aware Reduction of Tokens), which prunes tokens based on its duplication with other tokens, leading to significant and training-free acceleration. Concretely, DART selects a small subset of pivot tokens and then retains the tokens with low duplication to the pivots, ensuring minimal information loss during token pruning. Experiments demonstrate that DART can prune 88.9% vision tokens while maintaining comparable performance, leading to a 1.99$\times$ and 2.99$\times$ speed-up in total time and prefilling stage, respectively, with good compatibility to efficient attention operators. Our codes are available at https://github.com/ZichenWen1/DART.
Forward citations
Cited by 12 Pith papers
-
PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs
PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.
-
Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
SFPruner performs redundancy-aware visual token pruning in a single non-iterative pass using semantics-guided ridge leverage scores and directional masking, cutting selection latency from 112.4 ms to 2.5 ms at 512 tok...
-
CRISP: Pre-LLM Yet Text-Driven Visual Token Pruning for Efficient LVLM Inference
A training-free, text-driven pre-LLM pruning method that keeps ~99% of LVLM performance while retaining only ~11-22% of visual tokens.
-
Spectral Heat Flow for Conservative Token Condensation in Vision-Language Models
Training-free SpecFlow condenses VLM visual tokens via kNN heat diffusion, adaptive quadtree budgets, and coreset sinks, retaining 95.6% LLaVA-1.5 performance after pruning 88.9% of tokens.
-
ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attention
Frozen visual tokens with sparse cross-attention at selected layers preserve 98% accuracy while reducing vision-side FLOPs to 4% in LLaVA-1.5 models.
-
A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
Visual token pruning causes severe loss in discrete diffusion MLLMs; only from-scratch models on long-answer tasks recover via late denoising, so redundancy is recoverability, not dispensability.
-
Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
Post-training an LLM on economic reasoning problems improves accuracy on economic benchmarks and, without game-specific training, raises its Nash equilibrium frequency and win rates in strategic games.
-
QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design
A co-designed parallel decoder and KV-pruned group prefill pipeline speeds up long video LLM inference by over 3x, from 69.7 to 20.0 seconds on a 30-minute input.
-
SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models
Using the separator token as a position-free query, SepPrune prunes 80% of vision tokens while retaining 96.3% of Qwen2.5-VL-7B's original accuracy.
-
GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models
A training-free token pruning method that combines cosine-similarity saliency with greedy redundancy removal to preserve accuracy at high compression ratios.
-
EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models
EfficientVLA combines LLM layer pruning, task-aware visual token selection, and diffusion-head feature caching to cut CogACT's inference cost to 28.9% of baseline FLOPs with a 0.6% SIMPLER success drop.
-
FlexSelect: Flexible Token Selection for Efficient Long Video Understanding
FlexSelect selects a small fraction of query-relevant visual tokens using attention from an intermediate layer, improving long-video accuracy and inference speed across multiple VideoLLMs.
Discussion (0). Continue with ORCID to comment.