Pith. sign in

REVIEW 17 cited by

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.03003 v1 pith:QJH5QMMT submitted 2024-03-05 cs.CV

Gen Luo , Yiyi Zhou , Yuxin Zhang , Xiawu Zheng , Xiaoshuai Sun , Rongrong Ji This is my paper
classification cs.CV
keywords llava-hrmllmsvisualmixture-of-resolutionadaptationefficientexistinghigh-resolution
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Despite remarkable progress, existing multimodal large language models (MLLMs) are still inferior in granular visual recognition. Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming. Based on this observation, we propose a novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation (MRA). In particular, MRA adopts two visual pathways for images with different resolutions, where high-resolution visual information is embedded into the low-resolution pathway via the novel mixture-of-resolution adapters (MR-Adapters). This design also greatly reduces the input sequence length of MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL) tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g., +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR remain efficient with MRA, e.g., 20 training hours and 3$\times$ inference speed than LLaVA-1.5. Source codes are released at: https://github.com/luogen1996/LLaVA-HR.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HAFI-VLM: A Frequency Perspective for Diagnosing and Enhancing Visual Perception in Vision-Language Models

    cs.CV 2026-08 conditional novelty 7.0 of 10

    Pretrained vision encoders show spectral response rigidity, and HAFI-VLM injects text-conditioned low/mid/high frequency evidence to improve VLM perception on VQA, text-rich understanding, and hallucination robustness.

  2. PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    The work introduces the UAV Reasoning Segmentation task, the DRSeg benchmark dataset, and PixDLM as a baseline dual-path multimodal language model for reasoning-based segmentation in aerial imagery.

  3. Kwai Keye-VL 1.5 Technical Report

    cs.CV 2025-09 conditional novelty 6.0 of 10

    Keye-VL-1.5 combines similarity-based Slow-Fast video token allocation with progressive context extension and iterative RL, reporting leading video-understanding results among 8B-scale multimodal models.

  4. LLaPa: A Vision-Language Model Framework for Counterfactual-Aware Procedural Planning

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A VLM-based planner with task-oriented segmentation reranking and a clause-level condition retriever reports state-of-the-art scores on ActPlan-1K and ALFRED, though the reported ablation numbers are internally inconsistent.

  5. LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LIRA improves referring segmentation and reduces hallucination in multimodal LLMs by fusing semantic and pixel features and interleaving local image regions with text descriptions.

  6. Kwai Keye-VL Technical Report

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Kwai Keye-VL shows that a five-mode chain-of-thought cold-start plus mix-mode reinforcement learning can push an 8B multimodal model to strong short-video and general vision-language performance.

  7. LLaVA-SP: Enhancing Visual Representation with Visual Spatial Tokens for MLLMs

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LLaVA-SP adds six spatial tokens produced by multi-scale cropping or pooling and cross-attention to MLLMs, improving 10/11 benchmarks over LLaVA-1.5 with nearly unchanged latency.

  8. DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World

    cs.CV 2025-06 conditional novelty 6.0 of 10

    DenseWorld-1M provides one million images with detailed object captions, pixel masks, and spatial relations by chaining SAM, APE, RAM++, and VLMs through a three-stage labeling pipeline.

  9. Dense360: Dense Understanding from Omnidirectional Panoramas

    cs.CV 2025-06 reject novelty 6.0 of 10

    Introduces a 160K-panorama auto-annotated dataset, a dense captioning and grounding benchmark, and ERP-RoPE; fine-tuning Qwen2.5VL on the data lifts benchmark scores.

  10. Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 5.5 of 10

    A monolithic multimodal LLM that cuts pre-training data by 58% and first-token latency by up to 69% while matching or beating its predecessor on 15 benchmarks.

  11. Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization

    cs.LG 2025-10 conditional novelty 5.0 of 10

    A hierarchical ARPO+LLaRA framework that jointly sets image resolution, transmit power, and UAV trajectory reduces simulated latency for onboard VLM inference in low-altitude economy networks.

  12. A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A two-stage crop-and-predict framework improves high-resolution MLLM performance by using the model's own coarse localization to focus on a candidate region before final prediction.

  13. Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A new resolution-focused benchmark and an open-source native-resolution training framework show that preserving original image resolution improves VLM performance on fine-grained visual tasks.

  14. Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs

    cs.CV 2025-06 conditional novelty 5.0 of 10

    CDPruner prunes visual tokens in MLLMs by maximizing instruction-conditioned diversity via a determinantal point process, preserving accuracy at high reduction ratios.

  15. Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A 3B VLM trained with GRPO to call a zoom tool improves V*Bench accuracy by 5.7% over its base model but degrades TextVQA and HR-Bench performance.

  16. CyberV: Cybernetics for Test-time Scaling in Video Understanding

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A training-free test-time feedback loop, using attention drift to select key frames, improves video MLLM accuracy, with the largest gains on knowledge-heavy VideoMMMU.

  17. SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature Refinement

    cs.CV 2025-07 conditional novelty 4.0 of 10

    A three-stage coarse-to-fine training recipe for vision backbones produces consistent benchmark gains for lightweight multimodal LLMs.

Pith tools