Pith. sign in

REVIEW 5 cited by

Honeybee: Locality-enhanced Projector for Multimodal LLM

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.06742 v2 pith:P7VNWM6B submitted 2023-12-11 cs.CV cs.AIcs.CLcs.LG

Junbum Cha , Wooyoung Kang , Jonghwan Mun , Byungseok Roh This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.LG
keywords projectorvisualhoneybeecrucialdesigneffectivelyefficiencyllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In Multimodal Large Language Models (MLLMs), a visual projector plays a crucial role in bridging pre-trained vision encoders with LLMs, enabling profound visual understanding while harnessing the LLMs' robust capabilities. Despite the importance of the visual projector, it has been relatively less explored. In this study, we first identify two essential projector properties: (i) flexibility in managing the number of visual tokens, crucial for MLLMs' overall efficiency, and (ii) preservation of local context from visual features, vital for spatial understanding. Based on these findings, we propose a novel projector design that is both flexible and locality-enhanced, effectively satisfying the two desirable properties. Additionally, we present comprehensive strategies to effectively utilize multiple and multifaceted instruction datasets. Through extensive experiments, we examine the impact of individual design choices. Finally, our proposed MLLM, Honeybee, remarkably outperforms previous state-of-the-art methods across various benchmarks, including MME, MMBench, SEED-Bench, and LLaVA-Bench, achieving significantly higher efficiency. Code and models are available at https://github.com/kakaobrain/honeybee.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs

    cs.CV 2025-01 conditional novelty 7.0 of 10

    Decoder-only MLLMs tolerate simplified self-attention and FFN processing for visual tokens in about half of their layers, enabling a training-free FLOPs reduction method.

  2. LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    LLaSO releases a 3.8B speech-language model, 25.5M training instances, and an evaluation benchmark, claiming a normalized score of 0.72.

  3. ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A reinforcement-learned frame selection policy, trained with reward margins from a reference video-LLM, improves video QA accuracy of LLaVA-OV and InternVL3 across several benchmarks.

  4. Image Embedding Sampling Method for Diverse Captioning

    cs.CV 2025-02 conditional novelty 5.0 of 10

    A training-free hierarchical embedding sampling method (HBoP) lets a small BLIP model generate captions as diverse as human ones, beating much larger VLMs on diversity metrics.

  5. NanoVLMs: How small can we go and still make coherent Vision Language Models?

    cs.CV 2025-02 reject novelty 5.0 of 10

    NanoVLMs, 5M to 25M parameter vision-language models trained on simplified GPT-4o captions, are judged by GPT-4o as nearly as coherent as the 50x larger Kosmos-2 on a 25-sample test.

Pith tools