Pith. sign in

REVIEW 4 cited by

LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.02330 v4 pith:Y34AZWCY submitted 2024-01-04 cs.CV cs.CL

Yichen Zhu , Minjie Zhu , Ning Liu , Zhicai Ou , Xiaofeng Mou , Jian Tang This is my paper · ORCID
classification cs.CVcs.CL
keywords multi-modallanguagellava-phimodelmodelsassistantavailabledialogues
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

In this paper, we introduce LLaVA-$\phi$ (LLaVA-Phi), an efficient multi-modal assistant that harnesses the power of the recently advanced small language model, Phi-2, to facilitate multi-modal dialogues. LLaVA-Phi marks a notable advancement in the realm of compact multi-modal models. It demonstrates that even smaller language models, with as few as 2.7B parameters, can effectively engage in intricate dialogues that integrate both textual and visual elements, provided they are trained with high-quality corpora. Our model delivers commendable performance on publicly available benchmarks that encompass visual comprehension, reasoning, and knowledge-based perception. Beyond its remarkable performance in multi-modal dialogue tasks, our model opens new avenues for applications in time-sensitive environments and systems that require real-time interaction, such as embodied agents. It highlights the potential of smaller language models to achieve sophisticated levels of understanding and interaction, while maintaining greater resource efficiency.The project is available at {https://github.com/zhuyiche/llava-phi}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

    cs.AI 2026-05 conditional novelty 6.0 of 10

    A 24-dataset benchmark for inducing schema graphs from raw text, plus an auditable LLM-based pipeline that reports the highest scores on the benchmark's four schema-similarity metrics.

  2. Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution

    cs.AI 2025-09 conditional novelty 6.0 of 10

    Auras, a perception-generation disaggregation framework with a public context buffer and asynchronous pipeline executor, raises embodied-agent throughput by 2.54x on average without losing accuracy (102.7%).

  3. FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A 1.5B unified multimodal model trained with discrete flow matching and metric-induced probability paths matches autoregressive baselines of similar size on generation and understanding benchmarks.

  4. UniCMs: A Unified Consistency Model For Efficient Multimodal Generation and Understanding

    cs.CV 2025-02 conditional novelty 6.0 of 10

    UniCMs applies consistency distillation to a unified multimodal transformer, treating image mask-diffusion steps and text parallel-decoding steps as one shared denoising trajectory, enabling 2 to 8 step generation and...

Pith tools