Pith. sign in

REVIEW 11 cited by

Silkie: Preference Distillation for Large Visual Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.10665 v1 pith:KQA7DYW6 submitted 2023-12-17 cs.CV cs.CL

Lei Li , Zhihui Xie , Mukai Li , Shunian Chen , Peiyi Wang , Liang Chen , Yazheng Yang , Benyou Wang
show 1 more author
Lingpeng Kong
This is my paper · ORCID
classification cs.CVcs.CL
keywords preferencelvlmsmodelssilkievisualbenchmarkcognitiondataset
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper explores preference distillation for large vision language models (LVLMs), improving their ability to generate helpful and faithful responses anchoring the visual context. We first build a vision-language feedback (VLFeedback) dataset utilizing AI annotation. Specifically, responses are generated by models sampled from 12 LVLMs, conditioned on multi-modal instructions sourced from various datasets. We adopt GPT-4V to assess the generated outputs regarding helpfulness, visual faithfulness, and ethical considerations. Furthermore, the preference supervision is distilled into Qwen-VL-Chat through the direct preference optimization (DPO) method. The resulting model Silkie, achieves 6.9% and 9.5% relative improvement on the MME benchmark regarding the perception and cognition capabilities, respectively. Silkie also demonstrates reduced hallucination by setting a new state-of-the-art score of 3.02 on the MMHal-Bench benchmark. Further analysis shows that DPO with our VLFeedback dataset mainly boosts the fine-grained perception and complex cognition abilities of LVLMs, leading to more comprehensive improvements compared to human-annotated preference datasets.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LPOI: Listwise Preference Optimization for Vision Language Models

    cs.CV 2025-05 conditional novelty 7.0 of 10

    LPOI reduces VLM hallucination by training the model to prefer the original image over progressively masked versions of the same image, using a listwise ranking loss built from pairwise preference data.

  2. Improving Large Vision and Language Models by Learning from a Panel of Peers

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A panel of LVLMs that generate, evaluate, and learn from each other's outputs improves average benchmark scores by 9 points across 15 tasks.

  3. Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities

    cs.CR 2025-07 conditional novelty 6.0 of 10

    Vision-language models consistently recognize unsafe content better from text than from images, and a simplified reinforcement learning fine-tune narrows that gap.

  4. GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.

  5. MM-RLHF: The Next Step Forward in Multimodal LLM Alignment

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A human-annotated multimodal preference dataset plus critique-based reward modeling and reward-margin-weighted DPO improves MLLM performance across many benchmarks.

  6. Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs

    cs.CV 2026-02 conditional novelty 5.0 of 10

    Visual token compression (4x fewer tokens) plus a three-stage generative/contrastive/judge-curated training pipeline yields state-of-the-art MLLM-based retrieval accuracy at lower inference cost.

  7. PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MMGrounded-PostAlign trains MLLMs to produce a grounded object token or a rejection token plus selective rationales, improving hallucination and VQA benchmarks.

  8. LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs

    cs.CV 2025-06 conditional novelty 4.0 of 10

    LeanPO improves Video-LLM alignment by using a reference-free average-likelihood reward, self-generated winning/losing pairs, and dynamic label smoothing, yielding gains on six video benchmarks.

  9. DPO Learning with LLMs-Judge Signal for Computer Use Agents

    cs.AI 2025-06 conditional novelty 4.0 of 10

    An LLM-as-Judge pipeline that scores synthetic GUI interaction trajectories and fine-tunes a 2B model with DPO yields a local computer-use agent that beats its base model on 15-step OSWorld tasks.

  10. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

  11. ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning

    cs.CL 2025-05 reject novelty 2.0 of 10

    ASPO's adaptive sentence-level loss, by the paper's own definitions, reduces exactly to the standard DPO loss, leaving no difference in the optimization objective.

Pith tools