Pith. sign in

REVIEW 13 cited by

Woodpecker: Hallucination Correction for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.16045 v2 pith:ZJ5DO3WS submitted 2023-10-24 cs.CV cs.AIcs.CLcs.LG

Shukang Yin , Chaoyou Fu , Sirui Zhao , Tong Xu , Hao Wang , Dianbo Sui , Yunhang Shen , Ke Li
show 2 more authors
Xing Sun Enhong Chen
This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.LG
keywords woodpeckerhallucinationmodelscorrectiondifferentfivegeneratedhallucinations
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content. In order to mitigate hallucinations, existing studies mainly resort to an instruction-tuning manner that requires retraining the models with specific data. In this paper, we pave a different way, introducing a training-free method named Woodpecker. Like a woodpecker heals trees, it picks out and corrects hallucinations from the generated text. Concretely, Woodpecker consists of five stages: key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction. Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages. We evaluate Woodpecker both quantitatively and qualitatively and show the huge potential of this new paradigm. On the POPE benchmark, our method obtains a 30.66%/24.33% improvement in accuracy over the baseline MiniGPT-4/mPLUG-Owl. The source code is released at https://github.com/BradyFU/Woodpecker.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 22 citations worldwide. Full citation record

  1. Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers

    cs.CV 2026-07 accept novelty 6.0 of 10

    A single-prefill, white-box risk detector that estimates signed visual evidence from internal attention/value readouts and fuses it with confidence improves error ranking by 0.134 mean AP across 12 model–benchmark pairs.

  2. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  3. Controlling Multimodal LLMs via Reward-guided Decoding

    cs.CV 2025-08 conditional novelty 6.0 of 10

    MRGD guides MLLM decoding with a learned hallucination reward and a detector-based recall reward, allowing users to trade off object precision, recall, and test-time compute while reducing object hallucinations on CHA...

  4. ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A single-layer, single-query intervention that amplifies attention heads with high text-to-visual entropy reduces hallucination in LVLMs at about 1.07x the inference time of regular decoding.

  5. Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Griffon-R generates its own grounding hints and rationale before answering, achieving state-of-the-art visual reasoning on VSR and CLEVR while improving MMBench, ScienceQA, and TextVQA.

  6. Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    RVCD uses YOLO detections and retrieved single-concept AI images to adjust LVLM logits at decode time, cutting CHAIR hallucination rates by roughly half versus prior contrastive decoding baselines.

  7. ChartLens: Fine-grained Visual Attribution in Charts

    cs.CL 2025-05 conditional novelty 6.0 of 10

    ChartLens uses segmentation and set-of-marks prompting to attribute chart-based answers to specific visual elements, and the authors release a new benchmark for evaluating such attribution.

  8. Seeing It or Not? Interpretable Vision-aware Latent Steering to Mitigate Object Hallucinations

    cs.CV 2025-05 conditional novelty 6.0 of 10

    VaLSe uses attention-based visual contribution maps to steer an LVLM's latent features toward visually grounded content, reducing object hallucinations on several benchmarks while exposing flaws in CHAIR-style evaluation.

  9. Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Ensemble Decoding reduces object hallucination in large vision-language models by ensembling logits from attention-weighted image sub-images.

  10. Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models

    cs.CV 2025-02 conditional novelty 6.0 of 10

    DeGF reduces hallucinations in vision-language models by generating an image from the model's own response and using the divergence between predictions on original and generated images to switch between complementary ...

  11. The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering

    cs.CV 2025-02 conditional novelty 6.0 of 10

    VISTA reduces hallucination in vision-language models by adding a per-image visual steering vector to hidden states and blending in early-layer logits, cutting CHAIR object hallucination by about 40%.

  12. Energy-Guided Decoding for Object Hallucination Mitigation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    An energy-guided, training-free decoding rule that chooses the layer with minimal energy reduces object hallucination and yes-bias on several benchmarks.

  13. Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding

    cs.CV 2025-05 conditional novelty 5.0 of 10

    FarSight adds upper-triangular negative biases to the causal mask to absorb outlier-token attention, reducing hallucinations in MLLMs without training.

Pith tools