REVIEW 7 cited by
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Vision-Language Models (LVLMs) have advanced considerably, intertwining visual recognition and language understanding to generate content that is not only coherent but also contextually attuned. Despite their success, LVLMs still suffer from the issue of object hallucinations, where models generate plausible yet incorrect outputs that include objects that do not exist in the images. To mitigate this issue, we introduce Visual Contrastive Decoding (VCD), a simple and training-free method that contrasts output distributions derived from original and distorted visual inputs. The proposed VCD effectively reduces the over-reliance on statistical bias and unimodal priors, two essential causes of object hallucinations. This adjustment ensures the generated content is closely grounded to visual inputs, resulting in contextually accurate outputs. Our experiments show that VCD, without either additional training or the usage of external tools, significantly mitigates the object hallucination issue across different LVLM families. Beyond mitigating object hallucinations, VCD also excels in general LVLM benchmarks, highlighting its wide-ranging applicability.
Forward citations
Cited by 7 Pith papers
-
Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing
CGC+VTD identifies co-occurring image token clusters as a source of hallucinated objects in discrete-token LVLMs and suppresses clusters' absent-token signals in latent space, cutting hallucination rates across Chamel...
-
Adaptive Perturbation Selection for Contrastive Audio Decoding
A learned per-example router over a 105-perturbation audio library improves contrastive decoding for audio-LLM hallucination, with task-dependent best distortions (e.g., reverse audio for temporal order).
-
Improving Alignment in LVLMs with Debiased Self-Judgment
A contrastive self-judgment score that subtracts a model's image-free confidence from its visual confidence is used to guide decoding, safety moderation, and DPO training, improving hallucination and safety metrics ac...
-
INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling
INTER is a training-free logit-correction method that adds Harsanyi interaction scores to selected keyword tokens, lowering hallucination on six LVLM benchmarks.
-
Mitigating Object Hallucination via Robust Local Perception Search
A training-free decoding method that uses an MLLM's own local object descriptions as a reward prior, combined with CLIP similarity, to cut object hallucination, especially under adversarial image noise.
-
Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models
RVCD uses YOLO detections and retrieved single-concept AI images to adjust LVLM logits at decode time, cutting CHAIR hallucination rates by roughly half versus prior contrastive decoding baselines.
-
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
SHE lowers behavioral hallucination scores by about 10 percent by detecting low visual-textual similarity and projecting out the hallucinated direction in embedding space.
Discussion (0). Continue with ORCID to comment.