Pith. sign in

REVIEW 18 cited by

HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.00425 v2 pith:C4CRBQ34 submitted 2024-03-01 cs.CV cs.AIcs.LG

Zhaorun Chen , Zhuokai Zhao , Hongyin Luo , Huaxiu Yao , Bo Li , Jiawei Zhou This is my paper
classification cs.CVcs.AIcs.LG
keywords halclvlmsalgorithmcontextsdecodingobjectvision-languagewhile
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

While large vision-language models (LVLMs) have demonstrated impressive capabilities in interpreting multi-modal contexts, they invariably suffer from object hallucinations (OH). We introduce HALC, a novel decoding algorithm designed to mitigate OH in LVLMs. HALC leverages distinct fine-grained optimal visual information in vision-language tasks and operates on both local and global contexts simultaneously. Specifically, HALC integrates a robust auto-focal grounding mechanism (locally) to correct hallucinated tokens on the fly, and a specialized beam search algorithm (globally) to significantly reduce OH while preserving text generation quality. Additionally, HALC can be integrated into any LVLMs as a plug-and-play module without extra training. Extensive experimental studies demonstrate the effectiveness of HALC in reducing OH, outperforming state-of-the-arts across four benchmarks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

    cs.CV 2026-05 unverdicted novelty 6.5 of 10

    Object hallucinations in MLLMs track multi-head spatial inconsistency and temporal visual-attention fade; AFIP corrects both via cross-head enrichment and gated historical reinjection, reducing CHAIR/POPE rates training-free.

  2. Visual Token Compression Enhances Robustness of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Pruning visual tokens farthest from the text feature space at selected 'robust' layers improves MLLM jailbreak defense (average +13.29% RAR) and slightly reduces hallucination.

  3. Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs

    cs.CV 2025-08 conditional novelty 6.0 of 10

    Omission and fabrication hallucinations in MLLMs are claimed to have distinct causes, and the new VPFC method reduces omissions without adding fabrications.

  4. Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens

    cs.CV 2025-08 conditional novelty 6.0 of 10

    Modality bias, an imbalanced attention to text or image during hallucinated outputs, is shown to be mitigated by a training-free attention intervention plus contrastive decoding.

  5. INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling

    cs.CV 2025-07 conditional novelty 6.0 of 10

    INTER is a training-free logit-correction method that adds Harsanyi interaction scores to selected keyword tokens, lowering hallucination on six LVLM benchmarks.

  6. Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A dual-level attention intervention that boosts salient visual-token attention and suppresses text/system attention during decoding reduces hallucination rates in LLaVA, MiniGPT-4, and mPLUG-Owl2 on POPE and CHAIR.

  7. Mitigating Object Hallucination via Robust Local Perception Search

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A training-free decoding method that uses an MLLM's own local object descriptions as a reward prior, combined with CLIP similarity, to cut object hallucination, especially under adversarial image noise.

  8. CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention

    cs.CL 2025-06 conditional novelty 6.0 of 10

    An inference-time attention-shift intervention aligns non-English queries' cross-modal attention with English, cutting multilingual object hallucination in LVLMs on POPE and MME.

  9. Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    RVCD uses YOLO detections and retrieved single-concept AI images to adjust LVLM logits at decode time, cutting CHAIR hallucination rates by roughly half versus prior contrastive decoding baselines.

  10. From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization

    cs.AI 2025-05 conditional novelty 6.0 of 10

    EduVisAgent, a five-agent framework, outperforms all baseline AI models at generating pedagogically effective interactive visualizations for STEM problems, according to the new EduVisBench benchmark and its GPT-4o-bas...

  11. Disentangling Semantic Attention from Structural Bias in the Attention Manifold

    cs.CV 2026-07 conditional novelty 5.0 of 10

    SPAR removes a query-averaged structural bias from text-to-image attention and redistributes the reclaimed probability mass, reducing reported object and induced hallucinations in LLaVA models.

  12. OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination

    cs.AI 2025-08 conditional novelty 5.0 of 10

    OmniDPO extends direct preference optimization with audio-video alignment and modality-degradation preference pairs to reduce omni-modal hallucination.

  13. CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    CAI reduces object hallucination in LVLMs by injecting caption-query attention patterns into selected attention heads at inference time.

  14. PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MMGrounded-PostAlign trains MLLMs to produce a grounded object token or a rejection token plus selective rationales, improving hallucination and VQA benchmarks.

  15. MCA-LLaVA: Manhattan Causal Attention for Reducing Hallucination in Large Vision-Language Models

    cs.CV 2025-07 conditional novelty 4.0 of 10

    MCA-LLaVA reindexes image tokens by sums of mirrored 2D coordinates so instruction tokens attend across the whole image, reducing hallucination on POPE, CHAIR, and MME.

  16. Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

    cs.AI 2025-06 reject novelty 4.0 of 10

    SHE lowers behavioral hallucination scores by about 10 percent by detecting low visual-textual similarity and projecting out the hallucinated direction in embedding space.

  17. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

  18. ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning

    cs.CL 2025-05 reject novelty 2.0 of 10

    ASPO's adaptive sentence-level loss, by the paper's own definitions, reduces exactly to the standard DPO loss, leaving no difference in the optimization objective.

Pith tools