Pith. sign in

REVIEW 11 cited by

Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.03577 v2 pith:INN6OVCH submitted 2024-10-04 cs.CV

Xin Zou , Yizhou Wang , Yibo Yan , Yuanhuiyi Lyu , Kening Zheng , Sirui Huang , Junkai Chen , Peijie Jiang
show 3 more authors
Jia Liu Chang Tang Xuming Hu
This is my paper · ORCID
classification cs.CV
keywords visualmemvrmllmsbeforefactualhallucinationhallucinationslanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Despite their impressive capabilities, multimodal large language models (MLLMs) are prone to hallucinations, i.e., the generated content that is nonsensical or unfaithful to input sources. Unlike in LLMs, hallucinations in MLLMs often stem from the sensitivity of text decoder to visual tokens, leading to a phenomenon akin to "amnesia" about visual information. To address this issue, we propose MemVR, a novel decoding paradigm inspired by common cognition: when the memory of an image seen the moment before is forgotten, people will look at it again for factual answers. Following this principle, we treat visual tokens as supplementary evidence, re-injecting them into the MLLM through Feed Forward Network (FFN) as "key-value memory" at the middle trigger layer. This "look-twice" mechanism occurs when the model exhibits high uncertainty during inference, effectively enhancing factual alignment. Comprehensive experimental evaluations demonstrate that MemVR significantly mitigates hallucination across various MLLMs and excels in general benchmarks without incurring additional time overhead. The implementation is available from https://github.com/1zhou-Wang/MemVR

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. M$^3$Prune: Hierarchical Collaborative Pruning for Efficient Multi-Modal Multi-Agent Retrieval-Augmented Generation

    cs.MM 2026-08 conditional novelty 6.0 of 10

    M3Prune learns hierarchical communication graphs across text and image agents, prunes redundant links, and reports higher accuracy at lower token cost on three multimodal QA benchmarks.

  2. Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models

    cs.AI 2026-08 reject novelty 6.0 of 10

    The paper defines attribute-level MLLM unlearning and proposes CLRP, but the method's headline forgetting gains on cloze are partly produced by test-time logit subtraction applied only to the forget and test sets.

  3. SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance

    cs.CV 2026-08 conditional novelty 6.0 of 10

    SpatialAfford improves affordance grounding in a 4B VLM by first supervising cross-modal attention with the ground-truth region and then applying GRPO, outperforming several 7B+ baselines.

  4. Visual Access Boundaries in Vision-Language Model Reasoning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    CoT in VLMs extends language-side computation over early image-derived states rather than prolonging direct image-token access, with gains limited by perceptual readout reliability.

  5. UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models

    cs.CV 2026-02 conditional novelty 6.0 of 10

    Uncertainty-aware observation reinjection into FFN layers improves VLA manipulation success rates across LIBERO, SIMPLER, CALVIN, and real-robot tasks with no training.

  6. INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling

    cs.CV 2025-07 conditional novelty 6.0 of 10

    INTER is a training-free logit-correction method that adds Harsanyi interaction scores to selected keyword tokens, lowering hallucination on six LVLM benchmarks.

  7. Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A dual-level attention intervention that boosts salient visual-token attention and suppresses text/system attention during decoding reduces hallucination rates in LLaVA, MiniGPT-4, and mPLUG-Owl2 on POPE and CHAIR.

  8. Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis

    cs.CL 2025-05 conditional novelty 6.0 of 10

    PhantomCircuit traces knowledge overshadowing to attention circuits during training and prunes circuit edges to recover the overshadowed answer.

  9. Enhancing Multi-Robot Exploration Using Probabilistic Frontier Prioritization with Dirichlet Process Gaussian Mixtures

    cs.RO 2026-04 unverdicted novelty 5.0 of 10

    DP-GMM-based probabilistic frontier prioritization improves two multi-agent frontier explorers by roughly 10–14% across clutter, team size, and communication settings.

  10. Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Selecting the intermediate layer where image-conditioned and text-only predictions diverge most, and adding that layer's contrastive visual signal back to the final logits, reduces object hallucinations in four large ...

  11. CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A student-teacher multi-agent pipeline with positive-only feedback improves QWK agreement with human essay scores by 21% on a multimodal benchmark, with gains concentrated in traits where baselines were weakest.

Pith tools