Pith. sign in

REVIEW 4 major objections 3 minor 3 cited by

IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that in large vision-language models, visual attention fades as responses grow longer, that this decline drives the rise in hallucinations, and that a decoding-time logit-blending method called IKOD mitigates both without…

desk verdict Abstract-only read: promising inference-time hallucination fix, but the causal claim is explicitly a hypothesis and the full text is unreadable. read the letter →

arxiv 2508.03469 v1 pith:65B75BXJ submitted 2025-08-05 cs.CV

Jiabing Yang , Chenhang Cui , Yiyang Zhou , Yixiang Chen , Peng Xia , Ying Wei , Tao Yu , Yan Huang
show 1 more author
Liang Wang
This is my paper · ORCID
classification cs.CV
keywords largevision-languagemodelshallucinationmitigationvisualattentiondegradationdecoding-timeinterventionkey-valuecachemergingcollaborativedecodinglong-sequencebiasgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that as large vision-language models generate longer responses, the attention they pay to the image steadily declines, and this decline is a key cause of the well-known increase in hallucinations with sequence length. To counter it, the paper proposes IKOD, a decoding-time strategy that builds shorter, more image-focused sequences by merging the model's key-value cache and then combines their output scores with those of the original decoding. This intervention requires no additional training and no external tools, only access to the model's own attention and logits. If the claim is right, a lightweight runtime fix can make long generations more visually grounded and less prone to hallucination, while also improving general benchmark performance.

What carries the argument

The load-bearing mechanism is Image attention-guided Key-value merging cOllaborative Decoding (IKOD), a collaborative decoding strategy that uses attention to image tokens to guide how the key-value cache is merged. The key-value cache stores the key and value representations of all previously generated tokens, and merging it produces a shorter sequence that is more concentrated on visual content; decoding that shorter sequence yields logits from an image-focused branch. The final prediction combines these logits with the logits of the original autoregressive branch, so the model is steered back toward the image as generation proceeds. The measured per-token image-attention signal is what both documents the degradation and guides the merging.

What would settle it

A clean test is to decode the same prompt twice, once normally and once with attention to image tokens artificially clamped low after the first few tokens; if hallucination rates do not rise in lockstep with the clamp, the paper's causal story is wrong.

Watch

Extended reading notes

Core claim

The paper reports a consistent pattern across current large vision-language models: as a generated response grows longer, the attention directed at image tokens diminishes, and hallucination errors rise in parallel. It argues that this visual-attention degradation is a key mechanism behind the known long-term hallucination bias, not merely a byproduct of length. IKOD counters the degradation by maintaining, alongside the main decoding branch, a branch that merges the key-value cache into a shorter sequence with higher image attention, and the output scores from that image-focused branch are combined with the main branch's scores at each step. On hallucination and comprehensive benchmarks, the paper reports that this combination suppresses hallucinations and improves overall capacities across models. The method needs no additional training and no external tools.

Load-bearing premise

The load-bearing premise is that the decline in attention to image tokens is a cause of hallucinations, not merely a correlate, and that logits drawn from KV-merged shorter sequences still represent the model well enough to improve the final answer.

Editorial extensions

If this is right

  • If visual-attention degradation is a genuine mechanism, monitoring attention during decoding can flag when a model is about to drift into hallucination.
  • A retraining-free, tool-free intervention is available for deployment: any autoregressive large vision-language model with accessible attention and key-value cache can use the same idea.
  • Hallucination suppression and gains on comprehensive benchmarks can appear together, because keeping the model visually grounded helps the whole generation, not just factual accuracy.
  • The KV-merged branch gives the model a short, image-grounded preview of its own continuation, so longer generations, where attention drift is worst, should benefit most.
  • IKOD is complementary to other decoding interventions, since it changes neither weights nor prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves causality open: because it does not manipulate attention directly, the observed attention decline and hallucination rise could both be downstream of a third process, such as the model leaning more heavily on language priors as the response grows.
  • Extending beyond the paper, the optimal KV-merging strength is likely model-dependent; an adaptive recipe that measures image-attention retention per merging ratio could make the method more portable.
  • A testable extension is to combine the image-focused branch with contrastive decoding, using the merged short sequence as an anchor that keeps the long generation faithful to the image.
  • The image-attention statistic itself could be repurposed as a real-time hallucination early-warning signal during decoding, something the paper reports but does not explicitly propose as a monitoring metric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes IKOD (Image attention-guided Key-value merging cOllaborative Decoding), a training-free decoding strategy for large vision-language models (LVLMs) that aims to suppress hallucinations by counteracting visual attention degradation. The authors report observing that LVLM attention to image tokens decreases as generated sequences grow longer, hypothesize that this degradation is a key factor behind increased hallucination, and propose to combine logits from the original decoding with logits from KV-merged shorter sequences that exhibit higher image attention. The abstract claims extensive experiments on hallucination and comprehensive benchmarks, with no additional training or external tools. However, the supplied full text is a corrupted encoding and is largely unreadable, and the abstract provides no quantitative results, protocols, or causal evidence for the central hypothesis.

Significance. If the central claims were established, the paper would offer a practically useful, lightweight and model-agnostic intervention for hallucination reduction, and the documented attention-degradation phenomenon would be a valuable empirical finding for the LVLM community. The strengths of the paper are its falsifiable prediction about attention decline, its training-free and tool-free design, and the clarity of the proposed mechanism at the conceptual level. That said, the scientific significance hinges on the causal claim that attention degradation is a key factor behind hallucinations, which the abstract itself labels as a hypothesis; the method's mechanism could work for other reasons, and the supplied full text cannot be verified in its current form.

major comments (4)
  1. [Abstract (causal claim)] The abstract states that attention degradation is 'a key factor' behind hallucinations, but the relevant sentence explicitly says only 'which we hypothesize to be a key factor.' Even in the abstract, no causal intervention or mediation analysis is reported. Without an experiment that directly manipulates visual attention while controlling for sequence length, the observed correlation between attention decay and hallucination rate could be explained by other length-dependent mechanisms, such as positional bias, repetitive context, or a degrading language-model distribution. Because the paper's stated mechanism, and not merely its engineering heuristic, is the central claim, this missing causal test is load-bearing. A concrete causal experiment, such as attention intervention or a mediation analysis, should be added.
  2. [Full text (all sections)] The supplied full-text file is a corrupted encoding: equations, tables, and most prose are unreadable. As a result, the KV-merging algorithm, the logit combination rule, the hyperparameters (e.g., merging ratio and combination weight), the experimental protocols, and the numerical results cannot be verified. Since these are the core technical and empirical contents of the paper, a readable manuscript is required before the claims can be meaningfully assessed. This issue is not a presentation nitpick; it prevents verification of every load-bearing technical step.
  3. [Proposed method (IKOD)] IKOD selects shorter 'image-focused' sequences by using the same image-attention signal it seeks to restore, and its evaluation is on hallucination benchmarks. There is therefore a risk that the intervention is tuned to its own evaluation criterion. The authors should include control experiments, for example merging or truncating to sequences of the same length without attention guidance, or randomly merging KV pairs, to show that the gains are attributable to restoring image attention rather than to length priors or to the regularizing effect of KV merging. Without such controls, the current experiments would not establish that visual attention is the operative mechanism.
  4. [Proposed method (logit combination)] The method combines logits from the original decoding with logits from KV-merged shorter sequences. Because the merged sequences come from a modified model distribution, their logits may not be commensurable with the original logits. The paper does not appear to provide a calibration analysis or distributional evidence, such as perplexity comparisons or learned scaling factors, that would justify combining the two sets of logits. Without such evidence, the logit combination is an unprincipled heuristic, and the method's success could be due to the merge acting as a regularizer rather than to the proposed attention-restoration mechanism. A distributional analysis or a calibration experiment should be included.
minor comments (3)
  1. [Abstract] The phrase 'not incurring too much inference cost' is qualitative; please report the relative latency or FLOPs overhead in the experiments section.
  2. [Abstract] The reference to 'recent research' that found long-term bias should be replaced with explicit citations to those prior works.
  3. [Experimental setup (unreadable in full text)] Please define precisely how 'visual attention' is measured, for example mean attention weight to image tokens across layers and heads, and report error bars over multiple seeds once the manuscript is readable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity identified in the readable portions of the paper.

full rationale

The supplied full text is a corrupted encoding, so equations, tables, and the reference list could not be inspected. Based on the abstract, the central causal claim, that diminishing visual attention is a key factor in hallucinations, is explicitly presented as a hypothesis ("which we hypothesize to be a key factor"), not as a derived result. IKOD's hallucination and comprehensive-capacity claims are evaluated on external benchmarks, and no fitted parameter is renamed as a prediction. The attention-guided selection of shorter sequences makes the attention-restoration outcome a design property rather than a predicted discovery, but it does not force the external benchmark results. No load-bearing self-citation or ansatz-smuggling passage is visible in the readable text. Accordingly, no specific circular step can be quoted or exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

All ledger entries are inferred from the abstract because the supplied full text is corrupted and unreadable. The main free parameters and assumptions are those needed to make the IKOD mechanism work, not independently verified entities.

free parameters (2)
  • Logit combination weight = unknown
    IKOD combines logits from the KV-merged shorter sequence and the original decoding; the abstract does not state how this combination weight is set, so it is likely a hyperparameter tuned on a validation set.
  • KV merging ratio or block shape = unknown
    The abstract does not specify how many tokens are merged or the merge pattern; this is a design choice that could change the image-attention profile of the shortened sequence.
assumptions (3)
  • domain assumption Visual attention degradation is a key factor, not merely a correlation, behind increasing hallucinations in LVLMs.
    The abstract states this as a hypothesis; the entire IKOD design depends on it being at least actionable, and ideally causal.
  • domain assumption Shorter generated sequences consistently exhibit higher visual attention in current LVLMs.
    This underpins the use of KV-merged shorter sequences as the image-focused source; it is an empirical premise that is asserted but not demonstrated in the abstract.
  • ad hoc to paper Key-value merging into shorter sequences preserves enough logit quality to improve the final decode when combined with original logits.
    The abstract claims logits from merged sequences are used without showing that compression does not introduce serious distribution shift or new errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models." pith.science (2026). https://pith.science/paper/65B75BXJ

@misc{pith2026250803469,
  author       = {Pith},
  title        = {Pith review of: IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/65B75BXJ}},
  note         = {Machine review of arXiv:2508.03469}
}
read the original abstract

Recent advancements in Large Vision-Language Models (LVLMs) have demonstrated significant progress across multiple domains. However, these models still face the inherent challenge of integrating vision and language for collaborative inference, which often leads to "hallucinations", outputs that are not grounded in the corresponding images. Many efforts have been made to address these issues, but each comes with its own limitations, such as high computational cost or expensive dataset annotation. Recent research shows that LVLMs exhibit a long-term bias where hallucinations increase as the sequence length grows, yet the underlying cause remains poorly understood. Building on extensive research into attention mechanisms in LVLMs, we analyze the relationship between this long-term bias and visual attention. In our research, we identify a consistent phenomenon in current LVLMs: the model's attention to visual input diminishes as the generated sequence grows, which we hypothesize to be a key factor contributing to observed increasing hallucinations. Based on these insights, we propose Image attention-guided Key-value merging cOllaborative Decoding (IKOD), a collaborative decoding strategy generating more image-focused sequences. This method derives logits from shorter sequences with higher image attention through key-value merging and combines them with those from the original decoding, effectively mitigating attention degradation and suppressing hallucinations while not incurring too much inference cost. Extensive experiments on both hallucination and comprehensive benchmarks demonstrate IKOD's superior effectiveness in mitigating hallucinations and improving comprehensive capacities for LVLMs. Importantly, IKOD requires no additional training or external tools, making it a lightweight and efficient framework applicable to various models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. Visual Access Boundaries in Vision-Language Model Reasoning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    CoT in VLMs extends language-side computation over early image-derived states rather than prolonging direct image-token access, with gains limited by perceptual readout reliability.

  3. Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models

    cs.AI 2026-07 conditional novelty 6.0 of 10

    In GPT-2, OPT-1.3B, and LLaMA-3.2 models, boosting attention to function-token positions by 16-24% does not improve contextual retrieval, so short-range attention degradation is descriptive rather than causal.

Reference graph

Works this paper leans on

52 extracted references · 41 canonical work pages · cited by 3 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    An, W.; Tian, F.; Leng, S.; Nie, J.; Lin, H.; Wang, Q.; Chen, P.; Zhang, X.; and Lu, S. 2025. Mitigating object hallucinations in large vision-language models with assembly of global and local attention. In Proceedings of the Computer Vision and Pattern Recognition Conference, 29915--29926

    work page 2025

  4. [4]

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T.; Child, R.; Ramesh, A.; Ziegler, D.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.;...

    2020

  5. [5]

    Chen, J.; Yang, D.; Wu, T.; Jiang, Y.; Hou, X.; Li, M.; Wang, S.; Xiao, D.; Li, K.; and Zhang, L. 2024 a . Detecting and Evaluating Medical Hallucinations in Large Vision Language Models. CoRR

    work page 2024

  6. [6]

    Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B. 2024 b . An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, 19--35. Springer

    work page 2024

  7. [7]

    Chen, Z.; Zhao, Z.; Luo, H.; Yao, H.; Li, B.; and Zhou, J. 2024 c . HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding. In Forty-first International Conference on Machine Learning

    work page 2024

  8. [8]

    E.; Stoica, I.; and Xing, E

    Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90\

    work page 2023

Show all 52 references
  1. [9]

    R.; and He, P

    Chuang, Y.-S.; Xie, Y.; Luo, H.; Kim, Y.; Glass, J. R.; and He, P. 2023. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models. In The Twelfth International Conference on Learning Representations

    2023

  2. [10]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P. N.; and Hoi, S. 2024. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36

    2024

  3. [11]

    Ding, Y.; Geng, H.; Xu, C.; Fang, X.; Zhang, J.; Wei, S.; Dai, Q.; Zhang, Z.; and Wang, H. 2024. Open6DOR: Benchmarking open-instruction 6-DoF object rearrangement and a VLM-based approach. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 735...

    2024

  4. [12]

    Favero, A.; Zancato, L.; Trager, M.; Choudhary, S.; Perera, P.; Achille, A.; Swaminathan, A.; and Soatto, S. 2024. Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14303--14312

    2024

  5. [13]

    Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; Wu, Y.; and Ji, R. 2024. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. arXiv:2306.13394

    2024 arXiv

  6. [14]

    Huang, Q.; Dong, X.; Zhang, P.; Wang, B.; He, C.; Wang, J.; Lin, D.; Zhang, W.; and Yu, N. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  7. [15]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6700--6709

    2019

  8. [16]

    Kang, H.; and Liu, X.-Y. 2023. Deficiency of large language models in finance: An empirical examination of hallucination. In I Can't Believe It's Not Better Workshop: Failure Modes in the Age of Foundation Models

    2023

  9. [17]

    Leng, S.; Zhang, H.; Chen, G.; Li, X.; Lu, S.; Miao, C.; and Bing, L. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13872--13882

    2024

  10. [18]

    Li, L.; Xie, Z.; Li, M.; Chen, S.; Wang, P.; Chen, L.; Yang, Y.; Wang, B.; and Kong, L. 2023 a . Silkie: Preference Distillation for Large Visual Language Models. CoRR

    2023

  11. [19]

    L.; Holtzman, A.; Fried, D.; Liang, P.; Eisner, J.; Hashimoto, T.; Zettlemoyer, L.; and Lewis, M

    Li, X. L.; Holtzman, A.; Fried, D.; Liang, P.; Eisner, J.; Hashimoto, T.; Zettlemoyer, L.; and Lewis, M. 2023 b . Contrastive Decoding: Open-ended Text Generation as Optimization. In The 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  12. [20]

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, X.; and Wen, J.-R. 2023 c . Evaluating Object Hallucination in Large Vision-Language Models. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 292--305...

    2023

  13. [21]

    Lin, J.; Yin, H.; Ping, W.; Molchanov, P.; Shoeybi, M.; and Han, S. 2024 a . Vila: On pre-training for visual language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 26689--26699

    2024

  14. [22]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  15. [23]

    Lin, Z.; Lin, M.; Lin, L.; and Ji, R. 2024 b . Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference. arXiv preprint arXiv:2405.05803

    2024 arXiv

  16. [24]

    Liu, F.; Lin, K.; Li, L.; Wang, J.; Yacoob, Y.; and Wang, L. 2023 a . Aligning large multi-modal model with robust instruction tuning. CoRR

    2023

  17. [25]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296--26306

    2024

  18. [26]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023 b . Visual Instruction Tuning. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  19. [27]

    Liu, H.; Xue, W.; Chen, Y.; Chen, D.; Zhao, X.; Wang, K.; Hou, L.; Li, R.; and Peng, W. 2024 b . A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253

    2024 arXiv

  20. [28]

    Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Jiang, Q.; Li, C.; Yang, J.; Su, H.; et al. 2024 c . Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, 38--55. Springer

    2024

  21. [29]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; et al. 2024 d . Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, 216--233. Springer

    2024

  22. [30]

    Liu, Z.; Liu, B.; Wang, J.; Dong, Y.; Chen, G.; Rao, Y.; Krishna, R.; and Lu, J. 2024 e . Efficient inference of vision instruction-following models with elastic cache. In European Conference on Computer Vision, 54--69. Springer

    2024

  23. [31]

    Lu, P.; Mishra, S.; Xia, T.; Qiu, L.; Chang, K.-W.; Zhu, S.-C.; Tafjord, O.; Clark, P.; and Kalyan, A. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35: 2507--2521

    2022

  24. [32]

    OpenAI. 2023. GPT-4 Technical Report. ArXiv, abs/2303.08774

    2023 arXiv

  25. [33]

    A.; Burns, K.; Darrell, T.; and Saenko, K

    Rohrbach, A.; Hendricks, L. A.; Burns, K.; Darrell, T.; and Saenko, K. 2018. Object Hallucination in Image Captioning. In Riloff, E.; Chiang, D.; Hockenmaier, J.; and Tsujii, J., eds., Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 4035...

    2018

  26. [34]

    Schwenk, D.; Khandelwal, A.; Clark, C.; Marino, K.; and Mottaghi, R. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision, 146--162. Springer

    2022

  27. [35]

    Sun, Z.; Shen, S.; Cao, S.; Liu, H.; Li, C.; Shen, Y.; Gan, C.; Gui, L.-Y.; Wang, Y.-X.; Yang, Y.; et al. 2024. Aligning Large Multimodal Models with Factually Augmented RLHF. In Annual Meeting of the Association for Computational Linguistics

    2024

  28. [36]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  29. [37]

    Wang, X.; Pan, J.; Ding, L.; and Biemann, C. 2024. Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding. In ACL (Findings)

    2024

  30. [38]

    Woo, S.; Kim, D.; Jang, J.; Choi, Y.; and Kim, C. 2024. Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models. CoRR

    2024

  31. [39]

    Yang, S.; Ge, Y.; Li, Y.; Chen, Y.; Ge, Y.; Shan, Y.; and Chen, Y. 2024. SEED-Story: Multimodal Long Story Generation with Large Language Model. CoRR

    2024

  32. [40]

    Yin, S.; Fu, C.; Zhao, S.; Xu, T.; Wang, H.; Sui, D.; Shen, Y.; Li, K.; Sun, X.; and Chen, E. 2024. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12): 220105

    2024

  33. [41]

    Yu, R.; Yu, W.; and Wang, X. 2024. Attention prompting on image for large vision-language models. In European Conference on Computer Vision, 251--268. Springer

    2024

  34. [42]

    Yu, T.; Yao, Y.; Zhang, H.; He, T.; Han, Y.; Cui, G.; Hu, J.; Liu, Z.; Zheng, H.-T.; Sun, M.; et al. 2024 a . Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  35. [43]

    Yu, W.; Yang, Z.; Li, L.; Wang, J.; Lin, K.; Liu, Z.; Wang, X.; and Wang, L. 2024 b . MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities. In Forty-first International Conference on Machine Learning

    2024

  36. [44]

    Zhang, Y.-F.; Yu, W.; Wen, Q.; Wang, X.; Zhang, Z.; Wang, L.; Jin, R.; and Tan, T. 2024. Debiasing Multimodal Large Language Models. CoRR

    2024

  37. [45]

    Zhang, Z.; Sheng, Y.; Zhou, T.; Chen, T.; Zheng, L.; Cai, R.; Song, Z.; Tian, Y.; R \'e , C.; Barrett, C.; et al. 2023. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36: 34661--34710

    2023

  38. [46]

    Zhao, B.; Wu, B.; and Huang, T. 2023. SVIT: Scaling up Visual Instruction Tuning. CoRR

    2023

  39. [47]

    Zhou, G.; Yan, Y.; Zou, X.; Wang, K.; Liu, A.; and Hu, X. 2024 a . Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality. arXiv preprint arXiv:2410.04780

    2024 arXiv

  40. [48]

    Zhou, Y.; Cui, C.; Rafailov, R.; Finn, C.; and Yao, H. 2024 b . Aligning Modalities in Vision Large Language Models via Preference Fine-tuning. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  41. [49]

    Zhou, Y.; Cui, C.; Yoon, J.; Zhang, L.; Deng, Z.; Finn, C.; Bansal, M.; and Yao, H. 2023. Analyzing and Mitigating Object Hallucination in Large Vision-Language Models. In The Twelfth International Conference on Learning Representations

    2023

  42. [50]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models. In The Twelfth International Conference on Learning Representations

    2023

  43. [51]

    Zhu, H.; Qin, S.; Su, M.; Lin, C.; Li, A.; and Gao, J. 2024 a . Harnessing Large Vision and Language Models in Agriculture: A Review. arXiv preprint arXiv:2407.19679

    2024 arXiv

  44. [52]

    Zhu, Y.; Liu, J.; Gao, F.; Liu, W.; Wang, X.; Wang, P.; Huang, F.; Yao, C.; and Yang, Z. 2024 b . Visual Text Generation in the Wild. In European Conference on Computer Vision, 89--106

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.