Pith. sign in

REVIEW 2 major objections 5 minor 71 references

Vision-language models assemble visual evidence in a middle-layer relay window whose timing decides whether answers stay grounded.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 05:36 UTC pith:SBEKSNTK

load-bearing objection Solid mechanistic story plus a working inference-time controller; the VRW geometry is real enough to use, even if the causal isolation is incomplete. the 2 major comments →

arxiv 2607.11436 v1 pith:SBEKSNTK submitted 2026-07-13 cs.AI

The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning

Wencheng Ye , Yi Bin , Yujuan Ding , Hongye Fang , Zheng Wang , Xing Xu , Jingkuan Song , Yun Zhang
show 2 more authors
Sirui Da Heng Tao Shen
This is my paper
classification cs.AI
keywords vision-language modelsmultimodal reasoningVisual Relay Windowattention focushallucination mitigationinference-time controlTRACE
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models often lose reliable visual support once images enter the language stack, so answers drift toward language priors. This paper tracks how attention focus redistributes layer by layer and reports a stable three-stage pattern: early question-conditioned organization, a middle visual-dominant phase called the Visual Relay Window, and a late return to answer formation. The geometry of that middle window is not fixed; it lengthens or shortens with task demand, and mismatches produce unsupported answers. Causal patching shows that intervening inside the estimated window restores grounded responses far more often than intervening outside it. The authors turn the pattern into TRACE, a lightweight inference-time controller that predicts the window, schedules how much visual consolidation to allow during prefill, and anchors the selected visual support after handoff during decoding. Across four open backbones and seven benchmarks the same control improves both hallucination-sensitive and reasoning-heavy settings, casting those problems as one scheduling problem rather than separate fixes.

Core claim

Open vision-language models share a depth-wise three-stage redistribution of multimodal focus—early question-conditioned organization, a middle visual-dominant Visual Relay Window, and late answer formation—whose width and termination depth adapt to task demand and are causally linked to whether generation remains grounded. Controlling that window at inference time therefore strengthens evidence-grounded multimodal reasoning under a single mechanism.

What carries the argument

Visual Relay Window (VRW): the operational middle-layer interval where jointly normalized Image→Image attention dominates Answer→Query attention; TRACE predicts it from prefill statistics, expands or contracts visual-to-visual bias inside it, then depth-decays answer-to-visual anchoring after handoff.

Load-bearing premise

The claim rests on treating two normalized attention-mass probes and a fixed middle-depth search band as a sufficient definition of the functionally critical evidence-assembly interval that should be predicted and controlled.

What would settle it

If patching residual activations inside the estimated VRW no longer recovers grounded answers more often than same-width patches just outside it, or if TRACE’s measured width/end shifts reverse the claimed direction (broader/later for grounding tasks, shorter/earlier for reasoning) while still producing the reported gains, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper argues that VLMs exhibit a stable three-stage depth-wise redistribution of multimodal attention—early question-conditioned organization, a middle visual-dominant Visual Relay Window (VRW), and late answer formation—and that this middle interval is task-dependent and causally linked to grounded generation. VRW is operationalized from jointly normalized Answer→Query and Image→Image attention mass (Eqs. 2–8). Causal support comes from residual patching of visual-token activations inside the estimated window (Table 2: 41.4% recovery of unsupported branches vs ~9–11% outside and ~28% for same-width shifts). Building on this, TRACE uses a lightweight learned predictor, task-aware prefill V→V scheduling, and post-handoff answer→visual anchoring. Across four open-weight backbones and seven benchmarks, TRACE improves grounding-sensitive settings by 4.33 points on average (up to 6.6) and also improves reasoning-heavy tasks, with ablations and mechanism analyses (Fig. 6, Tables 6–8) supporting the intended relay reshaping.

Significance. If the result holds, the paper offers a unified mechanistic account of when visual evidence is assembled and handed off in VLMs, and converts that account into a practical inference-time controller. Strengths include multi-legged support (cross-model curves, task ANOVA, grounded/unsupported contrasts, controlled residual patching, Thinking-variant comparison), a capacity-matched generic-controller baseline (Table 6), component ablations, hyperparameter sensitivity, and public code. The contribution is relevant both to hallucination mitigation and to reasoning-oriented multimodal control, and is more structure-aware than purely decoding-stage contrastive or steering methods.

major comments (2)
  1. [§3.5, Table 2; §3.8] §3.5 and Table 2 establish that patching visual-token residual activations inside the estimated VRW recovers grounded answers more often than pre/post ranges or same-width shifts. That is strong evidence that the interval matters, but it does not yet show that the attention-mass geometry used to define VRW (Eqs. 4–8) is the operative mechanism rather than co-varying middle-layer pathways (MLP updates, non-visual sinks, cross-group residual streams). Because TRACE’s predictor is trained on pseudo-labels from the same rule-based estimator (§3.8, Eqs. 12–14), the claim that gains come from “scheduling the visual relay rhythm” remains partly under-determined. Please either (i) add at least one non-attention or alternative-probe control (e.g., patching non-visual residuals in the same layers, or defining a window from other probe pairs in Table 1), or (ii) clearly demote the mechanistic wordi
  2. [§3.3, Table 1, Table 9] Table 9 only varies normalization, search band, and δ for the same two probes. Given that the weakest load-bearing assumption is that Answer→Query vs Image→Image with ρ∈[0.2,0.8], δ=0.1 is a sufficient operational definition of the critical assembly interval (§3.3), the paper should report at least one alternative operationalization (e.g., Answer→Query/Answer→Image from Table 1, or a peak-only single-layer target) and show whether TRACE’s external gains and the grounded/unsupported separation remain directionally intact. Without that, the “stable three-stage relay” story risks being tied too tightly to one probe construction.
minor comments (5)
  1. [Figure 1] Figure 1 caption/label has “Intergrate”; fix spelling.
  2. [Table 4] Table 4 repeatedly prints “A verage” with a space; normalize to “Average”.
  3. [§3.2 vs §4.1] Clarify early that Fig. 1 uses ten models for the descriptive pattern while TRACE is evaluated on four backbones, so readers do not expect intervention results on all ten.
  4. [§3.9–3.10] Eq. (19) and surrounding text use bS / eS notation that is easy to misread in plain text; consider a clearer accent or boldface convention in the camera-ready version.
  5. [§3.1, §5] The conclusion correctly notes single-image and attention-probe limits; a short forward pointer in §3.1 would help set expectations earlier.

Circularity Check

1 steps flagged

Mild self-definitional loop only in mechanism verification (intervene on V→V mass, then report VRW shifts); core causal and benchmark claims remain external and non-circular.

specific steps
  1. self definitional [§3.3 Eqs. 4–8; §3.9 Eq. 19; §4.3 / Fig. 6]
    "R(l) = M(l)_I→I − M(l)_A→Q. ... During prefill, the scheduler directly reshapes visual-to-visual interactions in relay layers: bS(l)_ij = S(l)_pf,ij + λ g(l)_θ r_l m(l)_i m(l)_j , i, j ∈ V. ... Beyond benchmark gains, we ask whether the intervention reshapes computation in the intended way. Figure 6 shows a clear task dependence. ... TRACE improves performance by reshaping the same relay redistribution pattern identified in analysis"

    VRW geometry is defined from the signed dominance of Image→Image vs Answer→Query attention mass. TRACE’s prefill control adds a gated bias exactly to visual-to-visual logits inside the predicted window, which by construction changes M_I→I and therefore R(l), VRW-Width, and VRW-End. Reporting that those VRW statistics shift as intended is therefore partly a restatement that the intervention acted on its defining coordinates, not an independent measurement of a separate latent pathway. This tautology is limited to the mechanism-verification claim; external accuracy and residual-patching recovery are not forced by it.

full rationale

This is an empirical VLM-control paper, not a first-principles derivation. The three-stage pattern and VRW are operational constructs from attention probes (Eqs. 2–8), not claimed mathematical necessities. Causal support (Table 2 residual patching of visual tokens; grounded vs unsupported geometry in Fig. 4) and all main results (Table 4 accuracy on seven held-out benchmarks vs independent baselines) are external to the VRW definition and are not forced by construction. Training Dϕ on pseudo-labels from the offline VRW rule (Eqs. 13–14) is ordinary supervised recovery of an operational target, not a fitted quantity renamed as a prediction of a related scientific observable. There is no load-bearing self-citation uniqueness theorem, no ansatz smuggled via author-prior work, and no renaming of a known closed-form result. The only mild circularity is local to the mechanism-check in §4.3: TRACE’s prefill bias (Eq. 19) directly modulates the same Image→Image mass that enters R(l) and thus VRW-Width/End, so reporting that those statistics move in the intended direction partly verifies that the intervention fired rather than independently discovering a new internal rhythm. That does not collapse the performance or causal claims. Score 2 is proportionate.

Axiom & Free-Parameter Ledger

8 free parameters · 4 axioms · 2 invented entities

The central story rests on treating two attention-mass probes as a sufficient proxy for multimodal focus, on a hand-specified operational definition of the middle relay interval, and on a large set of inference-control hyperparameters. The invented VRW is operational rather than a new physical entity, but it is still a paper-introduced construct whose labels train the controller. External benchmark accuracy and causal patching provide partial independent checks; many knobs remain free.

free parameters (8)
  • VRW middle search band ρl ∈ [0.2, 0.8]
    Heuristic restriction of the relay peak search to network interior; variants are tested in Table 9 but the default is chosen by hand.
  • VRW tolerance δ = 0.1
    Defines how far R(l) may fall from the peak while remaining in the candidate interval (Eq. 6).
  • support ratio p = 0.3
    Top-p fraction of visual tokens retained in the dynamic support set; sensitivity shown in Table 8.
  • relay-exit threshold τr = 0.7
    Threshold on soft relay gate used to set handoff depth ê (Eq. 20).
  • anchoring loss weight λanchor = 0.05
    Balances next-token loss against post-handoff Anchor retention (Eq. 23–24).
  • prefill scheduling scale λ = 0.6
    Global strength of visual-to-visual bias inside predicted relay layers (Eq. 19).
  • gate sharpness/operating point α=13, β=0.5
    Map predictor scores to soft relay gate rl = σ(α(pl−β)).
  • relevance EMA μ=0.15; Anchor hinge η=0.7; decode bias γ0=0.4, κ=2.0
    Hand-fixed dynamics for support accumulation, retention hinge, and depth-decayed decode bias.
axioms (4)
  • domain assumption Layerwise Answer→Query and Image→Image attention mass are adequate proxies for question-conditioned organization vs visual consolidation.
    Introduced in §3.1 as the measurement basis for the entire three-stage analysis and VRW definition.
  • ad hoc to paper A contiguous middle interval where R(l)=M_I→I−M_A→Q is near its peak and preserves the handoff trend is the functionally critical evidence-assembly stage.
    Operational definition in §3.3 (Eqs. 4–8); justified by correlation and patching, not derived from a formal circuit theorem.
  • domain assumption Self-attention residual patching of visual-token activations at aligned decode steps is a valid causal test of whether a layer range supports grounded generation.
    Used in §3.5 Table 2; standard mechanistic-interpretability practice but still an assumption about intervention semantics.
  • domain assumption Lightweight modules trained with frozen backbone weights and next-token plus hinge anchoring losses can reshape relay without destroying general capabilities.
    Training setup in §3.11 and §4.1; supported by ablations but not guaranteed for all tasks or longer multi-image settings.
invented entities (2)
  • Visual Relay Window (VRW) no independent evidence
    purpose: Name and measure the middle visual-dominant phase via width and end depth statistics used for analysis, pseudo-labels, and control.
    Paper-defined operational construct from attention probes; independent_evidence is partial via causal patching and external accuracy, but the entity itself is not independently measured outside this framework.
  • TRACE controller (predictor + task-aware scheduler + post-handoff anchoring) independent evidence
    purpose: Inference-time mechanism that predicts relay layers, expands/contracts V→V bias with dynamic support selection, and anchors A→V attention after handoff.
    New engineered system; evaluated on public benchmarks, but the modules are defined by the paper’s own training objectives and hyperparameters.

pith-pipeline@v1.1.0-grok45 · 27402 in / 4054 out tokens · 41574 ms · 2026-07-14T05:36:05.222088+00:00 · methodology

0 comments
read the original abstract

Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening evidence-grounded reasoning. To understand this fragility, we examine the internal dynamics of VLMs through a mechanistic lens and uncover a stable three-stage redistribution of multimodal attention focus across depth: an early question-conditioned organization, a critical middle visual-dominant relay, and a late return to answer formation. We operationalize the middle phase as the Visual Relay Window (VRW), and show that its geometry varies with task demand, is causally tied to grounded generation, and distinguishes unsupported answers from stronger reasoning trajectories. Guided by this internal rhythm, we propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules. It reshapes relay allocation during prefill and preserves assembled visual support after handoff during decoding. Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings, improving them by 4.33 points on average and by up to 6.6 points, while also improving reasoning-heavy tasks. These results show that explicitly controlling multimodal focus across depth offers a unified and effective mechanism for strengthening evidence-grounded multimodal reasoning.

Figures

Figures reproduced from arXiv: 2607.11436 by Heng Tao Shen, Hongye Fang, Jingkuan Song, Sirui Da, Wencheng Ye, Xing Xu, Yi Bin, Yujuan Ding, Yun Zhang, Zheng Wang.

Figure 1
Figure 1. Figure 1: A unified three-stage relay pattern across vision-language models. Across ten open VLMs, the jointly normalized [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Task-dependent geometry of the Visual Relay Win [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Case study of relay bifurcation for the same image [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Unsupported answers are associated with narrower [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of TRACE. TRACE first predicts a task-specific Visual Relay Window from attention statistics. Within the predicted window, it dynamically schedules visual-to-visual attention bias to reinforce evidence relay. After the relay stage ends, it gradually shifts to answer-to-visual anchoring with a depth-decayed bias, preserving grounded visual evidence while supporting stable multimodal reasoning throu… view at source ↗
Figure 6
Figure 6. Figure 6: Task-wise relay reshaping on Qwen3-VL-4B. Posi [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

71 extracted references · 14 linked inside Pith

  1. [1]

    2026 , eprint=

    FineVision: Open Data Is All You Need , author=. 2026 , eprint=

    2026

  2. [2]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Llamav-o1: Rethinking step-by-step visual reasoning in llms , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  3. [3]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    When visualizing is the first step to reasoning: Mira, a benchmark for visual chain-of-thought , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  4. [4]

    International Conference on Learning Representations , volume=

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts , author=. International Conference on Learning Representations , volume=

  5. [5]

    arXiv preprint arXiv:2404.18930 , year=

    Hallucination of multimodal large language models: A survey , author=. arXiv preprint arXiv:2404.18930 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2402.00253 , year=

    A survey on hallucination in large vision-language models , author=. arXiv preprint arXiv:2402.00253 , year=

    Pith/arXiv arXiv

  7. [7]

    Journal of Computing and Information Science in Engineering , volume=

    Designqa: A multimodal benchmark for evaluating large language models’ understanding of engineering documentation , author=. Journal of Computing and Information Science in Engineering , volume=. 2025 , publisher=

    2025

  8. [8]

    arXiv preprint arXiv:2509.11986 , volume=

    Lost in embeddings: Information loss in vision-language models , author=. arXiv preprint arXiv:2509.11986 , volume=

    arXiv

  9. [9]

    Findings of the association for computational linguistics: emnlp 2024 , pages=

    Difficult task yes but simple task no: Unveiling the laziness in multimodal LLMs , author=. Findings of the association for computational linguistics: emnlp 2024 , pages=

    2024

  10. [10]

    Annual review of vision science , volume=

    Scene perception in the human brain , author=. Annual review of vision science , volume=. 2019 , publisher=

    2019

  11. [11]

    Computational brain & behavior , volume=

    The discovery and interpretation of evidence accumulation stages , author=. Computational brain & behavior , volume=. 2021 , publisher=

    2021

  12. [12]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  13. [13]

    arXiv preprint arXiv:2501.02189 , volume=

    Benchmark evaluations, applications, and challenges of large vision language models: A survey , author=. arXiv preprint arXiv:2501.02189 , volume=

    Pith/arXiv arXiv

  14. [14]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Circuit tracing in vision-language models: Understanding the internal mechanisms of multimodal thinking , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  15. [15]

    European Conference on Computer Vision , pages=

    Haloquest: A visual hallucination dataset for advancing multimodal reasoning , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  16. [16]

    arXiv preprint arXiv:2601.06521 , year=

    BabyVision: Visual Reasoning Beyond Language , author=. arXiv preprint arXiv:2601.06521 , year=

    Pith/arXiv arXiv

  17. [17]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  18. [18]

    arXiv preprint arXiv:2503.03321 , year=

    See what you are told: Visual attention sink in large multimodal models , author=. arXiv preprint arXiv:2503.03321 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Sharp: Steering hallucination in lvlms via representation engineering , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  20. [20]

    European Conference on Computer Vision , pages=

    Paying more attention to image: A training-free method for alleviating hallucination in lvlms , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  21. [21]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  22. [22]

    arXiv preprint arXiv:2203.11171 , year=

    Self-consistency improves chain of thought reasoning in language models , author=. arXiv preprint arXiv:2203.11171 , year=

    Pith/arXiv arXiv

  23. [23]

    Advances in Neural Information Processing Systems , volume=

    Countgd: Multi-modal open-world counting , author=. Advances in Neural Information Processing Systems , volume=

  24. [24]

    Proceedings of the Asian Conference on Computer Vision , pages=

    Vision language models are blind , author=. Proceedings of the Asian Conference on Computer Vision , pages=

  25. [25]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  26. [26]

    Grok-1.5 Vision Preview: Connecting the Digital and Physical Worlds with Our First Multimodal Model , year =

  27. [27]

    Findings of the association for computational linguistics: ACL 2022 , pages=

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning , author=. Findings of the association for computational linguistics: ACL 2022 , pages=

    2022

  28. [28]

    Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

    Docvqa: A dataset for vqa on document images , author=. Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

  29. [29]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Improved baselines with visual instruction tuning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  30. [30]

    5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=

    Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=. arXiv preprint arXiv:2508.18265 , year=

    Pith/arXiv arXiv

  31. [31]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Vision-language models for vision tasks: A survey , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2024 , publisher=

    2024

  32. [32]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    2025

  33. [33]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    Pith/arXiv arXiv

  34. [34]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  35. [35]

    arXiv preprint arXiv:2601.05547 , year=

    VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck , author=. arXiv preprint arXiv:2601.05547 , year=

    Pith/arXiv arXiv

  36. [36]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    AVAM: a Universal Training-free Adaptive Visual Anchoring Embedded into Multimodal Large Language Model for Multi-image Question Answering , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  37. [37]

    arXiv preprint arXiv:2605.18359 , year=

    RAVE: Re-Allocating Visual Attention in Large Multimodal Models , author=. arXiv preprint arXiv:2605.18359 , year=

    Pith/arXiv arXiv

  38. [38]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  39. [39]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  40. [40]

    International Journal on Digital Libraries , volume=

    Scienceqa: A novel resource for question answering on scholarly articles , author=. International Journal on Digital Libraries , volume=. 2022 , publisher=

    2022

  41. [41]

    Language Resources and Evaluation , volume=

    Ai2d-rst , author=. Language Resources and Evaluation , volume=. 2021 , publisher=

    2021

  42. [42]

    Proceedings of the 33rd ACM International Conference on Multimedia , pages=

    Vqa2: visual question answering for video quality assessment , author=. Proceedings of the 33rd ACM International Conference on Multimedia , pages=

  43. [43]

    International journal of computer vision , volume=

    Visual genome: Connecting language and vision using crowdsourced dense image annotations , author=. International journal of computer vision , volume=. 2017 , publisher=

    2017

  44. [44]

    Proceedings of the IEEE international conference on computer vision , pages=

    Vqa: Visual question answering , author=. Proceedings of the IEEE international conference on computer vision , pages=

  45. [45]

    5-coder technical report , author=

    Qwen2. 5-coder technical report , author=. arXiv preprint arXiv:2409.12186 , year=

    Pith/arXiv arXiv

  46. [46]

    Advances in neural information processing systems , volume=

    Learn to explain: Multimodal reasoning via thought chains for science question answering , author=. Advances in neural information processing systems , volume=

  47. [47]

    The Fourteenth International Conference on Learning Representations , year=

    Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models , author=. The Fourteenth International Conference on Learning Representations , year=

  48. [48]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    RFI: Rectified Flow Intervention for Mitigating Object Hallucination in Large Vision-Language Models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  49. [49]

    Proceedings of the ACM on Web Conference 2025 , pages=

    Adaptive activation steering: A tuning-free llm truthfulness improvement method for diverse hallucinations categories , author=. Proceedings of the ACM on Web Conference 2025 , pages=

    2025

  50. [50]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Activation steering decoding: Mitigating hallucination in large vision-language models through bidirectional hidden state intervention , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  51. [51]

    International Conference on Learning Representations , volume=

    Reducing hallucinations in large vision-language models via latent space steering , author=. International Conference on Learning Representations , volume=

  52. [52]

    arXiv preprint arXiv:2602.04268 , year=

    KVSmooth: Mitigating Hallucination in Multi-modal Large Language Models through Key-Value Smoothing , author=. arXiv preprint arXiv:2602.04268 , year=

    arXiv

  53. [53]

    arXiv preprint arXiv:2506.08391 , year=

    Second: Mitigating perceptual hallucination in vision-language models via selective and contrastive decoding , author=. arXiv preprint arXiv:2506.08391 , year=

    arXiv

  54. [54]

    arXiv e-prints , pages=

    Mitigating object hallucination in large vision-language models via classifier-free guidance , author=. arXiv e-prints , pages=

  55. [55]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  56. [56]

    Vocabulary Fixation Reveals Visual Attention Sink for Hallucination Mitigation in LVLMs , author=

  57. [57]

    arXiv preprint arXiv:2411.09968 , year=

    Seeing clearly by layer two: Enhancing attention heads to alleviate hallucination in lvlms , author=. arXiv preprint arXiv:2411.09968 , year=

    Pith/arXiv arXiv

  58. [58]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Response Wide Shut? Surprising Observations in Basic Vision Language Model Capabilities , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  59. [59]

    arXiv preprint arXiv:2601.09954 , year=

    The Spatial Blindspot of Vision-Language Models , author=. arXiv preprint arXiv:2601.09954 , year=

    arXiv

  60. [60]

    arXiv preprint arXiv:2605.05668 , year=

    Large vision-language models get lost in attention , author=. arXiv preprint arXiv:2605.05668 , year=

    Pith/arXiv arXiv

  61. [61]

    arXiv preprint arXiv:2602.07025 , year=

    The Geometry of Representational Failures in Vision Language Models , author=. arXiv preprint arXiv:2602.07025 , year=

    Pith/arXiv arXiv

  62. [62]

    From redundancy to relevance: Information flow in lvlms across reasoning tasks , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  63. [63]

    Advances in Neural Information Processing Systems , volume=

    Understanding information storage and transfer in multi-modal large language models , author=. Advances in Neural Information Processing Systems , volume=

  64. [64]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Causal tracing of object representations in large vision language models: Mechanistic interpretability and hallucination mitigation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  65. [65]

    Winter Conference on Applications of Computer Vision 2026 , year=

    FG-TRACER: Tracing Information Flow in Multimodal Large Language Models in Free-Form Generation , author=. Winter Conference on Applications of Computer Vision 2026 , year=

    2026

  66. [66]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  67. [67]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    What's in the Image? A Deep-Dive into the Vision of Vision Language Models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  68. [68]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Cross-modal information flow in multimodal large language models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  69. [69]

    Process-then-Retrieve: A Mechanistic Study of Cross-Modal Alignment in Vision-Language Models , author=

  70. [70]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Investigating and mitigating the multimodal hallucination snowballing in large vision-language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  71. [71]

    Advances in Neural Information Processing Systems , volume=

    More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models , author=. Advances in Neural Information Processing Systems , volume=