Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that hallucination in LVLMs can be mitigated at decoding time by selecting image tokens according to layer-2 attention and masking the rest, with contrastive decoding to amplify the selected evidence.

desk verdict This is a cleanly written incremental combo of FastV-style token selection and VCD-style contrastive decoding, but Table 1 lacks the K=576-with-CD control, so the paper never actually shows that token reduction itself helps. read the letter →

arxiv 2502.00717 v1 pith:A3OZFJGQ submitted 2025-02-02 cs.CV

Chao Wang , Jianming Yang , Yang Zhou This is my paper · ORCID
classification cs.CV
keywords hallucinationmitigationlargevision-languagemodelstokenreductionattentionredundancycontrastivedecodingtraining-freeobjectvisualperception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large vision-language models (LVLMs) understand an image in the first two decoder layers and then keep paying attention to irrelevant image tokens in deeper layers, wasting computation and inviting hallucinated answers. MINT is a training-free decoding strategy that exploits this redundancy: it reads the attention scores from the second decoder layer, keeps only the top-$K$ image tokens, masks the rest, and then applies contrastive decoding to amplify the chosen visual evidence. On POPE, MINT improves object-perception accuracy by roughly 4% over the original LLaVA models, and on CHAIR captioning it raises recall of annotated objects by about 5% while using fewer image tokens. The paper argues that hallucination from 'distracted perception' can be mitigated without retraining or external tools, making the method a cheap reliability boost for existing models.

What carries the argument

The carrying object is the second-layer attention criterion $\psi_{v_j}=\frac{1}{H}\sum_{i=1}^{H} a^{i,2}_{t,v_j}$ (Eq. 4), the average attention that the current token pays to image token $v_j$ across heads in decoder layer 2. This score selects the top-$K$ image tokens ($I_K = \arg\max_j \psi_{v_j}$, Eq. 5), and attention to all other image tokens is masked. The second half of the machinery is contrastive decoding with an adaptive plausibility constraint: logits from the selected-token run are combined with the negative logits of a run in which all image tokens are masked, giving $\tilde p_t = \rho[(1+\alpha)\logits_\phi(y_t\mid *,\tilde X_V) - \alpha \logits_\phi(y_t\mid *)]$, and the final token is sampled from the truncated vocabulary $V_h(y_{<t})$ (Eqs. 6-8).

What would settle it

An experiment that settles it: run MINT on a set of images where the answer hinges on a small, low-contrast object while recording layer-2 attention scores; if the object's tokens rank below the cutoff and the model answers incorrectly, the attention proxy is unreliable. A complementary check is to replace the layer-2 attention ranking with random token selection at the same budget and compare the gains.

Watch

Extended reading notes

Core claim

The central discovery is that attention in LVLMs is distributed unequally across layers: the model forms a rich image representation in the first two layers, after which most image tokens receive little or no task-relevant attention, and the residual attention goes to sinks like the system prompt. MINT converts this observation into a mechanism: at each decoding step it computes the average attention each image token receives in the second layer, selects the top-$K$ by that score, masks attention to all other image tokens, and then refines the logits by contrasting the selected-token distribution against a no-image distribution. The paper claims this reduces object hallucinations and increases the number of correctly mentioned visual objects, even when an image is represented by 75% or 50% of its original tokens.

Load-bearing premise

The load-bearing premise is that the second decoder layer's attention scores reliably identify the image tokens that actually matter for the current question, so masking every other token is safe.

Editorial extensions

If this is right

  • LVLMs can run with 25% fewer image tokens at no loss, and often a gain, on object-perception benchmarks, implying substantial token redundancy in the decoder.
  • Hallucinations caused by the model being distracted by irrelevant regions can be reduced by a purely decoding-time intervention, without retraining or extra models.
  • The same attention-based token selection may serve as a general lever for other decoding-time corrections, since it locates the visual evidence the model is actually using.
  • The method's gains on POPE and CHAIR transfer across LLaVA-1.5-7B and LLaVA-1.5-13B, and partially to Qwen-VL, but not to MiniGPT-4 on MME, where token selection can discard useful evidence.
  • Reducing image tokens also shortens the sequence processed in deep layers, which can translate into faster inference in addition to better accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ablation suggests the contrastive-decoding component, not the token selection alone, is what lifts accuracy: token selection without contrastive decoding at K=432 stays near baseline or drops at smaller K, while the combination gives the reported gains, so the improvement should not be attributed entirely to the attention-based selection.
  • The layer-2 attention heuristic is likely architecture-sensitive; the paper shows Qwen-VL benefits less, and a model that does not concentrate visual understanding in its second decoder layer may need a different layer choice.
  • A natural stress test is to feed images where the answer depends on a visually small but semantically central object; if layer-2 attention does not rank that object's tokens highly, MINT would be expected to fail, which would mark the boundary of the method.
  • The '5% more visual points' claim is measured as recall of annotated objects in CHAIR; whether such recall gains persist under more diverse open-ended prompts and longer generations remains an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MINT, a training-free decoding strategy for large vision-language models that combines (i) token selection, which keeps only the top-K image tokens according to the average attention they receive in the second decoder layer, and (ii) contrastive decoding with an adaptive plausibility constraint, which subtracts the text-only logits from the vision-conditioned logits. The authors report that MINT improves object-perception accuracy on POPE, perception accuracy on MME, and reduces object hallucination in CHAIR captioning across several LVLMs (LLaVA-1.5-7B/13B, Qwen-VL-7B, MiniGPT-4-7B), claiming roughly a 4% improvement in hallucination mitigation and a 5% increase in perceived visual points despite discarding a quarter of image tokens.

Significance. If the central claim were fully supported, MINT would be a useful plug-and-play inference-time method for reducing object hallucinations without retraining or external models. The paper is honest in reporting some negative results (MiniGPT-4 MME decline, Qwen-VL weakness on adversarial POPE, LLaVA-1.5-13B CHAIR_I degradation), which is a strength. However, the ablation design leaves the main attribution unresolved: the contribution of token selection per se is not isolated from the contrastive-decoding component, which is an existing technique. With the current evidence, the claimed advantages over VCD are not established. The work builds on FastV-style layer-2 attention selection and VCD-style contrastive decoding; its novelty is the specific combination and the emphasis on local perception, but the missing control cell prevents a clear assessment of the novel component's contribution.

major comments (3)
  1. [Table 1 and Section 4.1] The ablation in Table 1 omits the decisive condition K=576 with contrastive decoding (CD) enabled. From Appendix C (Table 8), the K=576 no-CD baseline gives POPE 82.13 and the K=432 no-CD run gives 82.38, i.e., token reduction alone changes accuracy by only +0.25 points, while adding CD to K=432 raises accuracy to 85.74. Without the K=576-with-CD cell, the reported gains cannot be attributed to the token-selection mechanism that is the paper's principal contribution; they may be entirely due to the existing contrastive-decoding component. I ask the authors to run this control and report it in Table 1, and to revisit the claims in Sections 4.2 and 5 accordingly.
  2. [Table 2 and Section 4.2] The headline improvements are not consistent with the full set of results. On adversarial POPE, Qwen-VL with MINT achieves 82.57% accuracy versus 83.53% for VCD, so MINT is worse than the VCD baseline. For LLaVA-1.5-13B on CHAIR (Table 4), MINT worsens CHAIR_I from 5.9 to 6.6 compared to the vanilla model. For MiniGPT-4 on MME (Tables 9-10), MINT causes broad declines in both accuracy and accuracy+ across most subsets, contrary to the paper's claim that MINT consistently improves or maintains performance. These negative results should be explicitly discussed and the abstract's '4% improvement' and '5% more visual points' claims should be qualified to the specific settings where they hold.
  3. [Section 3.3, Eq. (4)-(5)] The token-selection mechanism depends on a single untested assumption: that average attention in the second decoder layer is a reliable, per-generation-token indicator of task-relevant image tokens. The paper acknowledges the risk only indirectly through the K=72 collapse in Table 1, but it does not analyze the stability of the selected token sets across decoding steps or across different questions for the same image. Since the benchmark improvements may be driven by CD rather than by token selection, the paper needs either (i) an analysis showing that the layer-2 proxy selects tokens that are meaningful for the task, or (ii) an explicit statement that token selection is only a lightweight add-on whose benefit is not separately demonstrated.
minor comments (6)
  1. [Eq. (6)] The notation '∗' in logits_phi(y_t|∗, X_tilde_V) and logits_phi(y_t|∗) is ambiguous: the second term is described as text-only logits, but the equation does not make clear that the image tokens are entirely masked. Please use distinct symbols or explicitly define the two arguments.
  2. [Eq. (7)] The definitions of p_phi(y_t|⋆) and p_phi([T]|⋆) use the same star symbol with different meanings in the following sentence; this will confuse the reader. Define a notation for the full-input distribution versus the image-masked distribution.
  3. [Table 2 header] The header contains a typo: 'POPE M ODELS' should be 'POPE MODELS'.
  4. [Table 3 and Table 4] For the long-caption setting (512 new tokens), MINT with K=432 has CHAIR_I 14.2, which is worse than the vanilla model's 16.1 only by a small margin and worse than the K=288 variant. The claim of 'excellence in open-ended captioning' should be tempered, especially since the 13B model shows no improvement on CHAIR_I.
  5. [Section 4.1 (Datasets and Metrics)] The term 'accuracy+' is used without a formal definition beyond 'requires the LVLM to answer both questions correctly.' Please give the exact computation, as it is not standard in the MME literature.
  6. [Figure 3 and Section 3.2] The statement 'the attention sink suddenly emerges in the third layer' is based on a single aggregated visualization over 500 images; no error bars or statistical test are provided, so it is unclear whether the pattern is robust across random subsets or models.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MINT's reported gains are external-benchmark evaluations with fixed hyperparameters; the missing K=576+CD ablation is an attribution gap, not a circular step.

full rationale

The derivation chain is: (1) Sec. 3.2 empirically observes attention redundancy and early-layer image understanding; (2) Sec. 3.3 converts this into a fixed selection rule (Eq. 4-5, top-K by layer-2 attention, 25% token reduction applied uniformly across all models); (3) Sec. 3.4 adds contrastive decoding (Eq. 6-7) with fixed hyperparameters alpha=1, beta=0.1 inherited from the cited prior work; (4) Sec. 4 evaluates against external benchmarks (POPE, MME, CHAIR). No parameter is fitted to a dataset and then re-reported as a prediction of that same dataset; the reported numbers are genuine forward evaluations of a fixed procedure. The load-bearing premise (layer-2 attention ranks task-relevant image tokens) is not guaranteed by construction, and the paper itself documents failures (sharp POPE collapse at K=72 in Table 1, overall MME decline for MiniGPT-4, near-neutral Qwen-VL results), which demonstrates falsifiability rather than self-fulfillment. There is no self-citation chain: the method builds on FastV (Chen et al., 2025), VCD (Leng et al., 2024), PAI (Liu et al., 2025), and ICD (Wang et al., 2024), all by other research groups, and no uniqueness theorem or load-bearing self-citation is invoked. The genuine weakness is ablative attribution: Table 1 omits the K=576 with CD cell, so the contribution of the token-selection mechanism beyond the borrowed contrastive-decoding component is not isolated. That is a completeness/correctness concern about what causes the gains, not an equivalence of any reported result to its own input, and therefore does not constitute circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new entities are introduced. The method rests on four domain assumptions about attention behavior in LVLMs, each borrowed or inferred from prior work or from the paper's own visualizations, plus standard transformer math.

free parameters (4)
  • Image token retention ratio = 75% (i.e., 25% reduction)
    The number of kept tokens K is set to 75% of the original for all models (Section 4.1); the ablation varies K but the main results use 75%.
  • Contrastive decoding weight alpha = 1.0
    Fixed hyperparameter in Eq. 6, reported in Appendix A.2.
  • Adaptive plausibility threshold beta = 0.1
    Fixed hyperparameter in Eq. 7, reported in Appendix A.2.
  • Token selection layer index = 2
    Attention scores for selection are taken from the second decoder layer (Eq. 4); the choice is grounded in the empirical analysis but not swept.
assumptions (5)
  • domain assumption Attention to the system prompt is redundant and indicates attention sink.
    Section 3.2 infers from attention allocation that high system-prompt attention is redundant, and this motivates discarding non-image tokens from the model's focus.
  • domain assumption The model completes image comprehension in the first two decoder layers.
    Section 3.2 and Fig. 3-4 claim the LVLM stabilizes its image recognition in the first two layers; this underlies the choice to extract selection signals from layer 2.
  • domain assumption Second-layer attention to image tokens identifies task-relevant tokens.
    Equation 4 assumes that the average attention each image token receives in layer 2 is a valid salience criterion; no independent validation of this proxy is provided.
  • domain assumption Image-masked logits provide a language-prior distribution to subtract.
    Equation 6 adopts contrastive decoding and assumes that logits with all image tokens masked represent the model's language prior, following VCD.
  • standard math Standard transformer attention and softmax properties.
    Equations 1-3 rely on standard softmax and attention normalization; no new mathematical machinery is introduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction." pith.science (2026). https://pith.science/paper/A3OZFJGQ

@misc{pith2026250200717,
  author       = {Pith},
  title        = {Pith review of: MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A3OZFJGQ}},
  note         = {Machine review of arXiv:2502.00717}
}
read the original abstract

Hallucination has been a long-standing and inevitable problem that hinders the application of Large Vision-Language Models (LVLMs) in domains that require high reliability. Various methods focus on improvement depending on data annotations or training strategies, yet place less emphasis on LLM's inherent problems. To fill this gap, we delve into the attention mechanism of the decoding process in the LVLM. Intriguingly, our investigation uncovers the prevalent attention redundancy within the hierarchical architecture of the LVLM, manifesting as overextended image processing in deep layers and an overabundance of non-essential image tokens. Stemming from the observation, we thus propose MINT, a novel training-free decoding strategy, MItigating hallucinations via tokeN reducTion. Specifically, we dynamically intensify the LVLM's local perception capability by masking its attention to irrelevant image tokens. In addition, we use contrastive decoding that pushes the model to focus more on those key image regions. Our full method aims to guide the model in concentrating more on key visual elements during generation. Extensive experimental results on several popular public benchmarks show that our approach achieves a 4% improvement in mitigating hallucinations caused by distracted perception compared to original models. Meanwhile, our approach is demonstrated to make the model perceive 5% more visual points even though we reduce a suite of image tokens.

Figures

Figures reproduced from arXiv: 2502.00717 by the authors.

Figure 1
Figure 1. The overview of our method MINT. We first add a token selection module inside the LLM Decoder to enable control of the model’s dynamic concentrating on key image regions. Subsequently, we use contrastive decoding to amplify the impact of the selected image tokens, offering reference for logit refinement. The final calibrated logit can lead to more reliable token prediction for improved model performance. that do not… view at source ↗
Figure 3
Figure 3. The attention allocation in different layers. It can be found that the patterns in the first two layers are completely different from the ones in the 3rd and subsequent layers. major semantic information understanding of the image and user’s question in the first two layers, in turn. To figure out how the LVLM attends to the image in each layer when conducting generation, we further visualize the attention between i… view at source ↗
Figure 2
Figure 2. The overall attention allocation over different types of tokens. We randomly select 500 images from MSCOCO (Lin et al., 2014) and ask the LLaVA-1.5-7B model to respond according to the images. We set the maximum number of new tokens to 20 for simplicity and collect the average attention values using Eqn.3. As shown in Fig.2, it can be seen that the LVLM pays the greatest attention to the system prompt, which signifi… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The attention heatmap obtained from the 2nd and the 3rd layer of the LVLM when generating the token “[man]”. We select the heatmap in the first six heads with the largest average attention value. It can be seen that the model’s recognition of the image rapidly stabiliz…
Figure 5
Figure 5. Figure 5: A comparative evaluation result on MMEP . key visual information. In contrast, when sufficient tokens containing rich semantic information are available for selec￾tion, our approach dynamically amplifies key visual points, enabling the model to make more informed decis…
Figure 6
Figure 6. Figure 6: More visualization results of query-to-patch attention map in shallow layers. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: More visualization results of query-to-patch attention map in deep layers. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: One of the cases illustrates the effectiveness of our method. We use the prompt “Please describe the image in detail.” to get the caption. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: One of the cases illustrates the effectiveness of our method. We use the prompt “Please describe the image in detail.” to get the caption. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: One of the cases illustrates the effectiveness of our method. We use the prompt “Please describe the image in detail.” to get the caption. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Tracking positive shifts in visual attention over information-rich query words yields a saliency map that, when used to boost visual and query attention during decoding, reduces object hallucination on CHAIR, POPE, an...

Reference graph

Works this paper leans on

39 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen technical report

    Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023 a

    arXiv 2023

  2. [2]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023 b

    arXiv 2023

  3. [3]

    Y., Bhiwandiwalla, A., Tseng, S.-Y., Olson, M

    Ben Melech Stan, G., Aflalo, E., Rohekar, R. Y., Bhiwandiwalla, A., Tseng, S.-Y., Olson, M. L., Gurwicz, Y., Wu, C., Duan, N., and Lal, V. Lvlm-intrepret: An interpretability tool for large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 8182--8187, 2024

    work page 2024

  4. [4]

    Honeybee: Locality-enhanced projector for multimodal llm

    Cha, J., Kang, W., Mun, J., and Roh, B. Honeybee: Locality-enhanced projector for multimodal llm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 13817--13827, 2024

    2024

  5. [5]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Chen, B., Xu, Z., Kirmani, S., Ichter, B., Sadigh, D., Guibas, L., and Xia, F. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 14455--14465, 2024

    work page 2024

  6. [6]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., and Chang, B. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, pp.\ 19--35. Springer, 2025

    work page 2025

  7. [7]

    E., Stoica, I., and Xing, E

    Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90\ URL https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  8. [8]

    Vision transformers need registers

    Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

Show all 39 references
  1. [9]

    Intern LM - XC omposer2-4 KHD : A pioneering large vision-language model handling resolutions from 336 pixels to 4k HD

    Dong, X., Zhang, P., Zang, Y., Cao, Y., Wang, B., Ouyang, L., Zhang, S., Duan, H., Zhang, W., Li, Y., Yan, H., Gao, Y., Chen, Z., xinyue zhang, Li, W., Jingwen, L., Wang, W., Chen, K., He, C., ZHANG, X., Dai, J., Qiao, Y., Lin, D., and Wang, J. Intern LM - XC omposer2-4 KHD : ...

    2024

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  3. [11]

    Mme: A comprehensive evaluation benchmark for multimodal large language models

    Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023

    2023 arXiv

  4. [12]

    A., Ma, W.-C., and Krishna, R

    Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N. A., Ma, W.-C., and Krishna, R. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp.\ 148--166. Springer, 2025

    2025

  5. [13]

    Detecting and preventing hallucinations in large vision language models

    Gunjal, A., Yin, J., and Bas, E. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 18135--18143, 2024

    2024

  6. [14]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  7. [15]

    Lisa: Reasoning segmentation via large language model

    Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., and Jia, J. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9579--9589, 2024

    2024

  8. [16]

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding

    Leng, S., Zhang, H., Chen, G., Li, X., Lu, S., Miao, C., and Bing, L. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 13872--13882, 2024

    2024

  9. [17]

    L., Holtzman, A., Fried, D., Liang, P., Eisner, J., Hashimoto, T

    Li, X. L., Holtzman, A., Fried, D., Liang, P., Eisner, J., Hashimoto, T. B., Zettlemoyer, L., and Lewis, M. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2023

  10. [18]

    X., and Wen, J.-R

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., and Wen, J.-R. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 292--305, 2023 b

    2023

  11. [19]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp....

    2014

  12. [20]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 26296--26306, 2024 a

    2024

  13. [21]

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., and Lee, Y. J. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024 b . URL https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  14. [22]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. Advances in neural information processing systems, 36, 2024 c

    2024

  15. [23]

    Paying more attention to image: A training-free method for alleviating hallucination in lvlms

    Liu, S., Zheng, K., and Chen, W. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision, pp.\ 125--140. Springer, 2025

    2025

  16. [24]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.\ 8748--8763. PMLR, 2021

    2021

  17. [25]

    A., Burns, K., Darrell, T., and Saenko, K

    Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., and Saenko, K. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.\ 4035--4045, 2018

    2018

  18. [26]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  19. [27]

    Mitigating hallucinations in large vision-language models with instruction contrastive decoding

    Wang, X., Pan, J., Ding, L., and Biemann, C. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint arXiv:2403.18715, 2024

    2024 arXiv

  20. [28]

    Dilu: A knowledge-driven approach to autonomous driving with large language models

    Wen, L., Fu, D., Li, X., Cai, X., Tao, M., Cai, P., Dou, M., Shi, B., He, L., and Qiao, Y. Dilu: A knowledge-driven approach to autonomous driving with large language models. In The Twelfth International Conference on Learning Representations, 2023

    2023

  21. [29]

    Q-instruct: Improving low-level visual abilities for multi-modality foundation models

    Wu, H., Zhang, Z., Zhang, E., Chen, C., Liao, L., Wang, A., Xu, K., Li, C., Hou, J., Zhai, G., et al. Q-instruct: Improving low-level visual abilities for multi-modality foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp...

    2024

  22. [30]

    and Xie, S

    Wu, P. and Xie, S. V*: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 13084--13094, 2024

    2024

  23. [31]

    D., and Potts, C

    Wu, Z., Arora, A., Wang, Z., Geiger, A., Jurafsky, D., Manning, C. D., and Potts, C. Re FT : Representation finetuning for language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 b . URL https://openreview.net/forum?id=fykjplMc0V

    2024

  24. [32]

    Efficient streaming language models with attention sinks

    Xiao, G., Tian, Y., Chen, B., Han, S., and Lewis, M. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations, 2023

    2023

  25. [33]

    Multi-modal concept alignment pre-training for generative medical visual question answering

    Yan, Q., Duan, J., and Wang, J. Multi-modal concept alignment pre-training for generative medical visual question answering. In Findings of the Association for Computational Linguistics ACL 2024, pp.\ 5378--5389, 2024

    2024

  26. [34]

    Deco: Decoupling token compression from semantic abstraction in multimodal large language models

    Yao, L., Li, L., Ren, S., Wang, L., Liu, Y., Sun, X., and Hou, L. Deco: Decoupling token compression from semantic abstraction in multimodal large language models. arXiv preprint arXiv:2405.20985, 2024

    2024 arXiv

  27. [35]

    Attention prompting on image for large vision-language models

    Yu, R., Yu, W., and Wang, X. Attention prompting on image for large vision-language models. In European Conference on Computer Vision, pp.\ 251--268. Springer, 2025

    2025

  28. [36]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.-T., Sun, M., et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  29. [37]

    Llava-grounding: Grounded visual chat with large multimodal models

    Zhang, H., Li, H., Li, F., Ren, T., Zou, X., Liu, S., Huang, S., Gao, J., Li, C., Yang, J., et al. Llava-grounding: Grounded visual chat with large multimodal models. In European Conference on Computer Vision, pp.\ 19--35. Springer, 2025

    2025

  30. [38]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.