Pith. sign in

REVIEW 3 major objections 7 minor 4 cited by

Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HALLUCINOGEN, a new benchmark, shows LVLMs invent absent visual entities and fail near chance on latent medical entities.

desk verdict Genuinely new implicit hallucination benchmark, but the near-random accuracy claims depend on a fragile keyword-based response converter that needs validation. read the letter →

arxiv 2412.20622 v2 pith:T73NNZ6J submitted 2024-12-29 cs.CV cs.AI

Ashish Seth , Dinesh Manocha , Chirag Agarwal This is my paper · ORCID
classification cs.CVcs.AI
keywords hallucinationlargevision-languagemodelsvisualquestionansweringimplicitreasoningbenchmarklatententitieschestX-raychain-of-thoughtprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HALLUCINOGEN is a new benchmark for measuring when large vision-language models (LVLMs) invent visual entities. It sorts entities into salient ones, like a car, that are easy to see, and latent ones, like pneumonia on a chest X-ray, that require domain knowledge. Instead of asking "Is X present?", it asks indirect prompts that force the model to check for the entity first, such as locating, contextual, or counterfactual questions. Across eleven models, the benchmark reports that implicit prompts push accuracy down, and that on latent medical entities every tested model performs only slightly better than chance. The intended contribution is a harder, more realistic test of hallucination than the explicit yes/no benchmarks used previously.

What carries the argument

The carrying mechanism is the "implicit hallucination attack": a handcrafted open-ended prompt that references an entity without asking about its existence, forcing the model to presuppose the entity to answer. For salient entities the prompts ask for localization, surrounding context, or counterfactual removal of the object; for latent entities they ask for the region linked to a disease, radiographic signs, or what would remain if the disease were removed. The benchmark labels each response as yes/no by string-matching negative keywords and a GPT-4o judge, and reports accuracy against ground-truth entity presence labels.

What would settle it

Re-score the latent-entity responses by asking a radiologist or a blinded panel to label each raw response as asserting or not asserting the disease, and compare to the keyword-match labels. If accuracies on pneumonia questions rise substantially above random when correct answers are paraphrased without negative keywords, the near-random claim would not survive.

Watch

Extended reading notes

Core claim

The central claim is that current LVLMs hallucinate much more when the prompt does not name the entity as the direct object of a yes/no question. HALLUCINOGEN builds 90,000 image-prompt pairs from 3,000 salient COCO image-entity pairs and 3,000 latent chest X-ray disease pairs, with 15 implicit attack prompts per image. On these implicit attacks, average accuracy falls 6.8–29.0 percentage points below explicit-attack accuracy; on latent entities, all eleven models, including the medical specialist LLaVA-Med, land close to random guessing. The paper also reports that chain-of-thought reasoning lowers accuracy further, that post-prompting and trained mitigation methods barely move the numbers, and that attention analysis shows the models assigning near-zero weight to visual tokens while generating these responses.

Load-bearing premise

The headline accuracy figures depend on converting free-form answers into yes/no with a keyword search for "no", "not", "never", "none", and "nope" plus a GPT-4o judge, so a correct answer phrased without those words, such as "clear lungs", would be counted as a hallucination.

Editorial extensions

If this is right

  • Explicit yes/no benchmarks such as POPE systematically overstate how grounded LVLMs are.
  • Chain-of-thought prompting, often used to improve reasoning, increases hallucination on these attacks rather than reducing it.
  • Inference-time post-prompting and two trained mitigation strategies, RLHF and LURE, leave accuracy almost unchanged under HALLUCINOGEN attacks.
  • A model that passes explicit object-presence checks can still confidently describe the location, context, or removal of an object that is not there.
  • LVLMs are not currently reliable for answering indirect clinical questions about chest X-rays without external grounding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that if the scoring holds up, current LVLMs should not be trusted in radiology triage when clinicians ask anything beyond a direct "is disease X present" question, even from a medical specialist model.
  • The implicit-attack design transfers naturally to other domains where the target entity is latent, such as pathology, satellite imagery, or video surveillance; a similar near-random failure would suggest a general weakness rather than a medical-image quirk.
  • A testable extension the paper does not run: forcing models to output a grounding statement about the image before answering should reduce the yes-bias, and if it does, the near-zero visual attention result would point to a concrete fix.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces HALLUCINOGEN, a VQA benchmark for evaluating hallucination in large vision-language models (LVLMs). It categorizes visual entities into salient (COCO objects) and latent (NIH Chest X-ray diseases) types and designs three families of implicit hallucination attacks—localization, visual-context, and counterfactual reasoning—with 15 prompts per entity pair, yielding 90,000 image-prompt pairs. The authors evaluate eleven LVLMs and two mitigation strategies, converting open-ended responses to binary Yes/No labels via negative-keyword string matching and a GPT-4o judge. The main findings are that implicit attacks induce more hallucination than explicit ones, latent-entity accuracy is near random (≈50%) for all tested models, Chain-of-Thought increases hallucination, and post-prompting and existing mitigation methods fail to defend against the attacks.

Significance. If the findings are reliable, HALLUCINOGEN would be a valuable complement to existing explicit-attack benchmarks such as POPE and AMBER, especially for latent medical entities and counterfactual reasoning. The benchmark is constructed independently of the tested models: prompts are hand-crafted and ground-truth labels come from external datasets, with no parameters fitted to model responses. The paper also evaluates a wide range of models and defenses and includes a secondary GPT-4o-based response check. The main caveat is that the headline near-random accuracies depend on an unvalidated open-response conversion; the relative explicit-vs-implicit comparison is more robust than the absolute numbers. Releasing the benchmark and evaluation code would make the contribution reproducible.

major comments (3)
  1. [Section 4.1, Tables 7 and 8] The conversion of open-ended responses to binary labels uses only the negative keywords 'no', 'not', 'never', 'none', and 'nope'. This is too narrow: a correct denial such as 'The lungs are clear' or 'without evidence of pneumonia' contains none of these tokens and is scored as an affirmative hallucination, while a hallucinated response like 'There is no doubt the car is on the left' would be scored as correct. Because the latent-entity accuracies in Tables 7 and 8 are all within a few points of 50%, a small systematic parser bias can change the qualitative conclusion that all eleven LVLMs perform 'close to random guessing' (Section 4.2). The authors should validate the binarization on a human-annotated sample and either expand the keyword list or use a human-validated judge before the absolute accuracy numbers are used to support the central claim.
  2. [Section 4.1 and Appendix G.2] The GPT-4o judge check is presented as corroboration, but the paper only states that a 'high correlation' was observed; no correlation coefficient or agreement table is given, and the judge is itself an LVLM with its own potential biases. The prompt in Figure 6 asks whether the response 'suggests or assumes' the presence of a visual entity, which is not the same as establishing factual correctness and may be especially unreliable for latent medical entities. The authors should report agreement between string matching, GPT-4o, and human annotators on a random sample, separately for salient and latent categories.
  3. [Section 4.4, Table 4] The claim that Chain-of-Thought 'causes LVLMs to hallucinate more' is stronger than the data support. In the counterfactual rows, the differences between with and without CoT are small and sometimes within the reported standard errors, and no significance tests are provided. The authors should either restrict the claim to the localization and visual-context tasks, or report confidence intervals or significance tests for the differences across the five runs.
minor comments (7)
  1. [Appendix D] The sentence 'we provide the details on the prompt used for each category in HALLUCINOGEN for salient entities (see in Table 6) and latent entities (see in Table 9)' appears to have the table references reversed: Table 6 contains the latent X-ray/disease prompts and Table 9 contains the salient object prompts.
  2. [Appendix B] Appendix B says the authors evaluate 'eight leading-edge LVLMs,' but the main text evaluates eleven and includes LLaVA-Med, DeepSeek-VL2, and Gemini; the appendix list and count should be updated.
  3. [Section 4.1 and Appendix G.2] The paper reports a 'high correlation' between string matching and the GPT-4o judge but does not give the correlation coefficient or an agreement table; please include the actual numbers.
  4. [Section 3.1] Please state how many unique images and how many ground-truth 'Yes' versus 'No' queries are contained in the 90,000 image-prompt pairs, so that the 'close to random guessing' claim can be interpreted in light of the class balance.
  5. [Section F] The sentence 'All our experiments are conducted on one NVIDIA A6000 GPUs' should be singular, and the statement that manually created prompts 'avoid any potential harm or biases' overstates what manual curation can guarantee.
  6. [Table 7] Table 7 reports latent-entity accuracies without standard deviations, whereas the nearby tables include them; please add the error bars or explain the omission.
  7. [General] No code or data availability statement is provided; for a benchmark paper, releasing the prompt templates, benchmark splits, and evaluation scripts is important for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HALLUCINOGEN is an externally grounded empirical benchmark with no fitted parameters or self-cited uniqueness claims.

full rationale

The paper is an empirical evaluation, not a derivation. The HALLUCINOGEN benchmark is constructed from external ground truth (MS-COCO for salient entities, NIH Chest X-ray for latent entities), with hand-crafted prompts that do not depend on any LVLM output. No parameters are fitted to model responses, and no prediction is reverse-engineered from the data. The evaluation converts open-ended responses to binary labels via keyword matching and an LLM-as-judge; while this operationalization may introduce measurement bias (e.g., a correct answer phrased 'clear lungs' lacking the negative keywords would be scored as a hallucination), that is a validity concern about the metric, not a circularity: the ground-truth labels are external and the metric is not defined in terms of the target claim. The paper cites prior benchmarks for its evaluation conventions (POPE, NOPE) rather than relying on self-citations; no load-bearing argument invokes a uniqueness theorem or an ansatz from the authors' own prior work. The limitation section (Sec. 6) lists standard scope limitations and does not concede any circular step. Accordingly, the central claim that implicit attacks induce more hallucination than explicit attacks is not forced by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on assumptions about the completeness of external annotation datasets, the fidelity of the response-to-label conversion, and the interpretive rule that discussing an absent object's properties counts as asserting its presence. No free parameters are fitted to model outputs; the only hand-chosen numeric input is the negative keyword list used in evaluation.

free parameters (1)
  • Negative keyword list for string matching = ["no", "not", "never", "none", "nope"]
    Hand-chosen set used to binarize open-ended responses in Section 4.1; it directly determines reported accuracy and can misclassify valid negative phrases like 'clear' or 'without'.
assumptions (5)
  • domain assumption MS-COCO object annotations are exhaustive enough to treat unlisted entities as absent from an image.
    Section 3.1 sources 3,000 salient entity pairs from MS-COCO; if an entity is absent from the annotations, the paper assumes it is not in the image, but COCO is known to have incomplete instance annotations.
  • domain assumption NLP-extracted ChestX-ray8 labels are reliable enough to determine disease absence for latent entities.
    Section 3.1 uses the NIH Chest X-ray test set labels, which are produced by NLP from radiology reports with only an estimated >90% accuracy, so disease-absent labels may include false negatives.
  • domain assumption Open-ended responses can be faithfully converted to Yes/No labels by the chosen keyword matching, with LLM-as-judge as validation.
    Section 4.1 evaluation methodology; if this conversion fails, all reported accuracy numbers shift.
  • domain assumption A response that discusses the properties, position, or surrounding context of an absent entity constitutes a hallucination, even if the model does not explicitly say the entity exists.
    Section 3.2 defines the counterfactual and visual context attacks; this interpretive rule is reasonable but affects how many responses are scored as hallucinations.
  • standard math The 50% accuracy level corresponds to random guessing for binary labels.
    For two balanced answer classes, 50% is chance level; this is invoked in Section 4.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models." pith.science (2026). https://pith.science/paper/T73NNZ6J

@misc{pith2026241220622,
  author       = {Pith},
  title        = {Pith review of: Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T73NNZ6J}},
  note         = {Machine review of arXiv:2412.20622}
}
read the original abstract

Large Vision-Language Models (LVLMs) have demonstrated remarkable performance in complex multimodal tasks. However, these models still suffer from hallucinations, particularly when required to implicitly recognize or infer diverse visual entities from images for complex vision-language tasks. To address this challenge, we propose HALLUCINOGEN, a novel visual question answering (VQA) benchmark that employs contextual reasoning prompts as hallucination attacks to evaluate the extent of hallucination in state-of-the-art LVLMs. Our benchmark provides a comprehensive study of the implicit reasoning capabilities of these models by first categorizing visual entities based on the ease of recognition in an image as either salient (prominent, visibly recognizable objects such as a car) or latent entities (such as identifying a disease from a chest X-ray), which are not readily visible and require domain knowledge or contextual reasoning for accurate inference. Next, we design hallucination attacks for both types of entities to assess hallucinations in LVLMs while performing various vision-language tasks, such as locating or reasoning about specific entities within an image, where models must perform implicit reasoning by verifying the existence of the queried entity within the image before generating responses. Finally, our extensive evaluations of eleven LVLMs, including powerful open-source models (like LLaMA-3.2 and DeepSeek-V2), commercial models like Gemini, and two hallucination mitigation strategies across multiple datasets, demonstrate that current LVLMs remain susceptible to hallucination attacks.

Figures

Figures reproduced from arXiv: 2412.20622 by the authors.

Figure 1
Figure 1. Examples of different object hallucination attacks, where hallucination prompts from HALLUCINOGEN (right) are able to make the LVLM hallucinate response. (Left) When explicitly asked to identify a non-existent object, such as “person,” LVLMs like LLaVA1.5 (Liu et al., 2024b) generate a correct response. (Right) However, in the case of an implicit object hallucination attack, where the question requires first implici… view at source ↗
Figure 2
Figure 2. Illustration of various types of hallucination attacks in HALLUCINOGEN. We broadly define two categories of hallucination attacks: explicit and implicit attacks. An explicit attack involves directly prompting LVLMs to accurately identify the presence or absence of existing or non-existing visual entity. In contrast, an implicit attack employs more complex queries that do not explicitly inquire about a specific visua… view at source ↗
Figure 3
Figure 3. We benchmark eleven state-of-the-art LVLMs on the HALLUCINOGEN. Using image-entity pairs categorized as (top) salient and (bottom) latent entities, we evaluate these LVLMs across diverse tasks, including Localization (LOC), Visual Context (VC), and Counterfactual reasoning (CF). Lower accuracy reflects incorrectness in inferring the presence or absence of an object, which correlates with a higher degree of object ha… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparing attention scores for visual, query, and previously generated tokens while predicting the next tokens. The (left) plot illustrates the trend in attention scores for local￾ization tasks, while the (right) plot depicts the trend for coun￾terfactual reasoning tas…
Figure 5
Figure 5. Figure 5: Error Analysis on the incorrect responses gener￾ated by Qwen2VL (Yang et al., 2024) when evaluated across HALLUCINOGEN attack on diverse vision-language tasks. calization → Counterfactual), there is a steady rise in the number of “Yes” responses (72.2%–96.2%), while th…
Figure 6
Figure 6. Figure 6: Prompt used for the GPT-4o to convert open-ended responses into “Yes” or “No” Explicit Object Hallucination Attacks Question: Is the chair present in the image? Ground Truth: No No. There is no chair in the image Expected Response No, there is no chair in the image Gen…
Figure 7
Figure 7. Figure 7: Comparison of responses generated by LlaMa [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: We benchmark eleven state-of-the-art LVLMs on the HALLUCINOGEN. Using image-entity pairs categorized as (top) salient and (bottom) latent entities, we evaluate these LVLMs across diverse tasks, including Localization (LOC), Visual Context (VC), and Counterfactual reaso…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prior Bias in Vision Language Models on UML Diagram Interpretation

    cs.CV 2026-07 conditional novelty 6.5 of 10

    Reversing only the UML relation arrow while keeping class names and layout fixed cuts open-source VLM relation accuracy by about 33%, revealing prior-over-vision bias.

  2. ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

    cs.LG 2026-04 conditional novelty 6.5 of 10

    At matched accuracy, 85 of 210 open-weight LLM pairs have disjoint severity-tail slopes, so error rate alone cannot rank catastrophic-failure risk.

  3. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

  4. A comprehensive taxonomy of hallucinations in Large Language Models

    cs.CL 2025-08 conditional novelty 2.0 of 10

    A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.

Reference graph

Works this paper leans on

44 extracted references · 17 canonical work pages · cited by 4 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, and Erwin Cornejo. 2024. Hallumeasure: Fine-grained hallucination measurement using chain-of-thought reasoning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15020--15037

    work page 2024

  3. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

  4. [4]

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023

    arXiv 2023

  5. [5]

    S \'e bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv

    work page 2023

  6. [6]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology

    work page 2024

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv

    work page 2024

  8. [8]

    Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. 2023. https://arxiv.org/abs/2305.04790 Multimodal-gpt: A vision and language model for dialogue with humans . Preprint, arXiv:2305.04790

    arXiv 2023

Show all 44 references
  1. [9]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. 2023. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. ar...

    2023 arXiv

  2. [10]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. 2024. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In CVPR

    2024

  3. [11]

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3...

    2018

  4. [12]

    Hongyu Hu, Jiyuan Zhang, Minyi Zhao, and Zhenbang Sun. 2023. Ciem: Contrastive instruction evaluation method for better instruction tuning. arXiv preprint arXiv:2309.02301

    2023 arXiv

  5. [13]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Info...

    2023

  6. [14]

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In CVPR

    2024

  7. [15]

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In CVPR

    2024

  8. [16]

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2024. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36

    2024

  9. [17]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. arXiv

    2023

  10. [18]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, ...

    2014

  11. [19]

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024 a . A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253

    2024 arXiv

  12. [20]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 b . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  13. [21]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. NeurIPS

    2023

  14. [22]

    Shi Liu, Kecheng Zheng, and Wei Chen. 2025. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision, pages 125--140. Springer

    2025

  15. [23]

    Holy Lovenia, Wenliang Dai, Samuel Cahyawijaya, Ziwei Ji, and Pascale Fung. 2023. Negative object presence evaluation (nope) to measure object hallucination in vision-language models. arXiv

    2023

  16. [24]

    Junyu Luo, Cao Xiao, and Fenglong Ma. 2023. Zero-resource hallucination prevention for large language models. arXiv preprint arXiv:2309.02654

    2023 arXiv

  17. [25]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, and Trevor Darrell. 2023 a . Aligning large multimodal models with factually augmented rlhf. arXiv

    2023

  18. [26]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2023 b . Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525

    2023 arXiv

  19. [27]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  20. [28]

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. 2023. An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv

    2023

  21. [29]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  22. [30]

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. 2017. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In CVPR

    2017

  23. [31]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  24. [32]

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang,...

    2024 arXiv

  25. [33]

    Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. 2024. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models. IEEE TPAMI

    2024

  26. [34]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  27. [35]

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178

    2023 arXiv

  28. [36]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2024

  29. [37]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A survey on multimodal large language models. National Science Review, page nwae403

    2024

  30. [38]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  31. [39]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  32. [40]

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv

    2023

  33. [41]

    Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. 2024. Visual in-context learning for large vision-language models. arXiv

    2024

  34. [42]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv

    2023

  35. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.