{"id":"9965170f-7ebf-45bb-b6e7-a125d760e81d","arxiv_id":"2412.20622","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"HALLUCINOGEN benchmarks LVLMs with implicit contextual prompts and shows they hallucinate increasingly as prompt difficulty rises, reaching near-random accuracy on latent medical entities.","lead":"The paper introduces HALLUCINOGEN, a benchmark that tests whether vision-language models invent objects when asked indirect questions about images, such as locating or reasoning about things that are not present. It finds that these models hallucinate far more under such indirect prompts than under simple yes/no questions, and perform near randomly on medical X-ray tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Near-random latent-entity accuracies hinge on an unvalidated keyword binarization; correct absence answers like 'clear lungs' that contain none of the five negative keywords are scored as hallucinations.","rationale":"The reader and I identify the same load-bearing point: the accuracy numbers that support the strongest claim are produced by an unvalidated automatic conversion of free-form text into Yes/No labels. The benchmark idea is genuinely novel and the yes-bias pattern is plausible, but the Table 7 latent accuracies sit so close to 50% that even a small systematic parser error would move the central result. The paper's GPT-4o cross-check does not settle this because it is another automatic system rather than a human-validated gold standard. A human annotation study of a modest response sample would directly quantify the parser's error rate and show whether the near-random conclusion survives. Since this concern is exactly what the reader's conditional verdict already requires, I do not propose changing the verdict.","tokens_in":16341,"tokens_out":10976,"duration_ms":107673,"concrete_test":"Draw a stratified random sample of 1,000 model responses covering all 11 models, both entity types, and the three tasks, and have two independent annotators label each response as affirming or not affirming the queried entity, with adjudication of disagreements. Compare human labels against the string-matching and GPT-4o labels. If the mean absolute accuracy difference on latent entities exceeds 5 percentage points, or if the model ordering in Table 7 or Figure 3 changes, recompute the reported accuracies and the explicit-versus-implicit gap before accepting the near-random claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that all eleven LVLMs score near random on latent-entity HALLUCINOGEN attacks depends on converting open-ended responses into binary Yes/No labels. Section 4.1 does this by string-matching only the negative keywords 'no', 'not', 'never', 'none', and 'nope'; any response lacking these tokens is treated as an affirmation of presence. Because the reported hallucination attacks target absent entities (ground truth 'No' in Figs. 1 and 2), a correct answer phrased as 'The lungs are clear' or 'without evidence of pneumonia' would be scored as a hallucination. The paper reports a high correlation with a GPT-4o judge, but that judge is itself an unvalidated LVLM, and the appendix prompt asks only whether the response 'suggests or assumes' presence; no human agreement study is reported. Correlation between two automatic converters does not establish correctness. The Table 7 latent accuracies are all within about three points of 50%, so even a small parser bias can move models across the 'close to random guessing' threshold used in the headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HALLUCINOGEN, a VQA benchmark for evaluating hallucination in large vision-language models (LVLMs). It categorizes visual entities into salient (COCO objects) and latent (NIH Chest X-ray diseases) types and designs three families of implicit hallucination attacks—localization, visual-context, and counterfactual reasoning—with 15 prompts per entity pair, yielding 90,000 image-prompt pairs. The authors evaluate eleven LVLMs and two mitigation strategies, converting open-ended responses to binary Yes/No labels via negative-keyword string matching and a GPT-4o judge. The main findings are that implicit attacks induce more hallucination than explicit ones, latent-entity accuracy is near random (≈50%) for all tested models, Chain-of-Thought increases hallucination, and post-prompting and existing mitigation methods fail to defend against the attacks.","tokens_in":16442,"tokens_out":16507,"duration_ms":154520,"significance":"If the findings are reliable, HALLUCINOGEN would be a valuable complement to existing explicit-attack benchmarks such as POPE and AMBER, especially for latent medical entities and counterfactual reasoning. The benchmark is constructed independently of the tested models: prompts are hand-crafted and ground-truth labels come from external datasets, with no parameters fitted to model responses. The paper also evaluates a wide range of models and defenses and includes a secondary GPT-4o-based response check. The main caveat is that the headline near-random accuracies depend on an unvalidated open-response conversion; the relative explicit-vs-implicit comparison is more robust than the absolute numbers. Releasing the benchmark and evaluation code would make the contribution reproducible.","major_comments":[{"comment":"The conversion of open-ended responses to binary labels uses only the negative keywords 'no', 'not', 'never', 'none', and 'nope'. This is too narrow: a correct denial such as 'The lungs are clear' or 'without evidence of pneumonia' contains none of these tokens and is scored as an affirmative hallucination, while a hallucinated response like 'There is no doubt the car is on the left' would be scored as correct. Because the latent-entity accuracies in Tables 7 and 8 are all within a few points of 50%, a small systematic parser bias can change the qualitative conclusion that all eleven LVLMs perform 'close to random guessing' (Section 4.2). The authors should validate the binarization on a human-annotated sample and either expand the keyword list or use a human-validated judge before the absolute accuracy numbers are used to support the central claim.","section":"Section 4.1, Tables 7 and 8"},{"comment":"The GPT-4o judge check is presented as corroboration, but the paper only states that a 'high correlation' was observed; no correlation coefficient or agreement table is given, and the judge is itself an LVLM with its own potential biases. The prompt in Figure 6 asks whether the response 'suggests or assumes' the presence of a visual entity, which is not the same as establishing factual correctness and may be especially unreliable for latent medical entities. The authors should report agreement between string matching, GPT-4o, and human annotators on a random sample, separately for salient and latent categories.","section":"Section 4.1 and Appendix G.2"},{"comment":"The claim that Chain-of-Thought 'causes LVLMs to hallucinate more' is stronger than the data support. In the counterfactual rows, the differences between with and without CoT are small and sometimes within the reported standard errors, and no significance tests are provided. The authors should either restrict the claim to the localization and visual-context tasks, or report confidence intervals or significance tests for the differences across the five runs.","section":"Section 4.4, Table 4"}],"minor_comments":[{"comment":"The sentence 'we provide the details on the prompt used for each category in HALLUCINOGEN for salient entities (see in Table 6) and latent entities (see in Table 9)' appears to have the table references reversed: Table 6 contains the latent X-ray/disease prompts and Table 9 contains the salient object prompts.","section":"Appendix D"},{"comment":"Appendix B says the authors evaluate 'eight leading-edge LVLMs,' but the main text evaluates eleven and includes LLaVA-Med, DeepSeek-VL2, and Gemini; the appendix list and count should be updated.","section":"Appendix B"},{"comment":"The paper reports a 'high correlation' between string matching and the GPT-4o judge but does not give the correlation coefficient or an agreement table; please include the actual numbers.","section":"Section 4.1 and Appendix G.2"},{"comment":"Please state how many unique images and how many ground-truth 'Yes' versus 'No' queries are contained in the 90,000 image-prompt pairs, so that the 'close to random guessing' claim can be interpreted in light of the class balance.","section":"Section 3.1"},{"comment":"The sentence 'All our experiments are conducted on one NVIDIA A6000 GPUs' should be singular, and the statement that manually created prompts 'avoid any potential harm or biases' overstates what manual curation can guarantee.","section":"Section F"},{"comment":"Table 7 reports latent-entity accuracies without standard deviations, whereas the nearby tables include them; please add the error bars or explain the omission.","section":"Table 7"},{"comment":"No code or data availability statement is provided; for a benchmark paper, releasing the prompt templates, benchmark splits, and evaluation scripts is important for reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The benchmark is promising and within scope, but the central empirical results depend on an unvalidated response-conversion procedure. I would ask for human-validated conversion results and class-balance statistics before acceptance, and for the CoT claim to be tempered. The paper does not appear to have serious novelty or attribution problems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on the HALLUCINOGEN paper. The core idea is genuinely new: instead of asking 'Is X present?', it probes LVLMs with implicit prompts—localization, visual context, counterfactual reasoning—about entities that may not be in the image. The benchmark spans salient and latent (medical) entities across 90k prompts, runs eleven models and two mitigation strategies, and reports that implicit attacks, especially counterfactual and latent ones, push accuracies near chance. That finding is plausible and the benchmark is a real step beyond POPE, AMBER, and HallusionBench.\n\nWhat's well done: the task taxonomy is thoughtful, the extension to chest X-ray disease inference is important, and the broad model sweep gives the results generality. The observation that CoT makes hallucination worse is a useful counterpoint to the usual assumption. The paper is honestly written and includes a limitations section.\n\nThe soft spot is the response binarization. Section 4.1 converts open-ended outputs to Yes/No by string-matching only five negative keywords: 'no', 'not', 'never', 'none', 'nope'. A correct answer like 'The lungs are clear' or 'without evidence of pneumonia' contains none of those tokens and is scored as a hallucination. That means the absolute accuracies—particularly the near-random latent numbers—could be understated. The GPT-4o judge correlation is reassuring but the judge is itself an LVLM, and the prompt asks whether the response 'suggests or assumes' presence, not whether it is factually correct. No human agreement study is reported. Since latent accuracies sit near 50%, a small parser bias can move models across the 'close to random guessing' threshold. This doesn't invalidate the relative ordering—implicit vs explicit, counterfactual vs localization—but it blurs the headline claims.\n\nSecondary issues: COCO absence labels are incomplete, NIH disease labels are NLP-extracted (even if ~90% accurate), and no code or data are released, which hurts reproducibility.\n\nBottom line: this is a serious, useful contribution, but the evaluation pipeline needs a validation study (e.g., a human-annotated sample of the conversion) and a public release before the absolute numbers are trusted. I'd send it to review, with major revisions requested on the binarization and label noise. A reading group would get good discussion out of it.","headline":"Genuinely new implicit hallucination benchmark, but the near-random accuracy claims depend on a fragile keyword-based response converter that needs validation.","tokens_in":17057,"tokens_out":3449,"would_cite":true,"duration_ms":32168,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HALLUCINOGEN, a new benchmark, shows LVLMs invent absent visual entities and fail near chance on latent medical entities.","keywords":["hallucination","large vision-language models","visual question answering","implicit reasoning","benchmark","latent entities","chest X-ray","chain-of-thought prompting"],"falsifier":"Re-score the latent-entity responses by asking a radiologist or a blinded panel to label each raw response as asserting or not asserting the disease, and compare to the keyword-match labels. If accuracies on pneumonia questions rise substantially above random when correct answers are paraphrased without negative keywords, the near-random claim would not survive.","tokens_in":16054,"feed_emoji":"🩻","tokens_out":6353,"duration_ms":54954,"temperature":0.7,"pith_summary":"HALLUCINOGEN is a new benchmark for measuring when large vision-language models (LVLMs) invent visual entities. It sorts entities into salient ones, like a car, that are easy to see, and latent ones, like pneumonia on a chest X-ray, that require domain knowledge. Instead of asking \"Is X present?\", it asks indirect prompts that force the model to check for the entity first, such as locating, contextual, or counterfactual questions. Across eleven models, the benchmark reports that implicit prompts push accuracy down, and that on latent medical entities every tested model performs only slightly better than chance. The intended contribution is a harder, more realistic test of hallucination than the explicit yes/no benchmarks used previously.","feed_headline":"All 11 vision-language models fail new hidden-object test","feed_subtitle":"Indirect prompts about absent cars and pneumonias drop accuracy to near chance, even for a medical model.","key_machinery":"The carrying mechanism is the \"implicit hallucination attack\": a handcrafted open-ended prompt that references an entity without asking about its existence, forcing the model to presuppose the entity to answer. For salient entities the prompts ask for localization, surrounding context, or counterfactual removal of the object; for latent entities they ask for the region linked to a disease, radiographic signs, or what would remain if the disease were removed. The benchmark labels each response as yes/no by string-matching negative keywords and a GPT-4o judge, and reports accuracy against ground-truth entity presence labels.","core_discovery":"The central claim is that current LVLMs hallucinate much more when the prompt does not name the entity as the direct object of a yes/no question. HALLUCINOGEN builds 90,000 image-prompt pairs from 3,000 salient COCO image-entity pairs and 3,000 latent chest X-ray disease pairs, with 15 implicit attack prompts per image. On these implicit attacks, average accuracy falls 6.8–29.0 percentage points below explicit-attack accuracy; on latent entities, all eleven models, including the medical specialist LLaVA-Med, land close to random guessing. The paper also reports that chain-of-thought reasoning lowers accuracy further, that post-prompting and trained mitigation methods barely move the numbers, and that attention analysis shows the models assigning near-zero weight to visual tokens while generating these responses.","pith_inferences":["The paper leaves implicit that if the scoring holds up, current LVLMs should not be trusted in radiology triage when clinicians ask anything beyond a direct \"is disease X present\" question, even from a medical specialist model.","The implicit-attack design transfers naturally to other domains where the target entity is latent, such as pathology, satellite imagery, or video surveillance; a similar near-random failure would suggest a general weakness rather than a medical-image quirk.","A testable extension the paper does not run: forcing models to output a grounding statement about the image before answering should reduce the yes-bias, and if it does, the near-zero visual attention result would point to a concrete fix."],"forward_implications":["Explicit yes/no benchmarks such as POPE systematically overstate how grounded LVLMs are.","Chain-of-thought prompting, often used to improve reasoning, increases hallucination on these attacks rather than reducing it.","Inference-time post-prompting and two trained mitigation strategies, RLHF and LURE, leave accuracy almost unchanged under HALLUCINOGEN attacks.","A model that passes explicit object-presence checks can still confidently describe the location, context, or removal of an object that is not there.","LVLMs are not currently reliable for answering indirect clinical questions about chest X-rays without external grounding."],"supporting_citations":[{"why":"POPE; the explicit yes/no object-presence benchmark that HALLUCINOGEN extends and compares against.","marker":"(Li et al., 2023)"},{"why":"NIH ChestX-ray dataset; supplies the 3,000 latent-entity X-ray and disease pairs.","marker":"(Wang et al., 2017)"},{"why":"MS-COCO; supplies the 3,000 salient image-entity pairs for the benchmark.","marker":"(Lin et al., 2014)"},{"why":"NOPE; supplies the negative-keyword string-matching procedure used to convert open-ended answers into yes/no labels.","marker":"(Lovenia et al., 2023)"},{"why":"GPT-4 technical report; the LLM-as-judge model used to cross-check the string-matching labels.","marker":"(Achiam et al., 2023)"},{"why":"Chain-of-thought prompting; the reasoning defense the paper tests and finds increases hallucination.","marker":"(Wei et al., 2022)"},{"why":"LLaVA-Med; the medical-domain expert LVLM that the paper shows drops to near-random accuracy on latent entities.","marker":"(Li et al., 2024)"},{"why":"LURE; a trained hallucination mitigation method that HALLUCINOGEN shows does not restore accuracy.","marker":"(Zhou et al., 2023)"}],"fun_headline_variants":["LVLMs hallucinate on implicit visual queries, even medical models","HALLUCINOGEN benchmark: 11 LVLMs fail on indirect prompts","Vision-language models drop to near chance on hidden entities","Implicit prompts trip up all tested vision-language models","Medical AI also fools itself on unseen X-ray signs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline accuracy figures depend on converting free-form answers into yes/no with a keyword search for \"no\", \"not\", \"never\", \"none\", and \"nope\" plus a GPT-4o judge, so a correct answer phrased without those words, such as \"clear lungs\", would be counted as a hallucination.","fun_headline_variants_meta":{"raw":{"variants":["LVLMs hallucinate on implicit visual queries, even medical models","HALLUCINOGEN benchmark: 11 LVLMs fail on indirect prompts","Vision-language models drop to near chance on hidden entities","Implicit prompts trip up all tested vision-language models","Medical AI also fools itself on unseen X-ray signs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00064,"raw_usage":{"total_tokens":2962,"prompt_tokens":975,"completion_tokens":1987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":1901}},"tokens_in":591,"tokens_out":1987,"duration_ms":14541,"temperature":1.0,"reasoning_tokens":1901,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:14:58.683569+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-score the latent-entity responses by asking a radiologist or a blinded panel to label each raw response as asserting or not asserting the disease, and compare to the keyword-match labels. If accuracies on pneumonia questions rise substantially above random when correct answers are paraphrased without negative keywords, the near-random claim would not survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"NIH ChestX-ray dataset; supplies the 3,000 latent-entity X-ray and disease pairs."}],"review_version":1}