Pith. sign in

REVIEW 4 major objections 5 minor 10 cited by

A comprehensive taxonomy of hallucinations in Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A wide-ranging taxonomy of LLM hallucination argues, on the strength of a diagonalization argument, that any computable language model will inevitably produce outputs that diverge from a computable ground truth, no matter how it is…

desk verdict A useful but non-novel survey of LLM hallucination taxonomy that borrows its inevitability theorem from prior work and then overstates what that theorem actually proves. read the letter →

arxiv 2508.01781 v1 pith:ZV3ZLKND submitted 2025-08-03 cs.CL cs.AI

Manuel Cossio This is my paper
classification cs.CLcs.AI
keywords LLMhallucinationinevitabilitytheoremcomputabilitytaxonomyfactualityfaithfulnessretrieval-augmentedgenerationbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a wide-ranging taxonomy of LLM hallucination built around a formal thesis: hallucination is an inevitable feature of any computable language model, not a bug that better training can remove. It defines hallucination as a mismatch between a computable model $h$ and a computable ground-truth function $f$, and uses a diagonalization argument to prove that for every computable $h$ there is some $f$ on which it errs at every stage of training. The rest of the report organizes the phenomenon: intrinsic versus extrinsic, factuality versus faithfulness, specific manifestations from factual errors to code and multimodal hallucinations, causes in data, model, and prompt, and evaluation and mitigation strategies. A reader should care because the conclusion reframes the practical goal from eliminating hallucinations to detecting them, grounding outputs, and keeping humans in the loop.

What carries the argument

The load-bearing object is the formal world $G_f = \{(s,f(s)) : s \in S\}$ for a computable ground-truth function $f$, together with the hallucination condition $\forall i \in \mathbb{N}, \exists s \in S$ such that $h^{(i)}(s) \neq f(s)$. The argument is carried by diagonalization: one enumerates the states of all LLMs in a computably enumerable set and constructs $f$ so that at every stage it differs from each model on some input. This is what converts a list of failure examples into a proof that the failures cannot be trained or engineered away.

What would settle it

Run a deployed model on the same prompt one hundred times with nonzero sampling temperature: if the outputs differ, the system is not a total computable function of the prompt, directly breaking the premise on which the inevitability theorem is transferred to practice. Alternatively, find an agreed-upon hallucination that no computable ground-truth function could encode, such as a purely conventional or aesthetic error, which would show the formal definition does not cover all real hallucinations.

Watch

Extended reading notes

Core claim

The report sets out to show that hallucination is an innate limitation of computable LLMs, not a correctable defect. It adopts the formal setup of a ground-truth function $f$ over all finite strings and a model $h$ that is trained on samples from $f$; hallucination is defined by $\forall i \in \mathbb{N}, \exists s \in S$ with $h^{(i)}(s) \neq f(s)$. Using diagonalization, it presents three theorems: computably enumerable families of LLMs hallucinate, they hallucinate on infinitely many inputs, and any individual computable LLM has some computable $f$ on which every training stage errs, plus a corollary that no LLM can self-eliminate hallucination. Around this formal core, the paper organizes the phenomena: intrinsic versus extrinsic errors, factuality versus faithfulness, concrete manifestations from factual fabrications to code and multimodal errors, causes in data, model, and prompt, evaluation benchmarks and metrics, and mitigation through retrieval, tools, guardrails, and human oversight.

Load-bearing premise

The chain of results depends on modeling 'correctness' as a single computable function $f$ defined on every finite string and on treating deployed LLMs as total computable functions; if either fails for a real system, the theorem's conclusion does not automatically apply.

Editorial extensions

If this is right

  • Any computable LLM will hallucinate on at least one input at every training stage, so a perfectly hallucination-free model is not achievable by scaling, better data, or new architectures.
  • Internal self-correction methods such as chain-of-thought or self-refinement cannot by themselves eliminate hallucination; external aids such as retrieval, tools, guardrails, and human oversight are required.
  • Mathematical and logical outputs deserve special scrutiny because hard computational problems are exactly where hallucination is most likely.
  • Safety-critical applications cannot be left to autonomous LLM decisions; human-in-the-loop validation remains necessary.
  • Benchmarks and metrics must be taxonomy-aware and task-specific, because different hallucination types need different detectors and different mitigations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the inevitability theorem is accepted, adding a deterministic post-processing or guardrail layer does not itself escape the result, because the composition of two computable functions is still a computable function; only genuinely external, non-computable oversight would change the formal picture.
  • The report's overlapping taxonomies imply that reported hallucination rates are not comparable across benchmarks until the field agrees on one operational definition; the same error counted as intrinsic by one benchmark could be extrinsic or factual by another.
  • A testable prediction follows: retrieval-augmented systems should hallucinate least on questions whose answers are literally inside the retrieved corpus and most on questions that require synthesizing beyond it, since the external oracle is silent in exactly the places the theorem says errors must appear.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript is a broad survey of LLM hallucination phenomena. It opens with a formal definition and a set of theorems attributed to Xu et al. [100], claiming that hallucination is inevitable for any computable LLM, then presents taxonomies (intrinsic/extrinsic, factuality/faithfulness, and numerous specific categories), causes, human factors, benchmarks, mitigation strategies, and web resources. The practical conclusion is that complete elimination is impossible and that detection, mitigation, and human oversight are required.

Significance. The survey's encyclopedic scope is a strength: Table 2 and Table 3 organize a large literature, Section 7 covers benchmarks and metrics, Section 8 gives balanced mitigation strategies, and Section 9 documents monitoring resources. The manuscript is useful as a reference. However, the central 'inevitability' claim is presented as the intellectual anchor, and the interpretation of Theorem 3 in Section 2.2.2 is not justified by the theorem statement. The paper does not itself prove the theorems, so its credibility rests on faithful reporting; the over-reading of the quantifier structure and the citation errors weaken that anchor. The taxonomy and benchmark content remain valuable independently of the inevitability framing.

major comments (4)
  1. [2.2.2, 10.1] The theorem is reported as: for any individual computable LLM h, there exists a computable ground truth f such that every state of h hallucinates with respect to f. This is an existential statement over f (i.e., ∃f ∀i ∃s h[i](s)≠f(s)). The abstract, Section 2.2.2, and Section 10.1 read this as 'hallucination is an innate and inevitable limitation of any computable LLM regardless of architecture, training, prompting, or data' and 'complete elimination may be impossible regardless of architectural advancements or training refinements.' That inference is invalid: the existential quantifier over f is an adversarial witness chosen after h, not a fixed correctness criterion of a deployment. For any fixed computable ground truth f0, the program h0(s)=f0(s) is a total computable LLM that does not hallucinate with respect to f0 at any state. The theorem therefore shows only that no single model is correct for every computable f; it does not show that a model cannot be made correct for its actual target task. This is the load-bearing step for the paper's central claim and must be corrected.
  2. [2.2.1, 2.2.2] The formal model assumes a single computable ground-truth function f over all finite strings, and it treats real-world LLMs as total computable functions. Real-world correctness for open-ended tasks is not a single computable function, and deployed LLMs with stochastic sampling are not deterministic total functions. Consequently, the transfer of the theorem to 'general problem solvers', 'safety-critical decisions', and 'research and regulations' in Section 2.2.2 requires additional argument that the manuscript does not provide. At minimum, the practical implications should be explicitly conditioned on the formal world defined in Section 2.2.1.
  3. [5.4, 8] Section 5.4 attributes the theoretical inevitability of hallucination to [7] (HalluLens), and Section 8 attributes the same claim to [89] (Reflexion). Neither of these is the source of the theorems; Section 2.2.2 correctly cites [100] (Xu, Jain, and Kankanhalli). Since the inevitability claim is the paper's central thesis, the support must be cited consistently and correctly. These appear to be mechanical citation errors, but they matter for a claim that is used to justify the report's overall framing.
  4. [9.3.1] The text states that 'increased training compute leads to more accurate models, which are inherently less prone to hallucinate' and derives a 'direct implication for fighting hallucinations.' The Epoch AI dashboard reports accuracy on GPQA Diamond and MATH Level 5, not hallucination rates. Accuracy on these benchmarks is not a measure of hallucination, and the inference from benchmark accuracy to reduced hallucination propensity is not established in the cited data. Please rephrase this as a hypothesis for future validation or remove the direct causal claim.
minor comments (5)
  1. [Table 2] The table entries 'F actuality' and 'F aithfulness' contain spacing artifacts; they should read 'Factuality' and 'Faithfulness'.
  2. [9.3.2] The heading uses 'propietary'; this should be 'proprietary'.
  3. [7.1.3] The benchmark name is written as 'F ActScore' with an internal space; standard usage is 'FActScore'.
  4. [References] Reference [17] contains a duplicated author name ('ChenghaoZhu ChenghaoZhu'); the author list should be cleaned up.
  5. [9.1-9.4] The sample figures from web resources lack source documentation and retrieval dates in the captions; adding the source URL and retrieval date for each figure would improve reproducibility.

Circularity Check

1 steps flagged · score 5.0 of 10

The report's central inevitability claim rests on Theorem 3, whose ground-truth witness is constructed after the model to force a mismatch; the 'inevitable' conclusion is true by construction of f, not by failure on any fixed real-world correctness criterion.

  1. self definitional [Section 2.2.2 (Theorem 3 and 'Practical implications of inevitability'); echoed in Section 10.1]
    "Theorem 3: any computable LLM will hallucinate: this theorem generalizes the preceding findings. It asserts that for any individual computable LLM, there exists a computable ground truth function f such that every state of that LLM will hallucinate with respect to f. ... This is demonstrated by constructing a ground truth function f that is specifically designed to contradict the output of every LLM state along a diagonal enumeration of all LLM states and their outputs."

    The witness f in Theorem 3 is chosen after h and is 'specifically designed to contradict' h's outputs. With that f, the defining condition ∀i ∃s h[i](s) ≠ f(s) is satisfied by construction, so 'h hallucinates w.r.t. f' is an artifact of choosing f rather than a finding about h on a fixed task. The report nevertheless converts this existence statement into the load-bearing conclusion that hallucination is 'an innate and inevitable limitation' of computable LLMs 'irrespective of architectural design, learning algorithms, prompting techniques, or the specific training data'. The practical inevitability thus reduces to the theorem's own definition of f: it shows every model has some adversarial truth function on which it errs, not that models err on the actual ground truth of deployed tasks.

full rationale

The paper is a taxonomy/survey and performs no fitting or numerical derivation; most sections are independent literature synthesis. The one load-bearing theoretical step is the inevitability claim imported from [100] and restated in Section 2.2.2 and Section 10.1. Because that theorem's f is adversarially constructed after the model, the 'hallucination' it guarantees is definitional, so the strong practical conclusion goes beyond what the construction supports. This is a single definitional circularity in the central theoretical claim; the rest of the taxonomy (types, causes, benchmarks, mitigations) stands independently of it and is benchmarked against external literature, so the overall circularity score is moderate, not extreme.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper makes no fitted parameters and introduces no new entities. Its central claim is inherited from [100], so the main axioms are: the formal world with a single computable ground truth f captures hallucination, real LLMs are total computable functions, and the cited inevitability proof is correct.

assumptions (3)
  • domain assumption A formal world with a single computable ground-truth function f over all finite strings captures correctness.
    Section 2.2.1 defines hallucination as h[i](s) != f(s); this presumes every input has exactly one correct output and f is computable.
  • domain assumption Real-world LLMs are total computable functions from prompts to outputs.
    Section 2.2.2 states that real-world LLMs are considered a subset of total computable LLMs without argument.
  • standard math The inevitability theorems of Xu et al. [100] are correct and apply to the systems discussed.
    The report relies on the diagonalization proof in [100] and does not reproduce it; if that proof is flawed, the paper's central claim fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A comprehensive taxonomy of hallucinations in Large Language Models." pith.science (2026). https://pith.science/paper/ZV3ZLKND

@misc{pith2026250801781,
  author       = {Pith},
  title        = {Pith review of: A comprehensive taxonomy of hallucinations in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZV3ZLKND}},
  note         = {Machine review of arXiv:2508.01781}
}
read the original abstract

Large language models (LLMs) have revolutionized natural language processing, yet their propensity for hallucination, generating plausible but factually incorrect or fabricated content, remains a critical challenge. This report provides a comprehensive taxonomy of LLM hallucinations, beginning with a formal definition and a theoretical framework that posits its inherent inevitability in computable LLMs, irrespective of architecture or training. It explores core distinctions, differentiating between intrinsic (contradicting input context) and extrinsic (inconsistent with training data or reality), as well as factuality (absolute correctness) and faithfulness (adherence to input). The report then details specific manifestations, including factual errors, contextual and logical inconsistencies, temporal disorientation, ethical violations, and task-specific hallucinations across domains like code generation and multimodal applications. It analyzes the underlying causes, categorizing them into data-related issues, model-related factors, and prompt-related influences. Furthermore, the report examines cognitive and human factors influencing hallucination perception, surveys evaluation benchmarks and metrics for detection, and outlines architectural and systemic mitigation strategies. Finally, it introduces web-based resources for monitoring LLM releases and performance. This report underscores the complex, multifaceted nature of LLM hallucinations and emphasizes that, given their theoretical inevitability, future efforts must focus on robust detection, mitigation, and continuous human oversight for responsible and reliable deployment in critical applications.

Figures

Figures reproduced from arXiv: 2508.01781 by the authors.

Figure 1
Figure 1. Sample visualization of the AI Index, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p034_1.png] view at source ↗
Figure 2
Figure 2. Sample visualization of intelligence versus price, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p034_2.png] view at source ↗
Figure 3
Figure 3. Sample visualization of latency, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p035_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Sample visualization of text to image, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p035_4.png]
Figure 5
Figure 5. Figure 5: Sample visualization of code generation, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p036_5.png]
Figure 6
Figure 6. Figure 6: Sample visualization of audio generation (text to speech, word error rate), retrieved [PITH_FULL_IMAGE:figures/full_fig_p036_6.png]
Figure 7
Figure 7. Figure 7: Sample visualization of intelligence over time, retrieved on 28 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p036_7.png]
Figure 8
Figure 8. Figure 8: Sample visualization of grounded hallucinations rate using Hughes hallucination [PITH_FULL_IMAGE:figures/full_fig_p037_8.png]
Figure 9
Figure 9. Figure 9: Sample visualization of accuracy versus training compute, retrieved on 29 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p039_9.png]
Figure 10
Figure 10. Figure 10: Sample visualization of models with downloadable weights vs proprietary, retrieved [PITH_FULL_IMAGE:figures/full_fig_p040_10.png]
Figure 11
Figure 11. Figure 11: Sample visualization of US models vs non-US, retrieved on 29 June 2025 [PITH_FULL_IMAGE:figures/full_fig_p041_11.png]
Figure 12
Figure 12. Figure 12: Sample visualization of models performance on expert-level mathematics problems, [PITH_FULL_IMAGE:figures/full_fig_p042_12.png]
Figure 14
Figure 14. Figure 14: Sample visualization of mod￾els performance on web search for real￾time information, external knowledge, and grounded citations, retrieved on 9 July 2025 44 [PITH_FULL_IMAGE:figures/full_fig_p044_14.png]
Figure 16
Figure 16. Figure 16: Sample visualization of models performance on how well AI coding assis￾tants understand and generate code across various programming languages and tasks, retrieved on 9 July 2025 Although not focused exclusively on hallucination, LM Arena provides real-world insights …

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When to Trust the Answer: Question-Aligned Semantic Nearest Neighbor Entropy for Safer Surgical VQA

    cs.CV 2025-11 conditional novelty 7.0 of 10

    QA-SNNE adds question-answer alignment via bilateral gating to semantic nearest neighbor entropy, yielding higher AUROC for uncertainty detection in surgical VQA models under both standard and rephrased questions.

  2. PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    PseudoBench shows current LLM agents produce persuasive pseudoscientific reports with near-zero refusal rates and at most 27.4% resistance.

  3. Hypergraph Enterprise Agentic Reasoner over Heterogeneous Business Systems

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    HEAR uses a stratified hypergraph ontology to orchestrate evidence-driven multi-hop reasoning over heterogeneous business systems, reaching 94.7% accuracy on supply-chain root-cause tasks with open-weight models.

  4. Multi-TAP: Multi-criteria Target Adaptive Persona Modeling for Cross-Domain Recommendation

    cs.HC 2026-03 conditional novelty 6.0 of 10

    Modeling intra-domain preference heterogeneity with multi-criteria LLM personas and target-adaptive doppelganger transfer beats prior CDR methods on Amazon domain pairs.

  5. UCPO: Uncertainty-Aware Policy Optimization

    cs.AI 2026-01 conditional novelty 6.0 of 10

    UCPO uses decoupled advantage normalization and a dynamic uncertainty reward to train LLMs to express uncertainty, improving answer reliability (PAQ) but sometimes lowering F1.

  6. Topic Identification in LLM Input-Output Pairs through the Lens of Information Bottleneck

    cs.CL 2025-08 conditional novelty 6.0 of 10

    UDIB, an upper-bounded Deterministic Information Bottleneck clustering algorithm, is applied to LLM prompt and response embeddings to produce topic spaces that sharpen semantic divergence metrics for hallucination analysis.

  7. Divergent Recommendations, Convergent Diagnoses: Cross-Provider Failure-Mode Convergence in AI Commercial Recommendation

    cs.CY 2026-05 unverdicted novelty 4.0 of 10

    Two major AI providers diverge in which brands they recommend but converge on classifying the failure reasons, especially for low-prominence brands.

  8. Rethinking Agentic Reinforcement Learning In Large Language Models

    cs.AI 2026-04 unverdicted novelty 3.0 of 10

    The paper reviews conceptual foundations, methodological innovations, effective designs, critical challenges, and future directions for LLM-based Agentic Reinforcement Learning.

  9. Rethinking Agentic Reinforcement Learning In Large Language Models

    cs.AI 2026-04 unverdicted novelty 2.0 of 10

    The paper surveys the conceptual foundations, methodological innovations, challenges, and future directions of agentic reinforcement learning frameworks that embed cognitive capabilities like meta-reasoning and self-r...

  10. Rethinking Agentic Reinforcement Learning In Large Language Models

    cs.AI 2026-04 unverdicted novelty 2.0 of 10

    This review synthesizes conceptual foundations, methods, challenges, and future directions for agentic reinforcement learning in large language models.

Reference graph

Works this paper leans on

109 extracted references · 28 canonical work pages · cited by 8 Pith papers

  1. [100]

    Hallucination is inevitable: An innate limitation of large language models

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is inevitable: An innate limitation of large language models. arXiv preprint arXiv:2401.11817 , 2024

    arXiv 2024

  2. [7]

    Hallulens: Llm hallucination benchmark

    Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, and Pascale Fung. Hallulens: Llm hallucination benchmark. arXiv preprint arXiv:2504.17550, 2025

    arXiv 2025

  3. [89]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023

    2023

  4. [1]

    Exploring rag solutions to reduce hallucinations in llms

    Samar AboulEla, Paria Zabihitari, Nourhan Ibrahim, Majid Afshar, and Rasha Kashef. Exploring rag solutions to reduce hallucinations in llms. In 2025 IEEE International systems Conference (SysCon) , pages 1–8. IEEE, 2025. 46

    2025

  5. [2]

    Codemirage: Hallucinations in code generated by large language models

    Vibhor Agarwal, Yulong Pei, Salwa Alamir, and Xiaomo Liu. Codemirage: Hallucinations in code generated by large language models. arXiv preprint arXiv:2408.08333 , 2024

    arXiv 2024

  6. [3]

    Improving llm mathematical reasoning capabilities using external tools

    Jack Albright and Sheden Andemicael. Improving llm mathematical reasoning capabilities using external tools. Stanford CS224R, 2025

    2025

  7. [4]

    Make your llm fully utilize the context

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. Make your llm fully utilize the context. Advances in Neural Information Processing Systems, 37:62160–62188, 2024

    2024

  8. [5]

    Agentharm: A benchmark for measuring harmfulness of llm agents

    Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, et al. Agentharm: A benchmark for measuring harmfulness of llm agents. arXiv preprint arXiv:2410.09024, 2024

    arXiv

Show all 109 references
  1. [6]

    The internal state of an llm knows when it’s lying

    Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying. arXiv preprint arXiv:2304.13734, 2023

    2023 arXiv

  2. [8]

    Is incoherence surprising? targeted evaluation of coherence prediction from language models

    Anne Beyer, Sharid Lo´ aiciga, and David Schlangen. Is incoherence surprising? targeted evaluation of coherence prediction from language models. arXiv preprint arXiv:2105.03495, 2021

    2021 arXiv

  3. [9]

    ’it’s reducing a human being to a percentage’ perceptions of justice in algorithmic decisions

    Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shad- bolt. ’it’s reducing a human being to a percentage’ perceptions of justice in algorithmic decisions. In Proceedings of the 2018 Chi conference on human factors in computing systems, pages 1–14, 2018

    2018

  4. [10]

    Fact- controlled diagnosis of hallucinations in medical text summarization

    Suhas BN, Han-Chin Shing, Lei Xu, Mitch Strong, Jon Burnsky, Jessica Ofor, Jor- dan R Mason, Susan Chen, Sundararajan Srinivasan, Chaitanya Shivade, et al. Fact- controlled diagnosis of hallucinations in medical text summarization. arXiv preprint arXiv:2506.00448, 2025

    2025 arXiv

  5. [11]

    Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    S´ ebastien Bubeck, Varun Chadrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    2023

  6. [12]

    This reference does not exist: an explo- ration of llm citation accuracy and relevance

    Courtni Byun, Piper Vasicek, and Kevin Seppi. This reference does not exist: an explo- ration of llm citation accuracy and relevance. In Proceedings of the Third Workshop on Bridging Human–Computer Interaction and Natural Language Processing , pages 28–39, 2024

    2024

  7. [13]

    Hallucinated but factual! in- specting the factuality of hallucinations in abstractive summarization

    Meng Cao, Yue Dong, and Jackie Chi Kit Cheung. Hallucinated but factual! in- specting the factuality of hallucinations in abstractive summarization. arXiv preprint arXiv:2109.09784, 2021

    2021 arXiv

  8. [14]

    Detecting errors through ensembling prompts (deep): an end-to-end llm framework for detecting factual errors

    Alex Chandler, Devesh Surve, and Hui Su. Detecting errors through ensembling prompts (deep): an end-to-end llm framework for detecting factual errors. arXiv preprint arXiv:2406.13009, 2024. 47

    2024 arXiv

  9. [15]

    Softmax bottleneck makes language models unable to represent multi-mode word distributions

    Haw-Shiuan Chang and Andrew McCallum. Softmax bottleneck makes language models unable to represent multi-mode word distributions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , volume 1, 2022

    2022

  10. [16]

    Towards improving faithful- ness in abstractive summarization

    Xiuying Chen, Mingzhe Li, Xin Gao, and Xiangliang Zhang. Towards improving faithful- ness in abstractive summarization. Advances in Neural Information Processing Systems , 35:24516–24528, 2022

    2022

  11. [17]

    Is your llm outdated? a deep look at temporal generalization

    ChenghaoZhu ChenghaoZhu, Nuo Chen, Yufei Gao, Yunyi Zhang, Prayag Tiwari, and Benyou Wang. Is your llm outdated? a deep look at temporal generalization. In Proceed- ings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguist...

    2025

  12. [18]

    Envisioning legal mitigations for llm-based intentional and unintentional harms

    Inyoung Cheong, Aylin Caliskan, and Tadayoshi Kohno. Envisioning legal mitigations for llm-based intentional and unintentional harms. Adm. Law J , 2022

    2022

  13. [19]

    (a) i am not a lawyer, but...: engaging legal experts towards responsible llm policies for legal advice

    Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. (a) i am not a lawyer, but...: engaging legal experts towards responsible llm policies for legal advice. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2454–...

    2024

  14. [20]

    Mind the confidence gap: Overconfidence, calibration, and distractor effects in large language models

    Prateek Chhikara. Mind the confidence gap: Overconfidence, calibration, and distractor effects in large language models. arXiv preprint arXiv:2502.11028 , 2025

    2025

  15. [21]

    Automation bias in intelligent time critical decision support systems

    Mary L Cummings. Automation bias in intelligent time critical decision support systems. In Decision making in aviation , pages 289–294. Routledge, 2017

    2017

  16. [22]

    Bias and unfairness in information retrieval systems: New challenges in the llm era

    Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. Bias and unfairness in information retrieval systems: New challenges in the llm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 6437–6447, 2024

    2024

  17. [23]

    Building guardrails for large language models

    Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng, Wenjie Ruan, and Xiaowei Huang. Building guardrails for large language models. arXiv preprint arXiv:2402.01822, 2024

    2024 arXiv

  18. [24]

    Towards a rigorous science of interpretable machine learning

    Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 , 2017

    2017 arXiv

  19. [25]

    The role of trust in automation reliance

    Mary T Dzindolet, Scott A Peterson, Regina A Pomranky, Linda G Pierce, and Hall P Beck. The role of trust in automation reliance. International journal of human-computer studies, 58(6):697–718, 2003

    2003

  20. [26]

    On the origin of hallucinations in conversational models: Is it the datasets or the models? arXiv preprint arXiv:2204.07931, 2022

    Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, and Siva Reddy. On the origin of hallucinations in conversational models: Is it the datasets or the models? arXiv preprint arXiv:2204.07931, 2022

    2022 arXiv

  21. [27]

    Span-level hallucination detection for llm- generated answers

    Passant Elchafei and Mervet Abu-Elkheir. Span-level hallucination detection for llm- generated answers. arXiv preprint arXiv:2504.18639 , 2025. 48

    2025 arXiv

  22. [28]

    Ragas: Automated evaluation of retrieval augmented generation

    Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demon- strations, pages 150...

    2024

  23. [29]

    Enhancing criti- cal thinking in education by means of a socratic chatbot.arXiv preprint arXiv:2409.05511, 2024

    Lucile Favero, Juan Antonio P´ erez-Ortiz, Tanja K¨ aser, and Nuria Oliver. Enhancing criti- cal thinking in education by means of a socratic chatbot.arXiv preprint arXiv:2409.05511, 2024

    2024 arXiv

  24. [30]

    Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration

    Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration. arXiv preprint arXiv:2402.00367 , 2024

    2024 arXiv

  25. [31]

    Navigating the risks: A survey of security, privacy, and ethics threats in llm-based agents

    Yuyou Gan, Yong Yang, Zhe Ma, Ping He, Rui Zeng, Yiming Wang, Qingming Li, Chunyi Zhou, Songze Li, Ting Wang, et al. Navigating the risks: A survey of security, privacy, and ethics threats in llm-based agents. arXiv preprint arXiv:2411.09523 , 2024

    2024 arXiv

  26. [32]

    Impact of high data quality on llm hallucinations

    Ankush Ramprakash Gautam. Impact of high data quality on llm hallucinations. Inter- national Journal of Computer Applications , 975:8887, 2025

    2025

  27. [33]

    Trueteacher: Learning factual consistency evaluation with large language models

    Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. Trueteacher: Learning factual consistency evaluation with large language models. arXiv preprint arXiv:2305.11171, 2023

    2023 arXiv

  28. [34]

    Logical consistency of large language models in fact-checking

    Bishwamittra Ghosh, Sarah Hasan, Naheed Anjum Arafat, and Arijit Khan. Logical consistency of large language models in fact-checking. arXiv preprint arXiv:2412.16100 , 2024

    2024 arXiv

  29. [35]

    Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daum´ e III, Miro Dudik, and Hanna Wallach. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–16, 2019

    2019

  30. [36]

    True: Re- evaluating factual consistency evaluation

    Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. True: Re- evaluating factual consistency evaluation. arXiv preprint arXiv:2204.04991 , 2022

    2022 arXiv

  31. [37]

    q 2: Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering

    Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. q 2: Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering. arXiv preprint arXiv:2104.08202 , 2021

    2021 arXiv

  32. [38]

    A survey on halluci- nation in large language models: Principles, taxonomy, challenges, and open questions

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on halluci- nation in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Informat...

    2025

  33. [39]

    Leveraging passage retrieval with generative models for open domain question answering

    Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282 , 2020

    2007 arXiv

  34. [40]

    Navigating llm ethics: Advancements, challenges, and future directions

    Junfeng Jiao, Saleh Afroogh, Yiming Xu, and Connor Phillips. Navigating llm ethics: Advancements, challenges, and future directions. arXiv preprint arXiv:2406.18841, 2024. 49

    2024 arXiv

  35. [41]

    Can large language models infer causation from correlation? arXiv preprint arXiv:2306.05836 , 2023

    Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, and Bernhard Sch¨ olkopf. Can large language models infer causation from correlation? arXiv preprint arXiv:2306.05836 , 2023

    2023 arXiv

  36. [42]

    Mitigating llm hallucinations: A comprehensive review of techniques and architectures

    Satyadhar Joshi. Mitigating llm hallucinations: A comprehensive review of techniques and architectures. preprints.org, 2025

    2025

  37. [43]

    Understanding the effects of rlhf on llm generalisation and diversity

    Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on llm generalisation and diversity. arXiv preprint arXiv:2310.06452 , 2023

    arXiv

  38. [44]

    Evaluating the factual consistency of abstractive text summarization

    Wojciech Kry´ sci´ nski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. arXiv preprint arXiv:1910.12840 , 2019

    1910 arXiv

  39. [45]

    Summac: Re- visiting nli-based models for inconsistency detection in summarization

    Philippe Laban, Tobias Schnabel, Paul N Bennett, and Marti A Hearst. Summac: Re- visiting nli-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics , 10:163–177, 2022

    2022

  40. [46]

    Adversarial filters of dataset biases

    Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. Adversarial filters of dataset biases. In Inter- national conference on machine learning , pages 1078–1088. Pmlr, 2020

    2020

  41. [47]

    Hallucination by code generation llms: Taxonomy, benchmarks, mitigation, and challenges

    Yunseo Lee, John Youngeun Song, Dongsun Kim, Jindae Kim, Mijung Kim, and Jaechang Nam. Hallucination by code generation llms: Taxonomy, benchmarks, mitigation, and challenges. arXiv preprint arXiv:2504.20799 , 2025

    2025 arXiv

  42. [48]

    Retrieval- augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Na- man Goyal, Heinrich K¨ uttler, Mike Lewis, Wen-tau Yih, Tim Rockt¨ aschel, et al. Retrieval- augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing ...

    2020

  43. [49]

    Drift: Dynamic rule-based defense with injection isolation for securing llm agents

    Hao Li, Xiaogeng Liu, Hung-Chun Chiu, Dianqi Li, Ning Zhang, and Chaowei Xiao. Drift: Dynamic rule-based defense with injection isolation for securing llm agents. arXiv preprint arXiv:2506.12104, 2025

    2025

  44. [50]

    The dawn after the dark: An empirical study on factuality hallucination in large language models

    Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. The dawn after the dark: An empirical study on factuality hallucination in large language models. arXiv preprint arXiv:2401.03205 , 2024

    2024 arXiv

  45. [51]

    Detecting llm fact-conflicting hallucinations enhanced by temporal-logic-based reasoning

    Ningke Li, Yahui Song, Kailong Wang, Yuekang Li, Ling Shi, Yi Liu, and Haoyu Wang. Detecting llm fact-conflicting hallucinations enhanced by temporal-logic-based reasoning. arXiv preprint arXiv:2502.13416 , 2025

    2025 arXiv

  46. [52]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text sum- marization branches out , pages 74–81, 2004

    2004

  47. [53]

    Truthfulqa: Measuring how models mimic human falsehoods

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 , 2021

    2021 arXiv

  48. [54]

    Teaching models to express their uncer- tainty in words

    Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncer- tainty in words. arXiv preprint arXiv:2205.14334 , 2022. 50

    2022 arXiv

  49. [55]

    Mitigating the alignment tax of rlhf

    Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, et al. Mitigating the alignment tax of rlhf. arXiv preprint arXiv:2309.06256 , 2023

    2023 arXiv

  50. [56]

    Bias unveiled: Investigating so- cial bias in llm-generated code

    Lin Ling, Fazle Rabbi, Song Wang, and Jinqiu Yang. Bias unveiled: Investigating so- cial bias in llm-generated code. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27491–27499, 2025

    2025

  51. [57]

    Exploring and evaluating hallucinations in llm-powered code generation

    Fang Liu, Yang Liu, Lin Shi, Houkun Huang, Ruifeng Wang, Zhen Yang, Li Zhang, Zhongqi Li, and Yuchi Ma. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971 , 2024

    2024

  52. [58]

    Trustworthy llms: a survey and guideline for evaluating large language models’ alignment

    Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374, 2023

    2023 arXiv

  53. [59]

    Maximum hallucination standards for domain-specific large language models

    Tingmingke Lu. Maximum hallucination standards for domain-specific large language models. arXiv preprint arXiv:2503.05481 , 2025

    2025 arXiv

  54. [60]

    ” like having a really bad pa” the gulf between user expectation and experience of conversational agents

    Ewa Luger and Abigail Sellen. ” like having a really bad pa” the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI conference on human factors in computing systems , pages 5286–5297, 2016

    2016

  55. [61]

    A review of faithfulness metrics for hallucination assessment in large language models

    Ben Malin, Tatiana Kalganova, and Nikolaos Boulgouris. A review of faithfulness metrics for hallucination assessment in large language models. IEEE Journal of Selected Topics in Signal Processing, 2025

    2025

  56. [62]

    Coherence boosting: When your pre- trained language model is not paying enough attention

    Nikolay Malkin, Zhen Wang, and Nebojsa Jojic. Coherence boosting: When your pre- trained language model is not paying enough attention. arXiv preprint arXiv:2110.08294, 2021

    2021 arXiv

  57. [63]

    The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision

    Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv:1904.12584 , 2019

    1904 arXiv

  58. [64]

    On faithfulness and factuality in abstractive summarization

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661 , 2020

    2005 arXiv

  59. [65]

    Embracing the illusion of explanatory depth: a strategic framework for using iterative prompting for integrating large language models in health- care education

    Seysha Mehta and Neil Mehta. Embracing the illusion of explanatory depth: a strategic framework for using iterative prompting for integrating large language models in health- care education. Medical Teacher, 47(2):208–211, 2025

    2025

  60. [66]

    Factscore: Fine-grained atomic eval- uation of factual precision in long form text generation

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic eval- uation of factual precision in long form text generation. arXiv preprint arXiv:2305.14251, 2023

    2023 arXiv

  61. [67]

    Confirmation bias: A ubiquitous phenomenon in many guises

    Raymond S Nickerson. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology , 2(2):175–220, 1998. 51

    1998

  62. [68]

    Prevalence of hallucinations and their pathological associations in the general population

    Maurice M Ohayon. Prevalence of hallucinations and their pathological associations in the general population. Psychiatry research, 97(2-3):153–164, 2000

    2000

  63. [69]

    Benchmarking the confidence of large language models in answering clinical ques- tions: cross-sectional evaluation study

    Mahmud Omar, Reem Agbareia, Benjamin S Glicksberg, Girish N Nadkarni, and Eyal Klang. Benchmarking the confidence of large language models in answering clinical ques- tions: cross-sectional evaluation study. JMIR Medical Informatics , 13:e66917, 2025

    2025

  64. [70]

    Llms know more than they show: On the intrinsic representation of llm hallucinations

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. Llms know more than they show: On the intrinsic representation of llm hallucinations. arXiv preprint arXiv:2410.02707 , 2024

    arXiv

  65. [71]

    Confirmation and specificity biases in large language models: An explorative study

    Daniel E O’Leary. Confirmation and specificity biases in large language models: An explorative study. IEEE Intelligent Systems , 40(1):63–68, 2025

    2025

  66. [72]

    To what extent have llms reshaped the legal domain so far? a scoping literature review

    Bogdan Padiu, Radu Iacob, Traian Rebedea, and Mihai Dascalu. To what extent have llms reshaped the legal domain so far? a scoping literature review. Information, 15(11):662, 2024

    2024

  67. [73]

    Med-halt: Medical domain hallucination test for large language models

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Med-halt: Medical domain hallucination test for large language models. arXiv preprint arXiv:2307.15343 , 2023

    2023 arXiv

  68. [74]

    Medhallu: A comprehensive benchmark for detecting medical hallucina- tions in large language models

    Shrey Pandit, Jiawei Xu, Junyuan Hong, Zhangyang Wang, Tianlong Chen, Kaidi Xu, and Ying Ding. Medhallu: A comprehensive benchmark for detecting medical hallucina- tions in large language models. arXiv preprint arXiv:2502.14302 , 2025

    2025 arXiv

  69. [75]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318, 2002

    2002

  70. [76]

    Towards enhancing coherence in extractive summarization: Dataset and experiments with llms

    Mihir Parmar, Hanieh Deilamsalehy, Franck Dernoncourt, Seunghyun Yoon, Ryan A Rossi, and Trung Bui. Towards enhancing coherence in extractive summarization: Dataset and experiments with llms. arXiv preprint arXiv:2407.04855 , 2024

    2024 arXiv

  71. [77]

    Kilt: a benchmark for knowledge intensive language tasks

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, et al. Kilt: a benchmark for knowledge intensive language tasks. arXiv preprint arXiv:2009.02252 , 2020

    2009 arXiv

  72. [78]

    Mitigating exposure bias in large language model distillation: an imitation learning approach

    Andrea Pozzi, Alessandro Incremona, Daniele Tessera, and Daniele Toti. Mitigating exposure bias in large language model distillation: an imitation learning approach. Neural Computing and Applications , pages 1–17, 2025

    2025

  73. [79]

    Reducing extrinsic hallucination in mul- timodal abstractive summaries with post-processing technique

    Shaik Rafi, Lenin Laitonjam, and Ranjita Das. Reducing extrinsic hallucination in mul- timodal abstractive summaries with post-processing technique. Neural Computing and Applications, pages 1–21, 2025

    2025

  74. [80]

    Effects of perceptual fluency on judgments of truth

    Rolf Reber and Norbert Schwarz. Effects of perceptual fluency on judgments of truth. Consciousness and cognition , 8(3):338–342, 1999

    1999

  75. [81]

    Investigating the factual knowledge boundary of large language models with retrieval augmentation

    Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. Investigating the factual knowledge boundary of large language models with retrieval augmentation. arXiv preprint arXiv:2307.11019 , 2023. 52

    2023 arXiv

  76. [82]

    The clinicians’ guide to large language mod- els: A general perspective with a focus on hallucinations

    Dimitri Roustan, Fran¸ cois Bastardot, et al. The clinicians’ guide to large language mod- els: A general perspective with a focus on hallucinations. Interactive journal of medical research, 14(1):e59823, 2025

    2025

  77. [83]

    The misunderstood limits of folk science: An illusion of explanatory depth

    Leonid Rozenblit and Frank Keil. The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive science, 26(5):521–562, 2002

    2002

  78. [84]

    Ares: An auto- mated evaluation framework for retrieval-augmented generation systems

    Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. Ares: An auto- mated evaluation framework for retrieval-augmented generation systems. arXiv preprint arXiv:2311.09476, 2023

    2023 arXiv

  79. [85]

    Evaluating retrieval quality in retrieval-augmented generation

    Alireza Salemi and Hamed Zamani. Evaluating retrieval quality in retrieval-augmented generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages 2395–2400, 2024

    2024

  80. [86]

    Toolformer: Lan- guage models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess ` ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools. Advances in Neural Information Pro- cessing Systems, 36:68539–68...

    2023

  81. [87]

    Questeval: Summarization asks for fact-based evaluation

    Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Pi- wowarski, Jacopo Staiano, and Alex Wang. Questeval: Summarization asks for fact-based evaluation. arXiv preprint arXiv:2103.12693 , 2021

    2021 arXiv

  82. [88]

    Hallucinogen: A benchmark for evaluating object hallucination in large visual-language models

    Ashish Seth, Dinesh Manocha, and Chirag Agarwal. Hallucinogen: A benchmark for evaluating object hallucination in large visual-language models. arXiv preprint arXiv:2412.20622, 2024

    2024 arXiv

  83. [90]

    Retrieval augmentation reduces hallucination in conversation

    Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567 , 2021

    2021 arXiv

  84. [91]

    Large language models encode clinical knowledge

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620(7972):172–180, 2023

    2023

  85. [92]

    On early detection of hal- lucinations in factual question answering

    Ben Snyder, Marius Moisescu, and Muhammad Bilal Zafar. On early detection of hal- lucinations in factual question answering. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 2721–2732, 2024

    2024

  86. [93]

    Systematic biases in llm simulations of debates

    Amir Taubenfeld, Yaniv Dover, Roi Reichart, and Ariel Goldstein. Systematic biases in llm simulations of debates. arXiv preprint arXiv:2402.04049 , 2024

    2024 arXiv

  87. [94]

    Codehalu: Investigating code hallucinations in llms via execution-based verification

    Yuchen Tian, Weixiang Yan, Qian Yang, Xuandong Zhao, Qian Chen, Wen Wang, Ziyang Luo, Lei Ma, and Dawn Song. Codehalu: Investigating code hallucinations in llms via execution-based verification. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 39, pag...

    2025

  88. [95]

    Reasoning about concepts with llms: Inconsistencies abound

    Rosario Uceda-Sosa, Karthikeyan Natesan Ramamurthy, Maria Chang, and Moninder Singh. Reasoning about concepts with llms: Inconsistencies abound. arXiv preprint arXiv:2405.20163, 2024

    2024 arXiv

  89. [96]

    Faithfulness hallucination detection in healthcare ai

    Prathiksha Rumale Vishwanath, Simran Tiwari, Tejas Ganesh Naik, Sahil Gupta, Dung Ngoc Thai, Wenlong Zhao, SUNJAE KWON, Victor Ardulov, Karim Tarabishy, Andrew McCallum, et al. Faithfulness hallucination detection in healthcare ai. In Artifi- cial Intelligence and Data Science...

    2024

  90. [97]

    Videohal- lucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models

    Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. Videohal- lucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models. arXiv preprint arXiv:2406.16338 , 2024

    2024 arXiv

  91. [98]

    Combating multimodal llm hallucination via bottom-up holistic reasoning

    Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, and Tat- Seng Chua. Combating multimodal llm hallucination via bottom-up holistic reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8460– 8468, 2025

    2025

  92. [99]

    An llm can fool itself: A prompt-based adversarial attack

    Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan Kankanhalli. An llm can fool itself: A prompt-based adversarial attack. arXiv preprint arXiv:2310.13345, 2023

    2023 arXiv

  93. [101]

    Llm-driven instruction following: Progresses and concerns

    Wenpeng Yin, Qinyuan Ye, Pengfei Liu, Xiang Ren, and Hinrich Sch¨ utze. Llm-driven instruction following: Progresses and concerns. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts , pages 19–25, 2023

    2023

  94. [102]

    How much should we trust llm-based measures for accounting and finance research? Available at SSRN , 2024

    Minji Yoo. How much should we trust llm-based measures for accounting and finance research? Available at SSRN , 2024

    2024

  95. [103]

    Justilm: Few-shot justification generation for explainable fact-checking of real-world claims

    Fengzhu Zeng and Wei Gao. Justilm: Few-shot justification generation for explainable fact-checking of real-world claims. Transactions of the Association for Computational Linguistics, 12:334–354, 2024

    2024

  96. [104]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 , 2019

    1904 arXiv

  97. [105]

    Knowledge overshadowing causes amalgamated hallucination in large language models

    Yuji Zhang, Sha Li, Jiateng Liu, Pengfei Yu, Yi R Fung, Jing Li, Manling Li, and Heng Ji. Knowledge overshadowing causes amalgamated hallucination in large language models. arXiv preprint arXiv:2407.08039 , 2024

    2024 arXiv

  98. [106]

    The law of knowledge overshadowing: Towards understanding, predicting, and preventing llm hallucination

    Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R Fung, Kathleen McKeown, Chengxiang Zhai, Manling Li, et al. The law of knowledge overshadowing: Towards understanding, predicting, and preventing llm hallucination. arXiv preprint arXiv:2502.16143, 2025

    2025 arXiv

  99. [107]

    Making harmful behaviors unlearnable for large language models

    Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, and Xuanjing Huang. Making harmful behaviors unlearnable for large language models. arXiv preprint arXiv:2311.02105, 2023. 54

    2023 arXiv

  100. [108]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  101. [109]

    Medhallbench: A new benchmark for assessing hallucination in medical large language models

    Kaiwen Zuo and Yirui Jiang. Medhallbench: A new benchmark for assessing hallucination in medical large language models. arXiv preprint arXiv:2412.18947 , 2024. 55

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.