REVIEW 8 cited by
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Hallucination is often regarded as a major impediment for using large language models (LLMs), especially for knowledge-intensive tasks. Even when the training corpus consists solely of true statements, language models still generate hallucinations in the form of amalgamations of multiple facts. We coin this phenomenon as ``knowledge overshadowing'': when we query knowledge from a language model with multiple conditions, some conditions overshadow others, leading to hallucinated outputs. This phenomenon partially stems from training data imbalance, which we verify on both pretrained models and fine-tuned models, over a wide range of LM model families and sizes.From a theoretical point of view, knowledge overshadowing can be interpreted as over-generalization of the dominant conditions (patterns). We show that the hallucination rate grows with both the imbalance ratio (between the popular and unpopular condition) and the length of dominant condition description, consistent with our derived generalization bound. Finally, we propose to utilize overshadowing conditions as a signal to catch hallucination before it is produced, along with a training-free self-contrastive decoding method to alleviate hallucination during inference. Our proposed approach showcases up to 82% F1 for hallucination anticipation and 11.2% to 39.4% hallucination control, with different models and datasets.
Forward citations
Cited by 8 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models
Social bias is a statistically significant cause of faithfulness hallucinations in LLMs, with anti-stereotypical contexts increasing errors and pro-stereotypical contexts decreasing them.
-
Beyond Facts: Evaluating Intent Hallucination in Large Language Models
The paper proposes a query-centric evaluation of LLM "intent hallucination" via constraint decomposition, but the headline metric comparison is undermined by a self-referential human evaluation design.
-
Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
PhantomCircuit traces knowledge overshadowing to attention circuits during training and prunes circuit edges to recover the overshadowed answer.
-
Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor
A black-box hallucination watchdog that stores previously hallucinated queries in a vector database and flags new queries by embedding similarity and semantic entropy.
-
A Lightweight Multi-Expert Generative Language Model System for Engineering Information and Knowledge Extraction
A graph of small fine-tuned Llama experts with isolated training chunks scores 3x higher exact match than a single 8B model on Cessna repair-manual QA, but with lower ROUGE-L and METEOR.
-
Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.
-
A comprehensive taxonomy of hallucinations in Large Language Models
A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.
Discussion (0). Continue with ORCID to comment.