REVIEW 13 cited by
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This survey addresses the crucial issue of factuality in Large Language Models (LLMs). As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital. We define the Factuality Issue as the probability of LLMs to produce content inconsistent with established facts. We first delve into the implications of these inaccuracies, highlighting the potential consequences and challenges posed by factual errors in LLM outputs. Subsequently, we analyze the mechanisms through which LLMs store and process facts, seeking the primary causes of factual errors. Our discussion then transitions to methodologies for evaluating LLM factuality, emphasizing key metrics, benchmarks, and studies. We further explore strategies for enhancing LLM factuality, including approaches tailored for specific domains. We focus two primary LLM configurations standalone LLMs and Retrieval-Augmented LLMs that utilizes external data, we detail their unique challenges and potential enhancements. Our survey offers a structured guide for researchers aiming to fortify the factual reliability of LLMs.
Forward citations
Cited by 13 Pith papers
-
Conversational AI increases political knowledge as effectively as self-directed internet search
Using survey and experimental data, the paper reports that conversational AI is a common source of political information in the UK and that its effect on belief in true versus false statements statistically does not e...
-
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
A mixture-of-experts LLM trained with reinforcement learning to perform retrieval from its own parametric memory can replace external retrieval in some settings, at lower latency.
-
An Automated Attack Investigation Approach Leveraging Threat-Knowledge-Augmented Large Language Models
ANANKE reports 97.1% TPR and 0.2% FPR for LLM-based attack investigation, but its central evaluation is compromised by knowledge-base overlap with test scenarios.
-
RewardAnything: Generalizable Principle-Following Reward Models
RewardAnything follows natural-language reward principles at inference time and, with the new RABench benchmark, demonstrates that principle-conditioned listwise training beats fixed-preference reward models on held-o...
-
How Significant Are the Real Performance Gains? An Unbiased Evaluation Framework for GraphRAG
A new GraphRAG evaluation framework using graph-grounded questions and bias-correction yields much smaller win rates than earlier reports, casting doubt on reported GraphRAG gains.
-
Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency
A controlled study shows RETRO-style models need a minimum input-neighbor overlap to activate, and paraphrased synthetic context can speed up training by about 40% at a small perplexity cost.
-
SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidence
SelfElicit uses deep-layer attention to automatically highlight relevant evidence sentences in the input context, yielding consistent QA accuracy gains across six instruction-tuned LLMs.
-
Leveraging Knowledge Graphs and LLMs for Structured Generation of Misinformation
A knowledge graph-guided LLM pipeline generates misinformation by replacing objects in true triplets with structurally similar ones, and LLM judges often fail to flag the fakes as fake.
-
GC-KBVQA: A New Four-Stage Framework for Enhancing Knowledge Based Visual Question Answering Performance
A modular zero-shot KB-VQA framework using Grounding DINO, dual captioners, semantic caption filtering, and LLM prompting reports new state-of-the-art numbers on OK-VQA, A-OKVQA, and VQAv2.
-
Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
Explanations increase user reliance on both correct and incorrect LLM answers, while sources and inconsistent explanations reduce overreliance on incorrect answers in a controlled experiment.
-
Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA
Pre-fine-tuning scores on a three-task diagnostic can predict the direction of post-fine-tuning change in small LLMs for cybersecurity QA, but not the magnitude or rank-preservation, which is regime-dependent.
-
Confidence Estimation for Text-to-SQL in Large Language Models
Consistency-based methods are the most reliable confidence signal for text-to-SQL in black-box LLMs, and executing queries against a database adds a useful correctness signal.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
Discussion (0). Continue with ORCID to comment.