REVIEW 5 cited by
Mitigating Large Language Model Hallucination with Faithful Finetuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have demonstrated remarkable performance on various natural language processing tasks. However, they are prone to generating fluent yet untruthful responses, known as "hallucinations". Hallucinations can lead to the spread of misinformation and cause harm in critical applications. Mitigating hallucinations is challenging as they arise from factors such as noisy data, model overconfidence, lack of knowledge, and the generation process itself. Recent efforts have attempted to address this issue through representation editing and decoding algorithms, reducing hallucinations without major structural changes or retraining. However, these approaches either implicitly edit LLMs' behavior in latent space or suppress the tendency to output unfaithful results during decoding instead of explicitly modeling on hallucination. In this work, we introduce Faithful Finetuning (F2), a novel method that explicitly models the process of faithful question answering through carefully designed loss functions during fine-tuning. We conduct extensive experiments on popular datasets and demonstrate that F2 achieves significant improvements over vanilla models and baselines.
Forward citations
Cited by 5 Pith papers
-
Decomposed Entailment for Factuality Checking and Hallucination Detection
HallDetect detects source-grounded hallucinations by decomposing responses into atomic claims and verifying each with a compact NLI model over multi-scale source chunks, outperforming frugal generative baselines on th...
-
Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations
EAACD reduces hallucination in MoE LLMs by contrasting predictions of high-reliability expert groups against hallucination-amplified low-reliability expert groups.
-
Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models
Social bias is a statistically significant cause of faithfulness hallucinations in LLMs, with anti-stereotypical contexts increasing errors and pro-stereotypical contexts decreasing them.
-
ReqBrain: Task-Specific Instruction Tuning of LLMs for AI-Assisted Requirements Generation
ReqBrain, a LoRA-fine-tuned Zephyr-7b-beta model, produces software requirements that human evaluators could not reliably tell apart from human-authored ones, with automatic metrics favoring it over untuned ChatGPT-4o.
-
From Extraction to Synthesis: Entangled Heuristics for Agent-Augmented Strategic Reasoning
A generative strategy system that composes, rather than selects, historical heuristics via embedding-based interference and LLM narrative synthesis, demonstrated on the Meta vs. FTC case.
Discussion (0). Continue with ORCID to comment.