REVIEW 9 cited by
On the Risk of Misinformation Pollution with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact on information-intensive applications, particularly Open-Domain Question Answering (ODQA) systems. We establish a threat model and simulate potential misuse scenarios, both unintentional and intentional, to assess the extent to which LLMs can be utilized to produce misinformation. Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of ODQA systems. To mitigate the harm caused by LLM-generated misinformation, we explore three defense strategies: prompting, misinformation detection, and majority voting. While initial results show promising trends for these defensive strategies, much more work needs to be done to address the challenge of misinformation pollution. Our work highlights the need for further research and interdisciplinary collaboration to address LLM-generated misinformation and to promote responsible use of LLMs.
Forward citations
Cited by 9 Pith papers
-
Information Discernment in Large Language Models
LLMs update their stated numeric beliefs almost regardless of source reliability or whether a claim moves them closer to the truth, performing near chance on both dimensions.
-
DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)
DisarmRAG compromises the retriever to inject anti-self-correction instructions, achieving over 90% attack success across six LLMs while evading basic detection.
-
Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows
LLM agents frequently switch correct answers after one round of misleading feedback, and the new WAFER-QA benchmark measures this with web-backed critiques.
-
A Symbolic Adversarial Learning Framework for Evolving Fake News Generation and Detection
SALF uses adversarial LLM prompt-refinement and debate to make fake news harder to detect (detector F1 drops up to 53% in Chinese, 34% in English) while its oracle-informed detector improves by 7.7%.
-
An Audit and Analysis of LLM-Assisted Health Misinformation Jailbreaks Against LLMs
LLM-generated jailbreak prompts elicited health misinformation from GPT-3.5, Llama 3.1-8B, and Gemini 2.0 Flash at high rates, and both LLM judges and simple classifiers detected the resulting texts with high accuracy.
-
Can LLM Improve for Expert Forecast Combination? Evidence from the European Central Bank Survey
A zero-shot LLM prompt beats equal-weighted averaging for one-year ECB SPF forecasts in one regression, but the result is fragile, the comparison is asymmetric, and no code or data are provided.
-
Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training
Reinforcing the two experts most correlated with thinking tokens improves reasoning accuracy and efficiency in MoE large reasoning models, with gains of up to 10 points on AIME benchmarks.
-
Embodied AI: Emerging Risks and Opportunities for Policy Action
A policy analysis arguing that embodied AI risks are real, under-covered by current US/EU/UK frameworks, and best handled through certification, benchmarks, clarified liability, and economic adaptation.
-
A Survey on Data Security in Large Language Models
A survey of data security risks in LLMs that organizes threats, defenses, and evaluation datasets, with notable factual errors in its tables.
Discussion (0). Continue with ORCID to comment.