REVIEW 14 cited by
Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Generative language models have improved drastically, and can now produce realistic text outputs that are difficult to distinguish from human-written content. For malicious actors, these language models bring the promise of automating the creation of convincing and misleading text for use in influence operations. This report assesses how language models might change influence operations in the future, and what steps can be taken to mitigate this threat. We lay out possible changes to the actors, behaviors, and content of online influence operations, and provide a framework for stages of the language model-to-influence operations pipeline that mitigations could target (model construction, model access, content dissemination, and belief formation). While no reasonable mitigation can be expected to fully prevent the threat of AI-enabled influence operations, a combination of multiple mitigations may make an important difference.
Forward citations
Cited by 14 Pith papers
-
InfoOps Bench: A live information operations safety benchmark
Most of 17 frontier models can be co-opted to promote live state-backed information-operation claims, with integrity from 8.8% to 94.5% and large China-specific filtering in several Chinese-developed models.
-
On Capturing the Narrative: Social Media Manipulation Wargaming for Cyberliteracy
A 4,000-NPC, 108-team LLM-bot wargaming competition yielded no gain in participants' bot-detection confidence and an apparent decrease in their emotional discomfort with misinformation.
-
The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance
Conversational AI may evade human epistemic vigilance through 'honest non-signals'—genuine traits like fluency and helpfulness that no longer carry the trust information they do in humans.
-
Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm
A trigger-tag watermark embedded by fine-tuning lets modified LLMs mark their own phishing outputs for cheap detection.
-
Large language models can effectively convince people to believe conspiracies
In three experiments, GPT-4o instructed to argue for a conspiracy raised believers' confidence about as much as it lowered it when arguing against; a truth-constraining prompt and a corrective debrief largely undid the harm.
-
Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"
Across seven AI chatbots, Reddit users report mostly reliability failures, with each chatbot showing a distinct pattern of safety, privacy, and security complaints.
-
EPT Benchmark: Evaluation of Persian Trustworthiness in Large Language Models
A new Persian-Islamic trustworthiness benchmark ranks Claude highest and Qwen lowest across eight LLMs and finds safety is the weakest dimension.
-
When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems
In a 1,000-agent social simulation, decentralized groups of malicious AI agents spread more misinformation and commit more fraud than centralized groups, and they adapt to evade content moderation.
-
Sword and Shield: Uses and Strategies of LLMs in Navigating Disinformation
In a 25-participant Werewolf-style game, all roles used an LLM chatbot strategically, as a sword for disinformation and a shield against it.
-
AI Propaganda factories with language models
Small language models sustain political personas and become more ideologically extreme when replying to counter-arguments, according to a language-model judge.
-
Pruning General Large Language Models into Customized Expert Models
Cus-Prun identifies and removes neurons that are irrelevant to a user's target language, domain, and task, producing specialized expert models without post-training.
-
How does Misinformation Affect Large Language Model Behaviors and Preferences?
MisBench provides a 10.3M-example benchmark of styled, conflict-based misinformation and shows LLMs' detection accuracy depends strongly on conflict type and textual style.
-
Towards a Humanized Social-Media Ecosystem: AI-Augmented HCI Design Patterns for Safety, Agency & Well-Being
Five browser-side design patterns (rewriter, integrity meter, feed curator, micro-withdrawal, recovery mode) aim to give users control over social feeds through an explainable AI intermediary, with evaluation still pending.
-
Social Media Information Operations
A tutorial that frames social media information operations as an optimization problem and surveys the analytics, threat models, and countermeasures that support it.
Discussion (0). Sign in to comment.