The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
read the original abstract
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Finally, we discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.
This paper has not been read by Pith yet.
Forward citations
Cited by 24 Pith papers
-
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
The Pile is a newly constructed 825 GiB dataset from 22 diverse sources that enables language models to achieve better performance on academic, professional, and cross-domain tasks than models trained on Common Crawl ...
-
V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data
V.O.I.C.E is a new taxonomy that organizes synthetic voice risks into five categories and shows how they interact with exposure, visibility, and legal context using empirical incident data.
-
Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges
LLM agents reach only 35% average checkpoint completion on ten realistic CTF challenges in a new open benchmark with automated partial-credit scoring.
-
Improving Dictionary Learning with Gated Sparse Autoencoders
Gated SAEs decouple which features to use from how large their activations should be, applying the L1 penalty only to selection and thereby eliminating shrinkage while halving the number of firing features needed for ...
-
FedOT: Ownership Verification and Leakage Tracing via Watermarks for Federated LDMs
FedOT introduces chunked watermarks plus latent vector transformation to enable ownership verification, client tracing, and resistance to VAE replacement attacks in federated LDMs.
-
Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types
Harmful generation in LLMs relies on a compact, unified set of weights that alignment compresses and that are distinct from benign capabilities, explaining emergent misalignment.
-
Gaussian Shannon: High-Precision Diffusion Model Watermarking Based on Communication
Gaussian Shannon models diffusion as a noisy channel and uses error-correcting codes plus majority voting to recover watermark bits exactly from perturbed AI-generated images.
-
Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking
FIA uses contrastive concept saliency and temporal-spatial neuron identification to build unified masks that erase multiple target concepts while preserving general generation quality in diffusion models.
-
Unsolved Problems in ML Safety
The paper presents a roadmap that identifies four unsolved problems in ML safety: robustness against hazards, monitoring for hazards, alignment of model goals with human intent, and systemic safety.
-
CTRL: A Conditional Transformer Language Model for Controllable Generation
CTRL is a large conditional transformer language model that uses naturally occurring control codes to steer text generation style and content.
-
We Need No Pixels: Video Manipulation Detection Using Stream Descriptors
Video forgeries are detectable via binary classification on multimedia stream descriptors without pixel analysis.
-
AI Safety as Control of Irreversibility: A Systems Framework for Decision-Energy and Sovereignty Boundaries
AI safety requires stabilizing sovereignty boundaries to stop irreversible decision authority from concentrating in the most efficient AI nodes.
-
FSFM: A Biologically-Inspired Framework for Selective Forgetting of Agent Memory
FSFM is a biologically-inspired selective forgetting framework for LLM agents that claims to boost access efficiency by 8.49%, content quality by 29.2% signal-to-noise, and eliminate security risks entirely through a ...
-
MalGEN: A Testbed for Modeling and Evaluating Malware Behaviors
MalGEN generates 977 executable malware samples across 1920 settings, with 45.71% evading existing detection engines and exposing gaps in current defenses.
-
Investigating The Security of Modern AI and Cloud Infrastructure
Develops a taxonomy of security interaction levels in AI/cloud infrastructure and demonstrates practical attacks exploiting isolation assumptions.
-
When AI Meets Wall Street: A Survey on Trustworthy AI in Fintech
A survey that proposes a lifecycle-centric framework and the Financial AI Security and Robustness Taxonomy to organize 17 attack subtypes on AI pipelines in finance.
-
STRIKE: A Structured Taxonomy of Cybercrime for Risk, Impact, Knowledge, and Evolution
STRIKE is a proposed unified taxonomy for cybercrimes organized by attack vectors, tactics, societal impact, detection methods, and mitigation approaches.
-
Toward Accountable AI-Generated Content on Social Platforms: Steganographic Attribution and Multimodal Harm Detection
The proposed steganography-based attribution system with CLIP multimodal fusion achieves robust watermarking under distortions and 0.99 AUC-ROC for harm detection, enabling traceable AI content accountability.
-
We Need Strong Preconditions For Using Simulations In Policy
Societal-scale LLM agent simulations for policy need three preconditions: avoid neutral treatment of marginalized population simulations, require population participation, ensure accountability, plus development and d...
-
Preserving Decision Sovereignty in Military AI: A Trade-Secret-Safe Architectural Framework for Model Replaceability, Human Authority, and State Control
A trade-secret-safe layered architecture is specified to preserve decision sovereignty in military AI by making supplier models replaceable components under state-owned orchestration of policy, audit, and authorization.
-
The coordination gap in frontier AI safety policies
Frontier AI safety policies have a structural coordination gap caused by diffuse benefits and concentrated costs, which can be addressed by adapting precommitment and shared response protocols from other high-risk domains.
-
Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning
The paper outlines considerations, analogies from other fields, and recommendations for ML research norms on synthetic media to mitigate misuse risks without prescribing specific policies.
-
Taxing Artificial Intelligence
Taxation of AI activities can correct externalities, redistribute costs and gains, and support regulation, though instruments like corporate taxes, consumption taxes, and excises vary in feasibility, measurement chall...
-
The Role of Cooperation in Responsible AI Development
Competitive pressures in AI development create collective action problems that may require industry cooperation, with key factors and strategies identified to enable responsible outcomes.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.