REVIEW 31 cited by
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks. First, we demonstrate how to successfully leverage access to logprobs for jailbreaking: we initially design an adversarial prompt template (sometimes adapted to the target LLM), and then we apply random search on a suffix to maximize a target logprob (e.g., of the token "Sure"), potentially with multiple restarts. In this way, we achieve 100% attack success rate -- according to GPT-4 as a judge -- on Vicuna-13B, Mistral-7B, Phi-3-Mini, Nemotron-4-340B, Llama-2-Chat-7B/13B/70B, Llama-3-Instruct-8B, Gemma-7B, GPT-3.5, GPT-4o, and R2D2 from HarmBench that was adversarially trained against the GCG attack. We also show how to jailbreak all Claude models -- that do not expose logprobs -- via either a transfer or prefilling attack with a 100% success rate. In addition, we show how to use random search on a restricted set of tokens for finding trojan strings in poisoned models -- a task that shares many similarities with jailbreaking -- which is the algorithm that brought us the first place in the SaTML'24 Trojan Detection Competition. The common theme behind these attacks is that adaptivity is crucial: different models are vulnerable to different prompting templates (e.g., R2D2 is very sensitive to in-context learning prompts), some models have unique vulnerabilities based on their APIs (e.g., prefilling for Claude), and in some settings, it is crucial to restrict the token search space based on prior knowledge (e.g., for trojan detection). For reproducibility purposes, we provide the code, logs, and jailbreak artifacts in the JailbreakBench format at https://github.com/tml-epfl/llm-adaptive-attacks.
Forward citations
Cited by 31 Pith papers
-
External Data Extraction Attacks against Retrieval-Augmented Large Language Models
SECRET, an adaptive RAG extraction attack that fuses LLM-optimized jailbreak prompts with cluster-guided queries, extracts large portions of private databases from commercial and open LLMs, including ~35% of a sampled...
-
Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models
Persistent SAEs learn per-feature persistence coefficients from reconstruction, splitting features into fast local detectors and slow topic-tracking states that retain prompt-injection signals over long contexts.
-
AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models
Obfuscation alone reaches 76% success against intent-aware defenses; pairing it with emotional manipulation yields 97.6% success under a modeled composite evaluation of 250 prompts.
-
Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
Diffusion language models can revise away harmful intermediate text, and a step-wise internal refusal signal detects jailbreaks cheaply across autoregressive and diffusion models.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
Statutory Construction and Interpretation for Artificial Intelligence
Prompt-based legal canons and iterative rule refinement reduce disagreement among LLM judges about whether a response complies with natural-language rules.
-
JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
JADES judges jailbreak success by decomposing harmful prompts into weighted sub-questions and scoring each part, claiming 98.5% human agreement and showing prior attack success rates are inflated.
-
From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Training
Safe-completions train models to maximize helpfulness within safety constraints using a composite reward, and the paper reports safety and helpfulness gains over refusal-based training in GPT-5.
-
Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian
A multi-agent LLM assistant with DSPy-optimized prompts improved Romanian doctors' written communication quality and patient satisfaction in a live telemedicine deployment, but the evaluation is confounded by self-sel...
-
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
CAMO hides harmful instructions across text and image using masked keywords and math-puzzle clues, making several LVLMs answer banned queries while evading common safety filters.
-
SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
A security-aware prompt compressor that reveals the hidden intent of jailbreak prompts and injects it into the system prompt reduces average attack success from 35% to 1% with negligible overhead.
-
TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts
TwinBreak removes safety alignment by pruning the parameters whose activations differ most between paired harmful and harmless prompts, achieving high attack success on many open-source LLMs.
-
Rewriting the Budget: A General Framework for Black-Box Attacks Under Cost Asymmetry
A framework that adapts the search and gradient-estimation steps of decision-based attacks to minimize total cost under arbitrary ratios of high-cost to low-cost queries.
-
ATAG: AI-Agent Application Threat Assessment with Attack Graphs
ATAG extends the MulVAL attack graph generator with custom Datalog facts and interaction rules to model multi-step attacks on LLM-based multi-agent applications, demonstrated on a trip planner and an email responder.
-
Learning Safety Constraints for Large Language Models
A polytope learned in LLM representation space can detect unsafe regions and steer outputs back to safety at inference time, reducing jailbreak success across several models.
-
GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance
GeneBreaker, a new attack framework, steers DNA language models to generate sequences with over 90% identity to human pathogens, with success rates up to 60% on the largest Evo2 model.
-
Concealment of Intent: A Game-Theoretic Analysis
Intent-hiding adversarial prompting that mixes malicious intents with innocuous skills bypasses prompt and response filters, and a game-theoretic analysis quantifies the attacker's scaling advantage.
-
Lifelong Safety Alignment for Language Models
A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.
-
Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
Combining suffix-window representation finetuning with an ActGrad-pruned surrogate cuts latent-adversarial-training FLOPs per step by 48.1% with only 0.0118% trainable parameters, while accepting higher attack success rates.
-
How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributions
Attacks that break LLMs best are not the ones that improve safety most; a Shapley- and greedy-based framework that selects attack subsets by downstream defender utility outperforms attacker-centric and attribution-onl...
-
Stop Testing Attacks, Start Diagnosing Defenses: The Four-Checkpoint Framework Reveals Where LLM Safety Breaks
A graded-leakage measure raises reported LLM jailbreak success from 22.6% to 52.7%, with output-stage and intent-level defenses emerging as the weak checkpoints.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectories
Large reasoning models frequently ignore explicitly injected ground-truth solutions mid-trajectory and continue to overthink, often ending on incorrect answers.
-
InfoFlood: Jailbreaking Large Language Models with Information Overload
InfoFlood claims near-perfect jailbreak success on four frontier LLMs by rewriting harmful queries into verbose academic prose with fake citations, past-tense framing, and ethical disclaimers, without adversarial suffixes.
-
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
JailFlipBench and JailFlip attacks show that leading LLMs can be made to answer benign-looking questions with plausible but factually wrong and dangerous responses.
-
RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming
RedRFT is a new open-source benchmark with a unified PPO backbone, five reimplemented red teaming baselines, a proposed diversity metric, and ablation insights.
-
Fast Proxies for LLM Robustness Evaluation
Simple prompt-based and embedding-space attacks predict, with rank correlations up to 0.94, how open-source LLMs fare against a six-attack red-teaming ensemble, at roughly one thousandth of the compute.
-
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
Simple prompt-injection attacks against real-world LLM agents like Anthropic Computer Use, MultiOn, and ChemCrow succeed at leaking data and enabling harmful actions.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
-
Adversarial Preference Learning for Robust LLM Alignment
APL iteratively trains an attacker to generate adversarial prompt rewrites and a defender to resist them, using the defender's own preference probabilities as the attack signal.
-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.
Discussion (0). Continue with ORCID to comment.