REVIEW 12 cited by
Best-of-N Jailbreaking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce Best-of-N (BoN) Jailbreaking, a simple black-box algorithm that jailbreaks frontier AI systems across modalities. BoN Jailbreaking works by repeatedly sampling variations of a prompt with a combination of augmentations - such as random shuffling or capitalization for textual prompts - until a harmful response is elicited. We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3.5 Sonnet when sampling 10,000 augmented prompts. Further, it is similarly effective at circumventing state-of-the-art open-source defenses like circuit breakers. BoN also seamlessly extends to other modalities: it jailbreaks vision language models (VLMs) such as GPT-4o and audio language models (ALMs) like Gemini 1.5 Pro, using modality-specific augmentations. BoN reliably improves when we sample more augmented prompts. Across all modalities, ASR, as a function of the number of samples (N), empirically follows power-law-like behavior for many orders of magnitude. BoN Jailbreaking can also be composed with other black-box algorithms for even more effective attacks - combining BoN with an optimized prefix attack achieves up to a 35% increase in ASR. Overall, our work indicates that, despite their capability, language models are sensitive to seemingly innocuous changes to inputs, which attackers can exploit across modalities.
Forward citations
Cited by 12 Pith papers
-
Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
Evaluating LLM safety with one canonical prompt understates unsafe behavior; across five meaning-preserving reformulations, 5-13% of safe-on-canonical seeds become unsafe, and the union exceeds the worst single form f...
-
A Synonymous Variational Perspective on the Rate-Distortion-Perception Tradeoff
Synset-based reconstruction and synonymous variational inference are claimed to derive the distributional divergence in RDP and unify it with classical rate-distortion theory.
-
Best-of-N through the Smoothing Lens: KL Divergence and Regret Analysis
Smoothed Best-of-N has finite-sample KL and regret bounds under imperfect reward models, and tuning its temperature can make its regret bound beat hard Best-of-N in the overoptimization regime.
-
Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models
A comprehensive reanalysis finds that min-p sampling does not outperform top-p, top-k, or basic sampling once the original data are re-tested and hyperparameter budgets are equalized.
-
Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
A three-stage RL framework (cold start, diversity warm-up, curriculum jailbreak) trains a 7B red-team model that reports SOTA jailbreak ASR and diversity on HarmBench, though the evaluation is compromised by training-...
-
An Example Safety Case for Safeguards Against Misuse
A proposed framework, built around an 'uplift model' that translates red-team safeguard-evasion data into estimated risk, for justifying that AI misuse safeguards keep large-scale harm risk below a threshold.
-
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models
AJailBench is an open benchmark showing that large audio-language models can be jailbroken through TTS-converted text attacks and through subtle acoustic perturbations that preserve speech semantics.
-
SATORI: Static Test Oracle Generation for REST APIs
SATORI statically infers REST API test oracles from OpenAPI specs via LLMs, reporting F1 74.3%, above AGORA+'s 69.3%, with 18 confirmed bugs; the supplied full text, however, is a different paper.
-
MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
Concentrating synthetic preference paraphrases on low-margin pairs gives consistent but small reward-model and alignment gains in single-run experiments, while the abstract's semantic-aware, multi-benchmark claims are...
-
Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models
A survey that organizes audio and video AI security research into adversarial, backdoor, and jailbreak attacks, with extra attention to multimodal large language models.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
-
A Red Teaming Roadmap Towards System-Level Safety
A position paper from Scale AI argues that red teaming research should prioritize product-level safety specifications, realistic attacker models, and system-level monitoring over abstract model-level harm benchmarks.
Discussion (0). Sign in to comment.