REVIEW 11 cited by
Improving Alignment and Robustness with Circuit Breakers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
AI systems can take harmful actions and are highly vulnerable to adversarial attacks. We present an approach, inspired by recent advances in representation engineering, that interrupts the models as they respond with harmful outputs with "circuit breakers." Existing techniques aimed at improving alignment, such as refusal training, are often bypassed. Techniques such as adversarial training try to plug these holes by countering specific attacks. As an alternative to refusal training and adversarial training, circuit-breaking directly controls the representations that are responsible for harmful outputs in the first place. Our technique can be applied to both text-only and multimodal language models to prevent the generation of harmful outputs without sacrificing utility -- even in the presence of powerful unseen attacks. Notably, while adversarial robustness in standalone image recognition remains an open challenge, circuit breakers allow the larger multimodal system to reliably withstand image "hijacks" that aim to produce harmful content. Finally, we extend our approach to AI agents, demonstrating considerable reductions in the rate of harmful actions when they are under attack. Our approach represents a significant step forward in the development of reliable safeguards to harmful behavior and adversarial attacks.
Forward citations
Cited by 11 Pith papers
-
No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks
A new white-box defense, DSA, masks a model's strongest refusal neurons and applies dropout during training so safety is redistributed across many compensatory neurons, cutting attack success under heavy pruning from ...
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
Asymmetry by Design: Boosting Cyber Defenders with Differential Access to AI
A policy framework for shaping access to AI cyber tools to favor defenders, with three access approaches and implementation guidance.
-
Lifelong Safety Alignment for Language Models
A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.
-
Engineering Trustworthy Agentic AI for Critical Systems
A survey claiming that agentic AI trustworthiness is a single cross-domain problem and outlining a framework for graded, certifiable assurance.
-
When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shift
Standard train/test evaluation overstates prompt-attack classifier generalization by 8+ AUC points; leave-one-dataset-out reveals dataset shortcuts that CV hides.
-
Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods
Prepending a Hindi filler paragraph to WMDP-bio questions restores 57.3% accuracy in ELM-unlearned models, showing the unlearning is superficial output suppression rather than true knowledge removal.
-
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Conformal Arbitrage calibrates a score-gap threshold with conformal risk control so that a primary model can act when confident and defer to a guardian otherwise, with the expected guardrail loss bounded by a user-cho...
-
A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
Crescendo multi-turn jailbreak responses are represented by safety-tuned LLMs as benign rather than harmful, which helps explain why single-turn defenses fail.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
-
A Red Teaming Roadmap Towards System-Level Safety
A position paper from Scale AI argues that red teaming research should prioritize product-level safety specifications, realistic attacker models, and system-level monitoring over abstract model-level harm benchmarks.
Discussion (0). Sign in to comment.