REVIEW 34 cited by
Deliberative Alignment: Reasoning Enables Safer Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As large-scale language models increasingly impact safety-critical domains, ensuring their reliable adherence to well-defined principles remains a fundamental challenge. We introduce Deliberative Alignment, a new paradigm that directly teaches the model safety specifications and trains it to explicitly recall and accurately reason over the specifications before answering. We used this approach to align OpenAI's o-series models, and achieved highly precise adherence to OpenAI's safety policies, without requiring human-written chain-of-thoughts or answers. Deliberative Alignment pushes the Pareto frontier by simultaneously increasing robustness to jailbreaks while decreasing overrefusal rates, and also improves out-of-distribution generalization. We demonstrate that reasoning over explicitly specified policies enables more scalable, trustworthy, and interpretable alignment.
Forward citations
Cited by 34 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment
HARC couples harmfulness and refusal directions at prompt and response positions, yielding the best robustness-capability-usability trade-off among major safety methods.
-
Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs
TRIAL, a multi-turn jailbreak method that frames harmful actions as lesser-evil ethical choices, reports high jailbreak success rates across open and closed LLMs.
-
Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
Teacher language models transfer behavioral traits to students fine-tuned on semantically unrelated number, code, and reasoning traces, especially when the two models share initialization.
-
Underwriting the Agent Economy: The Blueprint for an AI Insurance Stack
Affirmative AI-agent insurance with billion-scale limits is achievable by 2030 solely through coordinated industry build-out of an eight-component stack spanning data, CAT models, standards, contracts, underwriting, p...
-
Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safety
Legal and ethical bans on CSAM access and generation break standard AI safety techniques, creating 15 open problems that demand new methods for dataset cleaning, concept fusion prevention, fine-tuning resilience, dete...
-
Constitutional Midtraining: Content Presence Drives Alignment Gains
Constitutional midtraining at 120B scale produces alignment gains that persist through instruction tuning and benign fine-tuning, especially reducing blackmail behavior, with no average capability cost.
-
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
SFT lessons — reason-based training, on-model replay, and wash-out robustness — transfer across toy models, model organisms, and alignment SFT, improving the capability–safety tradeoff.
-
QuantiBias: Benchmarking Quantization-Induced Bias in LLMs
Quantization leaves refusal and multiple-choice bias checks flat while open-ended stereotype endorsement remains high (~24–27% under an independent judge), a gap standard safety evaluations miss.
-
A Geometric Perspective on Stabilizing Value Conflict Resolution
An annealing-inspired chain-of-thought prompt lowers the sharpest loss-landscape curvature and improves moral reasoning benchmark scores in small LLMs.
-
Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models
Zero-RL multi-stage constructive safety alignment with SERL and long-context training lets a 14B model match much larger models on safety without collapsing helpfulness or style.
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
Think Before You Code: Dual Reasoning for the NLSafety-Utility Trade-Off in LLM Code Generation
Dual Reasoning with explicit safety audits improves the new SUDS metric by 1.32x to 3.42x over baselines on code generation benchmarks containing injected harmful keywords.
-
Reasoning Up the Instruction Ladder for Controllable Language Models
RLVR on ~7K verifiable system/user conflict examples teaches LLMs to prioritize higher-priority instructions, improving instruction-hierarchy and safety benchmarks.
-
SafeSearch: Automated Red-Teaming of LLM-Based Search Agents
An automated red-teaming framework and 300-case benchmark show that a single unreliable website can induce unsafe responses in LLM search agents, with attack success rates up to 90.5%.
-
Statutory Construction and Interpretation for Artificial Intelligence
Prompt-based legal canons and iterative rule refinement reduce disagreement among LLM judges about whether a response complies with natural-language rules.
-
IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement
IntentionReasoner adds four-level intent classification and targeted query rewriting to LLM guardrails, reporting state-of-the-art harm detection and near-zero jailbreak success.
-
Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI
Geo-alignment means matching an AI system's output distribution to the locally appropriate distribution for each query, location, and time, and the paper argues spatial structure makes that target learnable.
-
ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants
ASTRA is an automated red-teaming agent that uses knowledge-graph-guided spatial and temporal probing to find 11-66% more safety violations in AI coding assistants than existing tools.
-
Libra: Large Chinese-based Safeguard for AI Content
Libra-Guard, a Chinese-language AI content safety model, reaches 86.8% accuracy on the new 5,720-sample Libra-Test benchmark, beating open-source baselines and approaching GPT-4o and Claude-3.5-Sonnet.
-
SAND: Boosting LLM Agents with Self-Taught Action Deliberation
A self-training framework that uses self-consistency action sampling and execution-guided critiques to teach LLM agents to explicitly deliberate over candidate actions, improving average rewards by about 20% over supe...
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
Think Clearly: Improving Reasoning via Redundant Token Pruning
A training-free test-time method prunes low-attention reasoning tokens from the KV cache, guided by an injected end-of-thinking token, and reports accuracy gains on math competition benchmarks.
-
Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving
An outcome-guided teacher-student framework for VLM driving, where a teacher generates reflective reasoning from ground-truth waypoints, a student distills it, and a separate decoder converts the reasoning into waypoi...
-
Cost of Reasoning in non-English Languages: A Case Study on Japanese
Japanese reasoning-language control is feasible with CPT plus GRPO, but incurs a capability cost and does not free-improve cultural Japanese performance.
-
Towards terahertz nanomechanics
Suspended Lamb-wave resonators in lithium niobate films thinned from 300 nm to 67 nm reach 220 GHz, doubling the prior record.
-
Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
A jailbreak defense that reasons about hidden manipulations in attack prompts, trained with supervised fine-tuning plus entropy-guided reinforcement learning, generalizes to attacks never seen in training.
-
R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge
Adding an explicit 'is this harmful?' step to the reasoning chain, trained on just 1,000 examples, substantially reduces harmful responses from reasoning models while roughly preserving benchmark reasoning performance.
-
AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning
AlphaAlign uses pure reinforcement learning with a verifiable safety reward to make LLMs refuse harmful requests with explicit reasoning while preserving helpfulness on benign queries.
-
InfoFlood: Jailbreaking Large Language Models with Information Overload
InfoFlood claims near-perfect jailbreak success on four frontier LLMs by rewriting harmful queries into verbose academic prose with fake citations, past-tense framing, and ethical disclaimers, without adversarial suffixes.
-
SafeCoT: Improving VLM Safety with Minimal Reasoning
Training vision-language models to emit a short rule-based reasoning chain before refusing improves the safety-usefulness balance, with reported gains even at 100 training samples.
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
Crescendo multi-turn jailbreak responses are represented by safety-tuned LLMs as benign rather than harmful, which helps explain why single-turn defenses fail.
-
Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
A broad survey of LLM alignment that catalogs objectives, benchmarks, SFT/RLHF/DPO methods, and safety challenges, without contributing new experimental or theoretical results.
Discussion (0). Sign in to comment.