REVIEW 9 cited by
STAIR: Improving Safety Alignment with Introspective Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications. However, existing safety alignment methods typically suffer from safety-performance trade-offs and the susceptibility to jailbreak attacks, primarily due to their reliance on direct refusals for malicious queries. In this paper, we propose STAIR, a novel framework that integrates SafeTy Alignment with Itrospective Reasoning. We enable LLMs to identify safety risks through step-by-step analysis by self-improving chain-of-thought (CoT) reasoning with safety awareness. STAIR first equips the model with a structured reasoning capability and then advances safety alignment via iterative preference optimization on step-level reasoning data generated using our newly proposed Safety-Informed Monte Carlo Tree Search (SI-MCTS). We further train a process reward model on this data to guide test-time searches for improved responses. Extensive experiments show that STAIR effectively mitigates harmful outputs while better preserving helpfulness, compared to instinctive alignment strategies. With test-time scaling, STAIR achieves a safety performance comparable to Claude-3.5 against popular jailbreak attacks. Relevant resources in this work are available at https://github.com/thu-ml/STAIR.
Forward citations
Cited by 9 Pith papers
-
UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation
UniNDM detects sexual intent from early-stage diffusion noise and mitigates it via LLM-generated negative prompts and initial-noise optimization, across U-Net and DiT models.
-
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Inserting a short refusal cue into a harmful response flips many safety guards' verdicts from harmful to unharmful, and targeted masking of a few internal components suppresses most of this failure.
-
Think Before You Code: Dual Reasoning for the NLSafety-Utility Trade-Off in LLM Code Generation
Dual Reasoning with explicit safety audits improves the new SUDS metric by 1.32x to 3.42x over baselines on code generation benchmarks containing injected harmful keywords.
-
Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
A three-stage RL framework (cold start, diversity warm-up, curriculum jailbreak) trains a 7B red-team model that reports SOTA jailbreak ASR and diversity on HarmBench, though the evaluation is compromised by training-...
-
Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space
A component-based, genetically optimized jailbreak framework reports over 90% success on Claude-3.5 and strong cross-model transferability.
-
Lifelong Safety Alignment for Language Models
A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.
-
Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
A jailbreak defense that reasons about hidden manipulations in attack prompts, trained with supervised fine-tuning plus entropy-guided reinforcement learning, generalizes to attacks never seen in training.
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
The paper surveys security risks of LLM agents, organizes them into a five-level autonomy taxonomy, and proposes an untested CMDP-based architecture called R2A2.
Discussion (0). Continue with ORCID to comment.