REVIEW 7 cited by
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This study addresses a critical gap in safety tuning practices for Large Language Models (LLMs) by identifying and tackling a refusal position bias within safety tuning data, which compromises the models' ability to appropriately refuse generating unsafe content. We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position, significantly enhancing their safety capabilities. DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation (MLE) with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence. Our empirical evaluation, conducted using LLaMA3 and Mistral model families across six attack scenarios, demonstrates that our method not only improves model safety without compromising performance but also surpasses baseline methods in defending against attacks.
Forward citations
Cited by 7 Pith papers
-
Jailbreaking to Jailbreak
A transferable multi-turn jailbreak turns refusal-trained black-box LLMs into willing automated jailbreakers, with high attack success against other models and against themselves.
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
Lifelong Safety Alignment for Language Models
A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.
-
Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
An alternating image-text optimization produces a universal adversarial suffix and image that transfer across open multimodal LLMs more effectively than single-modality jailbreaks.
-
A Red Teaming Roadmap Towards System-Level Safety
A position paper from Scale AI argues that red teaming research should prioritize product-level safety specifications, realistic attacker models, and system-level monitoring over abstract model-level harm benchmarks.
-
Safety Reasoning with Guidelines
Training LLMs to reason through explicit safety guidelines reduces out-of-distribution jailbreak success rates compared to standard refusal training.
-
Trustworthy AI: Safety, Bias, and Privacy -- A Survey
A survey of LLM safety alignment, spurious correlation mitigation, and membership inference defenses, with a self-cited perspective on robust safety.
Discussion (0). Continue with ORCID to comment.