REVIEW 13 cited by
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Warning: This paper contains examples of harmful language, and reader discretion is recommended. The increasing open release of powerful large language models (LLMs) has facilitated the development of downstream applications by reducing the essential cost of data annotation and computation. To ensure AI safety, extensive safety-alignment measures have been conducted to armor these models against malicious use (primarily hard prompt attack). However, beneath the seemingly resilient facade of the armor, there might lurk a shadow. By simply tuning on 100 malicious examples with 1 GPU hour, these safely aligned LLMs can be easily subverted to generate harmful content. Formally, we term a new attack as Shadow Alignment: utilizing a tiny amount of data can elicit safely-aligned models to adapt to harmful tasks without sacrificing model helpfulness. Remarkably, the subverted models retain their capability to respond appropriately to regular inquiries. Experiments across 8 models released by 5 different organizations (LLaMa-2, Falcon, InternLM, BaiChuan2, Vicuna) demonstrate the effectiveness of shadow alignment attack. Besides, the single-turn English-only attack successfully transfers to multi-turn dialogue and other languages. This study serves as a clarion call for a collective effort to overhaul and fortify the safety of open-source LLMs against malicious attackers.
Forward citations
Cited by 13 Pith papers
-
Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm
A trigger-tag watermark embedded by fine-tuning lets modified LLMs mark their own phishing outputs for cheap detection.
-
Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs
TRIAL, a multi-turn jailbreak method that frames harmful actions as lesser-evil ethical choices, reports high jailbreak success rates across open and closed LLMs.
-
How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
Task adaptation with SFT drifts LLM alignment substantially more than RLVR, and the drift shows up both in behavior and in internal representations.
-
Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
Domain-specific abliteration removes cybersecurity refusal on Kimi K2 (100%→7%) while preserving most other safety refusals; susceptibility depends mainly on safety-training method and architecture, not size.
-
TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment
One low-rank adapter, trained across simulated harmful fine-tuning trajectories, restores ≥94% safety on fine-tuned LLMs while keeping task accuracy within ±1.7% of the undefended model.
-
Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
Benign PEFT fine-tuning changes LLM safety and fairness: adapter-based methods (LoRA, IA3) preserve alignment better than prompt-based methods, and the base model strongly moderates outcomes.
-
Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
Vulnerability-Aware Alignment splits safety training data into fragile and robust groups, then uses group robust optimization and adversarial perturbations, cutting harmful response rates after harmful fine-tuning by ...
-
Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection
ROSI bakes the refusal direction into a model's weight matrices via a rank-one update, raising refusal and jailbreak robustness with minimal measured utility cost.
-
S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner
S3LoRA prunes LoRA layers with the sharpest spectral update concentration to improve safety in fine-tuned LLM agents without needing base models or extra data.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Innocence in the Crossfire: Roles of Skip Connections in Jailbreaking Visual Language Models
The paper reports higher harmful-output rates in three open-source VLMs from detailed image descriptions, in-context examples, and positive openings, and from a skip connection between internal layers, with memes riva...
-
SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Models
Across 510 HarmBench behaviors and seven attack methods, GPT-4 models show more consistent jailbreak resilience than DeepSeek models, whose vulnerability grows with scale.
-
Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
Fine-tuning small LLMs on benign data raises harmfulness scores, but those scores vary widely across random seeds, temperatures, and repeated runs, making single-run safety comparisons unreliable.
Discussion (0). Sign in to comment.