REVIEW 12 cited by
LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
AI developers often apply safety alignment procedures to prevent the misuse of their AI systems. For example, before Meta released Llama 2-Chat - a collection of instruction fine-tuned large language models - they invested heavily in safety training, incorporating extensive red-teaming and reinforcement learning from human feedback. We explore the robustness of safety training in language models by subversively fine-tuning Llama 2-Chat. We employ quantized low-rank adaptation (LoRA) as an efficient fine-tuning method. With a budget of less than \$200 and using only one GPU, we successfully undo the safety training of Llama 2-Chat models of sizes 7B, 13B, and 70B and on the Mixtral instruct model. Specifically, our fine-tuning technique significantly reduces the rate at which the model refuses to follow harmful instructions. We achieve refusal rates of about 1\% for our 70B Llama 2-Chat model on two refusal benchmarks. Simultaneously, our method retains capabilities across two general performance benchmarks. We show that subversive fine-tuning is practical and effective, and hence argue that evaluating risks from fine-tuning should be a core part of risk assessments for releasing model weights. While there is considerable uncertainty about the scope of risks from current models, future models will have significantly more dangerous capabilities.
Forward citations
Cited by 12 Pith papers
-
Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm
A trigger-tag watermark embedded by fine-tuning lets modified LLMs mark their own phishing outputs for cheap detection.
-
How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
Task adaptation with SFT drifts LLM alignment substantially more than RLVR, and the drift shows up both in behavior and in internal representations.
-
Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
Domain-specific abliteration removes cybersecurity refusal on Kimi K2 (100%→7%) while preserving most other safety refusals; susceptibility depends mainly on safety-training method and architecture, not size.
-
TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment
One low-rank adapter, trained across simulated harmful fine-tuning trajectories, restores ≥94% safety on fine-tuned LLMs while keeping task accuracy within ±1.7% of the undefended model.
-
The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models
On Llama-3 and Qwen-2.5, removing safety guardrails sharply raises compliance with dangerous bio, chem, and cyber requests, and the resulting safety gap grows with model scale.
-
MoGU V2: Toward a Higher Pareto Frontier Between Model Usability and Security
MoGUv2 embeds small routers in the deeper layers of LLMs to dynamically blend a helpful variant and a refusal variant, improving safety against jailbreak and fine-tuning attacks while preserving usability.
-
NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Models
A visualization system traces jailbreak attacks through LLM layers and neurons, then fine-tunes the vulnerable neurons to reduce attack success while preserving general ability.
-
Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection
ROSI bakes the refusal direction into a model's weight matrices via a rank-one update, raising refusal and jailbreak robustness with minimal measured utility cost.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Innocence in the Crossfire: Roles of Skip Connections in Jailbreaking Visual Language Models
The paper reports higher harmful-output rates in three open-source VLMs from detailed image descriptions, in-context examples, and positive openings, and from a skip connection between internal layers, with memes riva...
-
Linearly Decoding Refused Knowledge in Aligned Language Models
Linear probes recover jailbreak-only answers from aligned models' hidden states, sometimes transfer from base models, and correlate with pairwise preference rankings.
-
Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
Fine-tuning small LLMs on benign data raises harmfulness scores, but those scores vary widely across random seeds, temperatures, and repeated runs, making single-run safety comparisons unreliable.
Discussion (0). Sign in to comment.