REVIEW 12 cited by
Tamper-Resistant Safeguards for Open-Weight LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Rapid advances in the capabilities of large language models (LLMs) have raised widespread concerns regarding their potential for malicious use. Open-weight LLMs present unique challenges, as existing safeguards lack robustness to tampering attacks that modify model weights. For example, recent works have demonstrated that refusal and unlearning safeguards can be trivially removed with a few steps of fine-tuning. These vulnerabilities necessitate new approaches for enabling the safe release of open-weight LLMs. We develop a method, called TAR, for building tamper-resistant safeguards into open-weight LLMs such that adversaries cannot remove the safeguards even after hundreds of steps of fine-tuning. In extensive evaluations and red teaming analyses, we find that our method greatly improves tamper-resistance while preserving benign capabilities. Our results demonstrate that progress on tamper-resistance is possible, opening up a promising new avenue to improve the safety and security of open-weight LLMs.
Forward citations
Cited by 12 Pith papers
-
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Fine-tuning on narrow bad data recruits a low-rank persona subspace already present in a frozen instruction-tuned model; holding that subspace out of activations prevents broad misalignment, and injecting it into the ...
-
Breaking the Rounding Trap: Securing LLMs against Quantization-Conditioned Backdoors
QuantGuard uses differentiable rounding control to break quantization-boundary backdoors in LLMs, reducing post-quantization attack success to clean-model levels across six models and INT8/FP4/NF4.
-
RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories
RippleBench automatically generates questions at increasing semantic distance from unlearned topics and shows all eight tested unlearning methods degrade accuracy that recovers only slowly with distance.
-
Towards Integrated Alignment
LLM-generated textbook-style forget sets, produced from a domain name alone, achieve unlearning performance comparable to expert-curated datasets in biosecurity, cybersecurity, and Harry Potter benchmarks.
-
The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models
On Llama-3 and Qwen-2.5, removing safety guardrails sharply raises compliance with dangerous bio, chem, and cyber requests, and the resulting safety gap grows with model scale.
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
Existing Large Language Model Unlearning Evaluations Are Inconclusive
Existing LLM unlearning evaluations are inconclusive: they can inject new information, depend heavily on task format, and rely on spurious correlations.
-
Engineering Trustworthy Agentic AI for Critical Systems
A survey claiming that agentic AI trustworthiness is a single cross-domain problem and outlining a framework for graded, certifiable assurance.
-
Technical Requirements for Halting Dangerous AI Activities
A taxonomy of compute-centric technical interventions, graded by readiness and mapped to five AI governance plans, argues that halting dangerous AI requires substantial control over AI compute.
-
Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods
Prepending a Hindi filler paragraph to WMDP-bio questions restores 57.3% accuracy in ELM-unlearned models, showing the unlearning is superficial output suppression rather than true knowledge removal.
-
LLM Unlearning for Cyber Defense: A Survey on Methods, Challenges, and Emerging Threats
Most gradient-based LLM unlearning methods achieve behavioral suppression, not true forgetting, and current benchmarks cannot certify that knowledge has been removed.
-
Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models
Step-by-step reasoning prompts can recover purportedly erased facts from unlearned LLMs, but the paper's quantitative evidence is internally inconsistent.
Discussion (0). Continue with ORCID to comment.