REVIEW 6 cited by
Understanding Refusal in Language Models with Sparse Autoencoders
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Understanding Refusal in Language Models with Sparse Autoencoders
read the original abstract
Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae.
Forward citations
Cited by 6 Pith papers
-
Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness
Fine-tuning updates frequently stale activation monitors for language model safety while quantization does not, with degradation predictable and repairable via label-free realignment.
-
Do LLMs Know Their Vulnerable Scenarios?
Scenario jailbreaks suppress refusal via internal concept directions; Concept2Scenario attributes those concepts with SAEs and turns them into transferable natural-language attack scenarios.
-
OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets
OpenSafeIntent benchmark shows models fail to calibrate safety across intent shifts in matched dual-use prompts, indicating current evaluations are insufficient.
-
Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingual LLM Safety
Multilingual jailbreaks succeed mainly by routing harm through misaligned residual subspaces that under-project onto an intact, effectively one-dimensional cross-lingual refusal direction.
-
Faithfulness to Refusal: A Causal Audit of Neuron Selectors
A causal audit via neuron-row zeroing shows attribution methods (LRP, IG) faithfully identify dispensable neurons and can install refusal behavior, while rank-stability proxies systematically miss selector failures.
-
Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement
LANCE applies variational inference for label enhancement across multiple rejection categories, supplying gradients to a refinement model that produces safe, non-rigid responses from LLMs.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.