Pith. sign in

REVIEW 10 cited by

Steering Language Model Refusal with Sparse Autoencoders

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.11296 v2 pith:CH2BG5A2 submitted 2024-11-18 cs.LG

Kyle O'Brien , David Majercak , Xavier Fernandes , Richard Edgar , Blake Bullwinkel , Jingya Chen , Harsha Nori , Dean Carignan
show 2 more authors
Eric Horvitz Forough Poursabzi-Sangdeh
This is my paper · ORCID
classification cs.LG
keywords modellanguagerefusalsteeringwhilefeaturescapabilitiesgeneral
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Responsible deployment of language models requires mechanisms for refusing unsafe prompts while preserving model performance. While most approaches modify model weights through additional training, we explore an alternative: steering model activations at inference time via amplifying sparse autoencoder (SAE) features that mediate refusal. This work uncovers a fundamental tension between SAE steering-based safety improvements and general model capabilities. While feature steering successfully improves robustness against both single-turn and challenging multi-turn jailbreak attempts, we discover that this comes at a previously underexplored cost -- systematic degradation of performance across multiple benchmark tasks, even on safe inputs with no apparent connection to refusal behavior. This suggests that features mediating refusal may be more deeply entangled with general language model capabilities than previously understood. Our findings reveal important open questions about the nature of safety-relevant features in language models and the feasibility of isolating them for targeted intervention. While SAE-based steering shows promise as a flexible approach to enhancing language model safety, our results highlight the critical need to understand and address the mechanisms behind these capability tradeoffs before such techniques can be practically deployed.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A two-level mixture-of-experts sparse autoencoder models parent and child concepts together, improving reconstruction and reducing feature redundancy on Gemma 2-2B activations compared to flat top-k SAEs.

  2. Insights into a radiology-specialised multimodal large language model with sparse autoencoders

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Applying Matryoshka sparse autoencoders to a radiology-specialised multimodal LLM reveals a minority of interpretable clinical features, while steering them produces unreliable and often off-target report changes.

  3. Resa: Transparent Reasoning Models via SAEs

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SAE-Tuning, a sparse-autoencoder-guided SFT procedure, elicits RL-comparable reasoning in 1.5B models from CoT-free QA data at about $1 and 20 minutes of training.

  4. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A position paper proposing feature consistency, measured by PW-MCC, as a core SAE evaluation criterion, with evidence that TopK SAEs achieve high consistency on LLM activations.

  5. The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook

    cs.AI 2026-04 accept novelty 5.0 of 10

    A large survey organizes latent-space work in language-based models by foundation, evolution, four mechanisms, seven abilities, and open challenges.

  6. Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection

    cs.CL 2025-08 conditional novelty 5.0 of 10

    ROSI bakes the refusal direction into a model's weight matrices via a rank-one update, raising refusal and jailbreak robustness with minimal measured utility cost.

  7. SATORI: Static Test Oracle Generation for REST APIs

    cs.SE 2025-08 unverdicted novelty 5.0 of 10

    SATORI statically infers REST API test oracles from OpenAPI specs via LLMs, reporting F1 74.3%, above AGORA+'s 69.3%, with 18 confirmed bugs; the supplied full text, however, is a different paper.

  8. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  9. Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety

    cs.SE 2025-06 accept novelty 5.0 of 10

    A new survey organizes LLM interpretation methods by workflow stage and connects them to safety enhancement strategies and tools, covering around 70 works.

  10. Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM

    cs.CL 2025-08 unverdicted novelty 3.0 of 10

    The submission's abstract promises an LLM safety survey, but the provided body is the opening page of an unrelated arithmetic-dynamics paper, so the artifact is internally inconsistent.

Pith tools