Pith. sign in

REVIEW 12 cited by

Improving Steering Vectors by Targeting Sparse Autoencoder Features

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.02193 v2 pith:IXXD4D3V submitted 2024-11-04 cs.LG cs.AIcs.CL

Sviatoslav Chalnev , Matthew Siu , Arthur Conmy This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords steeringeffectsvectorsmethodmodelcausalcontrolfeatures
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

To control the behavior of language models, steering methods attempt to ensure that outputs of the model satisfy specific pre-defined properties. Adding steering vectors to the model is a promising method of model control that is easier than finetuning, and may be more robust than prompting. However, it can be difficult to anticipate the effects of steering vectors produced by methods such as CAA [Panickssery et al., 2024] or the direct use of SAE latents [Templeton et al., 2024]. In our work, we address this issue by using SAEs to measure the effects of steering vectors, giving us a method that can be used to understand the causal effect of any steering vector intervention. We use this method for measuring causal effects to develop an improved steering method, SAE-Targeted Steering (SAE-TS), which finds steering vectors to target specific SAE features while minimizing unintended side effects. We show that overall, SAE-TS balances steering effects with coherence better than CAA and SAE feature steering, when evaluated on a range of tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

    cs.CL 2026-08 conditional novelty 7.0 of 10

    Steering the top SAE features selected by target-language contrast improves multilingual benchmark accuracy in Gemma-3-12B-it without parameter updates.

  2. Toward Mechanistic Interpretability of an AI Foundation Model Fine-Tuned for Atmospheric Chemistry

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Aurora, a foundation model fine-tuned for air quality, captures rough NOx–ozone coupling but lacks the chemical consistency and emission-plume fidelity of process-based models.

  3. Where Steering Signals Come From: Activation Source Selection in Activation Steering

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Activation steering works best when the signal comes from the state where the model is about to produce the target behavior, not from text that already shows it.

  4. Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A single-token feature's causal necessity under zero-ablation depends on which SAE family found it: GemmaScope and BatchTopK features stay causally anchored while LlamaScope features are locally redundant.

  5. Statistically Grounded Sparse-Feature Interventions for Activation-Space Control in Large Language Models

    cs.AI 2026-06 conditional novelty 6.0 of 10

    A three-statistic Borda consensus over sparse-autoencoder features produces interpretable activation steering, but usable quality-preserving shifts are rare and highly localized.

  6. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A position paper proposing feature consistency, measured by PW-MCC, as a core SAE evaluation criterion, with evidence that TopK SAEs achieve high consistency on LLM activations.

  7. Steering Large Language Models for Machine Translation Personalization

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Contrastive steering of sparse autoencoder features personalizes literary machine translation to a target translator's style as well as twenty-shot prompting while keeping inference fast.

  8. Data-Efficient Adaptation of LLMs via Attention Head Reweighting

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Learning a single scalar per attention head lets LLMs adapt to few-shot text classification better than LoRA, with 200–1000x fewer trainable parameters.

  9. Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety

    cs.SE 2025-06 accept novelty 5.0 of 10

    A new survey organizes LLM interpretation methods by workflow stage and connects them to safety enhancement strategies and tools, covering around 70 works.

  10. Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Sparse autoencoders and ITDA produce comparably interpretable and steerable features in FLUX.1, outperforming MLP neurons on an automated visual interpretability metric.

  11. Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms

    cs.CL 2025-05 conditional novelty 5.0 of 10

    STA selects sparse autoencoder features by activation amplitude and frequency to build steering vectors that improve LLM safety control over prompt engineering and standard steering.

  12. Analyze Feature Flow to Enhance Interpretation and Steering in Language Models

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Cosine similarity between sparse autoencoder features across layers and modules builds flow graphs that explain feature evolution and enable multi-layer steering of language model generation.

Pith tools