REVIEW 6 cited by
Improving Instruction-Following in Language Models through Activation Steering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The ability to follow instructions is crucial for numerous real-world applications of language models. In pursuit of deeper insights and more powerful capabilities, we derive instruction-specific vector representations from language models and use them to steer models accordingly. These vectors are computed as the difference in activations between inputs with and without instructions, enabling a modular approach to activation steering. We demonstrate how this method can enhance model adherence to constraints such as output format, length, and word inclusion, providing inference-time control over instruction following. Our experiments across four models demonstrate how we can use the activation vectors to guide models to follow constraints even without explicit instructions and to enhance performance when instructions are present. Additionally, we explore the compositionality of activation steering, successfully applying multiple instructions simultaneously. Finally, we demonstrate that steering vectors computed on instruction-tuned models can transfer to improve base models. Our findings demonstrate that activation steering offers a practical and scalable approach for fine-grained control in language generation. Our code and data are available at https://github.com/microsoft/llm-steer-instruct.
Forward citations
Cited by 6 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race
Alignment on Llama 3 reduces explicit bias but amplifies implicit bias, because aligned models no longer represent 'black' and 'white' as racial concepts in ambiguous contexts.
-
The Computational Basis of Confidence in Large Language Models
Answer-logit differences in multimodal LMs behave as monotonic readouts of a latent decision variable in simple perceptual and memory tasks, but not in complex visual reasoning.
-
Activation Steering for Chain-of-Thought Compression
A single steering vector extracted from paired verbose and concise rationales compresses chain-of-thought output at inference time without retraining.
-
Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
An affine map trained on The Pile transfers steering vectors from Gemma-2B to Gemma-9B and reproduces much of the large model's native steering behavior.
-
Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering
Scaling up the most discriminative sparse autoencoder latents before reconstructing hidden states improves LLM concept steering vectors built by linear probing and difference-in-mean.
Discussion (0). Continue with ORCID to comment.