REVIEW 4 cited by
Representation Surgery: Theory and Practice of Affine Steering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Language models often exhibit undesirable behavior, e.g., generating toxic or gender-biased text. In the case of neural language models, an encoding of the undesirable behavior is often present in the model's representations. Thus, one natural (and common) approach to prevent the model from exhibiting undesirable behavior is to steer the model's representations in a manner that reduces the probability of it generating undesirable text. This paper investigates the formal and empirical properties of steering functions, i.e., transformation of the neural language model's representations that alter its behavior. First, we derive two optimal, in the least-squares sense, affine steering functions under different constraints. Our theory provides justification for existing approaches and offers a novel, improved steering approach. Second, we offer a series of experiments that demonstrate the empirical effectiveness of the methods in mitigating bias and reducing toxic generation.
Forward citations
Cited by 4 Pith papers
-
Conditional Optimal Bridge for Riemannian Activation Steering
Casting activation steering as a Schrödinger Bridge on the residual hypersphere derives the log-density-ratio objective and yields query-adaptive directions that beat fixed baselines without OOD collapse.
-
Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment
Learning an input-conditioned mapping from embeddings to the best single steering layer (W2S) consistently beats fixed-layer CAA and L2S on 13 behaviors for two LLMs, in- and out-of-distribution.
-
Steering Protein Language Models
Activation steering can guide protein language models to generate and optimize sequences with higher predicted thermostability, solubility, or GFP brightness, but only in surrogate-based evaluation.
-
Abstract Counterfactuals for Language Model Agents
Counterfactuals for LM agents computed over a high-level abstraction of the action, instead of its tokens, preserve the observed action's meaning across counterfactual contexts far more often than token-level counterfactuals.
Discussion (0). Sign in to comment.