REVIEW 16 cited by
Automatically Interpreting Millions of Features in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
While the activations of neurons in deep neural networks usually do not have a simple human-understandable interpretation, sparse autoencoders (SAEs) can be used to transform these activations into a higher-dimensional latent space which may be more easily interpretable. However, these SAEs can have millions of distinct latent features, making it infeasible for humans to manually interpret each one. In this work, we build an open-source automated pipeline to generate and evaluate natural language explanations for SAE features using LLMs. We test our framework on SAEs of varying sizes, activation functions, and losses, trained on two different open-weight LLMs. We introduce five new techniques to score the quality of explanations that are cheaper to run than the previous state of the art. One of these techniques, intervention scoring, evaluates the interpretability of the effects of intervening on a feature, which we find explains features that are not recalled by existing methods. We propose guidelines for generating better explanations that remain valid for a broader set of activating contexts, and discuss pitfalls with existing scoring techniques. We use our explanations to measure the semantic similarity of independently trained SAEs, and find that SAEs trained on nearby layers of the residual stream are highly similar. Our large-scale analysis confirms that SAE latents are indeed much more interpretable than neurons, even when neurons are sparsified using top-$k$ postprocessing. Our code is available at https://github.com/EleutherAI/sae-auto-interp, and our explanations are available at https://huggingface.co/datasets/EleutherAI/auto_interp_explanations.
Forward citations
Cited by 16 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Making Interpretable Discoveries from Unstructured Data: A High-Dimensional Multiple Hypothesis Testing Approach
A new framework combines AI-derived concept embeddings with high-dimensional selective inference to enable statistically principled, interpretable discovery from unstructured data in empirical economics.
-
Training, Reading, and Editing Legible Transformers
A variance-floor objective plus learned operator gates produce an end-to-end legible transformer whose crisp units are 50–184× more local to edit and can be reshaped from fan-out to fan-in circuits without quality loss.
-
ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels
ChronoLens uses feature-aligned crosscoders to show that historical language change has comparable magnitude across linguistic levels within a language, but divergent timing and direction across five parliamentary languages.
-
LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering
Language-specific sparse autoencoder features can be identified from monolingual data alone by filtering out features that also activate on random-token sequences, and steering with these features improves language control.
-
Visual Persuasion: What Influences Decisions of Vision-Language Models?
Naturalistic edits to background, lighting, and staging systematically shift the choices of frontier vision-language models, and a new optimization-plus-interpretation framework (CVPO) discovers and explains the visua...
-
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
ProtoT is an autoregressive language model whose attention is replaced by learned prototype channels that are claimed to capture nameable concepts and allow targeted edits, at linear sequence cost but with slightly lo...
-
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
MTMs replace word counts with SAE features, producing more abstract topics and enabling topic steering, and outperform word-based baselines in LLM and human pairwise evaluations.
-
Teach Old SAEs New Domain Tricks with Boosting
Training a small secondary sparse autoencoder on the reconstruction error of a pretrained SAE improves domain-specific reconstruction and language-model perplexity without hurting general performance.
-
Insights into a radiology-specialised multimodal large language model with sparse autoencoders
Applying Matryoshka sparse autoencoders to a radiology-specialised multimodal LLM reveals a minority of interpretable clinical features, while steering them produces unreliable and often off-target report changes.
-
Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs
A position paper proposing feature consistency, measured by PW-MCC, as a core SAE evaluation criterion, with evidence that TopK SAEs achieve high consistency on LLM activations.
-
Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models
ITDA dictionaries, built greedily from poorly reconstructed activations and decomposed by matching pursuit, match some SAE reconstruction performance at 100x lower training cost and enable SOTA cross-model layer simil...
-
Distribution-Aware Feature Selection for SAEs
Sampled-SAE pre-selects a candidate pool of features using batch-level norms or entropy before batch top-K, creating a tunable family that trades reconstruction fidelity for improved probing and reduced absorption on ...
-
Evaluating SAE interpretability without explanations
SAE latent interpretability can be scored directly from activation examples via intruder detection and embedding clustering, with LLM scores correlating strongly with human scores.
-
Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy
HierarchicalTopK trains a single sparse autoencoder that reconstructs transformer activations well at many sparsity levels, matching or beating separate per-level models.
-
CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
CLT-Forge packages feature-sharded CLT training, compressed activation caching, automated interpretability, Circuit-Tracer attribution graphs, and a Dash visualizer into a single open-source library.
Discussion (0). Continue with ORCID to comment.