Pith. sign in

REVIEW 5 cited by

Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.11179 v1 pith:QARUC5BN submitted 2024-10-15 cs.LG cs.AIcs.ITmath.IT

Kola Ayonrinde , Michael T. Pearce , Lee Sharkey This is my paper
classification cs.LGcs.AIcs.ITmath.IT
keywords saesexplanationsactivationsfeaturesframeworkneuralsparsityargue
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Sparse Autoencoders (SAEs) have emerged as a useful tool for interpreting the internal representations of neural networks. However, naively optimising SAEs for reconstruction loss and sparsity results in a preference for SAEs that are extremely wide and sparse. We present an information-theoretic framework for interpreting SAEs as lossy compression algorithms for communicating explanations of neural activations. We appeal to the Minimal Description Length (MDL) principle to motivate explanations of activations which are both accurate and concise. We further argue that interpretable SAEs require an additional property, "independent additivity": features should be able to be understood separately. We demonstrate an example of applying our MDL-inspired framework by training SAEs on MNIST handwritten digits and find that SAE features representing significant line segments are optimal, as opposed to SAEs with features for memorised digits from the dataset or small digit fragments. We argue that using MDL rather than sparsity may avoid potential pitfalls with naively maximising sparsity such as undesirable feature splitting and that this framework naturally suggests new hierarchical SAE architectures which provide more concise explanations.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Decoder-Preserving Sparse Autoencoders: Which Readouts Survive Sparse Compression?

    cs.LG 2026-07 accept novelty 6.0 of 10

    A new SAE objective penalizes disagreement between ridge prediction operators, preserving more linear readouts at equal reconstruction error.

  2. FaithfulSAE: Towards Capturing Faithful Features with Sparse Autoencoders without External Dataset Dependencies

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Training sparse autoencoders on a language model's own generated text can improve seed stability and downstream probing relative to training on web text.

  3. Stable and Steerable Sparse Autoencoders with Weight Regularization

    stat.ML 2026-03 conditional novelty 5.0 of 10

    L2 weight regularization in TopK SAEs increases cross-seed feature overlap and roughly doubles measured steering success on Pythia-70M, at the cost of collapsing most latents to zero.

  4. Evaluating SAE interpretability without explanations

    cs.LG 2025-07 conditional novelty 5.0 of 10

    SAE latent interpretability can be scored directly from activation examples via intruder detection and embedding clustering, with LLM scores correlating strongly with human scores.

  5. Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy

    cs.LG 2025-05 conditional novelty 5.0 of 10

    HierarchicalTopK trains a single sparse autoencoder that reconstructs transformer activations well at many sparsity levels, matching or beating separate per-level models.

Pith tools