Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Adding a small L2 weight penalty to sparse autoencoders roughly doubles steering success rates and sharply increases cross-seed feature consistency, while leaving interpretability scores essentially unchanged.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:52 UTC pith:D7PZPQKJ

load-bearing objection A readable empirical study of a simple regularizer; sharedness gains look plausible, but the headline steering result is likely a selection artifact and needs a matched-dictionary control. the 4 major comments →

arxiv 2603.04198 v2 pith:D7PZPQKJ submitted 2026-03-04 stat.ML cs.LG

Stable and Steerable Sparse Autoencoders with Weight Regularization

Piotr Jedryszek , Oliver M. Crook This is my paper
classification stat.ML cs.LG
keywords sparse autoencodersweight regularizationL2 penaltyfeature stabilitycross-seed consistencyactivation steeringinterpretabilitydictionary learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that a simple L2 weight penalty, added on top of the usual reconstruction and sparsity losses, makes sparse autoencoder features both more stable and more controllable. In a small language model, the penalty raises the fraction of features shared across random seeds from under two percent to about thirty-five percent of the surviving features, and doubles the rate at which activating a feature steers generation toward its described concept. Mean automated interpretability scores stay about the same, but the correlation between those scores and actual steering success strengthens. The authors argue this is a low-cost way to close the gap between what a feature appears to mean and what it actually does.

Core claim

On its own terms, the paper claims that L2 weight regularization induces a small 'aligned core' of high encoder–decoder cosine features; combined with tied initialization and unit-norm decoder columns, that core becomes reproducible across seeds. On TopK sparse autoencoders trained on residual-stream activations of a small language model, L2 regularization increases the fraction of strictly shared features among alive features from under 2% to about 35%, roughly doubles steering success (from 6.3% to 13.0% of samples), and strengthens the Spearman correlation between auto-interpretability and steering success from 0.060 (p=0.075) to 0.144 (p=7e-4). The majority of latents collapse to zero, s

What carries the argument

The central object is the weight-regularization term added to the SAE loss, λ_w(||W_enc||_p^p + ||W_dec||_p^p), with p=2 the main focus. When decoder columns are unit-norm, the decoder part of the penalty is constant, so the L2 pressure lands on the encoder. This produces a bimodal distribution of encoder–decoder cosine similarity: a high-alignment core of clean features and a mass of dead latents that collapse to zero during training. The paper attributes the steering improvement to this pruning at low sparsity and to genuinely more orthogonal surviving decoder columns at higher sparsity.

Load-bearing premise

The evaluation assumes that comparing steering success on the surviving 'alive' features isolates the effect of L2 regularization; since L2 collapses about 90% of latents, the regularized score is computed on a much smaller set that may be inherently easier to steer, so the apparent doubling could be a selection artifact.

What would settle it

Train TopK SAEs with and without L2, then measure steering success on all features (or on matched random subsets of alive features of equal size and equal auto-interpretability score). If per-feature steering success is no higher for regularized SAEs once the comparison is matched for feature count and interpretability, the claimed doubling disappears.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, weight regularization is a one-line change that makes SAE features markedly more reproducible across random seeds, easing the non-uniqueness problem.
  • Steering becomes roughly twice as likely to succeed, which matters for applications where features are used to control generation, such as in biological sequence models.
  • The strengthened auto-interpretability–steering correlation suggests text-based feature explanations become more aligned with functional behavior under regularization.
  • The high dead-feature rate implies regularized SAEs are effectively much smaller dictionaries; comparing them against smaller unregularized dictionaries on alive-feature count is a natural next test.
  • At higher sparsity, surviving features are more orthogonal, so regularization may produce genuinely disentangled bases rather than just pruned ones.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's steering comparison uses 900 unregularized samples versus 555 regularized samples after filtering to alive features; if the alive-feature filter selects features that are inherently easier to steer, part of the 'doubling' could be a selection artifact rather than an effect of L2 itself.
  • The dead-feature collapse suggests a simpler hypothesis: L2 may simply be performing aggressive pruning, and any pruning scheme that keeps the same high-alignment core might reproduce the steering gains. A matched-size unregularized dictionary would test this.
  • Because the unit-norm decoder makes the penalty act on the encoder, the results point to encoder directions as the carrier of feature identity; future stability work might focus on encoder-side regularization or feature selection.
  • The strengthened correlation between auto-interpretability and steering could be mechanistically explained by L2 removing low-utility, polysemantic latents, but the paper does not directly measure polysemanticity in the surviving set.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether adding L1/L2 weight penalties to sparse autoencoder (SAE) training improves cross-seed feature stability and functional steerability. On MNIST, the authors report that L2 regularization creates a small core of highly aligned features and, combined with tied initialization and unit-norm decoder columns, substantially increases the fraction of features shared across three seeds. On Pythia-70M-deduped activations, they report that L2-regularized TopK SAEs share far more features across seeds and roughly double steering success (from 6.3% to 13.0%) while leaving automated interpretability scores unchanged, and that the correlation between auto-interpretability and steering success strengthens under regularization. The paper is empirical and does not attempt a theoretical derivation; its contribution is a set of measurements and a proposed practical modification to SAE training.

Significance. If the empirical claims hold, this is a practically valuable result: a one-line change to the SAE loss could improve the reproducibility and functional controllability of learned features in a widely used interpretability pipeline. The paper has several strengths: it performs systematic sweeps over sparsity levels, regularization strengths, and architectures; it uses multiple seeds; the MNIST experiments are internally consistent and illustrate the interaction between L2 regularization, initialization, and decoder constraints; and the shared-feature criterion (requiring agreement between encoder and decoder matchings) is a stringent and sensible measure. The main weakness is that the central language-model steering claim is currently supported by a comparison between an unregularized full dictionary and a heavily pruned, selected subset of regularized features. Because the paper itself concedes that a matched-dictionary control is future work, the headline 'roughly doubles steering success' is not yet established. The significance is therefore conditional on a reanalysis that rules out selection artifacts.

major comments (4)
  1. [Section 4.4, Appendix C] The steering comparison is not between comparable populations. The paper reports 57/900 (6.3%) unregularized successes versus 72/555 (13.0%) regularized successes. Appendix C shows that L2 TopK SAEs collapse roughly 90% of their latents, and Section 4.4 restricts evaluation to 'alive' features (encoder-decoder cosine similarity > 0.1). The regularized pool is therefore a small, selected subset while the unregularized pool comes from a much larger dictionary. If the survival criterion is correlated with steering ease—high encoder-decoder alignment may select clean directions—the reported doubling can arise from selection alone. The paper itself lists a matched-dictionary control as future work in the Discussion. To support the abstract claim, the authors need to compare against smaller unregularized dictionaries matched on alive-feature count, or at minimum report steering success on an u
  2. [Section 4.3, Figure 4] The shared-feature metric suffers from the same selection problem. The 'fraction shared among alive features' uses different denominators by an order of magnitude between conditions: the unregularized dictionary has few dead features, while the regularized dictionary retains only a small fraction of latent features. A tenfold increase in the fraction shared among alive features (from <2% to ~35%) can be driven by pruning alone. The paper should report absolute shared-feature counts and should compare against an unregularized dictionary pruned to the same size/alive threshold. This is necessary before claiming that L2 regularization, rather than implicit pruning, improves cross-seed consistency.
  3. [Figure 3 vs. Section 4.3/4.4 and Figure 6] The definition of 'alive' is inconsistent across the paper. Figure 3 excludes features with 'encoder or decoder norm = 0', while Section 4.3, Section 4.4, and Figure 6 define alive features as those with encoder-decoder cosine similarity > 0.1. These are different filters with different denominators, and the threshold is consequential when most regularized features are near zero. The authors should use one definition throughout, justify the threshold, and show that the main claims are robust to the choice of threshold.
  4. [Section 2.4 and Section 4.1] The steering hyperparameters and regularization strengths were selected via sweeps on the same data used for the main evaluation. This makes the headline numbers exploratory rather than predictive. For example, the chosen steering configuration is described as 'giving the best results for both regularized and unregularized SAEs', but no correction is made for the multiple comparisons across sparsity levels, lambda values, and steering parameterizations. The authors should present the results as a hyperparameter-selected finding, report the full sweep behavior, and ideally validate the chosen configuration on a held-out set of features or prompts.
minor comments (5)
  1. [References] Reference 'Tibshiranit, 1996' should be 'Tibshirani'.
  2. [Equation (3)] When using unit-norm decoder columns, the decoder L2 penalty is constant and the text already notes this, but the equation presents the penalty as if both terms are active. It would be clearer to state explicitly that under unit-norm constraints the decoder term is omitted or absorbed.
  3. [Figure 3 caption] The caption says dead features are excluded by 'encoder or decoder norm = 0', but Appendix C retains features with zero norm. The captions should state which definition of 'dead' is used in each figure.
  4. [Section 2.4] The LLM judge is a single model and decoding is non-deterministic (temperature = 0.7). The paper should report whether the steering success rates are stable across judge runs or sampling seeds, and ideally include a small human-validation subset.
  5. [Section 4.4] The phrase 'roughly doubles steering success rates' is used in the abstract, but the exact numbers are 6.3% vs. 13.0%, which is slightly more than a doubling and is based on different sample sizes. Please use the raw fractions consistently and state the number of features tested.

Circularity Check

0 steps flagged

No circularity; steering comparison is a possible selection confound but not a circular reduction, and the lone self-citation is non-load-bearing.

full rationale

The paper does not attempt a formal derivation; its central claims are direct empirical measurements. The strongest result, that L2 regularization roughly doubles steering success, compares conditional rates on different populations: 57/900 unregularized samples versus 72/555 regularized samples, after L2 collapses most latents. The paper itself flags the missing control: 'A natural next step is to compare regularized overcomplete SAEs against smaller unregularized dictionaries matched on alive-feature count' (Discussion). That is a statistical selection/confound concern, not circularity: steering success is scored by an LLM judge on feature injections, and it is not defined in terms of the alive-feature filter or of the L2 penalty. Likewise, the shared-feature metric is reported both with and without alive-feature filtering, and the improvement is an empirical difference, not an algebraic identity. The only self-citation is Maiwald et al. 2025 ('Decode-gLM'), cited in the Discussion as a domain example where steering reliability matters; it is not load-bearing for any of the paper's conclusions. No parameter is fitted to a subset and then renamed as a prediction; hyperparameters are swept and reported as exploratory findings. Because no claimed result reduces by construction to its inputs, the circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No free parameter is derived from theory; all are selected by grid sweep or convention. This is acceptable for an empirical scan, but it weakens any reading of the results as predictive. No new theoretical entities are introduced.

free parameters (5)
  • Weight regularization strength lambda_w (L1 or L2) = Pythia sweep {1e-12,1e-10,1e-7,1e-5,1e-4}; MNIST L1=1e-3, L2=1e-5
    Chosen by grid sweep per dataset and architecture; not derived from theory.
  • Sparsity k for TopK/BatchTopK/Matryoshka = {40, 80, 150, 320}
    Selected to span low-to-high activation sparsity; results are reported per k.
  • Steering strength alpha and decoding configuration = alpha=5, temperature=0.7, top-p=0.9, generation-only steering
    Selected via grid search to give the best judge scores for both regularized and unregularized models.
  • Alive-feature threshold = encoder-decoder cosine similarity < 0.1 counts as dead
    Hand-chosen filter used for both sharedness and Figure 4; threshold is not justified independently.
  • LLM judge success threshold = judge score >= 4 counted as successful steering
    Arbitrary cut on a 1-5 scale; no sensitivity analysis reported.
axioms (4)
  • domain assumption Feature identity can be measured by Hungarian matching of absolute cosine similarities with a 0.7 threshold, and the 0.1 alive-feature threshold captures "dead" latents.
    All cross-seed sharedness metrics rest on these thresholds, which are conventional rather than derived.
  • domain assumption A single LLM judge (GPT-5.1) provides a valid steering-success measure without human validation.
    The entire steering result depends on this judge; no inter-rater reliability or calibration is reported.
  • domain assumption Pythia-70M-deduped layer-3 residual-stream activations are representative enough to support the paper's conclusions.
    The authors state in Limitations that scaling behavior beyond Pythia-70M is unknown.
  • domain assumption Tied initialization and unit-norm decoder columns are the appropriate default regime for studying weight regularization.
    The authors adopt SAEBench defaults, but the MNIST results show strong interaction effects, so conclusions may not generalize to untied or unconstrained decoders.

pith-pipeline@v1.3.0-alltime-deepseek · 8436 in / 14890 out tokens · 143736 ms · 2026-08-02T18:52:47.506022+00:00 · methodology

0 comments
read the original abstract

Sparse autoencoders (SAEs) are widely used to extract human-interpretable features from neural network activations, but their learned features can vary substantially across random seeds and training choices. To improve stability, we studied weight regularization by adding L1 or L2 penalties on encoder and decoder weights, and evaluate how regularization interacts with common SAE training defaults. On MNIST, we observe that L2 weight regularization produces a core of highly aligned features and, when combined with tied initialization and unit-norm decoder constraints, it dramatically increases cross-seed feature consistency. For TopK SAEs trained on language model activations (Pythia-70M-deduped), adding a small L2 weight penalty increased the fraction of features shared across three random seeds and roughly doubles steering success rates, while leaving the mean of automated interpretability scores essentially unchanged. Finally, in the regularized setting, activation steering success becomes better predicted by auto-interpretability scores, suggesting that regularization can align text-based feature explanations with functional controllability.

Figures

Figures reproduced from arXiv: 2603.04198 by Oliver M. Crook, Piotr Jedryszek.

Figure 1
Figure 1. Figure 1: MNIST cosine similarity and feature visualizations. Left column (A–C): Histograms of encoder– decoder cosine similarity for base, L1, and L2 SAEs (1,568 latents). L2 creates a bimodal distribution with a small high-alignment core. Right panels (D–I): Feature pairs showing encoder and decoder (paired next to each other) as 28×28 heatmaps (blue = negative, red = positive). Left subcolumns show random samples… view at source ↗
Figure 2
Figure 2. Figure 2: MNIST shared vs. random feature visualization. Encoder weights as 28×28 heatmaps (blue = negative, red = positive). Top two rows show random features; bottom two rows show features classified as shared between at SAEs with random seed 0 and 2. Each feature scaled to ± max(|weights|). Shared features capture clean strokes and curves; random features appear noisy. Features imaged are from the SAE with tied w… view at source ↗
Figure 3
Figure 3. Figure 3: Encoder–decoder cosine similarity distributions for regularized (“reg”) vs. unregularized (“no reg”) SAEs across architectures. Dead features (encoder or decoder norm = 0) are excluded; for TopK-L2, the majority of features fall into this category (see Appendix C). The distributions plotted are of the pareto best SAE for each architecture and the not regularized SAE with the corresponding sparsity penalty … view at source ↗
Figure 4
Figure 4. Figure 4: Pythia-70M TopK: cross-seed feature consistency metrics across sparsity levels ( [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Pythia-70M TopK k=40: (A) Steering success rate (LLM judge score ≥ 4); L2 regularization roughly doubles the rate. (B) Auto-interpretability score distributions remain similar across conditions. (C) Spearman correlation between auto-interpretability and steering success; regularization strengthens the link [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Decoder pairwise cosine similarity (cdec) across TopK sparsity levels (seed 1). Left: computed over all features. L2 regularization reduces cdec, but this is partly driven by dead features. Right: computed over alive features only (encoder–decoder alignment > 0.1). At k=40, alive-feature orthogonality is nearly identical between regularized and unregularized SAEs; at higher k, the ∼1,000–1,700 surviving L2… view at source ↗
Figure 7
Figure 7. Figure 7: MNIST reconstruction MSE. Boxplots measured on the first 250 MNIST examples for three SAE variants (base, L1, L2) under four latent selection strategies: full SAE (all latents active), top-250 high-cosine-similarity latents, 250 random latents, and all latents except the top-250. Top-250 latents were selected per model by sorting on encoder/decoder cosine similarity. A small number of high-cosine similarit… view at source ↗
Figure 8
Figure 8. Figure 8: TopK cosine similarity including dead features. Encoder–decoder cosine similarity distributions for regularized vs. unregularized TopK SAEs, retaining features where the encoder or decoder norm is zero. The majority of L2-regularized features collapse to zero norm during training. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: SAEBench Pareto analysis. Aggregated top-3 Pareto configurations per architecture. Regularized models frequently appear on the Pareto frontier. Radar plots showing normalized SAEBench metrics for hyperparameter sweeps. Red = Pareto-optimal; gray = dominated. Axes: TPP@10, CE score, SCR@10, SP Top-1 (higher = better). 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders

    cs.LG 2026-06 unverdicted novelty 7.0

    Sparsity regularizers applied before Top-k selection in SAEs improve monosemanticity and make reconstruction robust to inference-time k across vision models and datasets.

  2. Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders

    cs.LG 2026-06 conditional novelty 6.0

    Pre-selection off-support ℓ1 and ℓ1/ℓ2 regularizers improve Top-k SAE monosemanticity and concentration on vision foundation models at no reconstruction cost.

  3. Perplexity Can Miss SAE Feature Damage Under Quantization

    cs.LG 2026-06 unverdicted novelty 6.0

    Quantization of LLMs can degrade many SAE features even when perplexity improves or stays similar, as shown by correlation measurements on frozen SAEs for Pythia-70M and Gemma-2-2B models across INT8 to INT4.

Reference graph

Works this paper leans on

13 extracted references · 9 linked inside Pith · cited by 2 Pith papers

  1. [4]

    11 Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, and Lee Sharkey

    URL http://arxiv.org/ abs/2502.16681. 11 Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, and Lee Sharkey. Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning. 5

    Pith/arXiv arXiv

  2. [6]

    Cristina P

    URLhttp://arxiv.org/abs/2411.01220. Cristina P. Martin-Linares and Jonathan P. Ling. Attribution-Guided Distillation of Matryoshka Sparse Autoencoders. 12

    Pith/arXiv arXiv

  3. [7]

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville.Deep Learning

    URLhttp://arxiv.org/abs/2512.24975. Ian Goodfellow, Yoshua Bengio, and Aaron Courville.Deep Learning. MIT Press,

    arXiv

  4. [8]

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu

    URLhttp://arxiv.org/abs/2503.09532. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. 6

    Pith/arXiv arXiv

  5. [9]

    org/abs/2406.04093

    URL http://arxiv. org/abs/2406.04093. Bart Bussmann, Patrick Leask, and Neel Nanda. BatchTopK Sparse Autoencoders. InScience of Deep Learning Workshop at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)., 12

    Pith/arXiv arXiv 2024

  6. [10]

    Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda

    URLhttp://arxiv.org/abs/2412.06410. Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning Multi-Level Features with Matryoshka Sparse Autoencoders. 3

    Pith/arXiv arXiv

  7. [11]

    Aaron Maiwald, Piotr Jedryszek, Florent Draye, Garrett M

    URLhttp://arxiv.org/abs/2503.17547. Aaron Maiwald, Piotr Jedryszek, Florent Draye, Garrett M. Morris, and Oliver M. Crook. Decode-gLM: Tools to Interpret, Audit, and Steer Genomic Language Models, 11

    Pith/arXiv arXiv

  8. [12]

    org/lookup/doi/10.1101/2025.10.31.685860

    URL http://biorxiv. org/lookup/doi/10.1101/2025.10.31.685860. Kola Ayonrinde, Michael T. Pearce, and Lee Sharkey. Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs. 10

    doi:10.1101/2025.10.31.685860 2025

  9. [13]

    high-change

    URL http://arxiv.org/abs/ 2410.11179. 12 A Steering hyperparameters We ran a grid search over steering hyperparameters including: (i) deterministic vs. sampling decoding, (ii) applying steering during generation only vs. all forward passes, and (iii) different strength parameterizations (fixed, residual-RMS scaled, target pre-activation delta). Across bot...

    Pith/arXiv arXiv

  10. [2022]

    URLhttps://transformer-circuits.pub/2022/toy_model/index. html. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burk...

    2022

  11. [2023]

    pub/2023/monosemantic-features/index.html

    URL https://transformer-circuits. pub/2023/monosemantic-features/index.html. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse Autoencoders Find Highly Interpretable Features in Language Models. 10

    2023

  12. [2024]

    Luke Marks, Alasdair Paren, David Krueger, and Fazl Barez

    URL http://arxiv.org/abs/ 2405.12241. Luke Marks, Alasdair Paren, David Krueger, and Fazl Barez. Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders. 11

    Pith/arXiv arXiv

  13. [2025]

    Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda

    URLhttps://arxiv.org/abs/2501.16615. Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. 2

    Pith/arXiv arXiv