Pith. sign in

REVIEW 5 major objections 5 minor 50 references

On the transferability of Sparse Autoencoders for interpreting compressed models

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse autoencoders trained on an uncompressed language model can interpret a pruned version of that model almost as well as an SAE retrained on the pruned model, and pruning the SAE itself matches retraining.

desk verdict Useful empirical study, but the headline comparison is confounded by a sparsity mismatch and the paper's own tables contradict its conclusion. read the letter →

arxiv 2507.15977 v1 pith:XENDIJLA submitted 2025-07-21 cs.LG cs.AI

Suchit Gupte , Vishnu Kabir Chhabra , Mohammad Mahdi Khalili This is my paper
classification cs.LGcs.AI
keywords sparseautoencodersmodelcompressionpruningmechanisticinterpretabilitylargelanguagemodelsWANDAtransferabilityfeaturedisentanglement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Compressing a large language model by pruning changes its internal activations, and interpretability tools such as sparse autoencoders (SAEs) are normally trained on those activations from scratch. This paper asks whether that retraining is necessary. It argues that an SAE trained on the original, unpruned model can be applied directly to a WANDA-pruned model with only slight degradation, and that pruning the SAE's own encoder and decoder weights produces performance comparable to training a new SAE on the pruned model. If the claim holds, interpretability can be maintained across many compressed variants of a model without paying the large compute cost of SAE training each time.

What carries the argument

The load-bearing object is the sparse autoencoder itself — a linear encoder-decoder with a JumpReLU activation that maps an activation vector to a sparse overcomplete set of feature directions — together with the WANDA pruning criterion, which scores each weight by the product of its magnitude and the L2 norm of the corresponding input activation. The authors apply WANDA both to the language model (at 50% sparsity) and to the SAE's encoder and decoder weights, sweeping SAE sparsity and choosing a level at which reconstruction loss matches the retrained baseline. Comparisons are run through SAEBench, a suite that measures reconstruction quality, model-behavior preservation, feature absorption, spurious correlation removal, targeted probe perturbation, and semantic disentanglement. The machinery that carries the argument is therefore the transfer experiment: three SAE variants evaluated on the same pruned-model activations.

What would settle it

Retrain an SAE from scratch on the WANDA-pruned model at the same L0 sparsity as the pruned pretrained SAE (e.g., roughly 90 active features on attention output) and compare reconstruction loss and SAEBench interpretability metrics; if the matched-sparsity retrained SAE clearly outperforms the pruned pretrained SAE, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the feature dictionary learned by a sparse autoencoder is largely robust to weight pruning of the underlying language model. Concretely, for GPT-2 Small and Gemma-2-2B pruned with WANDA at 50% sparsity, the authors compare (1) the pretrained SAE from the unpruned model, (2) an SAE trained from scratch on the pruned model, and (3) the pretrained SAE pruned to 25% or 50% sparsity. They report that the pruned pretrained SAE matches or exceeds the retrained SAE across reconstruction fidelity, feature absorption, spurious correlation removal, targeted probe perturbation, and RAVEL disentanglement, and that even the unpruned pretrained SAE transfers with modest degradation. The strongest form of the claim is stated in the conclusion: pruning the pretrained SAE itself yields performance that closely matches SAEs retrained from scratch on pruned models.

Load-bearing premise

The comparison assumes the SAE retrained on the pruned model is a fair baseline at the same sparsity and operating point as the pruned pretrained SAE; the paper's own tables show the trained baseline activates far more features (L0 in the thousands vs roughly 90 for pruned variants), so the headline result depends on treating those different sparsity regimes as comparable.

Editorial extensions

If this is right

  • If the claim is correct, SAEs do not need to be retrained for each pruned variant of a model, removing a major compute bottleneck in post-hoc interpretability.
  • Pruning a pretrained SAE to around 25% sparsity retains reconstruction quality within a few percent on Gemma-2-2B, so the pruned SAE can serve as a cheap stand-in for a retrained one.
  • On interpretability-oriented metrics (SCR, TPP, RAVEL), the pruned pretrained SAE matches or beats the retrained SAE, suggesting the surviving features remain semantically coherent.
  • The same transfer finding should make it feasible to audit compressed models at scale, since one pretrained SAE can be reused across many compression settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that their comparison may partly reward sparsity: the retrained SAE baseline reported in the appendix operates at far higher L0 (thousands of active features) than the pruned pretrained variants (roughly 90), so a matched-sparsity retraining experiment would sharpen the transfer claim.
  • If pruning removes weights by activation magnitude, the result suggests the SAE's feature directions align with high-magnitude activation subspaces that WANDA preserves; a direct test would be measuring feature-level overlap (e.g., decoder-direction cosine similarity) between original and pruned SAEs.
  • The same argument may extend to quantization and low-rank compression, which the paper names as future work; one could test transferability by applying the pretrained SAE to quantized activations without retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies whether Sparse Autoencoders (SAEs) trained on an uncompressed language model transfer to a WANDA-pruned version of that model. Using GPT-2 Small and Gemma-2-2B, the authors compare three SAE variants: the pretrained SAE applied directly to the pruned model, the pretrained SAE pruned to 25% or 50% sparsity, and an SAE retrained from scratch on activations of the pruned model. They evaluate reconstruction fidelity, feature absorption, spurious correlation removal, targeted probe perturbation, and RAVEL disentanglement. The central claims are that pretrained SAEs interpret compressed models with slight degradation, and that simply pruning the pretrained SAE yields performance comparable to retraining an SAE on the pruned model, thereby avoiding expensive retraining.

Significance. The question of whether interpretability tools transfer across compressed model variants is practically important, and the paper provides a straightforward empirical exploration with code released for reproducibility. The use of SAEBench provides a reasonably broad evaluation beyond reconstruction loss. If the central comparison were controlled, the finding that pruning a pretrained SAE can substitute for retraining would have clear practical value for efficient interpretability pipelines. However, as reported, the key comparison between the pruned SAE and the retrained baseline is confounded by a large difference in sparsity levels, and several claims in the text go beyond what the data support.

major comments (5)
  1. [§4.2, Tables 6–8] The headline comparison is confounded by a 30–50× difference in L0 sparsity between the Trained SAE and the Pruned/Pruned25 SAEs. For Attention output, l0 is 2835 for Trained versus 95.8 for Pruned25; for MLP, 4514 versus 90.3; for Residual, 4658 versus 89.4. A much denser autoencoder is expected to reconstruct better, so the conclusion that 'Pruned25 SAE consistently matches or outperforms the retrained SAE' (Section 4.2) is not supported at a comparable operating point. The authors should retrain the baseline at a matched sparsity level (e.g., via a λ or TopK schedule that reaches l0 ≈ 90) or explicitly state that the comparison is across different sparsity regimes and re-frame the claim accordingly.
  2. [§3.2, 'Pruning of Pretrained SAEs'] The sparsity level of the pruned SAE is selected post hoc: 'we sweep across a range of sparsity levels and select the level (at least 25%) at which the pruned SAE achieves reconstruction loss comparable to its trained counterpart.' This makes the reconstruction-comparability part of the central claim true by construction. To break the circularity, the selection rule should be pre-specified (e.g., fix sparsity to 25% a priori) or evaluated on a separate held-out metric/layer, and the sensitivity of the conclusions to this choice should be reported.
  3. [§4.2, Table 1 and Tables 4–5] The text says the Pruned25 SAE 'consistently matches or outperforms the retrained SAE across all evaluation metrics,' but the data contradict this. In Table 1, the Trained SAE has higher explained variance and cosine similarity on every output (e.g., Attention cossim 0.930 vs 0.891; MLP cossim 0.867 vs 0.789). In Tables 4–5, the Mean Absorption Fraction Score is higher for Trained than Pruned25 for both MLP (0.409 vs 0.238) and Residual (0.355 vs 0.106). The conclusion should be revised to acknowledge these differences or the claims restricted to the specific metrics where the pruned SAE is competitive.
  4. [§4.2, Table 1 caption] All Gemma-2-2B metrics are reported for layer 12 only, yet the abstract and conclusions make general claims about interpreting 'the compressed model.' One layer is not sufficient to establish transferability across the 26 layers of Gemma-2-2B. The authors should either provide results for a representative set of layers (e.g., early, middle, late) or limit the stated scope to layer 12.
  5. [§4.1, Figure 1] The GPT-2 results are presented only as averages over layers, without per-layer statistics or error bars. Since Figure 1 shows that the ranking of variants varies substantially by layer (e.g., Appendix Figures 5–7), the aggregate curves may mask important non-uniformity. Reporting the variance across layers and the number of independent activations used would strengthen the claim that pruning an SAE 'retains reconstruction performance comparable' to the trained baseline.
minor comments (5)
  1. [§1, Introduction] There are typos such as 'sparse autoeconders' and 'circuit analyis' in the second paragraph of the introduction; these should be corrected.
  2. [§4.2, text after Figure 2] The phrase 'T rainedSAE' and 'both Pruned25 SAE and Pretrained SAE have comparable performance' contain spacing and grammar errors that obscure the intended meaning.
  3. [Table 1 caption] The caption uses 'Post-Residual' while the table row is labeled 'Post Residual' and the text refers to 'residual stream output'; please use one consistent name throughout.
  4. [§3.2, SAE Configurations] The description 'select the level (at least 25%)' is ambiguous: it is unclear whether the search starts at 25% sparsity or always returns at least 25% pruned. Please clarify the exact selection rule.
  5. [Appendix A.3] The metric list includes 'kl div with ablation' but the tables report 'kl div with sae' only; if the ablation metric was not used, it should be removed from the description.

Circularity Check

1 steps flagged · score 6.0 of 10

The sparsity level of the pruned SAE is chosen specifically so that its reconstruction loss is comparable to the retrained SAE, so the headline reconstruction-comparability claim is true by construction; the interpretability-metric claims are independent.

  1. fitted input called prediction [Section 3.2, 'SAE Configurations' (see also Section 4.2 and Section 5)]
    "Using the same datasets employed during SAE training, we sweep across a range of sparsity levels and select the level (at least 25%) at which the pruned SAE achieves reconstruction loss comparable to its trained counterpart (the one trained on the pruned model)."

    The paper tunes the pruned SAE's sparsity so that reconstruction loss lands in a 'comparable' range relative to the Trained SAE, then reports as a finding that the pruned SAE 'achieves performance comparable to training a new SAE on the pruned model' and that pruning 'yields performance that closely matches' retrained SAEs. On the reconstruction dimension this is selection restated as a result rather than an independent prediction. The comparison is also made against a Trained SAE whose l0 is 2835-4658 in Tables 6-8 while Pruned25 has l0 of about 89-95, so the loss-matching target is a much denser operating point. The SAEBench interpretability metrics (absorption, SCR, TPP, RAVEL, sparse probing) are not part of the selection and are not affected by this circularity.

full rationale

This is an empirical study without a formal derivation chain, so most circularity patterns (self-definitional equations, uniqueness theorems, ansatz smuggling, renaming) do not apply. The one load-bearing step that reduces to its own inputs is the selection of the pruned SAE's sparsity in Section 3.2: the paper sweeps sparsity levels and picks the one whose reconstruction loss is 'comparable' to the SAE retrained on the pruned model, then presents reconstruction comparability as part of the headline result. For that component, the conclusion is enforced by the selection criterion rather than demonstrated. The l0 mismatch between the Trained baseline (roughly 2835-4658 active features) and the pruned variants (roughly 62-128 active features) is a correctness and confound concern about whether the comparison is at a matched operating point, not itself a circularity. The other evaluation dimensions (feature absorption, SCR, TPP, RAVEL, sparse probing) are independent of the selection and support the paper's broader transferability findings, which keeps the overall circularity partial rather than total. Score 6 reflects one central 'prediction' that is forced by construction, while leaving independent content in the remaining metrics.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central findings rest on several domain assumptions about SAEs and pruning, plus one fitted choice (25% pruned-SAE sparsity). No new entities are introduced. The claim that pruned SAEs are comparable is partly enforced by selecting the sparsity level on reconstruction loss.

free parameters (2)
  • pruned SAE sparsity level = 25%
    Swept over sparsity levels and selected the level where reconstruction loss is comparable to the SAE trained on the pruned model (Section 3.2). The headline result uses this selected value.
  • model pruning sparsity = 50%
    Chosen for both models; not varied for Gemma, so generalization to other sparsity levels is untested.
assumptions (4)
  • domain assumption Activation distributions of WANDA-pruned LLMs are close enough to the original that pretrained SAEs reconstruct them meaningfully.
    This is the transferability premise tested, assumed when applying pretrained SAEs to pruned activations (Section 3.2).
  • domain assumption SAEBench scores are valid measures of interpretability and reconstruction quality.
    The paper relies on SAEBench as ground truth without independent validation (Section 3.3).
  • domain assumption WANDA pruning preserves the features that SAEs encode.
    The method assumes pruning does not destroy the interpretable structure; this is part of the claim under test, not established prior.
  • domain assumption JumpReLU SAE configurations from prior work (Gemma Scope, sae_lens) are appropriate for pruned model activations.
    Pretrained SAEs are used without modification except pruning (Section 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the transferability of Sparse Autoencoders for interpreting compressed models." pith.science (2026). https://pith.science/paper/XENDIJLA

@misc{pith2026250715977,
  author       = {Pith},
  title        = {Pith review of: On the transferability of Sparse Autoencoders for interpreting compressed models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XENDIJLA}},
  note         = {Machine review of arXiv:2507.15977}
}
read the original abstract

Modern LLMs face inference efficiency challenges due to their scale. To address this, many compression methods have been proposed, such as pruning and quantization. However, the effect of compression on a model's interpretability remains elusive. While several model interpretation approaches exist, such as circuit discovery, Sparse Autoencoders (SAEs) have proven particularly effective in decomposing a model's activation space into its feature basis. In this work, we explore the differences in SAEs for the original and compressed models. We find that SAEs trained on the original model can interpret the compressed model albeit with slight performance degradation compared to the trained SAE on the compressed model. Furthermore, simply pruning the original SAE itself achieves performance comparable to training a new SAE on the pruned model. This finding enables us to mitigate the extensive training costs of SAEs.

Figures

Figures reproduced from arXiv: 2507.15977 by the authors.

Figure 1
Figure 1. Average reconstruction loss of different SAE variants across all layers of GPT-2 [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Absorption scores for Gemma-2-2B SAE variants. Higher Absorption and Full [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. SCR scores for Gemma-2-2B SAE variants. SCR measures the SAE’s ability to [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: TPP scores for Gemma-2-2B SAE variants. TPP evaluates how well each SAE local [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Reconstruction loss for GPT-2 small MLP output [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Reconstruction loss for GPT-2 small residual stream output [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Reconstruction loss for GPT-2 small attention output [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 13 canonical work pages

  1. [1]

    Refusal in language models is mediated by a single direction

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. arXiv preprint arXiv:2406.11717, 2024

    arXiv 2024

  2. [2]

    Batchtopk: A simple improvement for topk-saes, 2024a

    Bussmann B., Leask P., and N Nanda. Batchtopk: A simple improvement for topk-saes, 2024a. URL https://www.alignmentforum.org/posts/rKM9b6B2LqwSB5ToN/learning-multi-level-features-with-matryoshka-saes

    work page

  3. [3]

    Towards efficient post-training quantization of pre-trained language models

    Haoli Bai, Lu Hou, Lifeng Shang, Xin Jiang, Irwin King, and Michael R Lyu. Towards efficient post-training quantization of pre-trained language models. Advances in neural information processing systems, 35: 0 1405--1418, 2022

    work page 2022

  4. [4]

    A is for absorption: Studying feature splitting and absorption in sparse autoencoders, 2024

    David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders, 2024. URL https://arxiv.org/abs/2409.14507

    arXiv 2024

  5. [5]

    Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identification

    Vishnu Kabir Chhabra, Ding Zhu, and Mohammad Mahdi Khalili. Neuroplasticity and corruption in model mechanisms: A case study of indirect object identification. arXiv preprint arXiv:2503.01896, 2025

    work page Pith review arXiv 2025

  6. [6]

    Towards automated circuit discovery for mechanistic interpretability

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 0 16318--16352, 2023

    2023

  7. [7]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    arXiv 2023

  8. [8]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pp.\ 10323--10337. PMLR, 2023 a

    work page 2023

Show all 50 references
  1. [9]

    Sparsegpt: Massive language models can be accurately pruned in one-shot, 2023 b

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot, 2023 b . URL https://arxiv.org/abs/2301.00774

    2023 arXiv

  2. [10]

    The pile: An 800gb dataset of diverse text for language modeling, 2020

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling, 2020. URL https://arxiv.org/abs/2101.00027

    2020 arXiv

  3. [12]

    Scaling and evaluating sparse autoencoders, 2024 b

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024 b . URL https://arxiv.org/abs/2406.04093

    2024 arXiv

  4. [13]

    Openwebtext corpus

    Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwebtext corpus. http://Skylion007.github.io/OpenWebTextCorpus, 2019

    2019

  5. [14]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  6. [15]

    Finding neurons in a haystack: Case studies with sparse probing, 2023

    Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding neurons in a haystack: Case studies with sparse probing, 2023. URL https://arxiv.org/abs/2305.01610

    2023 arXiv

  7. [16]

    Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both weights and connections for efficient neural networks, 2015. URL https://arxiv.org/abs/1506.02626

    2015 arXiv

  8. [17]

    How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

    Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. Advances in Neural Information Processing Systems, 36: 0 76033--76060, 2023

    2023

  9. [18]

    Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression

    Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian Bartoldson, Ajay Jaiswal, Kaidi Xu, et al. Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression. arXiv preprint arXiv:2403...

    2024 arXiv

  10. [19]

    Language model compression with weighted low-rank factorization, 2022

    Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. Language model compression with weighted low-rank factorization, 2022. URL https://arxiv.org/abs/2207.00112

    2022 arXiv

  11. [20]

    Curt Tigges Joseph Bloom and David Chanin. Saelens. https://github.com/jbloomAus/SAELens, 2024

    2024

  12. [21]

    Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability, 2025 a

    Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Arthur Conmy, Samuel Marks, and Neel Nanda. Saebench: A comprehensive benchmark for sparse autoencoders in language mod...

    2025 arXiv

  13. [22]

    Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability, 2025 b

    Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Arthur Conmy, Samuel Marks, and Neel Nanda. Saebench: A comprehensive benchmark for sparse autoencoders in language mod...

    2025

  14. [23]

    Sparse autoencoders reveal universal feature spaces across large language models

    Michael Lan, Philip Torr, Austin Meek, Ashkan Khakzar, David Krueger, and Fazl Barez. Sparse autoencoders reveal universal feature spaces across large language models. arXiv preprint arXiv:2410.06981, 2024

    2024 arXiv

  15. [24]

    Taking features out of superposition with sparse autoencoders, 2022

    Beren Millidge Lee Sharkey, Dan Braun. Taking features out of superposition with sparse autoencoders, 2022. URL https://www.alignmentforum.org/posts/z6QQJbtpkEAX3Aojj/interim-research-report-taking-features-out-of-superposition

    2022

  16. [25]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024. URL https://arxiv.org/abs/2408.05147

    2024 arXiv

  17. [26]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--...

    2024

  18. [27]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36: 0 21702--21720, 2023

    2023

  19. [28]

    Towards principled evaluations of sparse autoencoders for interpretability and control, 2024

    Aleksandar Makelov, George Lange, and Neel Nanda. Towards principled evaluations of sparse autoencoders for interpretability and control, 2024. URL https://arxiv.org/abs/2405.08366

    2024 arXiv

  20. [29]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models, 2024. URL https://arxiv.org/abs/2403.19647

    2024 arXiv

  21. [30]

    Transformerlens

    Neel Nanda and Joseph Bloom. Transformerlens. https://github.com/TransformerLensOrg/TransformerLens, 2022

    2022

  22. [31]

    Sparse autoencoders trained on the same data learn different features, 2025

    Gonçalo Paulo and Nora Belrose. Sparse autoencoders trained on the same data learn different features, 2025. URL https://arxiv.org/abs/2501.16615

    2025 arXiv

  23. [32]

    Fine-tuning enhances existing mechanisms: A case study on entity tracking

    Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. Fine-tuning enhances existing mechanisms: A case study on entity tracking. arXiv preprint arXiv:2402.14811, 2024

    2024 arXiv

  24. [33]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019

    2019

  25. [34]

    Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024. URL https://arxiv.org/abs/2407.14435

    2024 arXiv

  26. [35]

    Goldsmith, and Mert Pilanci

    Rajarshi Saha, Naomi Sagan, Varun Srivastava, Andrea J. Goldsmith, and Mert Pilanci. Compressing large language models using low rank and low precision decomposition, 2024. URL https://arxiv.org/abs/2405.18886

    2024 arXiv

  27. [36]

    Omniquant: Omnidirectionally calibrated quantization for large language models

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models. arXiv preprint arXiv:2308.13137, 2023

    2023 arXiv

  28. [37]

    A simple and effective pruning approach for large language models

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695, 2023

    2023 arXiv

  29. [38]

    Jermyn A., Turner N., Anil C., Denison C., Lasenby R

    Bricken T., Templeton A., Batson J., Chen B., Conerly T. Jermyn A., Turner N., Anil C., Denison C., Lasenby R. Askell A., Wu Y., Kravec S., Schiefer N., Maxwell T., Joseph N., Hatfield-Dodds Z., Tamkin A., Nguyen K., McLean B., Burke J. E., Hume T., Henighan T. Carter S., and ...

    2023

  30. [39]

    G. M. Prolu Taggart. A nonlinearity for sparse autoencoders, 2024. URL https://www.alignmentforum.org/posts/HEpufTdakGTTKgoYF/prolu-a-nonlinearity-for-sparse-autoencoders

    2024

  31. [40]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  32. [41]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  33. [42]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurip...

    2017

  34. [43]

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593, 2022

    2022 arXiv

  35. [44]

    High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications

    John Wright and Yi Ma. High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications. Cambridge University Press, 2022

    2022

  36. [45]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp.\ 38087--38099. PMLR, 2023

    2023

  37. [46]

    Beyond perplexity: Multi-dimensional safety evaluation of llm compression, 2024

    Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. Beyond perplexity: Multi-dimensional safety evaluation of llm compression, 2024. URL https://arxiv.org/abs/2407.04965

    2024 arXiv

  38. [47]

    A survey on efficient inference for large language models

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, et al. A survey on efficient inference for large language models. arXiv preprint arXiv:2404.14294, 2024

    2024 arXiv

  39. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  40. [49]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  41. [50]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  42. [51]

    dzaL :3,cu' UW

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.