Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sparse autoencoders find thousands of interpretable features in protein language models.

desk verdict A solid transfer of SAE interpretability to protein language models with a useful evaluation framework, but the headline feature counts need a matched control before I'd trust the exact gap. read the letter →

arxiv 2412.12101 v1 pith:4ERCPANK submitted 2024-11-13 q-bio.BM cs.AIcs.LGq-bio.QM

Elana Simon , James Zou This is my paper · ORCID
classification q-bio.BMcs.AIcs.LGq-bio.QM
keywords proteinlanguagemodelssparseautoencodersmechanisticinterpretabilitysuperpositionESM-2biologicalconceptannotationsfeaturesteeringsequencegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what a protein language model actually knows about biology and argues that most of that knowledge is stored in a form that individual neurons do not expose. By training sparse autoencoders on the hidden embeddings of the protein language model ESM-2, the authors claim to recover up to 2,548 interpretable features per layer that match 143 known biological concepts such as binding sites, structural motifs, and functional domains. Raw neurons, by contrast, yield at most 46 matching features across 15 concepts, which the paper reads as evidence that ESM-2 represents biological concepts in superposition rather than in single neurons. The paper also shows that these features can flag missing database annotations and that activating a feature can steer sequence generation in interpretable ways. If correct, this turns the model into a source of both known and potentially novel protein biology.

What carries the argument

The central object is the sparse autoencoder dictionary: for each of ESM-2's six layers, the 320-dimensional embedding of every amino acid is reconstructed as a sparse combination of 10,240 feature vectors, with sparsity enforced by an L1 penalty during training. The features are normalized so activation values lie between 0 and 1. Concept matching uses a modified F1 metric: precision is scored per amino acid, while recall is scored per annotated domain, and the best activation threshold for each feature-concept pair is chosen on a validation split and then evaluated on a held-out split. The other load-bearing pieces are the large-language-model pipeline that converts activation examples into feature descriptions, and the steering procedure that clamps a feature's activation in the reconstructed embedding, lets the model continue processing, and measures changes in output probabilities.

What would settle it

Recompute the feature-concept F1 scores with recall counted per amino acid instead of per domain; if the number of sparse autoencoder features above 0.5 F1 falls to the level of raw neurons, the paper's headline gap is metric-driven. A second check is experimental: take a protein with a predicted but unannotated binding site from feature activation, assay whether the proposed residues actually bind the predicted ligand, and see whether the 'missing annotation' calls survive.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that sparse autoencoders trained on ESM-2's per-layer amino acid embeddings decompose the model's representations into a large dictionary of sparse, human-interpretable features, and that these features are the right unit for reading biology out of the model. The headline evidence is quantitative: using a modified F1 score that counts precision per amino acid but recall per annotated domain, up to 2,548 features in a layer exceed 0.5 F1 against one of 143 curated protein concepts on held-out data, while the raw neurons of the same embeddings reach only 46 such matches across 15 concepts. The same analysis on a model with randomly shuffled weights finds no biological concepts, indicating the features reflect learned weights rather than trivial sequence statistics. The paper further claims that features can identify missing database annotations, that a large language model can generate descriptions of features that predict activation on held-out proteins with a median Pearson correlation of 0.72, and that clamping a periodic-glycine feature on one residue raises the model's probability of glycine at a masked position three residues away. Together these results are offered as evidence that ESM-2 encodes rich, superpositioned biological knowledge that sparse autoencoder features can extract, interpret, and intervene on.

Load-bearing premise

The load-bearing assumption is that a feature matches a concept when it hits at least one amino acid in each annotated domain, so a feature covering two residues of a long domain counts as full recall; if that per-domain recall choice is unfair, the large advantage of sparse autoencoder features over neurons could be an artifact of the metric rather than a real difference in interpretability.

Editorial extensions

If this is right

  • Individual-neuron probes understate what protein language models know; most biological concepts in ESM-2 appear to be stored in superposition and only become visible through dictionary learning.
  • Sparse autoencoder features can propose edits to protein databases: activations on unannotated proteins point to missing motif, domain, and binding-site annotations that external databases corroborate.
  • Feature steering gives a handle on generation: activating a periodic-glycine feature at a single residue raises glycine probability at later masked positions, so features can act at a distance.
  • Large-language-model-generated descriptions extend interpretability beyond existing annotations, covering features that no curated concept labels, with activation predictions that correlate with measured values.
  • The release of an interactive platform makes these per-layer features explorable across sequence, structure, and annotation views.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's direct evidence, the same pipeline could be run on larger ESM variants to test whether the feature-count gap grows with model scale; the paper only analyzes the 8-million-parameter ESM-2.
  • The per-domain recall metric is the load-bearing choice; if a stricter amino-acid-level recall were applied, the 2,548-versus-46 gap might shrink, so an independent reanalysis with that metric would sharpen the claim.
  • Features that activate consistently but match no curated concept could be treated as predictions of novel motifs and tested experimentally, for example by mutating the activated residues and assaying binding or activity.
  • The steering result suggests feature-level control could become a protein engineering tool, but the paper tests only a few short sequences; broader sequence contexts are needed to know when the control holds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper trains sparse autoencoders (SAEs) on ESM-2-8M embeddings across all six layers and evaluates the interpretability of the resulting features against Swiss-Prot annotations. It reports that up to 2,548 SAE features per layer achieve F1 > 0.5 with 143 biological concepts on held-out data, versus 46 neurons and 15 concepts for raw ESM-2 neurons. Additional contributions include an LLM-based pipeline for describing features, examples of using feature activations to suggest missing database annotations, and steering experiments in which activating periodic-glycine features increases glycine probability at masked positions. The authors release code and an interactive dashboard (InterPLM).

Significance. If the main quantitative claim holds, the paper provides a substantial toolkit for interpreting protein language models and a new evaluation protocol for SAE features. The held-out split, the randomized-weight control, and the LLM description validation are credible design choices, and the public code and dashboard are valuable community resources. However, the headline SAE-versus-neuron comparison rests on a non-standard F1 metric and on validation-set threshold and feature selection that may systematically favor the much larger and sparser SAE dictionary, so the 2,548-versus-46 gap should be treated as provisional until additional controls are supplied.

major comments (4)
  1. [§3.3, Eqs. (1)-(3)] The modified F1 metric is asymmetric: precision is computed per amino acid while recall is computed per domain (Eq. 2). This can produce F1 = 1 for a feature that activates on exactly one residue inside each annotated domain, regardless of whether the feature encodes the concept. Because SAE features are trained to be sparse, they are far more likely than raw neuron activations to achieve this 'one hit per domain' pattern, so the gap between SAE features and neurons may be inflated by sparsity rather than by conceptual specificity. I request a label-permutation or activation-permutation control that reports the distribution of F1 > 0.5 counts under the null, and a secondary analysis using standard per-residue precision and recall so readers can see the effect of the domain-level recall adjustment.
  2. [§5.2.3] The validation set is used both to select the best feature per concept and to select the activation threshold for each feature-concept pair before held-out F1 is reported. Because the SAE dictionary has 10,240 features per layer versus 320 neurons for the identity-SAE baseline, the validation maximum over features is subject to a much larger winner's curse for SAEs. Although held-out F1 provides some protection, the validation and test sets are drawn from the same Swiss-Prot distribution and share systematic biases (for example, amino acid composition inside domains), so spurious validation winners may persist on test. The paper should report results with a fixed threshold (for example, 0.5) without per-pair threshold selection, or with a multiple-testing correction, or with a null distribution from permuted concept labels.
  3. [§4.1 and §3.3] The conclusion that PLMs store concepts in superposition is not directly supported by the current comparison, because the SAE dictionary is 32 times larger than the neuron set and the features are much sparser. The 2,548-versus-46 count is therefore consistent with a trivial 'more features, more hits' explanation. To support the superposition claim, the paper should demonstrate that individual neurons are polysemantic (for example, one neuron has high F1 for multiple unrelated concepts) and that SAE features are more monosemantic, ideally after matching the neuron baseline for dictionary size or sparsity. This analysis is load-bearing for the paper's central interpretability claim.
  4. [§3.7 and Figure 7] The steering experiments select features because they activate on periodic glycine patterns and then show that activating these features increases glycine probability; this is partly circular, since the selection criterion already encodes the periodic glycine property. The comparison against non-periodic glycine features is informative but those features were also selected for glycine association, so they do not form a matched control. A stronger control would include randomly selected features matched for activation frequency and sparsity, with steering effects reported on all amino acid positions, not only glycine. The paper's own limitation statement acknowledges the narrow scope, but the abstract's claim of 'targeted steering of protein sequence generation' needs this additional control.
minor comments (6)
  1. [§5.4.2] Steps 3 and 4 of the sequential/structural analysis are identical ('Generate null distributions through averaging 5 random permutations per protein'); one of them should be removed or the intended difference stated.
  2. [Figure 7 caption] The caption uses 'Steer %' while the text describes clamping values where '1x is the maximum observed activation value'; these units should be made consistent and precisely defined.
  3. [§3.2] The phrase 'These feature's characteristic activation modes' should be 'These features' characteristic activation modes'.
  4. [Appendix C.1 and C.2] The headings 'Swiss-Prot Metadata Categories' appear twice in immediate succession; renumber or merge these appendices.
  5. [§5.1.1 and §5.2.1] It would be helpful to state whether the 5M UniRef50 proteins used for SAE training overlap with the 50,000 Swiss-Prot proteins used for evaluation, and how any overlap was handled.
  6. [Figure 3a] The caption should explicitly state that the F1 score shown is the held-out F1 for the validation-selected best feature per concept, rather than a validation-set F1.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the SAE-to-concept comparison is evaluated on held-out Swiss-Prot labels with matched baselines, and no predicted quantity reduces to a fitted input by construction.

full rationale

The paper's central derivation chain associates unsupervised SAE features with external Swiss-Prot concepts. Features are trained without concept labels (Section 5.1), then scored on a held-out 25,000-protein test split using Eqs (1)-(3) after threshold and best-feature selection on the validation set (Sections 5.2.2-5.2.3); the reported F1 counts are recomputed on the held-out set, so the 2,548-vs-46 neuron comparison is not forced by the fitting procedure. Randomized-weight and identity-neuron baselines pass through the identical analysis pipeline, providing matched controls. The steering experiments test features that were selected by their activation patterns, which is a selection-bias concern rather than a definitional equivalence. The missing-annotation examples are checked against InterPro for the main cases, so they are not circular. The only self-citation ([5], a general methods primer) appears in background material and is not load-bearing. No equation or definition in the paper makes a predicted quantity equal to its input by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a small set of free choices: the SAE expansion factor and hyperparameters, the activation thresholds used to binarize features, the F1>0.5 interpretability cutoff, and the adjusted per-domain recall metric. These are not derived from theory and are tuned on validation data, though the held-out evaluation mitigates overfitting. The main domain assumptions are that ESM-2 embeddings are a valid substrate for biological concepts and that Swiss-Prot is a reliable gold standard; the latter is partially acknowledged as incomplete in Section 3.6. No new physical entities are introduced.

free parameters (5)
  • SAE expansion factor = 32x (dictionary size 10,240)
    Chosen by hand, not derived; determines the number of features extracted.
  • L1 penalty per layer = 0.07 to 0.2 range; final 0.08 to 0.1 per layer
    Hyperparameter search over L1 range; final model per layer selected based on Swiss-Prot concept F1 on a 20% validation subset (Section 5.2.3).
  • Learning rate per layer = 1e-7 for all selected models
    Sampled from 1e-4 to 1e-8; selected via the same concept-based model selection.
  • Activation threshold for binarizing features = 0, 0.15, 0.5, 0.6, 0.8; best per feature-concept pair
    Threshold is selected on validation set to maximize F1 for each pair, then applied to test; affects all counts.
  • Interpretable-feature cutoff = F1 > 0.5
    The headline counts (2,548 features, 143 concepts) depend on this arbitrary cutoff; changing it changes the result.
assumptions (4)
  • domain assumption ESM-2-8M embeddings contain biologically meaningful information that can be linearly decomposed into interpretable features.
    The entire SAE approach assumes the embedding geometry separates concepts along dictionary directions; cited in Section 2.2 and 3.1.
  • domain assumption Swiss-Prot annotations are a valid and sufficiently complete ground truth for biological concepts.
    Used as the gold standard for concept association in Section 5.2; incompleteness is acknowledged only later in Section 3.6.
  • ad hoc to paper The per-domain recall adjustment is an unbiased way to compare features with coarse annotations.
    Introduced in Section 3.3 to fix the granularity mismatch; no external validation that it preserves ranking across methods.
  • domain assumption Randomly shuffling ESM-2 weights produces a valid null model for feature interpretability.
    Used as the control baseline in Section 5.2.4; assumes shuffled weights destroy only learned structure, not data statistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders." pith.science (2026). https://pith.science/paper/4ERCPANK

@misc{pith2026241212101,
  author       = {Pith},
  title        = {Pith review of: InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ERCPANK}},
  note         = {Machine review of arXiv:2412.12101}
}
read the original abstract

Protein language models (PLMs) have demonstrated remarkable success in protein modeling and design, yet their internal mechanisms for predicting structure and function remain poorly understood. Here we present a systematic approach to extract and analyze interpretable features from PLMs using sparse autoencoders (SAEs). By training SAEs on embeddings from the PLM ESM-2, we identify up to 2,548 human-interpretable latent features per layer that strongly correlate with up to 143 known biological concepts such as binding sites, structural motifs, and functional domains. In contrast, examining individual neurons in ESM-2 reveals up to 46 neurons per layer with clear conceptual alignment across 15 known concepts, suggesting that PLMs represent most concepts in superposition. Beyond capturing known annotations, we show that ESM-2 learns coherent concepts that do not map onto existing annotations and propose a pipeline using language models to automatically interpret novel latent features learned by the SAEs. As practical applications, we demonstrate how these latent features can fill in missing annotations in protein databases and enable targeted steering of protein sequence generation. Our results demonstrate that PLMs encode rich, interpretable representations of protein biology and we propose a systematic framework to extract and analyze these latent features. In the process, we recover both known biology and potentially new protein motifs. As community resources, we introduce InterPLM (interPLM.ai), an interactive visualization platform for exploring and analyzing learned PLM features, and release code for training and analysis at github.com/ElanaPearl/interPLM.

Figures

Figures reproduced from arXiv: 2412.12101 by the authors.

Figure 1
Figure 1. Overview of SAE methodology and representative SAE features revealed through automated activation pattern analysis. a) Pipeline illustrating the extraction of embeddings, their conversion to features, and subsequent reinsertion of reconstructed embeddings into the PLM. b-c) Examples of features exhibiting interpretable activation patterns, both structural and conceptual. Each feature is visualized using a protein wh… view at source ↗
Figure 2
Figure 2. SAE feature analysis and visualizations reveal features with diverse and consistent activation patterns. a) Quantitative comparison of learned features through four complementary approaches: 1) Feature activation frequency distribution showing the relationship between proteome-wide prevalence (x-axis) and protein-specific activation strength (y-axis), revealing both ubiquitous and selective features 2) Structural vs… view at source ↗
Figure 3
Figure 3. SAE features have stronger associations with Swiss-Prot concepts than ESM neurons. Comparing the features of an SAE model trained on ESM-2 embeddings (pink), the original neurons of the ESM-2 embeddings (blue), and the features of an SAE trained on embeddings from an ESM-2 model with shuffled weights (green). Models are compared based on the F1 scores between features and Swiss-Prot concepts. (a) For each concept, s… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Clustering reveals groups of features with similar functional and structural roles but subtle differences in activation patterns. (a) UMAP of SAE features clustered based on their dictionary values. Features associated with one of the top 20 most commonly labeled Swiss…
Figure 5
Figure 5. Figure 5: Language models can generate automatic feature descriptions for SAE features. (a) Workflow for generating and validating descriptions with Claude-3.5 Sonnet (new). (b) Comparing measured maximum activation values in proteins to predicted maximum activation values via P…
Figure 6
Figure 6. Figure 6: Feature activation patterns can be used to identify missing and new protein annotations. (a) f/939 identifies missing motif annotation for Nudix box. It activates on a single amino acid in conserved position which is labeled in each structure. Right two proteins are ex…
Figure 7
Figure 7. Figure 7: Steering protein sequence generation for features that activates on glycines in periodic repeats of GXX. a) The maximally activated examples for 3 features that activate on periodic glycine repeats. Each maximally activates in collagen-like domains of complement C1q re…
Figure 8
Figure 8. Figure 8: Overview of SAE decomposition and training. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 1
Figure 1. Figure 1: Overview of process for training sparse autoencoders on PLM embeddings e equations: [PITH_FULL_IMAGE:figures/full_fig_p014_1.png]
Figure 9
Figure 9. Figure 9: Additional feature-concept analysis across layers [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Comparing quality of Swiss-Prot concept labels and accuracy of LLM predicted feature activation patterns [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Example of features in glycosylastransferase cluster (2244,6262,9047) with varied maximally activating [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Top row (a-c): Steering MGPP<mask>PP on each of the individual Periodic Glycine Features. Bottom row: [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Measuring the linear effect on the predicted probability of Glycine. Evaluated p(Glycine) at each position [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders

    cs.SD 2026-02 conditional novelty 6.0 of 10

    SAE features from Whisper and HuBERT are seed-stable, interpretable, and steerable: cutting false speech detections by 70% and correlating with EEG responses to speech.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Evolutionary-scale prediction of atomic-level protein structure with a language model

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, March 2023...

    work page 2023

  2. [2]

    Ruffolo, Eli N

    Erik Nijkamp, Jeffrey A. Ruffolo, Eli N. Weinstein, Nikhil Naik, and Ali Madani. ProGen2: Exploring the boundaries of protein language models. Cell Systems, 14(11):968–978.e3, November 2023. Publisher: Elsevier

    work page 2023

  3. [3]

    Wayment-Steele, Garyk Brixi, Haobo Wang, Dorothee Kern, and Sergey Ovchinnikov

    Zhidian Zhang, Hannah K. Wayment-Steele, Garyk Brixi, Haobo Wang, Dorothee Kern, and Sergey Ovchinnikov. Protein language models learn evolutionary statistics of interacting sequence motifs. Proceedings of the National 22 InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders Linear Effect on P(Glycine) when Steeri...

    work page 2024

  4. [4]

    The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    Anthropic. The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    work page 2024

  5. [5]

    Language models for biological research: a primer

    Elana Simon, Kyle Swanson, and James Zou. Language models for biological research: a primer. Nature Methods, 21(8):1422–1429, August 2024. Publisher: Nature Publishing Group

    work page 2024

  6. [6]

    Transformer protein language models are unsupervised structure learners

    Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, and Alexander Rives. Transformer protein language models are unsupervised structure learners. October 2020

    work page 2020

  7. [7]

    Varshney, Caiming Xiong, Richard Socher, and Nazneen Rajani

    Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, and Nazneen Rajani. BERTology Meets Biology: Interpreting Attention in Protein Language Models. October 2020

    work page 2020

  8. [8]

    Marks, Lucy J

    Debora S. Marks, Lucy J. Colwell, Robert Sheridan, Thomas A. Hopf, Andrea Pagnani, Riccardo Zecchina, and Chris Sander. Protein 3D Structure Computed from Evolutionary Sequence Variation. PLOS ONE, 6(12):e28766, December 2011. Publisher: Public Library of Science

    work page 2011

Show all 30 references
  1. [9]

    Ballard, Joshua Bambrick, Sebastian W

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Bodenstein, David A. Evans, Chia-Chun Hung, Michael O’Neill, David Reiman, Kathryn Tunyasuvunakool, Zachar...

    2024

  2. [10]

    Linear Algebraic Structure of Word Senses, with Applications to Polysemy, December 2018

    Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear Algebraic Structure of Word Senses, with Applications to Polysemy, December 2018. arXiv:1601.03764

    2018 arXiv

  3. [11]

    Zoom In: An Introduction to Circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom In: An Introduction to Circuits. Distill, 5(3):e00024.001, March 2020

    2020

  4. [12]

    Olshausen, and Yann LeCun

    Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors, April 2023. arXiv:2103.15949

    2023 arXiv

  5. [13]

    Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023. arXiv:2309.08600

    2023 arXiv

  6. [14]

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen,...

    2023

  7. [15]

    The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision, September 2024

    Liv Gorton. The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision, September 2024. arXiv:2406.03662

    2024 arXiv

  8. [16]

    In-context Learning and Induction Heads

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kam...

    2022

  9. [17]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse Feature Cir- cuits: Discovering and Editing Interpretable Causal Graphs in Language Models, March 2024. arXiv:2403.19647

    2024 arXiv

  10. [18]

    SAE Visualizer, 2024

    Callum McDougall. SAE Visualizer, 2024

    2024

  11. [19]

    Language models can explain neurons in language models, 2023

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023

    2023

  12. [20]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

    2024

  13. [21]

    Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models, October 2024

    Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks. Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models, October 2024. arXiv:2408.00113

    2024 arXiv

  14. [22]

    UMAP: Uniform Manifold Approximation and Projection

    Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. UMAP: Uniform Manifold Approximation and Projection. Journal of Open Source Software, 3(29):861, September 2018

    2018

  15. [23]

    On expert curation and scalability: UniProtKB/Swiss-Prot as a case study

    Sylvain Poux, Cecilia N Arighi, Michele Magrane, Alex Bateman, Chih-Hsuan Wei, Zhiyong Lu, Emmanuel Boutet, Hema Bye-A-Jee, Maria Livia Famiglietti, Bernd Roechert, and The UniProt Consortium. On expert curation and scalability: UniProtKB/Swiss-Prot as a case study. Bioinforma...

    2017

  16. [24]

    InterPro in 2022

    Typhaine Paysan-Lafosse, Matthias Blum, Sara Chuguransky, Tiago Grego, Beatriz Lázaro Pinto, Gustavo A Salazar, Maxwell L Bileschi, Peer Bork, Alan Bridge, Lucy Colwell, Julian Gough, Daniel H Haft, Ivica Letuni´c, Aron Marchler-Bauer, Huaiyu Mi, Darren A Natale, Christine A O...

    2022

  17. [25]

    Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. Density-Based Clustering Based on Hierarchical Density Estimates. In David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan,...

    2013

  18. [26]

    Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik Brinkmann, Caden Juang, Koyena Pal, Can Rager, Aaron Mueller, Samuel Marks, Arnab Sen Sharma, Francesca Lucchetti, Michael Ripa, Adam Belfki, Nikhil Prakash, Sumeet Multani, Carla Brodley, Arjun Guha, Jonathan Bell, ...

    2024 arXiv

  19. [27]

    Scaling and evaluating sparse autoencoders, June 2024

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, June 2024. arXiv:2406.04093

    2024 arXiv

  20. [28]

    Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders, July

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders, July

  21. [29]

    Transcoders Find Interpretable LLM Feature Circuits, November 2024

    Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders Find Interpretable LLM Feature Circuits, November 2024. arXiv:2406.11944 version: 2. 25

    2024 arXiv

  22. [2024]

    arXiv:2407.14435 version: 1

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.