Pith. sign in

REVIEW 3 major objections 7 minor 26 references

Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse dictionary learning can pull biologically meaningful concepts—cell types, genetic perturbations, and subtle morphological changes—out of an unsupervised microscopy foundation model, without text or labels, if representations are…

desk verdict A useful demonstration that dictionary learning can extract biologically meaningful features from microscopy MAEs; the main evaluation has a layer-selection leak that should be fixed before publication. read the letter →

arxiv 2412.16247 v3 pith:2THACNB5 submitted 2024-12-20 cs.LG cs.AIcs.CVstat.ML

Konstantin Donhauser , Kristina Ulicna , Gemma Elyse Moran , Aditya Ravuri , Kian Kenyon-Dean , Cian Eastwood , Jason Hartford This is my paper · ORCID
classification cs.LGcs.AIcs.CVstat.ML
keywords dictionarylearningmechanisticinterpretabilitymicroscopyfoundationmodelsmaskedautoencoderssparsecodingcellpaintingPCAwhiteningbiologicalconcepts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether sparse dictionary learning—the technique used to find interpretable features inside large language models—can also expose meaningful concepts inside an unsupervised vision model trained on cell images, where no text or labels guide the model. The authors propose combining a new dictionary-learning algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening step derived from unperturbed control cells, and report that the resulting features align with cell types, genetic perturbations, and functional gene groups. They find that ICFL features match the interpretability of expert-designed CellProfiler features, that linear probes on the sparse reconstructions retain most of the biological signal of the original representations, and that token-level heatmaps can separate perturbed from unperturbed cells in a single image. If these results hold, interpretability tools originally built for text models can become a discovery engine in a domain where experts do not yet know which concepts to look for.

What carries the argument

The central object is the learned dictionary, a matrix $W_{\text{dec}}$ whose columns are feature directions in the embedding space, together with sparse coefficient vectors $z$ such that a representation $x$ is approximately $W_{\text{dec}} z + b_{\text{pre}}$. The algorithm that carries the argument is ICFL, a batched variant of matching pursuit: at each iteration it selects the codebook columns most aligned with the current residual, solves for their coefficients, subtracts the reconstruction, and repeats, while the codebook itself is updated by gradient descent on the reconstruction loss. The other load-bearing component is a PCA whitening map computed from unperturbed control cells, which reweights the representation space so that dominant control-derived variance directions are downweighted and perturbation-relevant differences are amplified before ICFL is applied. This combination is what lets a decoder maximize reconstruction fidelity without being dominated by nuisance variation.

What would settle it

Run the same ICFL pipeline on a different microscopy dataset in which the control images come from one cell line and the perturbed images from another; if linear-probe accuracy on the non-control cell line's perturbations falls well below the unwhitened baseline while the control line's accuracy stays high, the control-derived whitening is removing cross-cell-type biological signal, and the reported generality would not transfer.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a masked autoencoder trained on microscopy images stores biologically meaningful concepts as sparse, linear feature directions, and that those directions can be recovered by dictionary learning if the encoder is paired with a control-derived whitening transform. The authors report retrieving features that correlate with specific cell types, individual siRNA and CRISPR perturbations, and functional gene groups such as adherens junctions, and they show that these features reveal morphology at a single-cell resolution in token-level heatmaps. They furthermore claim that the recovered features preserve a substantial fraction of the linear-probing signal of the original representations on five biological tasks, and that they attain selectivity comparable to hand-crafted CellProfiler features. The argument is empirical: the evidence is a set of selectivity scores, linear-probing accuracies, expert visualizations, and a small expert-annotated single-cell recall table.

Load-bearing premise

The load-bearing premise is that the variation present in unperturbed control cells is purely nuisance variation, and that this nuisance subspace is shared by all other cell types and perturbations, so that downweighting those directions amplifies only perturbation-relevant signal without removing any biology that distinguishes perturbations.

Editorial extensions

If this is right

  • Linear probes trained on ICFL reconstructions retain almost all of the biological signal of the original MAE representations for simple concepts like cell types and batch, and a substantial fraction for harder tasks like distinguishing hundreds of genetic perturbations.
  • ICFL features approach the selectivity of CellProfiler hand-crafted features and correlate with them strongly (Pearson coefficient 0.71), suggesting sparse directions capture morphology that experts already measure.
  • Token-level heatmaps aligned with a feature direction can identify which individual cells in a mixed image are perturbed versus control-like, with 87% recall on the expert-annotated set, in a supervision-free way.
  • ICFL avoids dead features far better than TopK SAEs and achieves higher reconstruction similarity at equal sparsity, particularly when combined with PCA whitening.
  • Larger models (MAE-G over MAE-L) improve ICFL feature selectivity on complex tasks, so the dictionary-learning route benefits from scaling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if feature directions genuinely correspond to morphological concepts, they could serve as automated hypothesis generators for drug mechanism-of-action, by linking a new perturbation's feature profile to known pathways without manual image review.
  • Beyond the paper: the reliance on a single control cell type suggests a testable extension: derive the whitening transform from multiple control conditions or cell types and measure whether feature selectivity on non-HUVEC lines improves or degrades, which would delimit how universal the nuisance subspace is.
  • Beyond the paper: ICFL's matching-pursuit structure could transfer to other scientific modalities where a 'control condition' defines nuisance variation, such as genomics or hyperspectral imaging, using the same control-whitening trick.
  • Beyond the paper: because the paper shows linear directions encode biology, one could attempt causal intervention experiments—steering the feature direction for adherens junctions and checking whether reconstructed images shift toward the expected morphology—to test whether the directions are causally used by the model, not merely correlated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Iterative Codebook Feature Learning (ICFL), a dictionary-learning method that combines batched matching pursuit with gradient updates of the decoder, and couples it with a PCA whitening transform estimated on unperturbed control cells. The authors apply this pipeline to two masked-autoencoder microscopy foundation models (MAE-L and MAE-G) and evaluate whether the resulting sparse features correspond to known biological concepts. They report that ICFL produces fewer dead features and better reconstruction cosine similarity than TopK sparse autoencoders, that features with high selectivity exist for cell types, batches, and genetic perturbations, that linear probes on ICFL reconstructions preserve a large fraction of the original representation's balanced test accuracy, and that domain experts can interpret selected features in terms of morphology and single-cell identity. The work is framed as a step toward mechanistic interpretability for scientific imaging data.

Significance. If the results are robust, this is a useful and timely contribution: it extends sparse dictionary learning from text and natural-image models to unsupervised scientific foundation models, proposes a practical control-data whitening preprocessing step, and introduces a five-task benchmark for measuring biological signal retention. The use of public datasets (RxRx1/RxRx3), comparison with the standard TopK SAE baseline, and expert-based case studies are strengths. However, the central quantitative claims are currently supported only by point estimates, and the functional-group preservation result is affected by an evaluation leak in layer selection. The method is not fully unsupervised, since the whitening step uses knowledge of control samples; this should be stated prominently.

major comments (3)
  1. [§5, Fig. 4A, Table 2] The token-extraction layer is selected by maximizing linear probing performance on the functional-group task (§5), and the same task (Task 5) is then used in Fig. 4A and Table 2 to demonstrate that ICFL reconstructions preserve genetic-perturbation signal. This makes the reported Task 5 BTA and selectivity counts selection-optimistic: the layer is a hyperparameter of the full pipeline, and the evaluation labels have been used to set it. I request a nested/held-out evaluation (layer selection on a validation split, reporting on a disjoint test split) or, failing that, a plot of the Task 5 BTA/selectivity across all candidate layers. This is a correctable evaluation leak, but it directly affects the headline claim that dictionary learning retrieves genetic perturbations.
  2. [§6.1–6.3, Fig. 4A, Table 2] The main quantitative comparisons (BTA in Fig. 4A, selectivity counts in Table 2, Pearson r=0.71 in Fig. 2C) are reported without confidence intervals or error bars. The differences that support claims such as 'ICFL retains more biological signal than TopK SAE' are often a few BTA points, and some counts are small (Task 5, threshold 0.2: 37 features). Please provide confidence intervals (e.g., bootstrap over wells/experiments or multiple training seeds) for the key numbers, so the reader can assess whether the reported improvements are within sampling noise.
  3. [§4.1] The PCA whitening transform is estimated on unperturbed HUVEC control cells and applied to all other cell types and perturbations. This assumes that the control covariance captures nuisance variation that is shared across conditions and cell types, and that downweighting these directions does not remove signal needed to distinguish perturbations or non-HUVEC cell types. The assumption is plausible but untested; the paper should report sensitivity, e.g., by comparing whitening transforms estimated on per-cell-type controls (for Task 1) or by showing that the top control variance directions are not informative for the perturbation tasks. The with/without-whitening ablation partially addresses this, but not the transfer across cell types.
minor comments (7)
  1. [§5] The sentence 'We selected this layer by finding the maximized linear probing performance on the functional group task from the original embeddings (§ 5)' refers to the section in which it appears; please specify the number of layers tried, the split used, and whether the selection was repeated for each model.
  2. [§6.1, Fig. 3] The procedure for selecting the feature directions displayed in Figure 3 is not described; please state whether these were chosen post hoc because they separate the two groups, and report how many features show separation of comparable quality.
  3. [§7, Table 3] The single-cell recall analysis is based on five images selected among the strongest-correlated images and a single expert annotator; please state the image-selection rule and consider reporting inter-annotator agreement.
  4. [§4.1] The symbol W denotes both the PCA whitening matrix in Section 4.1 and the dictionary matrix in Section 3; renaming one of these would remove ambiguity.
  5. [Table 1] The dead-feature criterion 'activated less than a fraction of 10^-5× during the last 1000 training steps' is ambiguous; please define the denominator and the activation threshold explicitly.
  6. [§6.2] For the CellProfiler comparison, please specify how the quantile thresholds α are chosen and whether the same α is applied to all labels; matching the sparsity only on average may not make the features equally comparable across labels.
  7. [Fig. 4A] Please clarify whether the 'original representation' baseline is the PCA-whitened or non-whitened representation, so that the with- and without-whitening panels can be interpreted correctly.

Circularity Check

1 steps flagged · score 2.0 of 10

Layer selection is tuned on the functional-group task used to evaluate the same biological-signal claim, adding mild selection optimism; the dictionary learning itself remains unsupervised and non-circular.

  1. fitted input called prediction [Section 5, 'Data source and foundation model' (layer selection); Section 6.1, 'Linear probing for biological signals']
    "We selected this layer by finding the maximized linear probing performance on the functional group task from the original embeddings (§ 5). ... For the difficult tasks of distinguishing between many genetic perturbations (3,5), a substantial amount of the linear signal is preserved."

    The token-extraction layer (16 for MAE-L, 33 for MAE-G) is chosen by maximizing linear-probe accuracy on the same functional gene group task (Task 5) that is later used in Section 6.1 and Figure 4A to demonstrate that ICFL reconstructions preserve genetic-perturbation signal. The reported Task 5 preservation is therefore a selection-optimistic estimate: the layer is the one where these exact labels are most linearly separable in the original embeddings, and no nested or held-out layer selection is reported. The reconstruction probing accuracy is not mathematically forced to match the original BTA, so this is a mild evaluation leak rather than a by-construction equivalence, but it does inflate the headline genetic-perturbation result.

full rationale

The core extraction pipeline is not circular: ICFL is trained unsupervised on MAE tokens, and biological labels are used only for evaluation, selectivity counting, and probe training. PCA whitening is fit to unperturbed control representations rather than to the perturbation labels, and the paper ablates its effect. The ICFL objective is a standard reconstruction loss with an OMP-style encoder; it is not defined in terms of the benchmark labels. The only load-bearing caveat is the layer-selection protocol: the same Task 5 (functional gene group) labels used to pick the layer are also used to measure the 'genetic perturbations preserved' result, making that particular evaluation optimistic. This does not reduce the derivation to its inputs because the learned dictionary must still generalize to held-out wells and the reconstruction probing is a nontrivial measurement. Self-citations to prior microscopy MAE and whitening work (Kraus et al., 2024; Kenyon-Dean et al., 2025) are not load-bearing because the method is implemented and ablated directly in this paper. Overall circularity is low.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The method rests on the linear representation hypothesis for MAEs, a domain assumption that the paper supports only indirectly. The PCA whitening transform, layer choice, sparsity, and codebook size are all fitted or hand-chosen values that materially affect the results. No new physical entities are introduced.

free parameters (7)
  • TopK sparsity K = 100
    Default sparsity for TopK SAE comparisons; affects all reported BTA and selectivity comparisons (Section 5).
  • ICFL iterations J and top-L per iteration = J=20, L=5
    Yields max sparsity 100; chosen by default, not tuned per task; affects reconstruction and selectivity.
  • Number of dictionary features M = 8192
    Codebook size; larger than embedding dimension; standard for sparse autoencoders.
  • Learning rate = 5e-5
    Used for all dictionary learning training; ablation in Appendix C shows limited impact.
  • Layer index for feature extraction = 33 (MAE-G), 16 (MAE-L)
    Selected by maximizing linear probing performance on the functional group task; a label-dependent choice that can bias downstream evaluation.
  • PCA whitening transform on control data = mean and covariance of HUVEC control representations (details not fully specified)
    Fitted to unperturbed control cells; downweights dominant nuisance directions; load-bearing preprocessing with no stated PCA rank or regularization.
  • Random-reset cosine threshold = 0.9
    Used to prevent feature collapse during training; practical hyperparameter.
assumptions (5)
  • domain assumption Superposition hypothesis: token representations are sparse linear combinations of a large overcomplete set of concepts.
    Underlies dictionary learning for interpretability; cited to Bricken et al., not proven for microscopy MAEs.
  • domain assumption Concepts are encoded as linear directions in intermediate MAE residual stream.
    Appendix A provides indirect evidence via row/null-space probing, but this is not a proof; if false, sparse features are correlations rather than model-intrinsic concepts.
  • domain assumption PCA whitening on unperturbed control cells removes nuisance variation while preserving perturbation signal.
    Section 4.1 states this as weak supervision; no validation that perturbation-relevant directions are not attenuated for non-HUVEC cell types.
  • ad hoc to paper The selected intermediate layer, chosen by functional-group linear probing, is representative.
    Selected using the target task; not derived from a standard principle.
  • domain assumption Selectivity scores and linear probing BTA are meaningful proxies for biological interpretability.
    Evaluation relies on known labels and expert visual inspection; no ground truth for concept quality beyond these proxies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models." pith.science (2026). https://pith.science/paper/2THACNB5

@misc{pith2026241216247,
  author       = {Pith},
  title        = {Pith review of: Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2THACNB5}},
  note         = {Machine review of arXiv:2412.16247}
}
read the original abstract

Sparse dictionary learning (DL) has emerged as a powerful approach to extract semantically meaningful concepts from the internals of large language models (LLMs) trained mainly in the text domain. In this work, we explore whether DL can extract meaningful concepts from less human-interpretable scientific data, such as vision foundation models trained on cell microscopy images, where limited prior knowledge exists about which high-level concepts should arise. We propose a novel combination of a sparse DL algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening pre-processing step derived from control data. Using this combined approach, we successfully retrieve biologically meaningful concepts, such as cell types and genetic perturbations. Moreover, we demonstrate how our method reveals subtle morphological changes arising from human-interpretable interventions, offering a promising new direction for scientific discovery via mechanistic interpretability in bioimaging.

Figures

Figures reproduced from arXiv: 2412.16247 by the authors.

Figure 1
Figure 1. Graphical abstract illustrating our algorithmic approach to biological concept interpretability from cell image data, consisting of (A) token-wise embeddings of cell images learned by a masked autoencoder (MAE), (B) reconstruction of the averaged token xi through a large codebook Wdec and sparse feature vector zi of sparsity K, which both get updated through (C) iterative codebook feature learning (ICFL) adapted fro… view at source ↗
Figure 2
Figure 2. Comparison of feature average selectivity scores from CellProfiler (CP) and ICFL for a subset of Task (3). (A) Max avg selectivity scores for each label in descending order, (B) averaged across labels at different thresholds for CP and sparsity levels for ICFL, as a function of the average number of non-zero values. (C) Correlation of max avg selectivity scores for each label between CP and ICFL. To remove the impac… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Comparison of ICFL with TopK SAEs. (A) Balanced test accuracy (BTA) of linear probes trained on the original representation (solid line) and reconstructions from ICFL and TopK SAEs with and without PCA whitening for five tasks from § 5. (B) Balanced test accuracy (BTA)…
Figure 5
Figure 5. Figure 5: Channel-specific features visualization from selected single-gene perturbations. Token heatmaps (top) are plotted above the composite 6-channel images (middle) and channel-specific staining images (bottom) of selected subcellular compartments, along with the channel-he…
Figure 6
Figure 6. Figure 6: Interpretation of single-cell identities from cell images through (A) composite images and (B) their actin-specific chan￾nel, which strongly correlates with a feature from the adherens junctions gene group. (C) Segmented single cells classified based on morphology and …
Figure 7
Figure 7. Figure 7: Decoding of tokens from intermediate transformer layers. (A) Sample reconstructed images when decoding from 3 different intermediate layers. (B) The relative linear probing accuracy when using the component from the null space, row space and a random 512-dimensional su…
Figure 8
Figure 8. Figure 8: The highest selectivity scores among all features for each label, ordered separately for each line starting with maximum score. 50 75 100 125 150 sparsity 0.4 0.6 0.8 cosine similarity 6 4 learning rate log 10 0.2 0.4 0.6 cosine similarity (a) w/ PCA whitening 50 75 10…
Figure 9
Figure 9. Figure 9: The cosine similarity between the original tokens and the reconstructed tokens for ICFL and TopK-SAE, (A) with PCA whitening and (B) without, as a function of the sparsity (first and third), i.e. # of non-zeros, and log10 learning rate (second and fourth). B. Interpret…
Figure 10
Figure 10. Figure 10: Sample visualizations of (A) composite images and their (B) nuclei- and (C) actin-staining channels which strongly correlate with a selected feature from a single functional gene group — adherens junctions. Plotted by side are the (D) cell category-specific single-cel…
Figure 11
Figure 11. Figure 11: Additional visualization of images most and least correlated with selected concept directions. For each row in [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Visualization of sample images aligned with a feature direction with plausible biological interpretation. This feature appears to be focusing on the endoplasmic reticuli and nucleoli channel (cyan area) surrounding the nucleus. These are expanded relative to the usual…
Figure 13
Figure 13. Figure 13: Visualization of sample images aligned with a feature direction with plausible biological interpretation. This feature appears to be firing for cells that are unusually large with spread out actin. Note that the feature focuses on the actin channel (red) surrounding t…
Figure 14
Figure 14. Figure 14: Visualization of sample images aligned with a feature direction with plausible biological interpretation. This feature appears to be active for long spindly cells, with the features are most aligned for the long “stretched out” section of the cells [PITH_FULL_IMAGE:f…
Figure 15
Figure 15. Figure 15: Visualization of sample images aligned with a feature direction with plausible biological interpretation. This feature is active for tightly clumped cells. The heatmaps are less clearly interpretable for these images, but appear to be active when neighboring nuclei ar…
Figure 16
Figure 16. Figure 16: Visualization of sample images aligned with a feature direction with plausible biological interpretation. This feature shows a similar behavior to the feature in [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Comparison of ICFL with TopK SAEs performance. (A) The BTA of linear probes trained on the original representation (solid lines) and reconstructions from ICFL features and TopK SAEs for representations taken from the residual stream and attention output of MAE-G (larg…
Figure 18
Figure 18. Figure 18: The selectivity scores as in [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: The selectivity scores as in [PITH_FULL_IMAGE:figures/full_fig_p023_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 12 canonical work pages

  1. [1]

    As observed, the null space component consistently yields the same probing accuracy as the entire token, while the row space component yields significantly lower accuracy

    across different layers. As observed, the null space component consistently yields the same probing accuracy as the entire token, while the row space component yields significantly lower accuracy. For comparison, we also show the relative probing accuracy when using a random dd-dimensional subspace (the same dimension as the row space), which consistently...

    work page 2023

  2. [4]

    interpretable

    and the attention output. benefits from scaling the model size. We further plot in Figure 19 the selectivity scores. For ICFL, we consistently observe improvements when increasing the model size, while for TopK SAE, we see a significant drop. Interestingly, this drop does not occur for the probing accuracy on the reconstructions in Figures 17a and 17b. Th...

    work page 2024

  3. [9]

    Hubel, D

    doi: 10.48550/arxiv.2212.08071. Hubel, D. H. and Wiesel, T. N. Receptive fields and func- tional architecture of monkey striate cortex. The Journal of physiology, 195(1):215–243,

    arXiv doi:10.48550/arxiv.2212.08071

  4. [10]

    On the origins of linear representations in large language models

    Jiang, Y ., Rajendran, G., Ravikumar, P., Aragam, B., and Veitch, V . On the origins of linear representations in large language models. arXiv preprint arXiv:2403.03867,

    arXiv

  5. [11]

    Madan, S., Henry, T., Dozier, J., Ho, H., Bhandari, N., Sasaki, T., Durand, F., Pfister, H., and Boix, X

    doi: 10.1038/s41588-024-01758-y. Madan, S., Henry, T., Dozier, J., Ho, H., Bhandari, N., Sasaki, T., Durand, F., Pfister, H., and Boix, X. When and how convolutional neural networks generalize to out- of-distribution category–viewpoint combinations. Nature Machine Intelligence, 4(2):146–153,

    work page doi:10.1038/s41588-024-01758-y

  6. [12]

    and Frey, B

    Makhzani, A. and Frey, B. K-sparse autoencoders. arXiv preprint arXiv:1312.5663,

    arXiv

  7. [17]

    Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery

    Rao, S., Mahajan, S., Böhle, M., and Schiele, B. Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery. arXiv preprint arXiv:2407.14499,

    arXiv

  8. [18]

    Tamkin, A., Taufeeque, M., and Goodman, N. D. Codebook features: Sparse and discrete interpretability for neural networks. arXiv preprint arXiv:2310.17230,

    arXiv

Show all 26 references
  1. [19]

    D., and Vidal, R

    Yellin, F., Haeffele, B. D., and Vidal, R. Blood cell detec- tion and counting in holographic lens-free imaging by convolutional sparse dictionary learning and coding. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 650–653. IEEE,

    2017

  2. [20]

    for an overview), there exists relatively little evidence for ViT MAEs besides the high linear probing accuracies on e.g., natural and microscopy image classification tasks (Huang et al., 2022; Alkin et al., 2024). In this section, we provide an argument further supporting the...

    2022

  3. [21]

    Both blocks are connected via an encoder-decoder projection matrix W : Rde×dd with, in our case, de = 1664(ViT-G model from (Zhai et al., 2022)) and dd =

    use two different embedding dimensions for the encoder block and the decoder block. Both blocks are connected via an encoder-decoder projection matrix W : Rde×dd with, in our case, de = 1664(ViT-G model from (Zhai et al., 2022)) and dd =

    2022

  4. [24]

    # of non-zeros, and log10 learning rate (second and fourth)

    The cosine similarity between the original tokens and the reconstructed tokens for ICFL and TopK-SAE,(A) with PCA whitening and (B) without, as a function of the sparsity (first and third), i.e. # of non-zeros, and log10 learning rate (second and fourth). B. Interpretable feat...

    2023

  5. [25]

    Ablations In this section we present ablations on type of token, model size, sparsity and learning rate

    C. Ablations In this section we present ablations on type of token, model size, sparsity and learning rate. If not further specified, we always use features extracted from ICFL using PCA whitening. Attention block It is common in the literature to use representations from the ...

    2023

  6. [512]

    register

    This projection matrix gives raise to a separation of the the tokens into the row-space and null space of W , x = xrow + xnull where only the information stored in xrow is passed to the decoder. ViTs and more generally transformer models have shown to align the basis across la...

    2024

  7. [1993]

    Linguistic regularities in continuous space word representations

    Mikolov, T., Yih, W.-t., and Zweig, G. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American chapter of the association for computational linguistics: Human language technologies, pp. 746–751,

    2013

  8. [1997]

    J., and Veitch, V

    Park, K., Choe, Y . J., and Veitch, V . The linear represen- tation hypothesis and the geometry of large language models. arXiv preprint arXiv:2311.03658,

    arXiv

  9. [1998]

    Gao, L., la Tour, T

    doi: 10.1038/35888. Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scal- ing and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093,

    arXiv

  10. [2006]

    Mim-refiner: A contrastive learning boost from in- termediate pre-trained representations

    Alkin, B., Miklautz, L., Hochreiter, S., and Brandstetter, J. Mim-refiner: A contrastive learning boost from in- termediate pre-trained representations. arXiv preprint arXiv:2402.10093,

    arXiv

  11. [2015]

    Emergent linear rep- resentations in world models of self-supervised sequence models

    Nanda, N., Lee, A., and Wattenberg, M. Emergent linear rep- resentations in world models of self-supervised sequence models. arXiv preprint arXiv:2309.00941,

    arXiv

  12. [2016]

    Vision transformers need registers

    Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers. arXiv preprint arXiv:2309.16588,

    arXiv

  13. [2018]

    Toy models of superposition

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.10652,

    arXiv

  14. [2019]

    The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision

    Gorton, L. The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision. arXiv preprint arXiv:2406.03662,

    arXiv

  15. [2021]

    Improving dictionary learning with gated sparse autoencoders

    Rajamanoharan, S., Conmy, A., Smith, L., Lieberum, T., Varma, V ., Kramár, J., Shah, R., and Nanda, N. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014, 2024a. 11 Towards scientific discovery with dictionary learning Rajamanoharan, ...

    arXiv

  16. [2022]

    M., Kraus, O., Victors, M., Arumugam, L., Vuggu- mudi, K., Urbanik, J., Hansen, K., Celik, S., Cernek, N., Jagannathan, G., et al

    Fay, M. M., Kraus, O., Victors, M., Arumugam, L., Vuggu- mudi, K., Urbanik, J., Hansen, K., Celik, S., Cernek, N., Jagannathan, G., et al. Rxrx3: Phenomics map of biology. bioRxiv, pp. 2023–02,

    2023

  17. [2023]

    Ferrando, J., Sarti, G., Bisazza, A., and Costa-jussà, M. R. A primer on the inner workings of transformer-based lan- guage models. arXiv preprint arXiv:2405.00208,

    arXiv

  18. [2024]

    doi: 10.1371/journal.pcbi. 1012463. Chen, X., Duan, Y ., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. InfoGAN: Interpretable representation learning by information maximizing generative adversar- ial nets. In Advances in Neural Information Processing Systems, vol...

    doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.