Pith. sign in

REVIEW 4 major objections 5 minor 7 references

Sparse autoencoders trained on Whisper and HuBERT expose stable, steerable audio features that cut false speech detections by 70 percent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:22 UTC pith:JHOOJR6W

load-bearing objection AudioSAE is the first large-scale SAE study of speech/audio encoders and ships real resources, but the headline hallucination-reduction claim is measured on an internal proxy and the reported numbers don't fully reconcile. the 4 major comments →

arxiv 2602.05027 v2 pith:JHOOJR6W submitted 2026-02-04 cs.SD cs.AI

AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders

Georgii Aparin , Tasnima Sadekova , Alexey Rukhovich , Assel Yermekova , Laida Kushnareva , Vadim Popov , Kristian Kuznetsov , Irina Piontkovskaya This is my paper
classification cs.SD cs.AI
keywords sparse autoencodersaudio interpretabilityWhisperHuBERTfeature steeringhallucination reductionEEG correlationphoneme disentanglement
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that sparse autoencoders, trained on the internal activations of audio-processing models like Whisper and HuBERT, recover a stable set of interpretable features: units that consistently activate for laughter, whispers, music, speech boundaries, and individual phonemes. It argues these features are robust across random training seeds, are sparse enough to serve as levers for behavior control, and can be removed or amplified to steer model outputs. The practical claim is that a linear steering intervention on SAE features reduces Whisper's false speech detections by 70% across non-speech benchmarks while barely changing word error rate. The paper also claims that a subset of these features is correlated with human EEG responses during speech listening, suggesting the SAE decomposition aligns partially with neural speech processing. If true, audio model internals become addressable with named, reusable features and cheap inference-time interventions.

Core claim

The central discovery, on the paper's own terms, is that sparse autoencoders trained layer by layer on the encoders of Whisper-small and HuBERT-base yield features that are stable and semantically meaningful. Over half of the features (as measured by IoU-based coverage) match across independently seeded SAEs on mid and late layers, duplicates are low, and reconstruction quality is preserved. These features specialize by domain, encode discrete events like laughter and sneezing, correspond to phonemes with high frame-level accuracy, and can be selectively unlearned: removing 19-27% of features erases a target vowel while preserving recognition of others. Applying a sign-flipped steering vecto

What carries the argument

The central object is the BatchTop-K sparse autoencoder, which decomposes each audio encoder layer's dense activations into a sparse linear combination of learned feature directions. The paper's analysis backbone is an Intersection-over-Union distributional similarity measure: two features are semantically similar if their binary activation patterns over a dataset substantially overlap, and this defines the coverage numbers that ground the stability claims. The steering mechanism is a minimal linear intervention: a logistic regression on SAE activations selects features predictive of hallucination, the selected coordinates are pushed in the opposite direction in the SAE latent space, and the

Load-bearing premise

The load-bearing premise is that Whisper's internal no_speech_prob parameter is a faithful proxy for whether the model is actually hallucinating spurious speech: all false-positive numbers and the 70% reduction claim are computed from shifts in this one scalar, never from the generated transcripts themselves.

What would settle it

Run Whisper on non-speech audio after applying the paper's SAE steering at alpha=1 and examine the actual decoded transcripts: if a large fraction of clips still contain spurious text despite no_speech_prob being pushed above 0.5, the central reduction claim collapses. As a second check, recompute the >50% seed-consistency coverage with an IoU threshold above 0.5; if coverage falls below a quarter, 'stable features' is an artifact of permissive matching.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If SAE features are stable and interpretable, audio model debugging can move from activation space to named, reusable features, analogous to text SAEs.
  • The hallucination-reduction result gives a no-training intervention for Whisper: a single steering vector applied at inference time reduces false speech detection by 70% on sound and noise datasets.
  • Selective unlearning of phonemes and concepts via feature removal suggests a route to targeted concept erasure in audio models without fine-tuning.
  • The EEG correlation opens a bridge between model internals and human neural responses, suggesting SAE features could serve as stimuli in neural decoding experiments.
  • Cross-layer coverage shows that later layers reuse earlier features, giving architects a concrete way to compare how different encoder layers reorganize acoustic information.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The no_speech_prob proxy should be checked against actual transcript output: if spurious transcripts persist above the 0.5 threshold, the 'hallucination reduction' is really a threshold shift and not a reduction in generated hallucinations.
  • The '>50% stable across seeds' number is threshold- and layer-dependent; recomputing coverage with an IoU threshold above 0.5 or on early layers may produce much lower values, so 'stable' should be read as 'stable for mid/late layers at a permissive matching threshold.'
  • The steering vector trained on FSD50k transfers to Musan and WHAM, suggesting cross-dataset generalization, but performance collapses at alpha=3; a calibration study of alpha robustness would be a natural follow-up.
  • The EEG result rests on one electrode (Pz) and a linear temporal-response model; testing other electrodes and non-linear TRFs would show whether the alignment is a general property of SAE features or an artifact of the linear fit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper trains BatchTopK sparse autoencoders (8x expansion, k=50) on all encoder layers of Whisper-small and HuBERT-base and evaluates them on reconstruction quality, feature stability across seeds/layers/models (via an IoU-based distributional metric), domain specialization, classification/top-k probing/unlearning, phoneme alignment, mel- and caption-based interpretation, steering for Whisper 'hallucination' reduction, and correlation with EEG. The central claims are that more than 50% of features are seed-consistent, SAE features are interpretable and disentangled, steering reduces Whisper's false speech detections by 70% with negligible WER increase, and some SAE features correlate with human EEG activity.

Significance. If the claims hold, this would be a useful contribution: it is one of the first broad SAE analyses for audio encoders, it releases code and checkpoints, and it combines several evaluation strategies (robustness, interpretability, interventions, neuroscience alignment). The paper is also careful in places, e.g., the EEG analysis uses a held-out test set and Holm-Bonferroni correction, and the hyperparameter choices are documented in detail. However, the headline practical claim is currently validated only against an internal no_speech_prob proxy rather than against actual hallucinated transcriptions, there is a numerical inconsistency between the main steering table and the appendix, and the seed-stability claim is overstated when layer and dataset are taken into account. These issues are fixable but currently prevent the paper from fully supporting its abstract.

major comments (4)
  1. [Section 3.4 and 4.7, Table 2] The DR/FPR metric is defined entirely from the internal parameter no_speech_prob < τ = 0.5. Section 1 defines hallucination as 'false speech predictions in non-speech segments', i.e., an output-level phenomenon. The paper never reports transcript-level measurements, such as the fraction of non-speech clips for which Whisper emits non-empty or hallucinated text, before and after steering. Shifting no_speech_prob toward 1 does not guarantee that spurious text disappears, and the LibriSpeech WER check does not cover non-speech behavior. The abstract's 'reduces Whisper's false speech detections by 70%' should be reworded as a change in an internal proxy, or the authors should add transcript-level validation on FSD50k/MUSAN/WHAM.
  2. [Table 2 vs. Appendix Table 10/Table 6] The main text claims that the SAE α=1 configuration reduces average FPR from 0.37 to 0.11, with per-dataset values 0.12/0.09/0.14. For the same stated configuration (FSD50k steering vector, top-k=100, α=1.0), Appendix Table 10 reports 0.16/0.09/0.20, and the full Table 6 agrees. The average differs (0.117 vs. 0.15), so the 70% reduction headline is not reproducible from the stated configuration. Please reconcile the tables, or clearly state any difference in dataset split, feature selection, or layer injection.
  3. [Abstract, Section 4.2, Appendix Table 4] The abstract claims that 'over 50% of the features remain consistent across random seeds'. This is not supported as stated by Table 4. For the two-seed comparison (Hub_Hub2), coverage on LibriSpeech is 419/6144 at layer 1 and 3164/6144 (51.5%) at layer 12; on FSD50k it is 1826/6144 (29.7%) and on MTG it is 2267/6144 (36.9%) at layer 12. The threshold of 50% is reached only for mid/late layers on LibriSpeech. The claim should be qualified by layer and dataset in both the abstract and the conclusion, or the analysis revised.
  4. [Appendix H.1 vs. Table 2 caption] The steering experiments are described in Appendix H.1 as being performed on 'activations after transformer block of AudioEncoder after 8th layer', while the Table 2 caption refers to 'Whisper with injected SAE on the last layer' for the No-Steer condition. This is a substantive ambiguity for a headline result, since SAE features differ by layer. Please state exactly at which encoder layer the SAE is injected for each condition, and whether the steering results in Table 2 use layer 8 or the last layer.
minor comments (5)
  1. [Section 2.3] Typo: 'improtant' should be 'important'.
  2. [Section 3.3.1] The IoU and coverage formulas are not numbered. Numbering them would make later references to 'Formula 3.3.1' clearer.
  3. [Appendix E.2] The unlearning results depend strongly on the logistic-regression regularization: standard L2 regularization requires only 3–6% of features, while no-regularization requires over 1000. The main-text Figure 4 uses the no-regularization setting, but this should be stated explicitly in the main text, since the abstract's '19–27%' figure is otherwise hard to interpret.
  4. [Section 4.8 / Appendix J] For the EEG result, reporting only 'around 1% of Whisper and 1.5% of HuBERT features' as significant makes it difficult to assess the strength of the effect. Please include effect sizes or confidence intervals for the significant temporal-response functions, and state explicitly that the development-set lag selection and test-set t-tests are on disjoint data (the current wording could be misread as double-dipping).
  5. [Table 1] The column headers 'Hub_Hub2', 'Hub_HubL+n', etc. are not defined in the table caption. The appendix version of the table is clearer and would be a better model for the main text.

Circularity Check

0 steps flagged

No circular derivation: empirical metrics, held-out steering/EEG tests, and no load-bearing self-citation

full rationale

The paper's derivation chain is self-contained. The stability metric (Section 3.3.1) is an explicitly defined IoU coverage, not a fitted parameter; the >50% seed-consistency claim is an empirical table reading (and its layer dependence is a reporting issue, not circularity). The steering experiment fits a logistic regression on no_speech_prob<τ and then measures DR defined by the same condition, but the intervention is causal (modifying SAE latents and decoding back through Whisper) and, crucially, the headline configuration is also evaluated on held-out datasets (Musan, WHAM) with vectors fit on FSD50k, so the reduction is not forced by construction. The paper explicitly calls no_speech_prob a 'proxy metric', which flags a validity gap (no transcript-level hallucination measurement) rather than a circular step. The EEG analysis (Section 3.5, Appendix J) uses a development set to select time lags and a held-out test set with Holm-Bonferroni correction, so it does not reduce to its fitting inputs. The only self-citation (Kuznetsov et al. 2025) is a related-work mention and is not load-bearing. No uniqueness theorem, ansatz-by-citation, or renaming of known results is load-bearing. Main weaknesses—proxy-based hallucination measurement and hyperparameter selection on evaluation datasets—are correctness risks, not circularity under the stated criteria.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

No new physical or ontological entities are postulated. The steering vector is an intervention direction in the SAE latent space, not a claimed component of the model. The burden of the paper sits in free hyperparameters (k, α, thresholds) and in assumptions inherited from the SAE and EEG literatures, all disclosed or locatable in the text.

free parameters (8)
  • SAE sparsity level k = 50
    Chosen from sweep over k ∈ {25, 50, 75, 100, 200}; k=50 adopted for all experiments (Sec 4.2, Fig 1).
  • SAE expansion factor = 8x
    Selected from 8x/32x sweep; 8x adopted for all experiments (Appendix A, Fig 8).
  • IoU coverage threshold θ = 0.5
    Defines 'semantically similar' features and hence all seed/layer stability and duplicate counts (Sec 3.3.1, Appendix B).
  • no_speech_prob threshold τ = 0.5
    Defines FPR/DR and hallucination labels for the steering experiment (Sec 3.4).
  • Steering strength α and top-k features = α=1, k=100
    'Optimal' configuration selected from a grid on the same Musan/FSD50K/WHAM datasets used to report the headline 70% FPR reduction (Appendix H, Tables 6/10).
  • Auto-interpretation activation threshold = 0.1
    Empirically selected; acknowledged in Appendix G.
  • Domain-specialization thresholds = frame: {0.2, 0.1, 0.04}; audio: {0.5, 0.3}
    Progressive thresholds chosen by hand to assign features to speech/music/sounds domains (Appendix C.1).
  • EEG pipeline choices = 1–8 Hz bandpass, 128 Hz resampling, 1000 features/model, 6-min dev / 9-min test
    Protocol-specific choices that determine which features show significant TRF correlation (Appendix J).
axioms (6)
  • domain assumption SAE features correspond to meaningful monosemantic concepts in audio models (superposition hypothesis; Cui et al. 2025 justification)
    Sec 3.1 justifies the entire interpretability program; if audio transformers do not exhibit the superposition-like structure assumed, the feature-interpretation findings would not mean what the paper claims.
  • domain assumption IoU over binary activation patterns is a valid proxy for semantic feature similarity
    Sec 3.3.1; underpins all seed/layer stability claims. High IoU could also arise from activation-frequency artifacts or dead features; duplicates are counted separately but not fully controlled.
  • ad hoc to paper Whisper's no_speech_prob is a valid monotone proxy for hallucinated speech detections
    Sec 3.4; the 70% FPR reduction claim is measured solely on this internal parameter, never on generated transcript content.
  • domain assumption The Broderick et al. (2018) TRF linear framework with electrode Pz is adequate for testing EEG–feature correlation
    Sec 3.5 and Appendix J inherit this external framework; the paper acknowledges in Limitations that only one electrode and linear models are used.
  • ad hoc to paper The audio captioning model plus LLM aggregation faithfully interprets feature content
    Sec 4.6/Appendix G; the paper admits phoneme-level features are mis-captioned (e.g., 'ba' becomes 'a man is speaking'), so auto-interpretation findings are partially unreliable.
  • domain assumption The training corpus mixture shapes the discovered feature taxonomy
    Appendix A Table 3; batches average ~40% speech, 45% music, 15% sounds. Claims about relative prevalence of music vs sound features are partly an artifact of this deliberately biased mixture.

pith-pipeline@v1.3.0-alltime-deepseek · 25551 in / 16738 out tokens · 166448 ms · 2026-08-03T04:22:43.326336+00:00 · methodology

0 comments
read the original abstract

Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo.

Figures

Figures reproduced from arXiv: 2602.05027 by Alexey Rukhovich, Assel Yermekova, Georgii Aparin, Irina Piontkovskaya, Kristian Kuznetsov, Laida Kushnareva, Tasnima Sadekova, Vadim Popov.

Figure 1
Figure 1. Figure 1: The trade-off between normalized reconstruc [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Layer-wise feature specialization ratio by [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Selective unlearning of letter ’A’ via iterative [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Top-k probing and unlearning for accent clas￾sification. More results in Appendix D. 4.5 Semantic Analysis Letter pronunciation classification. To demon￾strate disentanglement, we employ a vowel unlearn￾ing experiment: if disentangled features exist for different phonemes, we should be able to selec￾tively unlearn one phoneme class while preserving recognition of others. Using AVLetters2 (Cox et al., [PIT… view at source ↗
Figure 6
Figure 6. Figure 6: Activation of features responsible for the be [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Features identified via the label-based classifi [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Temporal response functions for different SAE features for HuBERT model. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Influence of the expansion rate to the number [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Connection between reconstruction quality [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Connection between sparsity and neurons survival rate B Feature robustness For this set of experiments, we use datasets Lib￾riSpeech, FSD and MTG for analysis of fea￾ture similarity (formulas 3.3.1 and 3.3.1). From these datasets we sample 500 audios with total of 749450, 74700 and 86220 frames, respectively. We use BatchTopK with k = 50 on inference, meaning that for a single input audio of n frames, top… view at source ↗
Figure 11
Figure 11. Figure 11: Frequency-based domain specialization. Audio-level (1st and 3rd rows from the top) and frame-level (2nd and 4th rows from the top) activation frequency versus activation magnitude for Whisper (1st and 2nd rows from the top) and HuBERT (3rd and 4th rows from the top) layers 6–7 (left and right columns respectively). Colors: red (speech), blue (sounds), green (music), gray (unassigned), black (dead). et al.… view at source ↗
Figure 12
Figure 12. Figure 12: Feature overlap for Whisper (layers 6 and 7): Venn diagrams for audio and frame levels. and preventing bias toward specific speakers or phonemes. E.2 Unlearning plots for various letters and regularization setups [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Top-k probing and unlearning for four classification tasks [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Additional features found in classification by label experiment. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Automatic Interpretation Pipeline [PITH_FULL_IMAGE:figures/full_fig_p023_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Features label frequencies for Whisper and [PITH_FULL_IMAGE:figures/full_fig_p023_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Distribution of no_speech_prob on the FSD50k dataset before and after applying steering vec￾tors. The post-steering distribution is skewed towards 1.0 [PITH_FULL_IMAGE:figures/full_fig_p024_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Distribution of no_speech_prob on the Mu [PITH_FULL_IMAGE:figures/full_fig_p024_18.png] view at source ↗
Figure 21
Figure 21. Figure 21: Features 3249 and 3081 of Hubert’s SAE from layer 11. J Details of EEG experiments We chose data from midline parietal electrode Pz collected from 19 subjects listening to 5 excerpts from audiobook each 3 minutes long resulting in 15 minutes for each participant. SAE features (stimuli s) were extracted from these excerpts with models trained on the last 12-th layer of HuBERT-base and Whisper-base models b… view at source ↗
Figure 20
Figure 20. Figure 20: Distribution of no_speech_prob on the LibriSpeech test-clean dataset before and after apply￾ing steering vectors. The post-steering distribution is skewed towards 1.0. difference between the speech domain and the mu￾sic and sound domains, based on frequencies from Domain Specialization experiments. Therefore, it can be concluded that detected features are strongly speech-specific. Moreover, they were pres… view at source ↗
Figure 22
Figure 22. Figure 22: t-SNE decomposition of SAE encoder weights for Whisper layer 6 in the audio-level setup. Each point [PITH_FULL_IMAGE:figures/full_fig_p028_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: t-SNE decomposition of SAE encoder weights for HuBERT layer 6 in the audio-level setup. Points [PITH_FULL_IMAGE:figures/full_fig_p029_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Unlearning plots for letters ’A’, ’E’ and ’I’ at the last layer of HuBERT model, using standard [PITH_FULL_IMAGE:figures/full_fig_p030_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Unlearning plots for letters ’O’ and ’U’ at the last layer of HuBERT model, using standard [PITH_FULL_IMAGE:figures/full_fig_p031_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Unlearning plots for letters ’A’, ’E’ and ’I’ at the last layer of HuBERT model, using [PITH_FULL_IMAGE:figures/full_fig_p032_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Unlearning plots for letters ’O’ and ’U’ at the last layer of HuBERT model, using [PITH_FULL_IMAGE:figures/full_fig_p033_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: K-probe vowel classification at layer 9. The curves show classification accuracy as the most informative features are added sequentially. Only the first 49 features are displayed; beyond this point, accuracy approaches perfect accuracy for all vowels [PITH_FULL_IMAGE:figures/full_fig_p034_28.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

7 extracted references · 4 linked inside Pith

  1. [3]

    Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi

    Revelio: Interpreting and leveraging se- mantic information in diffusion models.Preprint, arXiv:2411.16725. Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. 2023. Audiogen: Tex- tually guided audio generation. InThe Eleventh In- ternational Conference on Learning Represent...

    Pith/arXiv arXiv 2023

  2. [4]

    InFindings of the Association for Com- putational Linguistics: NAACL 2025, pages 1604– 1635, Albuquerque, New Mexico

    Decoding dark matter: Specialized sparse autoencoders for interpreting rare concepts in founda- tion models. InFindings of the Association for Com- putational Linguistics: NAACL 2025, pages 1604– 1635, Albuquerque, New Mexico. Association for Computational Linguistics. Tu Anh Nguyen, Wei-Ning Hsu, Antony d’Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani,...

    Pith/arXiv arXiv 2025

  3. [6]

    David Snyder, Guoguo Chen, and Daniel Povey

    Discovering and steering interpretable con- cepts in large generative music models.Preprint, arXiv:2505.18186. David Snyder, Guoguo Chen, and Daniel Povey. 2015. Musan: A music, speech, and noise corpus.arXiv preprint arXiv:1510.08484. Samuel Stevens, Wei-Lun Chao, Tanya Berger-Wolf, and Yu Su. 2025. Sparse autoencoders for sci- entifically rigorous inter...

    arXiv 2015

  4. [2018]

    Bart Bussmann, Patrick Leask, and Neel Nanda

    Electrophysiological correlates of seman- tic dissimilarity reflect the comprehension of natu- ral, narrative speech.Current Biology, 28(5):803– 809.e3. Bart Bussmann, Patrick Leask, and Neel Nanda. 2024. Batchtopk sparse autoencoders.arXiv preprint arXiv:2412.06410. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean-...

    Pith/arXiv arXiv 2024

  5. [2021]

    laughter

    Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset. InICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 920–924. IEEE. Appendix Table of Contents A Extended SAE training details 14 B Feature robustness 16 C Domain-level feature specialization 16 C.1 Ex...

    2021

  6. [2023]

    com/posts/z6QQJbtpkEAX3Aojj/interim-research- report-taking-features-out-of-superposition

    Taking features out of superposition with sparse autoencoders.URL https://www.lesswrong. com/posts/z6QQJbtpkEAX3Aojj/interim-research- report-taking-features-out-of-superposition. Elana Simon and James Zou. 2024. Interplm: Discovering interpretable features in protein lan- guage models via sparse autoencoders.Preprint, arXiv:2412.12101. Nikhil Singh, Manu...

    Pith/arXiv arXiv 2024

  7. [2025]

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey

    On the theoretical understanding of identifi- able sparse autoencoders and beyond.arXiv preprint arXiv:2506.15963. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023a. Sparse autoen- coders find highly interpretable features in language models.Preprint, arXiv:2309.08600. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben...

    arXiv 2025