Pith. sign in

REVIEW 5 major objections 6 minor 46 references

From Words to Waves: Analyzing Concept Formation in Speech and Text-Based Foundation Models

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Text models form linguistic concepts early; speech models build them late and with less capacity, and they encode negative sentiment more weakly.

desk verdict Mean-pooling of speech frames may be manufacturing the paper's central speech-to-linguistic transition; the resources and framing are useful, but the core claim needs re-validation with a different pooling method. read the letter →

arxiv 2506.01133 v1 pith:4ID7VOP3 submitted 2025-06-01 cs.CL cs.AIcs.SDeess.AS

As{i}m Ersoy , Basel Mousi , Shammur Chowdhury , Firoj Alam , Fahim Dalvi , Nadir Durrani This is my paper
classification cs.CLcs.AIcs.SDeess.AS
keywords conceptformationlatentanalysisspeechfoundationmodelstextmultimodallearninglinguistictaxonomiessentimentinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that concept formation in foundation models is modality-dependent: text models develop linguistic abstractions early and deeply, while speech models spend their early layers on acoustics and only later align with word-level linguistic taxonomies, always devoting less capacity to them. It matters because it offers a mechanistic explanation for a practical gap — speech models underperform text models on language understanding and on negative sentiment — and ties that gap to architecture and training objectives rather than to the input signal alone. The authors reach this conclusion by clustering layer-wise representations into interpretable concepts and measuring how well those concepts match human-defined linguistic categories in both unimodal and multimodal models. A direct consequence of their claim is that better speech reasoning may require not just more data but changes in how speech models are built and trained.

What carries the argument

The central mechanism is Latent Concept Analysis, an unsupervised pipeline that extracts contextualized embeddings from every transformer layer, clusters them with K-means into roughly 600 concepts, and scores each cluster against human-defined taxonomies using an alignment-and-coverage metric at a high match threshold ($\theta=0.9$). For speech input, frame-level embeddings are averaged within forced-aligned word boundaries to produce word-level acoustic word embeddings, which lets the same clustering and taxonomy alignment run on words in both modalities. The layer-wise alignment curves are the load-bearing evidence: their shape is what supports the acoustic-to-linguistic transition in speech models, the early-and-deep linguistic encoding in text models, and the distinctive plateau in shared-encoder multimodal models.

What would settle it

Re-run the full concept-discovery and alignment pipeline on the same speech data, but replace frame-averaged word embeddings with word embeddings produced by a learned pooling model or with hand-corrected word boundaries on a sample; if the layer-wise alignment curves for the speech model no longer show the late acoustic-to-linguistic transition, or if the negative-sentiment gap narrows, the central claims rest on the pooling choice rather than on the modality itself.

Watch

Extended reading notes

Core claim

The paper's central claim is that text models such as BERT directly encode linguistic taxonomies from the earliest layers, whereas speech models such as HuBERT transition gradually from acoustic to linguistic representations, with linguistic structure emerging in middle layers and peaking in upper layers. Multimodal models do not simply inherit the text-like pattern: SpeechT5's shared encoder, optimized for cross-modal consistency, shows weaker explicit linguistic taxonomy alignment in deeper layers than either BERT or the more loosely coupled Seamless model. The paper also reports a task-specific asymmetry: after fine-tuning for sentiment, both BERT and HuBERT develop polarity concepts in their final layers, but the speech model captures positive sentiment far better than negative, consistent with its lower accuracy on negative examples (87.48% vs 93.21% for BERT, while positive accuracy is comparable at 93.31% vs 94.98%). These findings collectively support the claim that speech models reserve substantial representational capacity for phonetic, prosodic, and speaker-specific features, leaving less room for the explicit linguistic and semantic taxonomies that text models internalize.

Load-bearing premise

The whole comparison depends on the assumption that the forced-alignment tool places word boundaries accurately enough that averaging the audio frames within each boundary yields a faithful word embedding; if boundaries are off, the speech clusters and every modality comparison built on them are corrupted.

Editorial extensions

If this is right

  • Speech foundation models will need additional capacity or pretraining objectives aimed at linguistic structure if they are to match text models on tasks that require syntax and semantics.
  • The late emergence of linguistic taxonomies in speech models implies that layer choice matters for probing: early layers carry acoustic content, while middle-to-upper layers carry the most linguistic signal.
  • Multimodal training that forces speech and text into one shared encoder can reduce the depth of explicit linguistic taxonomies relative to text-only or loosely coupled models, revealing a cost to cross-modal alignment.
  • Fine-tuned speech sentiment models are asymmetric: positive polarity is encoded robustly while negative polarity is under-represented, so speech sentiment systems should be evaluated on negative-heavy test sets separately.
  • Falling taxonomy alignment in the uppermost layers is interpreted not as lost structure but as a shift toward task-specific and holistic representations, which argues for evaluating final layers on task-level concepts rather than only linguistic taxonomies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves open is that the negative-sentiment gap in speech models may be partly inflated by the synthetic TTS audio used in the experiment, because synthetic prosody can flatten the acoustic cues that carry negative polarity; testing on natural spontaneous speech would separate alignment artifacts from intrinsic modality differences.
  • A testable extension is to replace uniform frame averaging with learned or attention-based pooling when building acoustic word embeddings; if the taxonomy-alignment curves shift, the pooling choice is part of the explanation rather than just the modality itself.
  • The paper's capacity explanation predicts that scaling up a speech model or giving it an explicit text-alignment bottleneck should increase linguistic taxonomy alignment; this can be checked by comparing base and large variants of the same speech model or by ablating SpeechT5's cross-modal objective.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper investigates how speech, text, and multimodal foundation models form latent concepts. Using Latent Concept Analysis, the authors cluster contextualized word representations (for speech, obtained by mean-pooling frame embeddings within forced-aligned word boundaries) and align the clusters with human-defined taxonomies such as POS, chunking, semantic annotations, suffixation, and sentiment polarity. They report that text models encode linguistic taxonomies from early layers while speech models gradually shift from acoustic to linguistic representations, that speech models allocate less capacity to linguistic and semantic taxonomies, and that fine-tuned speech models underperform in negative sentiment classification compared to text models. The paper releases code and a TTS-generated audio version of SST-2.

Significance. If the findings are robust, the paper provides a valuable layer-wise comparison of conceptual structure across speech and text foundation models, an underexplored area. Strengths include the public release of code and an audio SST-2 collection, use of an established unsupervised concept discovery framework, and concrete qualitative examples. However, the central methodological choice of mean-pooling frames and the absence of direct acoustic taxonomies currently leave the main claims vulnerable to alternative explanations; the manuscript would benefit from additional controls and statistical grounding.

major comments (5)
  1. [2.1 Speech Input] The word-level speech embeddings Ψw are derived by averaging frame embeddings within MFA word boundaries, and this aggregation is used in all subsequent clustering and alignment. The central layer-wise trend in Figure 2—low linguistic alignment in early HuBERT layers with a gradual rise—is interpreted as evidence that speech models transition from acoustic to linguistic representations. However, mean pooling over variable-length windows is a depth-dependent, lossy operation: early-layer frame embeddings are phonetically and speaker-varied within a word, making the average a poor word proxy, while later-layer frame embeddings are more uniform, making the average more faithful. The observed curve may therefore trace the pooling operator's interaction with layer depth rather than an internal conceptual shift. The citation [10] argues for learned pooling and does not support mean pooling. To support the central claim, the authors should show that the layer-wise trend is invariant to the aggregation operator (e.g., first/last frame, attention pooling, or learned pooling) or analyze models that produce word-level representations directly.
  2. [4.1 Comparing Modalities] The claim that speech models 'gradually transition from acoustic to linguistic representations' requires direct evidence of acoustic encoding in early layers and a decline of acoustic alignment in later layers. The paper only measures alignment with linguistic taxonomies (POS, chunking, semantics, suffixation). Low alignment with linguistic taxonomies in early layers is compatible with several alternatives—including the pooling artifact described above, lack of word-level structure, or simply the absence of those particular categories—and does not by itself demonstrate that the representations are acoustic. The authors should include acoustic or phonetic taxonomies (e.g., phoneme classes, speaker or prosodic properties) to directly test the transition.
  3. [4.1 Comparing Modalities] The conclusion that speech models 'allocate less capacity to linguistic and semantic taxonomies' is not supported by the alignment metric used. The metric measures the fraction of discovered clusters that match human-defined categories; it does not measure representational capacity. Differences across HuBERT, BERT, Seamless M4T, and SpeechT5 are confounded by architecture, depth, pretraining objective, and input tokenization, so the capacity interpretation is speculative. A more direct capacity measure (e.g., probing accuracy with controlled probe size, or information-theoretic estimates) or at least a matched comparison of models with similar size and objective would be needed.
  4. [2.2 Concept Alignment] The coverage term κθ(Cl) is defined with the same condition as the alignment term αθ(Ce): both require |Ce∩Cl|/|Ce| ≥ θ. For a coverage term, one would expect the condition to involve the fraction of the linguistic concept Cl that is covered by an encoded concept Ce, i.e., |Ce∩Cl|/|Cl|. As written, the two terms double-count the same condition and the metric does not measure taxonomy coverage as claimed. This error affects the reported alignment percentages and should be corrected, and the experiments re-run, before the quantitative claims can be accepted.
  5. [4.2 Task-specific Concepts] The sentiment analysis experiment reports accuracy values (87.48% versus 93.21% for negative; 93.31% versus 94.98% for positive) but does not provide details of the fine-tuning protocol (data split, optimizer, number of runs, seeds) and reports no error bars or significance tests. The claim that speech models struggle to capture negative sentiment rests on a single model (HuBERT) and a single TTS-generated audio corpus. Without additional models, natural speech data, and statistical validation, this conclusion is not robustly supported.
minor comments (6)
  1. [1 Introduction] The 'Index Terms' line in the header appears to be a formatting artifact and should be removed.
  2. [Figure 2] Figure 2 has no legend identifying the different taxonomy lines; a legend or a clear caption description is needed to interpret the plot.
  3. [3 Experimental Setup] The text refers to 'base and fine-tuned models' but does not specify which base model variants are used in the main analysis; please clarify.
  4. [7 References] Reference [14] for Seamless M4T lacks complete author and venue information; please provide the full citation.
  5. [4.1 Comparing Modalities] There is a typo: 'paralingusitic' should be 'paralinguistic'.
  6. [3 Experimental Setup] The choices K=600 and θ=0.9 are inherited from [11]; a brief sensitivity analysis or a justification for these values on the current datasets would strengthen the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central comparison is an empirical application of prior interpretability tools to new models and external taxonomies.

full rationale

The paper's derivation chain is: extract layer-wise representations from speech/text/multimodal models, cluster them using Latent Concept Analysis (LCA, cited to the authors' prior work), and measure alignment with externally defined linguistic taxonomies (POS, chunking, Parallel Meaning Bank, sentiment labels). The taxonomies and datasets are independent of the models and of the paper's claims. The alignment metric from [11] is an operationalization, not a fixed outcome; the layer-wise curves and modality differences are data-dependent and could have come out differently. The sentiment analysis uses task labels as ground truth for a post-hoc characterization, not as a prediction fitted to itself. The main validity concern—mean-pooling frame embeddings to word-level representations may interact with layer depth and could partly explain the speech 'transition' curve—is a methodological confound that might change the empirical conclusion, but it is not a circular reduction: the pooling rule is stated a priori, not fitted to the target result, and the authors do not define 'linguistic capacity' in terms of the alignment output. Self-citations are present, but they supply tools rather than the conclusion; the central finding depends on new model evaluations and external taxonomies, so the derivation is self-contained.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claims depend on several methodological assumptions: the forced aligner's accuracy, the validity of embedding averaging, the meaningfulness of K-means clusters under the chosen hyperparameters, the alignment threshold, the fidelity of synthetic audio, and the comparability of layer-wise analysis across architectures with different layer counts and training objectives.

free parameters (3)
  • Number of clusters K = 600
    Chosen following prior work [11]; not fitted to data but a hyperparameter affecting clustering granularity.
  • Alignment threshold theta = 0.9
    Threshold for considering a cluster aligned with a taxonomy, taken from prior work [11].
  • Minimum word frequency = 10
    Minimum frequency for including a representation, set to avoid noisy clusters, following [11].
assumptions (6)
  • domain assumption Montreal Forced Aligner provides accurate word boundaries for LibriSpeech and SST2-audio.
    Introduced in Section 2.1 'Speech Input' where word boundaries are obtained using the aligner.
  • domain assumption Averaging frame-level embeddings within word boundaries yields faithful word-level representations.
    Stated in Section 2.1 with reference to acoustic word embeddings [10].
  • domain assumption K-means clustering with K=600 produces meaningful latent concepts across models.
    Described in Section 3 'Concept Discovery and Annotation'.
  • domain assumption The alignment metric with theta=0.9 captures meaningful concept-taxonomy correspondence.
    Defined in Section 2.2 and used throughout the analysis.
  • domain assumption Synthetic audio from XTTSv2 preserves sentiment polarity and natural prosodic cues.
    Introduced in Section 3 'Data' when describing the creation of SST2-audio.
  • domain assumption Layer-wise comparisons are valid across models with different layer counts and training objectives.
    The paper compares models of different depths and pretraining schemes in Section 4.1 without controlling for these factors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Words to Waves: Analyzing Concept Formation in Speech and Text-Based Foundation Models." pith.science (2026). https://pith.science/paper/4ID7VOP3

@misc{pith2026250601133,
  author       = {Pith},
  title        = {Pith review of: From Words to Waves: Analyzing Concept Formation in Speech and Text-Based Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ID7VOP3}},
  note         = {Machine review of arXiv:2506.01133}
}
read the original abstract

The emergence of large language models (LLMs) has demonstrated that systems trained solely on text can acquire extensive world knowledge, develop reasoning capabilities, and internalize abstract semantic concepts--showcasing properties that can be associated with general intelligence. This raises an intriguing question: Do such concepts emerge in models trained on other modalities, such as speech? Furthermore, when models are trained jointly on multiple modalities: Do they develop a richer, more structured semantic understanding? To explore this, we analyze the conceptual structures learned by speech and textual models both individually and jointly. We employ Latent Concept Analysis, an unsupervised method for uncovering and interpreting latent representations in neural networks, to examine how semantic abstractions form across modalities. For reproducibility we made scripts and other resources available to the community.

Figures

Figures reproduced from arXiv: 2506.01133 by the authors.

Figure 1
Figure 1. Sample latent concepts from different models. Figures 1a & 1d show BERT concepts; 1b & 1c show HuBERT concepts. ranged based on distinct linguistic and task-specific concepts. Textual Input. Given a textual utterance U = [w1, . . . , wN ], we extract contextual embeddings at layer l: U Ml t −−→ Φl = [ϕ l 1, . . . , ϕl N ] where ϕ l i is the embedding of wi at layer l. Speech Input. For the corresponding speech utter… view at source ↗
Figure 2
Figure 2. Layer-wise concept alignment. Y-axis shows % of aligned concepts (using θ=0.9) per encoder layer (X-axis). 4. Findings and Analysis 4.1. Comparing Modalities In [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Alignment with task-specific polarity concepts els trained for sentiment classification. Using SST2-text and our SST2-audio, we extracted activation vectors and underly￾ing concepts from the models and aligned them with the output classes: positive and negative. Our results, presented in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 39 canonical work pages

  1. [10]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

  2. [1]

    Among these, large language models (LLMs) have demonstrated emer- gent capabilities once thought to require human intelligence

    Introduction Recent advances in artificial intelligence have led to the devel- opment of large neural models capable of processing and gen- erating language, vision, and speech [1, 2, 3, 4, 5]. Among these, large language models (LLMs) have demonstrated emer- gent capabilities once thought to require human intelligence. From commonsense reasoning to medic...

    work page

  3. [2]

    Methodology Our methodology is designed to uncover and compare the latent conceptual structures emerging within speech, text, and multi- modal foundation models. To achieve this, we employ Latent Concept Analysis (LCA) [8], an unsupervised approach that enables the discovery and interpretation of abstract represen- tations learned by neural networks. Our ...

    work page Pith review arXiv 2025

  4. [3]

    We investigate both unimodal and multimodal models, focusing on HuBERT, BERT, Seamless M4T, and SpeechT5

    Experimental Setup Models. We investigate both unimodal and multimodal models, focusing on HuBERT, BERT, Seamless M4T, and SpeechT5. HuBERT [12] is a self-supervised speech model that excels at learning speech representations through masked prediction 3Also known as the acoustic word embeddings [10] 4https://github.com/MontrealCorpusTools/Montreal-Forced-...

    work page

  5. [4]

    w_1", "w_2

    Findings and Analysis 4.1. Comparing Modalities In Figure 2, we illustrate how concepts learned by text, speech, and multimodal models align with the linguistic taxonomies studied in this paper. The alignment patterns across layers re- veal distinct processing strategies: speech and text models han- dle linguistic information differently. Specifically, sp...

    work page

  6. [5]

    Related Work The discovery and interpretation of latent concepts in deep mod- els remain crucial challenges in NLP, particularly in speech processing. Recent studies have focused on understanding the internal representations learned by these models, with an emphasis on layer-wise analysis and latent concept discov- ery [27, 28, 9, 29, 30]. These foundatio...

    work page

  7. [6]

    Conclusion In this study, we compared speech, text, and multimodal mod- els to understand how they represent linguistic concepts. Our findings suggest that text models, such as BERT, directly en- code linguistic structures from early layers, while speech mod- els, like HuBERT, gradually develop linguistic representations from acoustic features. Multimodal...

    work page

  8. [7]

    Sparks of artificial general intelligence: Early experiments with GPT-4,

    S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y . Zhang, “Sparks of artificial general intelligence: Early experiments with GPT-4,” Tech. Rep., 2023

    work page 2023

Show all 46 references
  1. [8]

    Holistic evaluation of language models,

    P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y . Zhang, D. Narayanan, Y . Wu, A. Kumar et al., “Holistic evaluation of language models,” arXiv preprint arXiv:2211.09110, 2022

    2022 arXiv

  2. [9]

    LLaMA: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMA: Open and efficient foundation language models,” arXiv:2302.13971, 2023

    2023 arXiv

  3. [11]

    Humanomni: A large vision-speech language model for human-centric video understanding,

    J. Zhao, Q. Yang, Y . Peng, D. Bai, S. Yao, B. Sun, X. Chen, S. Fu, X. Wei, L. Bo et al., “Humanomni: A large vision-speech language model for human-centric video understanding,” arXiv preprint arXiv:2501.15111, 2025

    2025 arXiv

  4. [12]

    Chatgpt makes medicine easy to swallow: An exploratory case study on simplified radiology reports,

    K. Jeblick, B. Schachtner, J. Dexl, A. Mittermeier, A. T. St ¨uber, J. Topalis, T. Weber, P. Wesp, B. Sabel, J. Ricke, and M. Ingrisch, “Chatgpt makes medicine easy to swallow: An exploratory case study on simplified radiology reports,” 2022

    2022

  5. [13]

    Where do you know what you know? the representation of semantic knowledge in the human brain,

    K. E. Patterson, P. J. Nestor, and T. T. Rogers, “Where do you know what you know? the representation of semantic knowledge in the human brain,” Nature Reviews Neuroscience, 2007

    2007

  6. [14]

    Discovering latent concepts learned in BERT,

    F. Dalvi, A. R. Khan, F. Alam, N. Durrani, J. Xu, and H. Sajjad, “Discovering latent concepts learned in BERT,” inProc. of ICLR, 2022

    2022

  7. [15]

    Asking without telling: Exploring latent ontologies in contextual representations,

    J. Michael, J. A. Botha, and I. Tenney, “Asking without telling: Exploring latent ontologies in contextual representations,” in Proc. of EMNLP, 2020

    2020

  8. [16]

    Acoustic word embeddings for untranscribed target languages with con- tinued pretraining and learned pooling,

    R. Sanabria, O. Klejch, H. Tang, and S. Goldwater, “Acoustic word embeddings for untranscribed target languages with con- tinued pretraining and learned pooling,” in Proc. of Interspeech, 2023

    2023

  9. [17]

    Scaling up discovery of latent concepts in deep NLP models,

    M. Hawasly, F. Dalvi, and N. Durrani, “Scaling up discovery of latent concepts in deep NLP models,” in Proc. of EACL, 2024

    2024

  10. [18]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,” 2021

    2021

  11. [19]

    BERT: Pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language under- standing,” in Proc. of NAACL, 2019

    2019

  12. [20]

    Seamlessm4t: Massively multilingual & multimodal machine translation,

    S. Communication, “Seamlessm4t: Massively multilingual & multimodal machine translation,” 2023

    2023

  13. [21]

    Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing,

    J. Ao, R. Wang, L. Zhou, C. Wang, S. Ren, Y . Wu, S. Liu, T. Ko, Q. Li, Y . Zhang, Z. Wei, Y . Qian, J. Li, and F. Wei, “Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing,” 2022

    2022

  14. [22]

    Building a large annotated corpus of English: The Penn Treebank,

    M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large annotated corpus of English: The Penn Treebank,” Com- putational Linguistics, 1993

    1993

  15. [23]

    Introduction to the CoNLL-2000 shared task chunking,

    E. F. Tjong Kim Sang and S. Buchholz, “Introduction to the CoNLL-2000 shared task chunking,” in Proc. of CoNLL and the Second Learning Language in Logic Workshop, 2000

    2000

  16. [24]

    The parallel meaning bank: Towards a multilingual corpus of translations annotated with compositional meaning representations,

    L. Abzianidze, J. Bjerva, K. Evang, H. Haagsma, R. van Noord, P. Ludmann, D.-D. Nguyen, and J. Bos, “The parallel meaning bank: Towards a multilingual corpus of translations annotated with compositional meaning representations,” in Proc. of EACL, 2017

    2017

  17. [25]

    Recursive deep models for semantic composition- ality over a sentiment treebank,

    R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic composition- ality over a sentiment treebank,” in Proceedings of the 2013 Con- ference on Empirical Methods in Natural Language Processing , 2013

    2013

  18. [26]

    Lib- rispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” in Proc. of ICASSP, 2015

    2015

  19. [27]

    Neurox library for neuron analysis of deep nlp models,

    F. Dalvi, N. Durrani, and H. Sajjad, “Neurox library for neuron analysis of deep nlp models,” in Proc. of ACL, 2023

    2023

  20. [28]

    GLUE: A multi-task benchmark and analysis platform for natu- ral language understanding,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman, “GLUE: A multi-task benchmark and analysis platform for natu- ral language understanding,” in Proc. of Blackbox NLP, 2018

    2018

  21. [29]

    Slue: New benchmark tasks for spoken language un- derstanding evaluation on natural speech,

    S. Shon, A. Pasad, F. Wu, P. Brusco, Y . Artzi, K. Livescu, and K. J. Han, “Slue: New benchmark tasks for spoken language un- derstanding evaluation on natural speech,” 2022

    2022

  22. [30]

    What happens to BERT embeddings during fine-tuning?

    A. Merchant, E. Rahimtoroghi, E. Pavlick, and I. Tenney, “What happens to BERT embeddings during fine-tuning?” in Proc of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, Nov. 2020

    2020

  23. [31]

    How transfer learning im- pacts linguistic knowledge in deep NLP models?

    N. Durrani, H. Sajjad, and F. Dalvi, “How transfer learning im- pacts linguistic knowledge in deep NLP models?” in Findings of the ACL-IJCNLP 2021

    2021

  24. [32]

    Can llms facilitate interpreta- tion of pre-trained language models?

    B. Mousi, N. Durrani, and F. Dalvi, “Can llms facilitate interpreta- tion of pre-trained language models?” in Proc. of EMNLP, 2023

    2023

  25. [33]

    BERT rediscovers the classical NLP pipeline,

    I. Tenney, D. Das, and E. Pavlick, “BERT rediscovers the classical NLP pipeline,” in Proc. of ACL, 2019

    2019

  26. [34]

    What does BERT learn about the structure of language?

    G. Jawahar, B. Sagot, and D. Seddah, “What does BERT learn about the structure of language?” in Proc. of ACL, 2019

    2019

  27. [35]

    Analyzing encoded concepts in transformer language models,

    H. Sajjad, N. Durrani, F. Dalvi, F. Alam, A. Khan, and J. Xu, “Analyzing encoded concepts in transformer language models,” in Proc. of NAACL, 2022

    2022

  28. [36]

    On the transforma- tion of latent space in fine-tuned nlp models,

    N. Durrani, H. Sajjad, F. Dalvi, and F. Alam, “On the transforma- tion of latent space in fine-tuned nlp models,” inProc. of EMNLP, 2022

    2022

  29. [37]

    What do end-to- end speech models learn about speaker, language and channel information? a layer-wise and neuron-level analysis,

    S. A. Chowdhury, N. Durrani, and A. Ali, “What do end-to- end speech models learn about speaker, language and channel information? a layer-wise and neuron-level analysis,” Computer Speech & Language, 2024

    2024

  30. [38]

    What do speech foundation models not learn about speech?

    A. Waheed, H. Atwany, B. Raj, and R. Singh, “What do speech foundation models not learn about speech?” arXiv preprint arXiv:2410.12948, 2024

    2024 arXiv

  31. [39]

    Speech representation analysis based on inter-and intra-model similarities,

    Y . El Kheir, A. Ali, and S. A. Chowdhury, “Speech representation analysis based on inter-and intra-model similarities,” in Proc. of ICASSP Workshop, 2024

    2024

  32. [40]

    How do mul- timodal foundation models encode text and speech? an analysis of cross-lingual and cross-modal representations,

    H. Lee, D. Liu, S. Sinhamahapatra, and J. Niehues, “How do mul- timodal foundation models encode text and speech? an analysis of cross-lingual and cross-modal representations,” arXiv preprint arXiv:2411.17666, 2024

    2024 arXiv

  33. [41]

    Human-like linguistic biases in neural speech models: Phonetic categorization and phonotactic constraints in wav2vec2. 0,

    M. de Heer Kloots and W. Zuidema, “Human-like linguistic biases in neural speech models: Phonetic categorization and phonotactic constraints in wav2vec2. 0,” in Proc. INTERSPEECH, 2024

    2024

  34. [42]

    Probing self- supervised speech models for phonetic and phonemic informa- tion: A case study in aspiration,

    K. Martin, J. Gauthier, C. Breiss, and R. Levy, “Probing self- supervised speech models for phonetic and phonemic informa- tion: A case study in aspiration,” in Proc. Interspeech, 2023

    2023

  35. [43]

    SD-HuBERT: Sentence-level self-distillation in- duces syllabic organization in hubert,

    C. J. Cho, A. Mohamed, S.-W. Li, A. W. Black, and G. K. Anu- manchipalli, “SD-HuBERT: Sentence-level self-distillation in- duces syllabic organization in hubert,” in Proc. of ICASSP, 2024

    2024

  36. [44]

    Phonetic analysis of self- supervised representations of english speech,

    D. Wells, H. Tang, and K. Richmond, “Phonetic analysis of self- supervised representations of english speech,” in Proc. of Inter- speech, 2022

    2022

  37. [45]

    Self-supervised speech representations are more phonetic than semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-supervised speech representations are more phonetic than semantic,” arXiv preprint arXiv:2406.08619, 2024

    2024 arXiv

  38. [46]

    What do self- supervised speech models know about words?

    A. Pasad, C.-M. Chien, S. Settle, and K. Livescu, “What do self- supervised speech models know about words?” TACL, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.