Phoneme embeddings in self-supervised ASR models show both random variance and systematic bias as sources of demographic unfairness, with variance hindering fairness more than bias.
A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Self-supervised learning (SSL) has transformed speech processing, yet its reliance on massive pre-training datasets remains a bottleneck. While robustness is often attributed to scale and diversity, the role of the data distribution is less understood. We systematically examine how curated subsets of pre-training data influence Automatic Speech Recognition (ASR) performance. Surprisingly, optimizing for acoustic, speaker, or linguistic diversity yields no clear improvements over random sampling. Instead, we find that prioritizing the longest utterances achieves superior ASR results while using only half the original dataset, reducing pre-training time by 24% on a large corpora. These findings suggest that for pre-training speech SSL models, data length is a more critical factor than either data diversity or overall data quantity for performance and efficiency, offering a new perspective for data selection strategies in SSL speech processing.
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Prioritizing longest utterances in SSL speech pre-training data outperforms random or diversity-based sampling for ASR performance while using half the data volume.
citing papers explorer
-
Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition models
Phoneme embeddings in self-supervised ASR models show both random variance and systematic bias as sources of demographic unfairness, with variance hindering fairness more than bias.
-
A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models
Prioritizing longest utterances in SSL speech pre-training data outperforms random or diversity-based sampling for ASR performance while using half the data volume.