Pith. sign in

REVIEW 3 major objections 4 minor 47 references

OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fully open speech models, trained on a publicly cleaned 166,000-hour web-audio corpus, match or beat Whisper and MMS across multilingual ASR, language identification, and long-form English recognition.

desk verdict A genuinely useful open-data scaling paper whose strongest claim—long-form WER beating Whisper-large-v3—is compromised by using the same web-presentation corpus for threshold selection and final evaluation. read the letter →

arxiv 2506.00338 v1 pith:FEVTGIWV submitted 2025-05-31 cs.CL cs.SDeess.AS

Yifan Peng , Shakeel Muhammad , Yui Sudo , William Chen , Jinchuan Tian , Chyi-Jiunn Lin , Shinji Watanabe This is my paper
classification cs.CLcs.SDeess.AS
keywords speechfoundationmodelsdatacleaningweb-crawledmultilingualASRtranslationlanguageidentificationYODASopen-source
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the real bottleneck for open speech foundation models is not the volume of available data but its quality: web-crawled audio arrives with wrong language labels and misaligned text, and a cheap, fully public cleaning pipeline can repair it. The authors apply that pipeline to YODAS, a Creative Commons corpus of 370,000 hours in 149 languages, and retain 166,000 hours across 75 languages. Models trained on the cleaned data plus the existing OWSM corpus — about 320,000 hours total — consistently and substantially beat every previous fully open OWSM model on multilingual ASR, language identification, and English benchmarks. The headline result is the 1-billion-parameter OWSM-CTC v4 model reaching a 3.3% word error rate on long-form English speech, slightly better than Whisper-large-v3, which has 50% more parameters and was trained on roughly fifteen times more data. If the claims hold, academic-scale groups can match frontier industrial speech models by cleaning openly licensed data instead of accumulating proprietary data.

What carries the argument

The carrying mechanism is the three-stage data-cleaning pipeline applied to YODAS, since every downstream gain is attributed to the quality of its 166,000-hour output. Stage one, resegmentation, uses the CTC-segmentation algorithm with the publicly available OWSM-CTC v3.2 model to realign text to audio, split long-form recordings into utterances of at most 30 seconds, and emit a per-utterance confidence score quantifying alignment quality. Stage two, LID filtering, retains only utterances whose original language label matches both the text-based prediction (fastText) and the audio-based prediction (ECAPA-TDNN). Stage three, CTC-score filtering, ranks the confidence scores within each language and removes any long-form utterance containing a short utterance in the lowest 10% quantile; the threshold is chosen empirically by fine-tuning a small OWSM v3.1 model on subsets filtered at different $\theta_{\text{CTC}}$ values and evaluating on Common Voice and a long-form web presentation corpus, where unfiltered data drives WERs above 100% due to token repetition. The pipeline's design is deliberately scalable: every component is a public pretrained model or toolkit, so the same recipe can be re-run as YODAS grows.

What would settle it

Audit the cleaned subset in a language the teacher model does not support: hand-align a random sample of retained utterances, compare each text to its audio, and measure the mismatch rate against the same measurement on raw YODAS; if the cleaned data is not clearly better aligned, the pipeline's benefit in unsupported languages is not established. A cheaper proxy is to compare per-language FLEURS WER gains between teacher-supported and teacher-unsupported languages, since a systematic gap would implicate the teacher-coverage assumption directly.

Watch

Extended reading notes

Core claim

The central claim, stated in the paper's own framing, is that large-scale web-crawled data contains two systematic defects — incorrect language labels and audio-text misalignments — and that a scalable pipeline built entirely from public tools can fix both well enough for trained models to match frontier industrial systems. The pipeline realigns each long-form recording's text to audio with CTC segmentation using the public OWSM-CTC v3.2 model, which also assigns each resulting short utterance a confidence score; it then keeps only utterances whose original language label agrees with both a text-based (fastText) and an audio-based (ECAPA-TDNN) language identifier; finally it discards whole long-form utterances that contain any short utterance in the lowest per-language $\theta_{\text{CTC}}$ confidence quantile, with $\theta_{\text{CTC}} = 0.10$ chosen by fine-tuning a small OWSM v3.1 model and evaluating at several thresholds. The resulting 166,000-hour, 75-language subset, trained together with existing OWSM data (320,000 hours total), yields the OWSM v4 series, which outperforms all previous OWSM versions on every compared benchmark family — FLEURS, MLS, the Open ASR leaderboard, long-form English, and LID — and matches or surpasses Whisper and MMS in multiple scenarios, most notably the best long-form English WER (3.3%) among all compared models. The authors also report that without adding any new speech-translation data, the v4 models improve CoVoST-2 X-En scores and keep En-X scores comparable to v3.2, showing the extra ASR data does not hurt translation.

Load-bearing premise

The load-bearing premise is that the teacher model used to realign the audio and score its own alignment quality — OWSM-CTC v3.2, which the paper itself notes covers only a subset of YODAS's 149 languages — is reliable enough in every language that survives the pipeline; if its alignments are wrong in the unsupported languages, cleaning would either delete correctly transcribed data or keep misaligned data in exactly the low-resource languages the dataset is meant to serve.

Editorial extensions

If this is right

  • Fully open models can now compete with Whisper-large-v3 on long-form English ASR: OWSM-CTC v4 records 3.3% WER against Whisper's 3.4%, with 50% fewer parameters and roughly fifteen times less training data.
  • Data volume is subordinate to data quality: filtering the raw YODAS corpus from 284,000 to 166,000 hours improved rather than degraded downstream ASR, since unfiltered data drove WERs above 100% on Common Voice.
  • Cleaning transfers across benchmarks: the v4 series improves over v3.x on FLEURS in all 102 languages for the CTC model, on all eight MLS languages, on six of eight Open ASR leaderboard test sets, and on language identification (95.6% for v4 medium).
  • Speech translation is not harmed by the extra ASR data: OWSM-CTC v4 improves all four CoVoST-2 X-En test sets while keeping En-X scores comparable to v3.2, without adding any new ST data.
  • The complete recipe — cleaning scripts, cleaned YODAS data, training code, model weights, and training logs — is released publicly, making the result reproducible with academic-scale compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same three-stage recipe should transfer to other web-crawled corpora, including future YODAS releases, so long as a teacher model, a text LID model, and an audio LID model exist for the target languages; the teacher-coverage gap the paper notes is the main constraint on that generalization.
  • Because the confidence threshold is a per-language quantile, languages where the teacher model aligns poorly should either lose most of their data or retain misaligned utterances; a testable prediction is that per-language FLEURS WER gains correlate with the teacher's per-language alignment quality, a pattern the paper does not report.
  • The 41% retention rate (166,000 of 370,000 hours) suggests an economically attractive scaling strategy for open speech research: invest in cheap cleaning passes that convert a large corpus into a smaller, higher-precision one rather than accumulating ever more raw hours.
  • The released cleaned YODAS subset is a reusable asset independent of the OWSM training recipe, potentially serving as pretraining or evaluation material for other multilingual ASR and LID efforts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces OWSM v4, a series of fully open Whisper-style speech models (three AED models and one CTC model) trained on a cleaned version of the YODAS dataset combined with earlier OWSM data. The proposed data-cleaning pipeline has three stages: CTC-based resegmentation/realignment, language identification (LID) filtering, and CTC-confidence-score filtering. The authors report consistent improvements over previous OWSM versions on FLEURS, MLS, and English ASR benchmarks, and claim competitive or superior performance relative to industrial models such as Whisper and MMS. They also state that cleaned data, models, and scripts will be publicly released.

Significance. If the claims withstand scrutiny, this is a valuable contribution to open speech-model research: it demonstrates that a scalable, publicly documented cleaning pipeline can turn a large noisy web-crawled corpus into a useful training resource, and it provides fully open models that narrow the gap with proprietary systems. The explicit commitment to open weights, data details, and training logs is a strength, as is the evaluation across multiple benchmarks. However, the headline long-form result and the attribution of improvements to data scaling/cleaning depend on resolving the issues below, so the significance is conditional.

major comments (3)
  1. [Section 2.1.3 / Table 2 vs. Table 5] The CTC confidence threshold θCTC is selected by evaluating a fine-tuned OWSM v3.1 small model on Common Voice and 'a web presentation corpus' (Table 2). The headline long-form English ASR result in Table 5 is also reported on 'a web presentation corpus.' Unless these are explicitly identified as different corpora, this constitutes test-set leakage: the corpus used to tune θCTC is the same corpus used to claim OWSM-CTC v4 outperforms Whisper-large-v3 (3.3% vs. 3.4%). The paper must either clearly distinguish the two corpora or provide a held-out evaluation on a different long-form English test set before this central claim can be accepted.
  2. [Section 2.2 / Mel filterbank change] The v4 models increase the number of mel filterbanks from 80 to 128 relative to OWSM v3.1/v3.2, following Whisper-large-v3. Because the comparison 'v4 vs. previous OWSM' therefore involves an architectural change as well as data scaling/cleaning, the observed improvements cannot be attributed solely to the data pipeline. The paper should either provide an ablation that isolates the filterbank change (e.g., train a v3.2-style model with 128 mel features) or explicitly acknowledge that part of the gain may be architectural.
  3. [Section 2.1.1 / Teacher language coverage] The paper states that the OWSM-CTC v3.2 model used for resegmentation and confidence scoring 'supports only a subset of the languages present in YODAS.' If the teacher's alignment and confidence scores are unreliable for languages outside this subset, the cleaning pipeline could discard correctly transcribed data or retain misaligned data, especially for the lower-resource languages that the dataset is intended to cover. The paper should report how many of the 75 retained languages are actually supported by the teacher, or provide per-language quality checks, to substantiate the claim that the cleaned dataset is uniformly reliable.
minor comments (4)
  1. [Tables 5, 6, 7] No error bars, confidence intervals, or significance tests are reported. Several comparisons (e.g., Table 5's 3.3% vs. 3.4% long-form WER, Table 7's average WERs) are within a range that may not be statistically meaningful without variance estimates.
  2. [Table 7] The underline notation says 'Our v4 model outperforms previous OWSM,' but the comparison is ambiguous: for Gigaspeech, OWSM-CTC v4 (13.89) is worse than OWSM-CTC v3.2 (13.50), so the claim should specify which predecessor (v3.1 or v3.2) is the reference.
  3. [Table 5 / Section 3.1] The dataset name 'FLERUS' appears to be a typo for 'FLEURS' in the table caption; please correct it.
  4. [Section 2.2] The training description says '700k steps, i.e., around three epochs,' but the relation between steps and epochs depends on the effective dataset size; a more precise calculation would help reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Long-form WER headline uses a corpus that also tuned the CTC confidence threshold.

  1. fitted input called prediction [Section 2.1.3 (threshold selection) and Section 3.3 / Table 5]
    "To identify a suitable threshold, we fine-tune a pre-trained small-sized OWSM v3.1 (367M) [14] on the cleaned YODAS data filtered at different thresholds. We then evaluate them on Common Voice [35] for short-form ASR and a web presentation corpus for long-form ASR, as shown in Table 2. ... Table 5 shows long-form English ASR results... OWSM-CTC v4 achieves the lowest long-form WER of 3.3%, slightly outperforming Whisper-large-v3."

    The CTC confidence threshold θCTC is selected in §2.1.3 using the LF (long-form) column on 'a web presentation corpus' (Table 2). The same phrase is used in Table 5, which reports the final OWSM v4 models' long-form English ASR WER on 'a web presentation corpus,' and this is the basis for the claim that OWSM-CTC v4 achieves the lowest long-form WER. The paper does not state that these are different corpora. Since the threshold controls the cleaned training data, the final model's long-form WER is not an unbiased evaluation on an unseen corpus; it is partially optimized by construction because the threshold was chosen, in part, to minimize WER on that same corpus. This is a fitted input (θCTC) called a prediction (long-form WER).

full rationale

Most of the paper's derivation chain is self-contained: the data-cleaning pipeline uses public models (fastText, ECAPA-TDNN, CTC-segmentation) and the trained models are evaluated on independent external benchmarks (FLEURS, MLS, CoVoST-2, Hugging Face Open ASR Leaderboard). The prior OWSM models are used as teachers or baselines, but they are not invoked as unfalsifiable uniqueness theorems. The one significant circularity is the threshold-selection step: the CTC confidence threshold is tuned on a web presentation corpus (Table 2), and the same named corpus is then used to report the headline long-form WER (Table 5). This makes the specific claim of superiority over Whisper-large-v3 on long-form English ASR statistically forced rather than independent. Other comparisons (e.g., MLS, FLEURS) remain valid, so the paper is only partially circular. The increase of mel filterbanks from 80 to 128 is a confounding variable but is not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper's empirical claims rest on a sequence of cleaning decisions: the reliability of the teacher model (OWSM-CTC v3.2), the accuracy of external LID tools, and the generalization of a chosen threshold. There are no new theoretical entities, but there are several free parameters and domain assumptions that a replication would need to test.

free parameters (1)
  • CTC filtering threshold theta_CTC = 0.10
    Chosen by grid search over values {0.0, 0.10, 0.15, 0.20, 0.30} using a fine-tuned OWSM v3.1 small model evaluated on Common Voice and a web presentation corpus (Table 2). A global per-language quantile is applied to all 75 languages.
assumptions (4)
  • domain assumption The CTC confidence scores from OWSM-CTC v3.2 accurately reflect audio-text alignment quality across all 75 target languages.
    Invoked in Section 2.1.1 where CTC segmentation and confidence scores are computed for the entire YODAS corpus, despite the model supporting only a subset of YODAS languages.
  • domain assumption The fastText and ECAPA-TDNN LID models used in Section 2.1.2 have sufficient accuracy, and requiring agreement with the original label does not systematically exclude valid data.
    The LID filtering step retains utterances only when the original label matches both models; errors in these external models would propagate into the cleaned dataset.
  • domain assumption The threshold theta_CTC chosen using a small fine-tuned model generalizes to the larger final v4 models and to all languages in the 75-language set.
    The paper selects the threshold with a 367M-parameter model (Section 2.1.3) but trains much larger models for 700k steps; no per-language or per-size threshold search is performed.
  • domain assumption The audio and text in YODAS, after resegmentation, are correctly paired, and the original timestamps are inaccurate only in ways the CTC resegmentation can fix.
    Section 2.1.1 assumes the realignment step resolves timestamp inaccuracies and that utterances containing only non-speech elements are safely removed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning." pith.science (2026). https://pith.science/paper/FEVTGIWV

@misc{pith2026250600338,
  author       = {Pith},
  title        = {Pith review of: OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FEVTGIWV}},
  note         = {Machine review of arXiv:2506.00338}
}
read the original abstract

The Open Whisper-style Speech Models (OWSM) project has developed a series of fully open speech foundation models using academic-scale resources, but their training data remains insufficient. This work enhances OWSM by integrating YODAS, a large-scale web-crawled dataset with a Creative Commons license. However, incorporating YODAS is nontrivial due to its wild nature, which introduces challenges such as incorrect language labels and audio-text misalignments. To address this, we develop a scalable data-cleaning pipeline using public toolkits, yielding a dataset with 166,000 hours of speech across 75 languages. Our new series of OWSM v4 models, trained on this curated dataset alongside existing OWSM data, significantly outperform previous versions on multilingual benchmarks. Our models even match or surpass frontier industrial models like Whisper and MMS in multiple scenarios. We will publicly release the cleaned YODAS data, pre-trained models, and all associated scripts via the ESPnet toolkit.

Figures

Figures reproduced from arXiv: 2506.00338 by the authors.

Figure 1
Figure 1. Our data-cleaning pipeline consists of three steps: (1) realign audio and text using a pre-trained OWSM-CTC model, (2) filter data based on LID results, and (3) filter data based on CTC confidence scores [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 38 canonical work pages

  1. [1]

    Introduction Speech foundation models (SFMs), typically trained on large amounts of data, have demonstrated state-of-the-art (SOTA) performance in various speech processing tasks [1–4]. A no- table example is OpenAI’s Whisper [1], which is trained on 680 thousand to 5 million hours of audio data and supports mul- tilingual automatic speech recognition (AS...

    work page Pith review arXiv 2024

  2. [2]

    YODAS data cleaning The raw YODAS data has not undergone a rigorous cleaning process and may contain annotation errors [24]

    Proposed Method 2.1. YODAS data cleaning The raw YODAS data has not undergone a rigorous cleaning process and may contain annotation errors [24]. Common is- sues include mismatched language labels and misalignment be- tween audio and text. Therefore, data cleaning is essential to ensure accuracy and reliability. Figure 1 illustrates our data- cleaning pip...

    work page

  3. [3]

    transcription

    Experimental Results We evaluate our OWSM v4 models on multilingual ASR, LID, and ST benchmarks using greedy decoding unless otherwise specified. While we include results from models developed by well-resourced industry entities such as OpenAI’s Whisper and Meta’s MMS, our primary comparisons are against baselines from academic institutions, given our con...

    work page arXiv

  4. [4]

    We reveal that large-scale web-crawled data contains incorrect lan- guage labels and audio-text misalignments

    Conclusion We improve fully open speech-to-text foundation models via data scaling and cleaning using academic-scale resources. We reveal that large-scale web-crawled data contains incorrect lan- guage labels and audio-text misalignments. To mitigate these issues, we develop a scalable data-cleaning pipeline using pub- lic models and toolkits. Applying it...

    work page

  5. [5]

    Acknowledgements We use PSC Bridges2 and NCSA Delta via ACCESS CIS210014, by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296

    work page

  6. [6]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, et al. , “Robust speech recognition via large-scale weak supervision,” in Proc. ICML, 2023

    work page 2023

  7. [7]

    Google USM: Scal- ing automatic speech recognition beyond 100 languages,

    Y . Zhang, W. Han, J. Qin, Y . Wang, et al., “Google USM: Scal- ing automatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023

    arXiv 2023

  8. [8]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, et al. , “Scaling speech technology to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    work page 2024

Show all 47 references
  1. [9]

    Less is more: Accurate speech recognition & translation without web- scale data,

    K. C. Puvvada, P. ˙Zelasko, H. Huang, O. Hrinchuk, et al., “Less is more: Accurate speech recognition & translation without web- scale data,” in Proc. Interspeech, 2024

    2024

  2. [10]

    Reproducing Whisper-Style Training Using an Open-Source Toolkit and Pub- licly Available Data,

    Y . Peng, J. Tian, B. Yan, D. Berrebbi, et al. , “Reproducing Whisper-Style Training Using an Open-Source Toolkit and Pub- licly Available Data,” inProc. ASRU, 2023

    2023

  3. [11]

    ESPnet: End- to-End Speech Processing Toolkit,

    S. Watanabe, T. Hori, S. Karita, T. Hayashi, et al., “ESPnet: End- to-End Speech Processing Toolkit,” inProc. Interspeech, 2018

    2018

  4. [12]

    Conformer: Convolution-augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, et al. , “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proc. Interspeech, 2020

    2020

  5. [13]

    Branchformer: Parallel MLP-attention architectures to capture local and global context for speech recognition and understanding,

    Y . Peng, S. Dalmia, I. Lane, and S. Watanabe, “Branchformer: Parallel MLP-attention architectures to capture local and global context for speech recognition and understanding,” in Proc. ICML, 2022

    2022

  6. [14]

    Zipformer: A faster and better encoder for automatic speech recognition,

    Z. Yao, L. Guo, X. Yang, W. Kang, et al., “Zipformer: A faster and better encoder for automatic speech recognition,” in Proc. ICLR, 2023

    2023

  7. [15]

    Atten- tion is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, et al., “Atten- tion is all you need,” in Proc. NeurIPS, 2017

    2017

  8. [16]

    Squeezeformer: An efficient transformer for automatic speech recognition,

    S. Kim, A. Gholami, A. E. Shaw, N. Lee, et al., “Squeezeformer: An efficient transformer for automatic speech recognition,” in Proc. NeurIPS, 2022

    2022

  9. [17]

    Fast conformer with linearly scalable attention for efficient speech recognition,

    D. Rekesh, N. R. Koluguri, S. Kriman, S. Majumdar, et al., “Fast conformer with linearly scalable attention for efficient speech recognition,” in Proc. ASRU, 2023

    2023

  10. [18]

    Sum- maryMixing: A linear-complexity alternative to self-attention for speech recognition and understanding,

    T. Parcollet, R. van Dalen, S. Zhang, and S. Bhattacharya, “Sum- maryMixing: A linear-complexity alternative to self-attention for speech recognition and understanding,” inProc. Interspeech, 2024

    2024

  11. [19]

    OWSM v3.1: Bet- ter and faster open whisper-style speech models based on E- Branchformer,

    Y . Peng, J. Tian, W. Chen, S. Arora, et al., “OWSM v3.1: Bet- ter and faster open whisper-style speech models based on E- Branchformer,” in Proc. Interspeech, 2024

    2024

  12. [20]

    E-Branchformer: Branch- former with enhanced merging for speech recognition,

    K. Kim, F. Wu, Y . Peng, J. Pan, et al., “E-Branchformer: Branch- former with enhanced merging for speech recognition,” inProc. SLT, 2023

    2023

  13. [21]

    A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Transla- tion, and Understanding Tasks,

    Y . Peng, K. Kim, F. Wu, B. Yan, et al., “A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Transla- tion, and Understanding Tasks,” inProc. Interspeech, 2023

    2023

  14. [22]

    OWSM-CTC: An open encoder-only speech foundation model for speech recognition, translation, and language identification,

    Y . Peng, Y . Sudo, M. Shakeel, and S. Watanabe, “OWSM-CTC: An open encoder-only speech foundation model for speech recognition, translation, and language identification,” in Proc. ACL, 2024

    2024

  15. [23]

    Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. J. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in Proc. ICML, 2006

    2006

  16. [24]

    Unsupervised data selection via discrete speech representation for ASR,

    Z. Lu, Y . Wang, Y . Zhang, W. Han, et al., “Unsupervised data selection via discrete speech representation for ASR,” in Proc. Interspeech, 2022

    2022

  17. [25]

    Unsupervised data selec- tion for speech recognition with contrastive loss ratios,

    C. Park, R. Ahmad, and T. Hain, “Unsupervised data selec- tion for speech recognition with contrastive loss ratios,” inProc. ICASSP, 2022

    2022

  18. [26]

    Spgispeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition,

    P. K. O’Neill, V . Lavrukhin, S. Majumdar, V . Noroozi, et al., “Spgispeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition,” in Proc. Interspeech, 2021

    2021

  19. [27]

    Gigaspeech: An evolv- ing, multi-domain ASR corpus with 10, 000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, et al., “Gigaspeech: An evolv- ing, multi-domain ASR corpus with 10, 000 hours of transcribed audio,” in Proc. Interspeech, 2021

    2021

  20. [28]

    The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage,

    D. Galvez, G. Diamos, J. Ciro, J. F. Cer ´on, et al., “The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage,” CoRR, vol. abs/2111.09344, 2021

    2021 arXiv

  21. [29]

    YODAS: Youtube-Oriented Dataset for Audio and Speech,

    X. Li, S. Takamichi, T. Saeki, W. Chen, et al. , “YODAS: Youtube-Oriented Dataset for Audio and Speech,” in Proc. ASRU, 2023

    2023

  22. [30]

    On the effects of het- erogeneous data sources on speech-to-text foundation models,

    J. Tian, Y . Peng, W. Chen, K. Choi, et al., “On the effects of het- erogeneous data sources on speech-to-text foundation models,” in Proc. Interspeech, 2024

    2024

  23. [31]

    Speechstew: Simply mix all available speech recognition data to train one large neural network,

    W. Chan, D. Park, C. Lee, Y . Zhang, et al., “Speechstew: Simply mix all available speech recognition data to train one large neural network,” arXiv preprint arXiv:2104.02133, 2021

    2021 arXiv

  24. [32]

    MSR-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research,

    S. Li, Y . You, X. Wang, Z. Tian, et al., “MSR-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research,” in Proc. Interspeech, 2024

    2024

  25. [33]

    Libriheavy: A 50,000 hours asr corpus with punctuation casing and context,

    W. Kang, X. Yang, Z. Yao, F. Kuang, et al. , “Libriheavy: A 50,000 hours asr corpus with punctuation casing and context,” in Proc. ICASSP, 2024

    2024

  26. [34]

    Gigaspeech 2: An evolving, large-scale and multi-domain asr corpus for low- resource languages with automated crawling, transcription and refinement,

    Y . Yang, Z. Song, J. Zhuo, M. Cui, et al. , “Gigaspeech 2: An evolving, large-scale and multi-domain asr corpus for low- resource languages with automated crawling, transcription and refinement,” arXiv preprint arXiv:2406.11546, 2024

    2024 arXiv

  27. [35]

    MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Founda- tion Model Training on EU Languages,

    M. Gaido, S. Papi, L. Bentivogli, A. Brutti, et al. , “MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Founda- tion Model Training on EU Languages,” inProc. EMNLP, 2024

    2024

  28. [36]

    CTC-Segmentation of Large Corpora for German End-to-End Speech Recognition,

    L. K ¨urzinger, D. Winkelbauer, L. Li, T. Watzel, and G. Rigoll, “CTC-Segmentation of Large Corpora for German End-to-End Speech Recognition,” in Speech and Computer, 2020, pp. 267– 278

    2020

  29. [37]

    Bag of tricks for efficient text classification,

    A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov, “Bag of tricks for efficient text classification,” arXiv preprint arXiv:1607.01759, 2016

    2016 arXiv

  30. [38]

    Fast- text.zip: Compressing text classification models,

    A. Joulin, E. Grave, P. Bojanowski, M. Douze, et al. , “Fast- text.zip: Compressing text classification models,”arXiv preprint arXiv:1612.03651, 2016

    2016 arXiv

  31. [39]

    Ravanelli, T

    M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, et al., Speech- Brain: A general-purpose speech toolkit , arXiv:2106.04624, 2021

    2021 arXiv

  32. [40]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila et al. , “Common voice: A massively-multilingual speech corpus,” arXiv:1912.06670, 2019

    1912 arXiv

  33. [41]

    Pytorch: An imperative style, high- performance deep learning library,

    A. Paszke et al. , “Pytorch: An imperative style, high- performance deep learning library,” inProc. NeurIPS, 2019

    2019

  34. [42]

    FlashAttention-2: Faster Attention with Better Paral- lelism and Work Partitioning,

    T. Dao, “FlashAttention-2: Faster Attention with Better Paral- lelism and Work Partitioning,” inProc. ICLR, 2024

    2024

  35. [43]

    Decoupled weight decay regular- ization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regular- ization,” in Proc. ICLR, 2019

    2019

  36. [44]

    CoV oST 2 and Massively Multilingual Speech Translation,

    C. Wang et al., “CoV oST 2 and Massively Multilingual Speech Translation,” in Proc. Interspeech, 2021

    2021

  37. [45]

    FLEURS: Few-Shot Learning Evaluation of Universal Representations of Speech,

    A. Conneau et al., “FLEURS: Few-Shot Learning Evaluation of Universal Representations of Speech,” in Proc. SLT, 2022

    2022

  38. [46]

    MLS: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A large-scale multilingual dataset for speech research,” arXiv:2012.03411, 2020

    2012 arXiv

  39. [47]

    Srivastav, S

    V . Srivastav, S. Majumdar, N. Koluguri, A. Moumen, S. Gandhi, et al., Open automatic speech recognition leaderboard, https: //huggingface.co/spaces/hf-audio/open_asr_ leaderboard, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.