Pith. sign in

REVIEW 18 cited by

Scaling Speech Technology to 1,000+ Languages

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.13516 v1 pith:HFJQH56S submitted 2023-05-22 cs.CL cs.SDeess.AS

Vineel Pratap , Andros Tjandra , Bowen Shi , Paden Tomasello , Arun Babu , Sayani Kundu , Ali Elkahky , Zhaoheng Ni
show 8 more authors
Apoorv Vyas Maryam Fazel-Zarandi Alexei Baevski Yossi Adi Xiaohui Zhang Wei-Ning Hsu Alexis Conneau Michael Auli
This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords languagesspeechmodelmultilingualtechnologyfractionlanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Expanding the language coverage of speech technology has the potential to improve access to information for many more people. However, current speech technology is restricted to about one hundred languages which is a small fraction of the over 7,000 languages spoken around the world. The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. The main ingredients are a new dataset based on readings of publicly available religious texts and effectively leveraging self-supervised learning. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, as well as a language identification model for 4,017 languages. Experiments show that our multilingual speech recognition model more than halves the word error rate of Whisper on 54 languages of the FLEURS benchmark while being trained on a small fraction of the labeled data.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 116 citations worldwide. Full citation record

  1. Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper

    cs.CL 2025-01 conditional novelty 7.0 of 10

    Embedding Layer Surgery and Task-wise Beam Search reduce catastrophic forgetting when adding new languages to Whisper, lowering old-language AWER from 14.2% to 11.9% versus Experience Replay.

  2. Motor, Cognitive, or Corpus? What Survives Cross-Lingual Transfer in Speech-Based Parkinsons Disease Detection

    cs.CL 2026-08 conditional novelty 6.0 of 10

    Frozen self-supervised speech representations transfer poorly across languages and corpora; Parkinson's-trained classifiers treat dementia speech like Parkinson's, indicating corpus-driven rather than disease-specific signal.

  3. CHILDES-Aligned: A Curated Children's Speech Dataset via Multi-Model Timestamp Ensembling

    eess.AS 2026-07 conditional novelty 6.0 of 10

    Multi-model ASR consensus (BEACON) curates 413 h of CHILDES with corrected timestamps; the 283 h ASR subset yields up to 19.5% relative WER reduction on four held-out child benchmarks.

  4. On Barriers to Archival Audio Processing

    cs.SD 2025-07 conditional novelty 6.0 of 10

    On archival radio audio, Whisper V3 identifies languages with 91.3% accuracy, but speaker embeddings lose similarity across ages and languages, making speaker recognition unreliable for indexing.

  5. Double Entendre: Robust Audio-Based AI-Generated Lyrics Detection via Multi-View Fusion

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A late-fusion model that combines ASR-transcribed lyrics and speech embeddings detects AI-written lyrics from audio alone, achieving 94.9% recall in-domain and staying robust to attacks.

  6. Speech-to-Speech Translation Pipelines for Conversations in Low-Resource Languages

    cs.CL 2025-06 conditional novelty 6.0 of 10

    For Turkish-French and Pashto-French conversational speech translation, the best cascaded pipelines combine Whisper or Microsoft ASR with Google or Microsoft MT, and component rankings are mostly stable across pipelines.

  7. Improving Language and Modality Transfer in Translation by Character-level Modeling

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A character-level encoder distilled from SONAR embeddings improves cross-lingual transfer, and a pretrained adapter connects MMS speech recognition to it for competitive zero-shot speech translation.

  8. OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A new open suite of 13 multilingual speech models, up to 18B parameters, yields empirical scaling laws for ASR and speech translation performance.

  9. Contrastive Learning for Task-Independent SpeechLLM-Pretraining

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Contrastive pre-training that aligns speech and text across all model layers beats ASR-based pre-training and, with 10% of task data, matches or exceeds specialized models on translation and question answering.

  10. Towards Maximum Likelihood Training for Transducer-based Streaming Speech Recognition

    eess.AS 2024-11 reject novelty 6.0 of 10

    A learned future-audio density ratio, FoCCE, is inserted into the streaming transducer forward recursion during training and modestly reduces word error rates.

  11. k2SSL: A Faster and Better Framework for Self-Supervised Speech Representation Learning

    eess.AS 2024-11 conditional novelty 6.0 of 10

    Zipformer-based self-supervised pretraining is up to 3.5x faster than HuBERT and yields lower ASR word error rates on LibriSpeech.

  12. Towards Digital Preservation of Efik: TTS for a Low-Resource African Language

    cs.CL 2026-07 conditional novelty 5.5 of 10

    First end-to-end Efik TTS baseline: a 3-hour single-speaker corpus and four fine-tuned models, with MMS-TTS best at MOS 3.80±0.63 but residual tonal errors.

  13. Context-Driven Dynamic Pruning for Large Speech Foundation Models

    eess.AS 2025-05 conditional novelty 5.0 of 10

    A per-layer gate predictor using speaker and acoustic-event embeddings dynamically prunes an OWSM speech model, cutting encoder GFLOPs by 56.7 while improving Europarl-ST BLEU by about 26% relative.

  14. From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognition

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Fine-tuning TTS models on tens of hours of real audio enables generation of 500,000 hours of synthetic speech that reduces ASR error rates by over 30% on Whisper-large-v3.

  15. Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Typhoon 2 improves Thai LLM performance through continual pre-training on curated Thai data and post-training, releasing text, vision, audio, and safety models.

  16. DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages

    cs.CL 2026-07 reject novelty 4.0 of 10

    Open w2v-BERT ASR base models for 27 African languages, with a two-step annealing recipe and prefix-frame language conditioning.

  17. FreeSVC: Towards Zero-shot Multilingual Singing Voice Conversion

    cs.SD 2025-01 conditional novelty 4.0 of 10

    FreeSVC combines a multilingual SPIN content extractor, ECAPA2 speaker embeddings, and language embeddings to improve zero-shot cross-lingual singing voice conversion over a ContentVec baseline.

  18. A Hybrid Machine Learning Framework for Optimizing Crop Selection via Agronomic and Economic Forecasting

    cs.LG 2025-07 reject novelty 3.0 of 10

    A two-stage ML pipeline that recommends crops by predicted market price after filtering for agronomic suitability, delivered through a Kannada voice interface.

Pith tools