SpeechBrain: A General-Purpose Speech Toolkit
read the original abstract
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.
This paper has not been read by Pith yet.
Forward citations
Cited by 32 Pith papers
-
SpeechDx: A Multi-Task Benchmark for Clinical Speech AI
SpeechDx is a multi-task benchmark with 12 datasets and 27 tasks across health conditions, structured by conceptualization, formulation, and articulation stages, showing that no current audio encoder generalizes reliably.
-
Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India
Voice of India is a new 536-hour benchmark of real telephonic conversations in 15 Indian languages with variant-aware transcripts for more realistic ASR evaluation.
-
Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
NodePFN pre-trains on synthetic graphs with controllable homophily and causal feature-label models to achieve 71.27 average accuracy on 23 node classification benchmarks without graph-specific training.
-
Hierarchical Codec Diffusion for Video-to-Speech Generation
HiCoDiT generates speech from video by conditioning low-level RVQ tokens on speaker identity and high-level tokens on facial expressions via a dual-scale normalized diffusion transformer.
-
MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks
MECAT is a multi-expert benchmark for audio AI offering fine-grained captions and QA pairs generated via expert models and LLM reasoning, paired with the DATE metric that combines semantic similarity and cross-sample ...
-
DASB - Discrete Audio and Speech Benchmark
DASB is a new benchmark for discrete audio tokens showing semantic tokens outperform acoustic ones but discrete representations remain less robust than continuous features across domains.
-
LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish
LuxEmo is a new 21-hour conversational expressive speech corpus for Luxembourgish with 4 emotion categories, created via semi-automatic curation from RTL broadcasts and used to benchmark five TTS systems.
-
LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish
LuxEmo is a new 21-hour Luxembourgish expressive speech corpus with emotion labels, built from RTL broadcasts using automated detection plus human validation, and used to benchmark five TTS systems.
-
What Counts as an Error? Dual-Reference Benchmarking for Atypical ASR
Dual-reference benchmarking on atypical stuttered speech reveals disparities in ASR model performance and rankings between verbatim and intended transcriptions.
-
Mechanisms of Misgeneralization in Physical Sequence Modeling
Generative sequence models for physical tasks exhibit physical misgeneralization where local prediction errors propagate through physical measurements to distort aggregate distributions over quantities like distance o...
-
Evaluating voice anonymisation using similarity rank disclosure
SRD provides a threshold-independent, representation-level privacy assessment for voice anonymization that reveals system weaknesses not detected by equal error rate evaluation.
-
A Paradigm for Interpreting Metrics and Identifying Critical Errors in Automatic Speech Recognition
A paradigm converts any metric into a minimum edit distance equivalent to interpret ASR errors in terms of human perception and severity.
-
Text-To-Speech with Chain-of-Details: modeling temporal dynamics in speech generation
Chain-of-Details (CoD) is a cascaded TTS method that explicitly models temporal coarse-to-fine dynamics with a shared decoder, achieving competitive performance using significantly fewer parameters.
-
Contextual Biasing for ASR in Speech LLM with Common Word Cues and Bias Word Position Prediction
Common-word acoustic cues and bias-word position prediction in speech LLMs cut rare-word transcription errors by 16.3% versus baselines, including out-of-domain cases.
-
FAC-FACodec: Controllable Zero-Shot Foreign Accent Conversion with Factorized Speech Codec
FAC-FACodec is a controllable zero-shot foreign accent conversion framework using a factorized speech codec that adds an explicit parameter for adjusting pronunciation-level accent modification strength.
-
From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
Mamba matches Conformer accuracy for ASR in seven South African languages with lower compute, multilingual training improves results, and language embeddings aid cross-corpus robustness but do not capture typological ...
-
LISE : Listenable Interpretable Speaker Embeddings
LISE decomposes pretrained speaker embeddings into components that preserve ASV performance with negligible EER degradation and enable listeners to distinguish speakers at 83.9% accuracy.
-
Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition
Event-driven SpeechMamba with FATReLU reaches over 60% sparsity and spiking version over 70% sparsity with <1% accuracy drop while cutting parameters 30% versus prior SNNs, plus a new cycle-accurate simulator.
-
Syllabic-Structure Decoder for Automatic Speech Recognition in Vietnamese
A phoneme-based syllabic decoder for Vietnamese ASR outperforms larger-vocabulary baselines like PhoWhisper on standard and multi-dialect benchmarks while using a compact inventory.
-
PashtoTTS-Bench: automated screening for low-resource non-Latin-script text-to-speech
Introduces INSV-A automated screening benchmark for Pashto TTS systems reporting WER, script fidelity, and LID results across five systems on FLEURS and Common Voice prompts.
-
HATS: An Open data set Integrating Human Perception Applied to the Evaluation of Automatic Speech Recognition Metrics
HATS supplies human side-by-side preference judgments on ASR transcripts to measure correlation with lexical and embedding-based evaluation metrics.
-
A Toolkit for Detecting Spurious Correlations in Speech Datasets
A toolkit flags spurious correlations in speech datasets by checking if non-speech regions predict the target class better than chance.
-
Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India
A 536-hour, 15-language, 139-cluster telephonic ASR benchmark for Indian languages with spelling-variation-aware transcripts and geographic performance analysis.
-
DeepFense: A Unified, Modular, and Extensible Framework for Robust Deepfake Audio Detection
DeepFense supplies a unified toolkit and large-scale benchmarks showing that pre-trained front-end feature extractors drive most performance differences while top models exhibit strong biases by audio quality, speaker...
-
A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models
Prioritizing longest utterances in SSL speech pre-training data outperforms random or diversity-based sampling for ASR performance while using half the data volume.
-
ESPnet3: Infrastructure for Scalable Speech and Audio Research in the Foundation Model Era
ESPnet3 introduces a new modular architecture with DataOrganizer and sharding to cut training time and simplify model integration for speech research.
-
Montreal Forced Aligner and the state of speech-to-text alignment in 2026
MFA version 3.0 reaches state-of-the-art or near state-of-the-art results on forced alignment benchmarks for English, Japanese, and Korean with average boundary errors under 15 milliseconds.
-
Beyond ROC-AUC: Operating-Point Performance Reporting for Biometric Verification
Full ROC-AUC can hide or reverse low-FMR performance differences, as FaceNet outperforms ArcFace on AUC but underperforms at FMR=10^-3 with statistical significance.
-
Phonetic Modeling of Dialectal Variation in Vietnamese Speech
Dialect-specific phonetic vocabulary and joint decoder for Vietnamese ASR that matches wav2vec2-base-vi-250h performance with fewer parameters and no pretraining on the UIT-ViMD dataset.
-
Enhancing Speaker Verification with Whispered Speech via Post-Processing
Post-processing with an encoder-decoder model yields 22% relative EER reduction on normal-vs-whispered trials and 1.88% EER on whispered-vs-whispered, outperforming ReDimNet-B2.
-
SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization
SEDTalker uses frame-level speech emotion diarization to condition a hybrid Transformer-Mamba model for fine-grained, temporally continuous emotion control in 3D facial animation.
-
KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026
KIT's IWSLT 2026 submission adapts a multilingual TTS model with language prompting, RL fine-tuning, and reference-conditioned lexical matching, reporting largest gains from prompting.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.