REVIEW 8 cited by
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization). The adoption of these methods is still limited by the fact that most speech synthesis datasets are read, severely limiting spontaneity and expressivity. Here, we introduce Expresso, a high-quality expressive speech dataset for textless speech synthesis that includes both read speech and improvised dialogues rendered in 26 spontaneous expressive styles. We illustrate the challenges and potentials of this dataset with an expressive resynthesis benchmark where the task is to encode the input in low-bitrate units and resynthesize it in a target voice while preserving content and style. We evaluate resynthesis quality with automatic metrics for different self-supervised discrete encoders, and explore tradeoffs between quality, bitrate and invariance to speaker and style. All the dataset, evaluation metrics and baseline models are open source
Forward citations
Cited by 8 Pith papers
-
Dialogs: a studio-quality expressive conversational Russian speech corpus for dialog assistants
Dialogs is a new 20.6-hour studio-quality Russian conversational speech corpus with emotion labels and a VITS2 proof-of-concept.
-
AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders
SAE features from Whisper and HuBERT are seed-stable, interpretable, and steerable: cutting false speech detections by 70% and correlating with EEG responses to speech.
-
NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizations
A Mandarin speech pipeline that tags non-word vocalizations (laughter, breath, filled pauses) for ASR and controls their generation in TTS, backed by a claimed 573-hour, 174,179-utterance word-level annotated corpus.
-
NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech
The authors release NonverbalTTS, a 17-hour open dataset with nonverbal vocalization and emotion annotations, and show that fine-tuning an open TTS model on it yields performance comparable to CosyVoice2.
-
WHISTRESS: Enriching Transcriptions with Sentence Stress Detection
WHISTRESS extends Whisper with a token-level stress classifier trained on a new synthetic dataset, and shows zero-shot transfer to natural speech benchmarks.
-
Beyond One-Size-Fits-All: Personalized and Culturally Adaptive Emotional TTS via Interactive Optimization of Individual Emotion Perception Spaces
An interactive genetic algorithm tunes arousal-valence coordinates per listener in emotional TTS, and personalized or culture-specific coordinates beat a generic U.S.-average baseline in small A/B tests.
-
X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
X3-OPD improves audio-grounded reasoning by training the audio student on its own rollouts with token-level teacher feedback, using a three-tier paired text-audio corpus.
-
Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
A FreeVC-style conditional VAE with mHuBERT-147 discrete units, mixed-style layer normalization, an augmentation similarity loss, and F0 cross-attention reports better emotion transfer and less source leakage than thr...
Discussion (0). Continue with ORCID to comment.