REVIEW 13 cited by
Better speech synthesis through scaling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In recent years, the field of image generation has been revolutionized by the application of autoregressive transformers and DDPMs. These approaches model the process of image generation as a step-wise probabilistic processes and leverage large amounts of compute and data to learn the image distribution. This methodology of improving performance need not be confined to images. This paper describes a way to apply advances in the image generative domain to speech synthesis. The result is TorToise -- an expressive, multi-voice text-to-speech system. All model code and trained weights have been open-sourced at https://github.com/neonbjb/tortoise-tts.
Forward citations
Cited by 13 Pith papers
-
Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training
Jointly training the watermark embedder/detector with the source separator enables ~1% bit-error-rate recovery of per-stem watermarks after mixing and separation, where independent training yields 15–35%.
-
Next Tokens Denoising for Speech Synthesis
Dragon-FM generates speech autoregressively over two-second chunks while using flow matching inside each chunk, achieving fast synthesis at 12.5 discrete audio tokens per second.
-
SpeechFake: A Large-Scale Multilingual Speech Deepfake Dataset Incorporating Cutting-Edge Generation Methods
SpeechFake is a large-scale multilingual deepfake speech dataset with baseline experiments showing improved generalization to unseen generation methods.
-
TTS-1 Technical Report
A new TTS system family combines pre-training, supervised fine-tuning, and GRPO reinforcement learning with a 48 kHz codec to produce multilingual speech with in-context voice cloning.
-
Semantics-Aware Human Motion Generation from Audio Instructions
An end-to-end audio-conditioned model generates 3D human motion from spoken instructions, with performance close to text-conditioned baselines on synthetic speech datasets.
-
Large Language Model Data Generation for Enhanced Intent Recognition in German Speech
LLM-generated German text data improves intent recognition for elderly German speakers, and the smaller German-focused LeoLM outperforms the much larger ChatGPT as a data generator.
-
Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space
A four-modality contrastive model with token-level alignment improves text, video, and newly introduced audio-to-motion retrieval on HumanML3D and KIT-ML.
-
Quantize More, Lose Less: Autoregressive Generation from Residually Quantized Speech Representations
QTTS models speech as sequences from a multi-codebook RVQ audio codec whose first codebook is trained with ASR supervision, aiming for higher-fidelity TTS than single-codebook systems.
-
De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks
Existing voice-protection perturbations succeed only against naive attackers; a phoneme-guided purification-refinement pipeline restores cloneability of protected speech for most VC models.
-
Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis
Rectified flows with a tunable sampling temperature offer the best naturalness-diversity trade-off among stochastic prosody predictors for text-to-speech.
-
StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decoding
StreamFlow achieves streaming speech token decoding with 180 ms first-packet latency by using hierarchical block-wise attention masks in a DiT flow matching model.
-
MPE-TTS: Customized Emotion Zero-Shot Text-To-Speech Using Multi-Modal Prompt
A multi-modal emotion prompt encoder and prosody predictor let MPE-TTS control emotion from speech, text, or image while preserving speaker timbre.
-
JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1
A paper announcing a large-scale whole-body talking avatar benchmark and evaluation protocol, but with insufficient details to verify the dataset or the joint audio-video evaluation.
Discussion (0). Sign in to comment.