REVIEW 8 cited by
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large-scale generative models such as GPT and DALL-E have revolutionized the research community. These models not only generate high fidelity outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are not filtered or enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster. Audio samples can be found in \url{https://voicebox.metademolab.com}.
Forward citations
Cited by 8 Pith papers
-
Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training
Jointly training the watermark embedder/detector with the source separator enables ~1% bit-error-rate recovery of per-stem watermarks after mixing and separation, where independent training yields 15–35%.
-
VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation
VoiceStar uses a progress-based rotary position embedding and mixed prompt training to give zero-shot voice cloning precise duration control and much longer output than training clips.
-
X-Translator: A Real-Time Multilingual Speaker-Aware Speech-to-Speech Translation System
An open, modular cascaded system (streaming ASR + MT + prompt-conditioned TTS) preserves speaker identity in long-form multi-speaker translation, at higher latency and slightly lower translation quality than proprietary APIs.
-
Unlocking Speech Instruction Data Potential with Query Rewriting
A multi-LLM rewriting and multi-agent validation pipeline makes text-to-speech synthesized speech instruction data far more usable and improves downstream speech instruction following.
-
Optimal Self-Distillation for Rectified Flow via Linear Probing
For linear rectified flow with ridge regression on fixed interpolants, optimally mixed self-distillation strictly improves velocity risk whenever the teacher is off the ridge stationary point, with a closed-form mixin...
-
A Survey on Generative Model Unlearning: Fundamentals, Taxonomy, Evaluation, and Future Direction
A survey and framework that categorizes generative model unlearning by point-wise versus concept-wise objectives, parameter-based versus non-parametric methods, and completeness/utility/efficiency evaluation.
-
Technical report: Impact of Duration Prediction on Speaker-specific TTS for Indian Languages
In a five-language zero-shot TTS study, no single duration prediction strategy dominates: speaker-prompted durations help some languages, infilling durations help others, and results vary by metric.
-
EZ-VC: Easy Zero-shot Any-to-Any Voice Conversion
EZ-VC combines discrete units from a multilingual self-supervised encoder (Xeus) with an F5-TTS flow-matching decoder to achieve zero-shot any-to-any voice conversion, without text labels or multiple disentangling encoders.
Discussion (0). Continue with ORCID to comment.