REVIEW 25 cited by
Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in neural vocoding are predominantly driven by Generative Adversarial Networks (GANs) operating in the time-domain. While effective, this approach neglects the inductive bias offered by time-frequency representations, resulting in reduntant and computionally-intensive upsampling operations. Fourier-based time-frequency representation is an appealing alternative, aligning more accurately with human auditory perception, and benefitting from well-established fast algorithms for its computation. Nevertheless, direct reconstruction of complex-valued spectrograms has been historically problematic, primarily due to phase recovery issues. This study seeks to close this gap by presenting Vocos, a new model that directly generates Fourier spectral coefficients. Vocos not only matches the state-of-the-art in audio quality, as demonstrated in our evaluations, but it also substantially improves computational efficiency, achieving an order of magnitude increase in speed compared to prevailing time-domain neural vocoding approaches. The source code and model weights have been open-sourced at https://github.com/gemelo-ai/vocos.
Forward citations
Cited by 25 Pith papers
-
TS3-Codec: Transformer-Based Simple Streaming Single Codec
A convolution-free, transformer-only streaming codec with a single codebook matches or beats a strong convolutional baseline at 12% compute and 77% bitrate.
-
Phoenix TTS: High-Fidelity Synthesis and Voice Conversion via Flow-Matching-Driven Speech Tokenization
A jointly trained speech tokenizer and flow-matching decoder achieve strong zero-shot TTS and voice conversion on English and Mandarin, with WER below ground truth in several tests.
-
Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer
Depth-only pruning of a flow-matching Hindi TTS teacher, followed by staged re-fine-tuning, produces 131–190M students with ASR-WER close to the teacher and real-time laptop inference.
-
ZipL-Dialog: Memory-Efficient Long-Form Spoken Dialog Synthesis via Latent Flow Matching
ZipL-Dialog cuts peak GPU memory 11.22× and speeds inference 2.23× for multi-minute zero-shot dialog TTS by doing conditional flow matching in a 4× compressed latent space while keeping perceptual naturalness.
-
Unified Audio Intelligence Without Regressing on Text Intelligence
A unified 30B MoE audio-text LLM achieves state-of-the-art audio understanding, generation, and speech tasks while preserving text reasoning comparable to its text-only backbone.
-
FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech
FlowTTS-GRPO fine-tunes open-source flow-matching TTS models with multi-objective online RL via ODE-to-SDE conversion, improving speaker similarity and quality on CosyVoice 3.0 and F5-TTS.
-
UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations
UniPASE extends the PASE framework with DeWavLM-Omni to convert degraded speech into high-fidelity, low-hallucination audio across sampling rates via phonetic enhancement, acoustic adaptation, and multi-rate vocoding.
-
HH-Codec: High Compression High-fidelity Discrete Neural Codec for Spoken Language Modeling
HH-Codec reaches 24 tokens/s and 0.3 kbps for 24 kHz speech with single-quantizer inference and reports reconstruction metrics close to much higher-bandwidth codecs.
-
DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis
Reinforcement learning on duration prediction improves intelligibility and speaker similarity in a 4-step distilled text-to-speech model, and teacher-guided sampling recovers prosodic diversity.
-
Autoregressive Speech Enhancement via Acoustic Tokens
Acoustic tokens outperform semantic tokens on speaker identity in speech enhancement, an autoregressive transducer helps in some settings, but discrete representations still lag continuous ones.
-
XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs
XY-Tokenizer is a 1 kbps dual-channel speech codec that reports simultaneously strong text alignment and high speaker similarity, comparable to specialized codecs at similar bitrates.
-
SpeechRefiner: Towards Perceptual Quality Refinement for Front-End Algorithms
SpeechRefiner, a conformer-based conditional flow matching model, improves SIGMOS perceptual quality scores on speech processed by various front-ends, including unseen systems.
-
Mel-McNet: A Mel-Scale Framework for Online Multichannel Speech Enhancement
Mel-McNet performs online multichannel speech enhancement in the Mel domain, reducing FLOPs by roughly 60% versus McNet while keeping speech quality and ASR accuracy comparable.
-
FlowTSE: Target Speaker Extraction with Flow Matching
Conditional flow matching on mel-spectrograms with a phase-conditioned vocoder matches or beats published TSE baselines on Libri2Mix.
-
GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modeling
GenSE enhances speech by first denoising semantic tokens with a language model and then generating acoustic tokens from a single-quantizer codec, reporting higher DNSMOS, speaker similarity, and lower WER than prior systems.
-
ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram
ESTVocoder synthesizes speech by transforming the amplitude and phase spectra of an F0-derived harmonic excitation into speech spectra with a ConvNeXt v2 neural filter, improving several objective metrics over HiFi-GA...
-
Traceable TTS: Toward Watermark-Free TTS with Strong Traceability
A joint training loop makes an F5-TTS model produce audio that a paired wav2vec 2.0/LCNN discriminator can recognize, enabling watermark-free attribution; however, the reported generalization gain is not isolated from...
-
MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation
A single-layer streaming Transformer codec with masked Gaussian noise injection during training reports state-of-the-art reconstruction and better downstream generation and understanding in 16 kHz English speech.
-
FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching
FlowSE applies rectified flow matching with a DiT backbone to speech enhancement, reporting better DNSMOS and WER results and a much lower real-time factor than diffusion baselines.
-
From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognition
Fine-tuning TTS models on tens of hours of real audio enables generation of 500,000 hours of synthetic speech that reduces ASR error rates by over 30% on Whisper-large-v3.
-
OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
OZSpeech is a one-step zero-shot TTS system that starts from learned content and mean-style codes and uses flow matching to refine them, achieving very low word error rates with a small model.
-
Metis: A Foundation Speech Generation Model with Masked Generative Pre-training
A masked generative model pre-trained on unlabeled speech then fine-tuned per task matches or beats task-specific systems across TTS, voice conversion, speaker extraction, enhancement, and lip-to-speech.
-
A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions
A two-stage neural vocoder predicts noisy amplitude and phase spectra from a noisy mel-spectrogram, denoises them with MP-SENet, and reconstructs clean speech, outperforming HiFi-GAN and Vocos on VoiceBank+DEMAND.
-
Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders
Denoising audio embeddings from a generative pretrained encoder before vocoder synthesis yields efficient speech enhancement with better speaker fidelity than discriminative encoders.
-
DS-Codec: Dual-Stage Training with Mirror-to-NonMirror Architecture Switching for Speech Codec
DS-Codec improves low-bitrate speech codec quality by first training a mirrored codec and then switching to a non-mirrored decoder, while using product quantization to form one large codebook.
Discussion (0). Continue with ORCID to comment.