Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 123M-parameter flow-matching TTS matches state-of-the-art zero-shot quality while running far faster.

desk verdict Solid compact zero-shot TTS system with believable results; the real question is how much the average-upsampling alignment assumption limits robustness beyond English and Chinese. read the letter →

arxiv 2506.13053 v3 pith:VQ342OQI submitted 2025-06-16 eess.AS cs.SD

Han Zhu , Wei Kang , Zengwei Yao , Liyong Guo , Fangjun Kuang , Zhaoqing Li , Weiji Zhuang , Long Lin
show 1 more author
Daniel Povey
This is my paper
classification eess.AScs.SD
keywords zero-shottext-to-speechflowmatchingZipformerspeech-textalignmentaverageupsamplingdistillationefficientinferencenon-autoregressiveTTS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a compact, non-autoregressive text-to-speech system can deliver the quality of much larger zero-shot voice-cloning models without their inference cost. ZipVoice, a 123-million-parameter model trained on 100k hours of multilingual speech, is built on a Zipformer backbone, uses parameter-free average upsampling to align text with speech, and is distilled so it needs only 4 to 16 sampling steps with no classifier-free-guidance overhead. The authors report that it matches or beats larger systems on intelligibility, speaker similarity, and naturalness, while being 3 times smaller and up to 30 times faster than the F5-TTS baseline, and close to real time on a single CPU thread. If correct, the claim matters because it suggests state-of-the-art zero-shot TTS no longer requires multi-hundred-million-parameter models or expensive iterative sampling.

What carries the argument

The machinery is conditional flow matching, which trains a vector field $v_t$ to push Gaussian noise toward speech features, plus three ZipVoice-specific choices that carry the result. The vector field estimator and text encoder are Zipformer stacks, an efficient transformer-style encoder originally built for speech recognition whose U-Net-like multi-resolution structure, convolutional modules, and attention-weight reuse supply modeling capacity within 123M parameters. Alignment is handled by average upsampling: each text embedding is repeated $d = \lfloor T/N \rfloor$ times, and leftover frames are filled with filler embeddings, giving the flow model a stable initial text-speech correspondence without any duration predictor. Finally, flow distillation builds a teacher vector field from two CFG-guided teacher steps and regresses the student onto it while conditioning the student on the CFG strength, so the student inherits CFG's quality gain with one evaluation per step.

What would settle it

Run ZipVoice on a corpus with strongly non-uniform token durations, such as Mandarin with long contrastive pauses or English sentences with heavily reduced function words, and measure WER against forced-alignment ground truth; if WER climbs toward the 20.19 observed when average upsampling is removed instead of staying near 1.5 to 1.7, the uniform-duration alignment, not the model size, is doing the load-bearing work for intelligibility.

Watch

Extended reading notes

Core claim

ZipVoice is a flow-matching zero-shot TTS model whose central claim is that a carefully chosen efficient architecture, a simple alignment prior, and a distillation step can erase the usual speed-quality trade-off. With roughly 123M parameters, a Zipformer-based vector field estimator and text encoder, and an average upsampling rule that repeats each text embedding for $\lfloor T/N \rfloor$ frames, the paper reports intelligibility, speaker similarity, and naturalness comparable to existing state-of-the-art systems such as MaskGCT, E2-TTS, and F5-TTS that have 155M to 1048M parameters. The distilled variant runs at 4 to 8 sampling steps, avoids the second classifier-free-guidance pass by taking the CFG strength as a model input, and achieves a real-time factor of 0.0125 on GPU and 1.22 on a single CPU thread, which is close to real-time synthesis.

Load-bearing premise

The system assumes every text token in a sentence is spoken for the same length of time (the paper's $\lfloor T/N \rfloor$ rule, with leftover frames filled in), and if real durations vary, the alignment and intelligibility benefit can collapse.

Editorial extensions

If this is right

  • If the results hold, zero-shot TTS with quality comparable to large models can run on a single CPU thread near real time, which changes the deployment envelope to laptops and edge devices.
  • The 3x smaller model at comparable WER and UTMOS suggests that parameter count is not the binding constraint on flow-matching TTS quality; architecture and alignment matter as much as scale.
  • Flow distillation with CFG-strength conditioning removes the doubled inference pass of classifier-free guidance, so the speedup compounds with reduced sampling steps.
  • The average upsampling ablation, where removing it raises WER from 1.69 to 20.19, indicates that a crude uniform-duration prior can substitute for explicit alignment in non-autoregressive TTS without a duration model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uniform-duration assumption in the paper's Eq. (5) is the natural stress point: languages with heavy rhythm variation, very long sentences, or expressive prosody may expose it, and a testable extension is to replace floor-based padding with a learned duration predictor and check whether intelligibility holds while the speed advantage shrinks.
  • Because the paper uses ASR-based WER as the intelligibility proxy, the matched-quality claim could shift with a different ASR model or under noisy conditions; extending evaluation to human listening with prosodic and cross-lingual emphasis would clarify the boundary.
  • The CFG-strength-as-input trick used in distillation could transfer to other conditional flow or diffusion generation tasks that currently pay a doubled inference cost for classifier-free guidance.
  • Zipformer's success as a TTS backbone suggests that speech recognition and speech generation share enough structure that ASR-oriented efficient encoders are a reusable resource for generation, not just recognition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. ZipVoice is a zero-shot text-to-speech system built on conditional flow matching. Its three main designs are: (1) a Zipformer-based vector field estimator and text encoder, which the authors argue gives strong modeling capacity at 123M parameters; (2) an average-upsampling speech-text alignment (Eq. 5) that repeats each text token uniformly floor(T/N) times and pads the remainder with filler embeddings, avoiding explicit duration prediction; and (3) a flow-distillation method that lets a student model absorb classifier-free guidance (CFG) into its inputs, so that inference uses fewer sampling steps and only one network evaluation per step. The model is trained on 100k hours of Emilia and 585 hours of LibriTTS, then evaluated on LibriSpeech-PC test-clean, Seed-TTS test-en, and Seed-TTS test-zh. The paper reports that ZipVoice matches or improves on larger SOTA systems in WER, SIM-o, and UTMOS while being up to 30x faster than F5-TTS on CPU, and it provides extensive ablations for each architectural choice.

Significance. If the reported results are reliable, ZipVoice is a practically significant contribution: it shows that a compact Zipformer-based flow-matching model can reach SOTA-level zero-shot TTS quality while running close to real time on a single CPU core. The flow-distillation method that removes CFG's extra inference pass is a useful engineering idea, and the parameter-free average upsampling alignment is simple and reproducible. The paper ships code, checkpoints, and demo samples, and the ablations (Tables IV-VI) isolate the contribution of each component rather than relying on a single end-to-end comparison. External benchmarks against official or widely used checkpoints, rather than only self-reported numbers, strengthen the credibility of the comparison. The main risks are the untested robustness of the duration estimation at inference and the absence of confidence intervals for the headline metric comparisons; neither is a fatal flaw, but both need to be addressed before the SOTA-quality claim is fully supported.

major comments (3)
  1. [II-F, Eq. (11)] The inference-time duration estimate T_synthesis = T_prompt * (|y_synthesis| / |y_prompt|) determines the length of the average-upsampled text condition used by the vector field estimator, yet the model is trained with ground-truth speech lengths. The paper does not validate the accuracy of Eq. (11) on the test sets, nor does it study how WER or SIM-o degrade when T_synthesis is perturbed. This matters because Table IV shows that the average-upsampling alignment is the primary carrier of intelligibility: removing it raises WER from 1.69 to 20.19, and replacing it with ConvNeXt still leaves WER at 15.49. If Eq. (11) is inaccurate for a given language, speaker, or prosodic style, the text condition is systematically misaligned and intelligibility should collapse in a similar way. To support the SOTA-quality claim, please report the distribution of estimated versus ground-truth durations on the evaluation sets, and ideally include a sensitivity analysis (e.g., WER and SIM-o as a function of multiplicative or additive errors in T_synthesis).
  2. [V-A, Table I] All metrics in Table I are point estimates with no confidence intervals or significance tests. The headline claim of being 'comparable to SOTA' rests on small numeric differences, such as ZipVoice's WER 1.64 versus F5-TTS 1.89 on LibriSpeech-PC and SIM-o 0.668 versus MaskGCT 0.691. Without bootstrap confidence intervals or a significance test (e.g., a paired test over the evaluation samples), the reader cannot tell whether these differences are meaningful or within sampling noise. Since this is the central claim of the paper, please provide confidence intervals or significance tests for at least the key comparisons among ZipVoice, F5-TTS, and MaskGCT on each benchmark.
  3. [IV-E] The metric terminology in Section IV-E is reversed. WER, SIM-o, and UTMOS are objective, reproducible metrics computed by automatic models, while CMOS and SMOS are subjective metrics collected from human listeners. The paragraph labels WER/SIM-o/UTMOS as 'subjective' and CMOS/SMOS as 'objective,' which misdescribes the evidence basis. This is more than a wording issue because it appears in the section that defines the evaluation protocol and could mislead readers about which results rely on human ratings. Please correct this paragraph and make the labels in Table I's column headers consistent with the corrected terminology.
minor comments (4)
  1. [II-D] The text states 'Under the practically valid assumption T >= N, the minimum token duration is 1.' However, the inference-time duration estimate in Eq. (11) can in principle yield T_synthesis < N if the prompt contains many more tokens than the target text. Please state how this case is handled (e.g., clamping d to 1 and using a shorter total length) or justify that it does not occur in practice.
  2. [II-F] The time-dependent CFG strategy is described only qualitatively ('in early NFEs, only the text condition is dropped... in later steps, both text and audio conditions are dropped'). To make the inference procedure reproducible, please specify the exact NFE thresholds and the number of steps in each phase.
  3. [Throughout] There are several typos and formatting inconsistencies: 'V ocos' should be 'Vocos', 'Vocder' should be 'vocoder' (Table II), 'filter tokens' should be 'filler tokens' in Section II-D, and 'with with NFEs' appears in Section V-E. The paper also uses 'ZipV oice' with a spurious space in the abstract and running text, likely from LaTeX line-breaking, which should be normalized to 'ZipVoice'.
  4. [Table I] The row for E2-TTS is marked as an unofficial implementation and trained on 100K Emilia. Since E2-TTS's original training data and recipe differ, please clarify whether this checkpoint is a re-training by the authors or an externally provided unofficial checkpoint, and whether the comparison is fair in terms of training data and steps.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: ZipVoice's quality and speed claims are backed by external benchmarks and in-paper ablations; the only self-citation (Zipformer) is non-load-bearing.

full rationale

ZipVoice is a systems paper whose central claims—SOTA-comparable intelligibility, speaker similarity, and naturalness at 123M parameters and 4–16 NFE—are verified against externally released checkpoints (F5-TTS official and retrained, MaskGCT official, E2-TTS unofficial) using independent measurement pipelines (Whisper-large-v3, Paraformer-zh, Hubert-based ASR, WavLM-ECAPA cosine similarity, UTMOS). None of the headline numbers is a fitted parameter renamed as a prediction. Eq. (5) (average upsampling, d = floor(T/N)) and Eq. (11) (duration ratio) are explicit heuristics, and the paper candidly admits Eq. (5) is "theoretically simplistic and has a considerable gap between real durations" (Sec. II-D), then justifies it empirically via the Table IV ablation (WER 1.69 vs 20.19/15.49 without it). The flow-distillation objective (Eqs. 6–9) regresses the student onto a teacher-constructed vector field; the resulting speedup claim rests on measured RTF (Table II) and externally evaluated WER/UTMOS (Tables I, VI), not on the distillation equations by themselves. The only self-citations of note are Zipformer [16] and LibriHeavy [9]; the Zipformer backbone's TTS-specific value is demonstrated by the paper's own Table V ablations (e.g., removing bypass raises WER to 98.89), so the self-citation is not the load-bearing evidence. Genuine limitations, flagged for correctness risk rather than circularity: the uniform-duration assumption's robustness to duration mismatch is untested (no WER vs. T_synthesis-error sweep, and Sec. VI-E concedes low-NFE performance is limited by the absence of token-level duration), the "multilingual" claim is evaluated only on English and Chinese, and the CFG strength schedule is unreported. These do not make any claimed result equivalent to its inputs by construction. Score 2 reflects only the presence of a minor, non-load-bearing self-citation; the derivation chain is otherwise self-contained.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests primarily on standard flow-matching theory, the speech-infilling task from Voicebox, and the paper's own uniform-duration alignment assumption. No invented entities are introduced. The flow-distillation method adds hyperparameters (Δt_max, ω range, EMA decay) whose values are not reported, which slightly raises the burden on the reader to reproduce the method.

free parameters (5)
  • Teacher step-size bound Δt_max
    Used in Eq. (6)-(7) to sample tmid and tdest; value is not reported, yet it controls the distillation trajectory.
  • CFG strength range [ω_min, ω_max]
    Teacher vector field in Eq. (8) is built with ω sampled uniformly from this interval; the range is not reported.
  • EMA decay factor β
    Second distillation phase uses an EMA teacher in Eq. (10); the value is not reported.
  • Speech masking ratio = 70%-100%
    Randomly sampled masking length for the speech-infilling task; a hand-chosen training choice that shapes zero-shot speaker transfer.
  • CFG text-drop probability = 20%
    Standard CFG dropout during flow-matching training; affects the unconditional predictions used by the teacher.
assumptions (4)
  • standard math Conditional flow matching with the straight-line path x_t = (1-t)x0 + t x1 is a valid generative objective when trained via Eq. (1).
    Foundation of the method; adopted from Lipman et al. [12] without re-derivation.
  • ad hoc to paper Uniform token duration: every text token spans d = floor(T/N) frames, with filler embeddings for leftovers.
    Introduced in Section II-D, Eq. (5); the authors explicitly call it theoretically simplistic, and it is load-bearing for alignment.
  • domain assumption Speech infilling with a random temporal mask teaches zero-shot voice transfer from the unmasked prompt.
    Adopted from Voicebox [2]; the paper does not independently validate this assumption beyond the final benchmark results.
  • domain assumption WER, SIM-o, and UTMOS are treated as valid proxies for intelligibility, speaker similarity, and naturalness.
    Section IV-E uses ASR models, a speaker-embedding model, and UTMOS; the paper even mislabels these as subjective metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching." pith.science (2026). https://pith.science/paper/VQ342OQI

@misc{pith2026250613053,
  author       = {Pith},
  title        = {Pith review of: ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQ342OQI}},
  note         = {Machine review of arXiv:2506.13053}
}
read the original abstract

Existing large-scale zero-shot text-to-speech (TTS) models deliver high speech quality but suffer from slow inference speeds due to massive parameters. To address this issue, this paper introduces ZipVoice, a high-quality flow-matching-based zero-shot TTS model with a compact model size and fast inference speed. Key designs include: 1) a Zipformer-based vector field estimator to maintain adequate modeling capabilities under constrained size; 2) Average upsampling-based initial speech-text alignment and Zipformer-based text encoder to improve speech intelligibility; 3) A flow distillation method to reduce sampling steps and eliminate the inference overhead associated with classifier-free guidance. Experiments on 100k hours multilingual datasets show that ZipVoice matches state-of-the-art models in speech quality, while being 3 times smaller and up to 30 times faster than a DiT-based flow-matching baseline. Codes, model checkpoints and demo samples are publicly available at https://github.com/k2-fsa/ZipVoice.

Figures

Figures reproduced from arXiv: 2506.13053 by the authors.

Figure 1
Figure 1. Illustration of ZipVoice training (left) and inference (right). [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models

    cs.SD 2026-07 conditional novelty 6.5 of 10

    Hierarchical multi-prompt representation generation plus generalized flow matching yields high-quality single-stage waveform diffusion from 12.5 Hz latents and efficient LDM TTS.

  2. FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesis

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A compact tokenizer-free non-autoregressive flow-matching DiT synthesizes Turkish speech in frozen AudioVAE2 latents at WER 8.0% / CER 3.0%, beating larger open cloners while running at RTF 0.11 on consumer GPUs.

  3. Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model

    eess.AS 2026-07 conditional novelty 5.0 of 10

    A lightweight face adapter plus soft-tuning aligns face embeddings to a frozen StyleTTS 2 style space, yielding natural zero-shot face-to-speech and language-agnostic transfer to Spanish.

  4. Universal Speech Content Factorization

    eess.AS 2026-03 conditional novelty 5.0 of 10

    A universal least-squares speech-to-content map plus few-second speaker transforms yields open-set, low-rank, timbre-suppressed features competitive for zero-shot VC and TTS.

Reference graph

Works this paper leans on

55 extracted references · 36 canonical work pages · cited by 4 Pith papers

  1. [1]

    Neural codec language models are zero-shot text to speech synthesizers,

    S. Chen, C. Wang, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li et al. , “Neural codec language models are zero-shot text to speech synthesizers,” IEEE Transactions on Audio, Speech and Language Processing, 2025

    work page 2025

  2. [2]

    V oicebox: Text-guided multilingual universal speech generation at scale,

    M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V . Manohar, Y . Adi, J. Mahadeokar et al. , “V oicebox: Text-guided multilingual universal speech generation at scale,” Advances in neural information processing systems , vol. 36, pp. 14 005–14 034, 2023

    2023

  3. [3]

    E2 tts: Embarrassingly easy fully non- autoregressive zero-shot tts,

    S. E. Eskimez, X. Wang, M. Thakker, C. Li, C.-H. Tsai, Z. Xiao, H. Yang, Z. Zhu, M. Tang, X. Tan et al. , “E2 tts: Embarrassingly easy fully non- autoregressive zero-shot tts,” in 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 682–689

    work page 2024

  4. [4]

    F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,

    Y . Chen, Z. Niu, Z. Ma, K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen, “F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,” arXiv preprint arXiv:2410.06885 , 2024

    arXiv 2024

  5. [5]

    MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,

    Y . Wang, H. Zhan, L. Liu, R. Zeng, H. Guo, J. Zheng, Q. Zhang, X. Zhang, S. Zhang, and Z. Wu, “MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,” in The Thirteenth International Conference on Learning Representations , 2025

    work page 2025

  6. [6]

    Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications,

    H.-H. Guo, Y . Hu, K. Liu, F.-Y . Shen, X. Tang, Y .-C. Wu, F.-L. Xie, K. Xie, and K.-T. Xu, “Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications,” arXiv preprint arXiv:2409.03283, 2024

    arXiv 2024

  7. [7]

    Seed-tts: A family of high-quality versatile speech generation models,

    P. Anastassiou, J. Chen, J. Chen, Y . Chen, Z. Chen, Z. Chen, J. Cong, L. Deng, C. Ding, L. Gao et al. , “Seed-tts: A family of high-quality versatile speech generation models,” arXiv preprint arXiv:2406.02430 , 2024

    arXiv 2024

  8. [8]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” in Proc. Interspeech 2019 , 2019, pp. 1526–1530

    work page 2019

Show all 55 references
  1. [9]

    Libriheavy: A 50,000 hours asr corpus with punctuation casing and context,

    W. Kang, X. Yang, Z. Yao, F. Kuang, Y . Yang, L. Guo, L. Lin, and D. Povey, “Libriheavy: A 50,000 hours asr corpus with punctuation casing and context,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 10 9...

    2024

  2. [10]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,

    H. He, Z. Shang, C. Wang, X. Li, Y . Gu, H. Hua, L. Liu, C. Yang, J. Li, P. Shi et al. , “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,” in 2024 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2024, pp. 885–890

    2024

  3. [11]

    Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,

    K. Shen, Z. Ju, X. Tan, E. Liu, Y . Leng, L. He, T. Qin, S. Zhao, and J. Bian, “Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,” in ICLR, 2024

    2024

  4. [12]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in The Eleventh International Conference on Learning Representations , 2023

    2023

  5. [13]

    Sf- speech: Straightened flow for zero-shot voice clone,

    X. Li, Z. Shang, H. Hua, P. Shi, C. Yang, L. Wang, and P. Zhang, “Sf- speech: Straightened flow for zero-shot voice clone,” IEEE Transactions on Audio, Speech and Language Processing , 2025

    2025

  6. [14]

    P-flow: A fast and data-efficient zero-shot tts through speech prompting,

    S. Kim, K. Shih, J. F. Santos, E. Bakhturina, M. Desta, R. Valle, S. Yoon, B. Catanzaro et al., “P-flow: A fast and data-efficient zero-shot tts through speech prompting,” Advances in Neural Information Processing Systems , vol. 36, pp. 74 213–74 228, 2023

    2023

  7. [15]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  8. [16]

    Zipformer: A faster and better encoder for automatic speech recognition,

    Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “Zipformer: A faster and better encoder for automatic speech recognition,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  9. [17]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applica- tions, 2021

    2021

  10. [18]

    Freeu: Free lunch in diffusion u-net,

    C. Si, Z. Huang, Y . Jiang, and Z. Liu, “Freeu: Free lunch in diffusion u-net,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 4733–4743

    2024

  11. [19]

    U-dits: Downsample tokens in u-shaped diffusion transformers,

    Y . Tian, Z. Tu, H. Chen, J. Hu, C. Xu, and Y . Wang, “U-dits: Downsample tokens in u-shaped diffusion transformers,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  12. [20]

    Fastspeech: Fast, robust and controllable text to speech,

    Y . Ren, Y . Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech: Fast, robust and controllable text to speech,” Advances in neural information processing systems , vol. 32, 2019

    2019

  13. [21]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu et al. , “Conformer: Convolution-augmented transformer for speech recognition,” in Proc. Interspeech 2020 , 2020, pp. 5036–5040

    2020

  14. [22]

    Glow-tts: A generative flow for text-to-speech via monotonic alignment search,

    J. Kim, S. Kim, J. Kong, and S. Yoon, “Glow-tts: A generative flow for text-to-speech via monotonic alignment search,” Advances in Neural Information Processing Systems , vol. 33, pp. 8067–8077, 2020

    2020

  15. [23]

    Flow-tts: A non-autoregressive network for text to speech based on flow,

    C. Miao, S. Liang, M. Chen, J. Ma, S. Wang, and J. Xiao, “Flow-tts: A non-autoregressive network for text to speech based on flow,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 7209–7213

    2020

  16. [24]

    Simple- speech: Towards simple and efficient text-to-speech with scalar latent transformer diffusion models,

    D. Yang, D. Wang, H. Guo, X. Chen, X. Wu, and H. Meng, “Simple- speech: Towards simple and efficient text-to-speech with scalar latent transformer diffusion models,” Proc. INTERSPEECH, 2024

    2024

  17. [25]

    DiTTo-TTS: Diffusion transformers for scalable text-to-speech without domain-specific factors,

    K. Lee, D. W. Kim, J. Kim, S. Chung, and J. Cho, “DiTTo-TTS: Diffusion transformers for scalable text-to-speech without domain-specific factors,” in The Thirteenth International Conference on Learning Representations , 2025

    2025

  18. [26]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders,

    S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, “Convnext v2: Co-designing and scaling convnets with masked autoencoders,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 16 133–16 142

    2023

  19. [27]

    On distillation of guided diffusion models,

    C. Meng, R. Rombach, R. Gao, D. Kingma, S. Ermon, J. Ho, and T. Sal- imans, “On distillation of guided diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 14 297–14 306

    2023

  20. [28]

    Tacotron: Towards end-to- end speech synthesis,

    Y . Wang, R. Skerry-Ryan, D. Stanton, Y . Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y . Xiao, Z. Chen, S. Bengio et al. , “Tacotron: Towards end-to- end speech synthesis,” in Proc. Interspeech 2017 , 2017, pp. 4006–4010

    2017

  21. [29]

    Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,

    J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y . Zhang, Y . Wang, R. Skerrv-Ryan et al. , “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in 2018 IEEE international conference on acoustics, speech and signal processin...

    2018

  22. [30]

    Revisiting over- smoothness in text to speech,

    Y . Ren, X. Tan, T. Qin, Z. Zhao, and T.-Y . Liu, “Revisiting over- smoothness in text to speech,” arXiv preprint arXiv:2202.13066 , 2022

    2022 arXiv

  23. [31]

    Grad- tts: A diffusion probabilistic model for text-to-speech,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad- tts: A diffusion probabilistic model for text-to-speech,” in International Conference on Machine Learning . PMLR, 2021, pp. 8599–8608

    2021

  24. [32]

    Matcha-tts: A fast tts architecture with conditional flow matching,

    S. Mehta, R. Tu, J. Beskow, É. Székely, and G. E. Henter, “Matcha-tts: A fast tts architecture with conditional flow matching,” in ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 341–11 345

    2024

  25. [33]

    Consistency models,

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” in International Conference on Machine Learning . PMLR, 2023, pp. 32 211–32 252

    2023

  26. [34]

    Flow straight and fast: Learning to generate and transfer data with rectified flow,

    X. Liu, C. Gong et al. , “Flow straight and fast: Learning to generate and transfer data with rectified flow,” in The Eleventh International Conference on Learning Representations , 2023

    2023

  27. [35]

    Comospeech: One-step speech and singing voice synthesis via consistency model,

    Z. Ye, W. Xue, X. Tan, J. Chen, Q. Liu, and Y . Guo, “Comospeech: One-step speech and singing voice synthesis via consistency model,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 1831–1839

    2023

  28. [36]

    Reflow- tts: A rectified flow model for high-fidelity text-to-speech,

    W. Guan, Q. Su, H. Zhou, S. Miao, X. Xie, L. Li, and Q. Hong, “Reflow- tts: A rectified flow model for high-fidelity text-to-speech,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 10 501–10 505

    2024

  29. [37]

    V oiceflow: Efficient text- to-speech with rectified flow matching,

    Y . Guo, C. Du, Z. Ma, X. Chen, and K. Yu, “V oiceflow: Efficient text- to-speech with rectified flow matching,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 121–11 125

    2024

  30. [38]

    Flashspeech: Efficient zero-shot speech synthesis,

    Z. Ye, Z. Ju, H. Liu, X. Tan, J. Chen, Y . Lu, P. Sun, J. Pan, W. Bian, S. He et al. , “Flashspeech: Efficient zero-shot speech synthesis,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 6998–7007

    2024

  31. [39]

    Slimspeech: Lightweight and efficient text-to-speech with slim rectified flow,

    K. Wang, W. Guan, S. Lu, J. Yao, L. Li, and Q. Hong, “Slimspeech: Lightweight and efficient text-to-speech with slim rectified flow,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025, pp. 1–5

    2025

  32. [40]

    Lightspeech: Lightweight and fast text to speech with neural architecture search,

    R. Luo, X. Tan, R. Wang, T. Qin, J. Li, S. Zhao, E. Chen, and T.-Y . Liu, “Lightspeech: Lightweight and fast text to speech with neural architecture search,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp....

    2021

  33. [41]

    Librispeech-pc: Benchmark for evaluation of punctuation and capitalization capabilities of end-to-end asr models,

    A. Meister, M. Novikov, N. Karpov, E. Bakhturina, V . Lavrukhin, and B. Ginsburg, “Librispeech-pc: Benchmark for evaluation of punctuation and capitalization capabilities of end-to-end asr models,” in 2023 IEEE automatic speech recognition and understanding workshop (ASRU) . I...

    2023

  34. [42]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference , 2020, pp. 4218–4222

    2020

  35. [43]

    Didispeech: A large scale mandarin speech corpus,

    T. Guo, C. Wen, D. Jiang, N. Luo, R. Zhang, S. Zhao, W. Li, C. Gong, W. Zou, K. Hanet al., “Didispeech: A large scale mandarin speech corpus,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 6968–6972

    2021

  36. [44]

    V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,

    H. Siuzdak, “V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,” in The Twelfth International Conference on Learning Representations , 2024

    2024

  37. [45]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

  38. [46]

    Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition,

    Z. Gao, S. Zhang, I. McLoughlin, and Z. Yan, “Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition,” in Proc. Interspeech 2022 , 2022, pp. 2063–2067

    2022

  39. [47]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  40. [48]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  41. [49]

    Ecapa-tdnn: Em- phasized channel attention, propagation and aggregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Em- phasized channel attention, propagation and aggregation in tdnn based speaker verification,” in Proc. Interspeech 2020 , 2020, pp. 3830–3834

    2020

  42. [50]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “Utmos: Utokyo-sarulab system for voicemos challenge 2022,” Interspeech 2022 , 2022

    2022

  43. [51]

    Cosyvoice: A scalable multilingual zero-shot text-to- speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Maet al., “Cosyvoice: A scalable multilingual zero-shot text-to- speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  44. [52]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models,

    Z. Du, Y . Wang, Q. Chen, X. Shi, X. Lv, T. Zhao, Z. Gao, Y . Yang, C. Gao, H. Wang et al. , “Cosyvoice 2: Scalable streaming speech synthesis with large language models,” arXiv preprint arXiv:2412.10117 , 2024

    2024 arXiv

  45. [53]

    Spark-tts: An efficient llm-based text-to- speech model with single-stream decoupled speech tokens,

    X. Wang, M. Jiang, Z. Ma, Z. Zhang, S. Liu, L. Li, Z. Liang, Q. Zheng, R. Wang, X. Feng et al. , “Spark-tts: An efficient llm-based text-to- speech model with single-stream decoupled speech tokens,”arXiv preprint arXiv:2503.01710, 2025

    2025 arXiv

  46. [54]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,

    E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. Gölge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,” in International conference on machine learning. PMLR, 2022, pp. 2709–2720

    2022

  47. [55]

    Amphion: an open-source audio, music, and speech generation toolkit,

    X. Zhang, L. Xue, Y . Gu, Y . Wang, J. Li, H. He, C. Wang, S. Liu, X. Chen, J. Zhang et al. , “Amphion: an open-source audio, music, and speech generation toolkit,” in 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 879–884

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.