Pith. sign in

REVIEW 4 major objections 5 minor 53 references

OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A learned prior lets zero-shot TTS finish in one sampling step, with word error rate near 0.05.

desk verdict OZSpeech makes a plausible practical case for one-step zero-shot TTS with a learned prior, but an undefined tau in the flow-matching derivation and missing key baselines keep it from being reproducible as written. read the letter →

arxiv 2505.12800 v1 pith:YVHIMSIO submitted 2025-05-19 cs.SD cs.AIeess.AS

Hieu-Nghia Huynh-Nguyen , Ngoc Son Nguyen , Huynh Nguyen Dang , Thieu Vo , Truong-Son Hy , Van Nguyen This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords zero-shottext-to-speechflowmatchingone-stepsamplinglearnedprioroptimaltransportfactorizedneuralcodecFACodecspeechsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that text-to-speech can start not from Gaussian noise but from a learned prior code that already carries the content, and then reach the target speech in a single flow-matching step. It argues that conditioning the flow on this learned prior removes the need for multi-step diffusion sampling, and the result is a zero-shot voice-cloning system whose word error rate stays near 0.05 even as audio prompts get longer or noisier. A careful reader should care because this is the efficiency claim that matters for real-time TTS: the model is 2.7 to 6.5 times faster at inference and only 29% to 71% the size of competing systems, trained on only 500 hours of speech.

What carries the argument

The load-bearing object is the learned prior code $x_{\mathrm{pr}}$: six quantizer sequences from FACodec that decompose speech into prosody, content, and acoustic detail, predicted by a cascaded feed-forward transformer conditioned on phoneme embeddings. Around this the paper builds three mechanisms: the Prior Codes Generator $f_\psi$ with a duration predictor, the one-step OT-CFM objective $\mathcal{L}_{\mathrm{CFM}}(\theta)=\mathbb{E}_{x_{\mathrm{pr}},x_1}\left\|v_\theta(x_{\mathrm{pr}},\tau)-\frac{x_1-x_{\mathrm{pr}}}{1-\tau}\right\|^2$ in which the interpolation time $t$ is replaced by the unspecified prior-dependent $\tau$, and the Anchor Loss that prevents embedding collapse and estimates $\tilde{x}_1=x_{\mathrm{pr}}+(1-\tau)v_\theta$. A folding operation and quantizer embeddings let all six code sequences be modeled in one Transformer pass instead of sequentially.

What would settle it

Retrain the model with $\tau$ fixed to a few concrete values, such as 0, 0.5, and 0.9, and report whether one-step sampling still gives WER near 0.05 on the LibriSpeech test-clean evaluation set. If no fixed $\tau$ reproduces the reported results, or if the model only works when $\tau$ is chosen per sample in a way the paper does not specify, the central reformulation is not reproducible as stated.

Watch

Extended reading notes

Core claim

The central discovery is a reformulation of optimal transport conditional flow matching in which the source distribution is not the standard Gaussian $x_0$ but a learned prior $x_{\mathrm{pr}}$ produced by a cascaded code generator aligned to phonemes. The vector field estimator $v_\theta$ then regresses the velocity between $x_{\mathrm{pr}}$ and $x_1$, with the usual interpolation time replaced by an unknown prior-dependent time variable $\tau$. The authors report that one evaluation of this field is enough to synthesize intelligible speech, and they measure a consistent word error rate of about 0.05 across 1s, 3s, and 5s prompts and across prompt signal-to-noise ratios from clean down to 0 dB, with multi-fold WER improvements over F5-TTS, VoiceCraft, NaturalSpeech 2, and VALL-E at the cost of somewhat lower naturalness scores.

Load-bearing premise

The whole method rests on a number $\tau$ that the paper never defines concretely: it stands in for how far the learned prior is from real speech, but the paper does not say what value is used when training or when generating in one step.

Editorial extensions

If this is right

  • At inference, one evaluation of the vector field replaces the 32 to 200 function evaluations used by F5-TTS and NaturalSpeech 2, giving a real-time factor of 0.26 on an A100 GPU.
  • The model keeps WER near 0.05 for 1s, 3s, and 5s prompts, while baseline WERs rise with prompt length, so content accuracy no longer depends on prompt duration.
  • With noisy prompts at SNR 0 to 12 dB, OZSpeech's WER stays near 0.05 without fine-tuning; fine-tuning on noisy prompts further lifts naturalness and prosody metrics without hurting WER.
  • Training on arbitrary prompt segments instead of first segments improves all metrics, indicating that hiding the prompt position is important for attribute transfer.
  • A 100M-parameter variant performs almost as well as the 145M base model, so the efficiency gain is not tied to the larger configuration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the learned prior is doing most of the generation work, the flow-matching component may be closer to a residual correction than a generative model; a clean ablation that replaces $v_\theta$ with a simple predictor would reveal how much of the one-step success comes from the prior rather than the flow.
  • The unspecified $\tau$ suggests a testable extension: infer $\tau$ from $(x_{\mathrm{pr}}, x_1)$ with a small regressor or schedule it during training; if any fixed $\tau$ works as well as the reported results, the prior-dependent-time claim would be strengthened.
  • The noise-tolerance result points to a broader use: the same conditioning scheme could be applied to speech enhancement or voice conversion, where the prompt is noisy by construction and the content codes are masked.
  • Because WER stays flat while other metrics degrade with prompt length, the disentangled codec is likely isolating content from prosody; this separation could be verified by probing whether content codes are invariant to prompt noise.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes OZSpeech, a zero-shot TTS system that combines a learned prior code generator with a one-step optimal-transport conditional flow matching (OT-CFM) module. Speech is decomposed with FACodec into six quantizer sequences (prosody, content, acoustic details) plus a timbre vector; the Prior Codes Generator produces a phoneme-conditioned starting point, and a transformer-based Vector Field Estimator predicts the residual velocity toward the target codes in a single sampling step. The authors report experiments on LibriTTS/LibriSpeech comparing with F5-TTS, VoiceCraft, NaturalSpeech 2, and VALL-E, plus ablations on prompt selection and noise robustness. The main claimed advantages are a multi-fold WER reduction, stable WER across prompt lengths and noise levels, a 2.7-6.5x faster inference speed, and a 29%-71% model-size reduction relative to baselines.

Significance. If the central reformulation is valid, the paper makes a useful practical contribution: it demonstrates that a learned prior derived from a codec can serve as the initialization for a one-step flow-matching TTS, with a favorable efficiency/size trade-off and a noise-robustness analysis that few prior TTS papers provide. The empirical results, especially the consistently low WER and the controlled noise experiment, are potentially impactful for deployment-oriented zero-shot TTS. However, the significance is currently bounded by two gaps: the one-step flow-matching derivation rests on an underspecified quantity tau, and the evaluation omits the strongest published baseline (NaturalSpeech 3), which the paper itself discusses as a canonical factorized-codec system.

major comments (4)
  1. [Section 3.3, Eq. (3) and Eq. (13)] The reformulation of the OT-CFM loss is not well-defined as written. The original loss (Eq. (2)) couples the state xt and the interpolation parameter t through the linear path xt = t*x1 + (1-t)*x0; replacing (xt, t) with (xpr, tau) is only valid if xpr lies on the corresponding linear path for the same tau used in the denominator. The paper never states how xpr relates to such a path, how tau is sampled during training, or what value of tau is used at inference in the update x1_hat = xpr + (1-tau) * F^{-1}(v_theta(...)). The statement that the model "implicitly learns the optimal t for each prior" is not a mechanism, since tau is a conditioning input to the network and is not produced as a network output or estimated by an auxiliary module. The training objective and the one-step sampling claim therefore rest on an unstated premise; please specify the training distribution of tau and the inference-time choice, or reformulate the loss so that the target does not depend on an unobserved interpolation parameter.
  2. [Section 4.1 and Table 1] The baseline set omits NaturalSpeech 3 (Ju et al., 2024b), which is cited in the paper as the source of FACodec and which the text identifies as a state-of-the-art zero-shot TTS system built on the same factorized codec representation. Without this comparison, the central claim of "multi-fold WER improvement" and the implied SOTA claim are not supported. Please add the published NaturalSpeech 3 numbers (or a reproduction under the same evaluation protocol) to Tables 1 and 2, or explicitly justify its exclusion.
  3. [Tables 1, 3, 4 and Section F] No variance or confidence-interval information is reported for any metric, which matters especially for the WER claims. The paper reports identical WER values (0.05) across 1s/3s/5s prompts and across all noise levels (0.05 or 0.06), which is surprising for an ASR-based metric on a test set of 2.2 hours; such stability could arise from a degenerate ASR output (e.g., empty hypotheses or a systematic mis-transcription) rather than robust synthesis. Please report per-utterance score distributions (or at least standard errors and the number of test utterances), and include a worked example or failure analysis that confirms the low WER is not a scoring artifact. The boxplots in Figure 2 are a step in this direction, but they do not report N or the fraction of utterances contributing to the near-zero WER box.
  4. [Section 3.3, 'One-Step Optimal Transport Flow Matching Formulation' and Appendix C] The joint training of the Prior Codes Generator and the Vector Field Estimator is described, but the interaction between the prior and the flow objective is not specified precisely. In particular, Eq. (3) treats xpr as a random variable, yet Appendix C states that the two components are "jointly trained" with a total loss Ltotal = Lprior + Ldur + LCFM + Lanchor. If xpr is generated by f_psi during training and gradients flow back through the prior into LCFM, the distribution of xpr is being shaped by the same objective that later depends on tau; if instead xpr is detached, the loss reduces to a regression toward x1 with a bias determined by the prior. The paper does not state which of these regimes is used; please clarify the gradient flow (stop-gradient or not) and explain how the claimed "learned prior" behavior is achieved without a separate distillation stage beyond what is already in Lanchor.
minor comments (5)
  1. [Section 4.1, abbreviations] The table footnote lists '[star] and [clubs] mean results inferred from official and ufficial checkpoints' — 'ufficial' is a typo and should be 'official' or 'unofficial', depending on intent.
  2. [Table 4, SNR=0dB row for VALL-E] The F0 RMSE value is listed as '102,68' (with a comma); this should be '102.68' for consistency with the decimal-point convention used elsewhere in the table.
  3. [Appendix A.2, Eq. (12) and Eq. (2)] The paper presents two versions of the conditional flow matching loss (Eq. (12) with the target x1 - x0, and Eq. (2) with the target (x1 - xt)/(1-t)). Please verify the equivalence and consistency of the notation across these equations, since Eq. (2) as written appears to omit the factor that maps the conditional velocity to the unconditional target.
  4. [Section 4.2, retrained F5-TTS discussion] The sentence describing the retrained F5-TTS reports a WER exceeding 0.95 and excludes the checkpoint from Table 1. This is an important negative result; please provide more detail in an appendix (training steps, data, and the specific failure mode), because as written it is a single unreproduced observation that could be affected by hyperparameter choices.
  5. [Appendix E, Table 5] The claim that the Small model 'outperforms the Base model by 13.6% in F0 RMSE for 1s' needs a unit or direction reference: the table reports lower RMSE as better, so please make explicit that relative improvement means a decrease in RMSE, and similarly for the other directional statements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central derivation is self-contained; the undefined tau in Eq. (3) is an incomplete specification, not a circular reduction.

full rationale

OZSpeech is trained with supervised losses against external FACodec targets and evaluated on held-out LibriSpeech test-clean data. The learned prior is a normal modeling choice: Lprior supervises the prior generator to predict ground-truth codes, and LCFM supervises the vector field to correct the residual between the prior and target. This makes the residual smaller by construction, but that is a design objective, not a logical circularity. Eq. (3) replaces the tied pair (xt, t) with the unobserved (xpr, tau) and does not state how tau is sampled or set at inference; this is an under-specified training/inference scheme and a correctness/reproducibility concern, not a case where the output is equivalent to the input by definition. No load-bearing self-citation, imported uniqueness theorem, or renamed known result was found.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the FACodec factorization, the OT-CFM background, and two paper-specific assumptions: the near-target prior and the well-definedness of tau. No new physical or conceptual entities are introduced.

free parameters (1)
  • tau (interpolation parameter)
    Eq. (3) uses tau in the denominator of the flow matching target, but the paper does not specify how tau is obtained during training or sampling; it is effectively a free variable in the loss.
assumptions (4)
  • domain assumption FACodec provides a faithful disentangled representation of speech into content, prosody, acoustic detail, and timbre streams.
    The method relies on FACodec (Ju et al., 2024) for factorization; if the factorization is imperfect, masking content codes may leak content or fail to transfer style. Invoked in Section 3.1 and Appendix A.1.
  • standard math Optimal transport conditional flow matching with linear paths is a valid generative objective.
    The OT-CFM linear path and conditional flow matching losses from Lipman et al. (2023) and Liu et al. (2023) are used as the starting point for Eq. (2).
  • ad hoc to paper The learned prior distribution from the Prior Codes Generator is close enough to the target distribution for one-step residual regression.
    The empirical premise for one-step sampling is that xpr is a good starting point; if the prior is not close, the single-step correction cannot recover the target. Stated in Section 3.2 as reducing the distance between xpr and x1.
  • ad hoc to paper The interpolation parameter tau is a well-defined, learnable or samplable quantity.
    The reformulated loss treats t as an unknown prior-dependent time variable tau, but never defines its distribution, training objective, or inference value; the tractability of Eq. (3) depends on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching." pith.science (2026). https://pith.science/paper/YVHIMSIO

@misc{pith2026250512800,
  author       = {Pith},
  title        = {Pith review of: OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YVHIMSIO}},
  note         = {Machine review of arXiv:2505.12800}
}
read the original abstract

Text-to-speech (TTS) systems have seen significant advancements in recent years, driven by improvements in deep learning and neural network architectures. Viewing the output speech as a data distribution, previous approaches often employ traditional speech representations, such as waveforms or spectrograms, within the Flow Matching framework. However, these methods have limitations, including overlooking various speech attributes and incurring high computational costs due to additional constraints introduced during training. To address these challenges, we introduce OZSpeech, the first TTS method to explore optimal transport conditional flow matching with one-step sampling and a learned prior as the condition, effectively disregarding preceding states and reducing the number of sampling steps. Our approach operates on disentangled, factorized components of speech in token format, enabling accurate modeling of each speech attribute, which enhances the TTS system's ability to precisely clone the prompt speech. Experimental results show that our method achieves promising performance over existing methods in content accuracy, naturalness, prosody generation, and speaker style preservation. Audio samples are available at our demo page https://ozspeech.github.io/OZSpeech_Web/.

Figures

Figures reproduced from arXiv: 2505.12800 by the authors.

Figure 1
Figure 1. Overview of OZSpeech: (a) The overall architecture: The text prompt is converted to phonemes and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Boxplots showing the distributions of performance metrics (WER, UTMOS, and SIM-O) on the Lib [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 19 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  4. [4]

    Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Zhao You, and Zhiyong Yan. 2021. https://doi.org/10.21437/Interspeech.2021-1965 Gigaspeech: An evolving, multi-domain...

    doi:10.21437/interspeech.2021-1965 2021

  5. [6]

    Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. 2024 b . Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370

    arXiv 2024

  6. [7]

    Sanyuan Chen, Chengyi Wang, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2025. https://doi.org/10.1109/TASLPRO.2025.3530270 Neural codec language models are zero-shot text to speech synthesizers . IEEE Transactions on Audio, Speech and Language Processing, pages 1--15

    arXiv 2025

  7. [8]

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024 c . https://arxiv.org/abs/2410.06885 F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching . Preprint, arXiv:2410.06885

    arXiv 2024

  8. [9]

    David Dean, Sridha Sridharan, Robert Vogt, and Michael Mason. 2010. The qut-noise-timit corpus for evaluation of voice activity detection algorithms. In Proceedings of the 11th annual conference of the international speech communication association, pages 3110--3113. International Speech Communication Association

    work page 2010

Show all 53 references
  1. [10]

    Alexandre D \'e fossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2023. https://openreview.net/forum?id=ivCd8z8zR2 High fidelity neural audio compression . Transactions on Machine Learning Research. Featured Certification, Reproducibility Certification

    2023

  2. [11]

    Chenpeng Du, Yiwei Guo, Feiyu Shen, Zhijun Liu, Zheng Liang, Xie Chen, Shuai Wang, Hui Zhang, and Kai Yu. 2024 a . https://doi.org/10.1609/aaai.v38i16.29747 Unicats: A unified context-aware text-to-speech framework with contextual vq-diffusion and vocoding . Proceedings of the...

    2024 doi

  3. [12]

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. 2024 b . Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407

    2024 arXiv

  4. [13]

    Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, Yanqing Liu, Sheng Zhao, and Naoyuki Kanda. 2024. https://arxiv.org/abs/2406.18009 E2 TTS : Embarrassingly easy fully non-autoregressive zero-sho...

    2024 arXiv

  5. [14]

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. 2024. https://arxiv.org/abs/2410.12557 One step diffusion via shortcut models . Preprint, arXiv:2410.12557

    2024 arXiv

  6. [15]

    Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. 2024. https://doi.org/10.18653/v1/2024.naacl-long.261 Empowering diffusion models on the embedding space for text generation . In Proceedings of the 2024 Conference of the North American Chapt...

    2024 doi

  7. [16]

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. 2023. https://openreview.net/forum?id=jQj-_rLVXsj Diffuseq: Sequence to sequence text generation with diffusion models . In The Eleventh International Conference on Learning Representations

    2023

  8. [17]

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. https://doi.org/10.21437/Interspeech.2020-3015 Conformer: Convolution-augmented transformer for speech recognition . In Inters...

    2020 doi

  9. [18]

    Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xu Tan. 2022. https://arxiv.org/abs/2211.12171 Prompttts: Controllable text-to-speech with text descriptions . Preprint, arXiv:2211.12171

    2022 arXiv

  10. [19]

    Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. 2024. Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment. arXiv preprint arXiv:2406.07855

    2024 arXiv

  11. [20]

    Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generat...

    2024

  12. [21]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf Denoising diffusion probabilistic models . In Advances in Neural Information Processing Systems, volume 33, pages 6840--6851. ...

    2020

  13. [22]

    Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Bj \"o rn Ommer, and Cees Snoek. 2024. https://aclanthology.org/2024.eacl-short.33 Flow matching for conditional text generation in a few sampling steps . In Proceedings of the 18th Conference of the European Chapt...

    2024

  14. [23]

    Shengpeng Ji, Ziyue Jiang, Hanting Wang, Jialong Zuo, and Zhou Zhao. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.733 M obile S peech: A fast and high-fidelity framework for mobile zero-shot text-to-speech . In Proceedings of the 62nd Annual Meeting of the Association fo...

    2024 doi

  15. [25]

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Eric Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiangyang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, and sheng zhao. 2024 a . https://openreview.net/forum?id=dVhrnjZJad...

    2024

  16. [26]

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Eric Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiangyang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, and Sheng Zhao. 2024 b . https://proceedings.mlr.press/v235/ju24b.h...

    2024

  17. [27]

    Minki Kang, Wooseok Han, Sung Ju Hwang, and Eunho Yang. 2023. https://doi.org/10.21437/Interspeech.2023-754 Zet-speech: Zero-shot adaptive emotion-controllable text-to-speech synthesis with diffusion and style-based models . In Interspeech 2023, pages 4339--4343

    2023 doi

  18. [28]

    Eugene Kharitonov, Damien Vincent, Zal \'a n Borsos, Rapha \"e l Marinier, Sertan Girgin, Olivier Pietquin, Matt Sharifi, Marco Tagliasacchi, and Neil Zeghidour. 2023. https://doi.org/10.1162/tacl_a_00618 Speak, read and prompt: High-fidelity text-to-speech with minimal superv...

    2023 doi

  19. [29]

    Jaehyeon Kim, Sungwon Kim, Jungil Kong, and Sungroh Yoon. 2020. https://api.semanticscholar.org/CorpusID:218862956 Glow-tts: A generative flow for text-to-speech via monotonic alignment search . ArXiv, abs/2005.11129

    2020 arXiv

  20. [30]

    Shih, Rohan Badlani, Joao Felipe Santos, Evelina Bakhturina, Mikyas T

    Sungwon Kim, Kevin J. Shih, Rohan Badlani, Joao Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, and Bryan Catanzaro. 2023. https://openreview.net/forum?id=zNA7u7wtIN P-flow: A fast and data-efficient zero-shot TTS through speech prompting . In T...

    2023

  21. [31]

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. https://openreview.net/forum?id=gzCS252hCO Voicebox: Text-guided multilingual universal speech generation at scale . I...

    2023

  22. [32]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. 2023. https://openreview.net/forum?id=PqvMRDCJT9t Flow matching for generative modeling . In The Eleventh International Conference on Learning Representations

    2023

  23. [33]

    Xingchao Liu, Chengyue Gong, and qiang liu. 2023. https://openreview.net/forum?id=XVjTT1nw5z Flow straight and fast: Learning to generate and transfer data with rectified flow . In The Eleventh International Conference on Learning Representations

    2023

  24. [34]

    Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. https://doi.org/10.21437/Interspeech.2017-1386 Montreal forced aligner: Trainable text-speech alignment using kaldi . In Interspeech 2017, pages 498--502

    2017 doi

  25. [35]

    Shivam Mehta, Ruibo Tu, Jonas Beskow, \'Eva Sz\'ekely, and Gustav Eje Henter. 2024. https://doi.org/10.1109/ICASSP48485.2024.10448291 Matcha-tts: A fast tts architecture with conditional flow matching . In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech a...

    2024

  26. [36]

    Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, and Furu Wei. 2024. https://doi.org/10.48550/arXiv.2407.08551 Autoregressive speech synthesis without vector quantization . CoRR, abs/2407.08551

    2024 doi arXiv

  27. [37]

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. https://doi.org/10.1109/ICASSP.2015.7178964 Librispeech: An asr corpus based on public domain audio books . In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), page...

    2015

  28. [38]

    Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, and David Harwath. 2024. https://doi.org/10.18653/v1/2024.acl-long.673 V oice C raft: Zero-shot speech editing and text-to-speech in the wild . In Proceedings of the 62nd Annual Meeting of the Association for Comput...

    2024 doi

  29. [39]

    Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019 a . https://proceedings.neurips.cc/paper_files/paper/2019/file/f63f65b503e22cb970527f23c9ad7db1-Paper.pdf Fastspeech: Fast, robust and controllable text to speech . In Advances in Neural Inform...

    2019

  30. [40]

    Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019 b . FastSpeech: fast, robust and controllable text to speech. Curran Associates Inc., Red Hook, NY, USA

    2019

  31. [41]

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. https://arxiv.org/abs/2204.02152 Utmos: Utokyo-sarulab system for voicemos challenge 2022 . Preprint, arXiv:2204.02152

    2022 arXiv

  32. [42]

    Kai Shen, Zeqian Ju, Xu Tan, Eric Liu, Yichong Leng, Lei He, Tao Qin, sheng zhao, and Jiang Bian. 2024. https://openreview.net/forum?id=Rc7dAwVL3v Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers . In The Twelfth International ...

    2024

  33. [43]

    Hubert Siuzdak. 2024. https://arxiv.org/abs/2306.00814 Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis . Preprint, arXiv:2306.00814

    2024 arXiv

  34. [44]

    Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. 2024. https://arxiv.org/abs/2401.07333 Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering . Preprint, arXiv:2401.07333

    2024 arXiv

  35. [45]

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. https://proceedings.mlr.press/v202/song23a.html Consistency models . In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 32211-...

    2023

  36. [46]

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021. https://openreview.net/forum?id=PxTIG12RRHS Score-based generative modeling through stochastic differential equations . In International Conference on Learning Representations

    2021

  37. [47]

    Chung Tran, Chi Mai Luong, and Sakriani Sakti. 2023. https://doi.org/10.21437/Interspeech.2023-2243 Sten-tts: Improving zero-shot cross-lingual transfer for multi-lingual tts with style-enhanced normalization diffusion framework . In Interspeech 2023, pages 4464--4468

    2023 doi

  38. [48]

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural discrete representation learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 6309–6318, Red Hook, NY, USA. Curran Associates Inc

    2017

  39. [49]

    Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, Chris Summers, ...

    2023 arXiv

  40. [50]

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111

    2023 arXiv

  41. [51]

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2022. https://doi.org/10.1109/TASLP.2021.3129994 Soundstream: An end-to-end neural audio codec . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495--507

    2022

  42. [52]

    Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu

    Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. https://doi.org/10.21437/Interspeech.2019-2441 Libritts: A corpus derived from librispeech for text-to-speech . In Interspeech 2019, pages 1526--1530

    2019 doi

  43. [53]

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. 2024 a . https://openreview.net/forum?id=AF9Q8Vip84 Speechtokenizer: Unified speech tokenizer for speech language models . In The Twelfth International Conference on Learning Representations

    2024

  44. [54]

    Xueyao Zhang, Liumeng Xue, Yicheng Gu, Yuancheng Wang, Jiaqi Li, Haorui He, Chaoren Wang, Ting Song, Xi Chen, Zihao Fang, Haopeng Chen, Junan Zhang, Tze Ying Tang, Lexiao Zou, Mingxuan Wang, Jun Han, Kai Chen, Haizhou Li, and Zhizheng Wu. 2024 b . Amphion: An open-source audio...

    2024

  45. [55]

    Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. arXiv preprint arXiv:2303.03926

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.