Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting

Changhun Kim; Eunho Yang; Minki Kang; Wooseok Han

REVIEW 3 major objections 5 minor 25 references

Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Clean prosody from pre-training data keeps cloned voices clear.

desk verdict A useful incremental TTS paper with a strong WER result, but the ablation overstates the role of prior prosody prompting. read the letter →

arxiv 2412.20155 v1 pith:W6RFKARU submitted 2024-12-28 cs.SD cs.AIeess.AS

Wooseok Han , Minki Kang , Changhun Kim , Eunho Yang This is my paper · ORCID

classification cs.SDcs.AIeess.AS

keywords speaker-adaptiveTTSvoicecloningprosodypromptingdiffusionmodelprior-preservationlosslow-resourcenoisyspeechrobustness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to make speaker-adaptive text-to-speech reliable when only a few seconds of target speech are available, especially when that speech is noisy. It proposes Stable-TTS, which uses clean prior samples from the pre-training data as prosody prompts, keeping rhythm and intonation stable, while fine-tuning a diffusion model with a prior-preservation loss so the model does not forget how to produce clean audio. The central claim is that this combination keeps word error rates below 1% even with a single target sample and cuts word errors by roughly 90% on noisy recordings compared with two recent baselines, while roughly preserving speaker similarity.

What carries the argument

The two load-bearing components are the prosody language model, an autoregressive model that predicts discrete phoneme-level prosody codes given a prompt speech sample, and the prior-preservation loss, a mean-squared-error term between the noise estimates of the frozen pre-trained diffusion model and the fine-tuned one computed on clean prior samples. The prior samples—a small hand-picked subset of the clean pre-training corpus, one male and one female voice—supply the prompt that drives the prosody language model and the anchor that the fine-tuned model must not drift away from.

What would settle it

A concrete test would be to take one of the two prior samples and synthesize a set of emotionally charged or prosodically varied sentences across many target speakers, then measure listening naturalness and WER; if the fixed prior sample makes all outputs sound like the prior speaker's intonation—for example, flat newsreader prosody for questions or exclamations—the central claim of prosody consistency fails. More directly, sampling many different prior samples and showing that WER and MOS swing widely would falsify the claim that a carefully chosen prior is sufficient.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that prosody—the part of speech that carries rhythm, stress, and intonation—can be decoupled from the target speaker's timbre and sourced from clean pre-training data, making speaker adaptation stable under conditions that break previous methods. Stable-TTS's prosody language model predicts discrete prosody codes from a clean prior sample rather than from the noisy target audio, and a prior-preservation loss anchors the fine-tuned diffusion model to the pre-trained distribution. The paper reports that this yields word error rates below 1% with as few as one target sample and around 90% relative WER reduction on the noisy VoxCeleb set compared with Grad-StyleSpeech and UnitSpeech, with speaker similarity largely intact.

Load-bearing premise

The whole approach rests on the assumption that a single fixed clean prosody sample per gender can supply prosody that transfers to any input text and any target speaker without sounding mismatched.

Editorial extensions

If this is right

On clean test sets, Stable-TTS reports word error rates below 1% even when fine-tuned on a single target sample, whereas a baseline's WER rises to nearly 49% in the same setting.
On the noisy VoxCeleb set, the reported WER is roughly 92% lower than that of the stronger baseline and 93% lower than that of the weaker baseline, while speaker similarity scores stay within about two to three points.
Fine-tuning remains stable across step counts, with 500 steps described as the sweet spot, so practitioners do not need careful early stopping.
The model works in a zero-shot setting as well as after fine-tuning, meaning the same weights can be deployed before any target-speaker data arrives.
Removing either the prosody language model or the prior-preservation loss degrades MOS, SMOS, and WER in every setting tested, so both components are presented as necessary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: if the prior-prosody prompt genuinely generalizes, the same framework could be applied to cross-lingual speaker adaptation, where a prior sample's language provides prosody templates for new languages—something the paper does not test.
Editorial extension: the reliance on a single hand-picked prior per gender suggests a stress test: vary the prior sample across speakers, emotions, and speaking rates and measure WER and speaker-similarity variance; the paper's fixed choice leaves this sensitivity unquantified.
Editorial extension: the prior-preservation loss is a direct analogue of a regularization technique from image generation, so one could expect it to also help other fine-tuning scenarios in audio, such as singing-voice synthesis or accent adaptation, though those domains are not demonstrated here.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Take: This is a competent, incremental speaker-adaptive TTS paper whose headline result—large WER reductions under limited and noisy samples—looks real, but whose central causal claim about prior prosody prompting is not backed by its own ablation table.

What's new: the paper combines two known ideas in a new place: using clean pre-training samples as the prompt for the prosody language model (prior prosody prompting) and a DreamBooth-style prior-preservation loss during fine-tuning. That is a legitimate extension of Mega-TTS and DreamBooth, and it is the first application I know of to TTS fine-tuning. The full system beats Grad-StyleSpeech and UnitSpeech by a large margin on WER on all three datasets—roughly an 80–90% reduction—while keeping speaker similarity about level or better. The limited-data scaling experiment with 1, 5, 20, and 100 target samples is a nice addition; WER stays under 1% while Grad-StyleSpeech degrades badly.

Where the paper stumbles: the ablation in Table I directly contradicts the claim in Section III-C that removing any component degrades MOS, SMOS, and WER across all scenarios. Removing the prior prompt actually gives lower WER than the full model on LibriTTS (0.83 vs. 1.13) and VoxCeleb (1.02 vs. 1.32), and higher MOS on LibriTTS (3.65 vs. 3.37). Only SMOS improves consistently with the prompt. So the prior prosody prompting is not what drives the headline robustness result; the prior-preservation loss likely does most of the work. And the prompt is a single hand-picked utterance per gender, with no criteria and no sensitivity analysis—if that utterance changes, we have no idea what happens. No significance tests and only two baselines are also weaknesses. No code is released, which makes it harder to check the claims.

These are serious but fixable issues. The paper still shows a working system, and the prior-preservation loss transfer is a useful contribution. A competent referee could push the authors to vary the prior sample, report significance, and rewrite the ablation section to match their own numbers. That would make the paper much stronger. The current version overstates the role of the prior prompt and needs revision, but it deserves referee time.

Referee Report

3 major / 5 minor

Summary. The paper proposes Stable-TTS, a speaker-adaptive text-to-speech framework that combines a prosody language model (PLM) with a prosody encoder, a timbre encoder, and a prior-preservation loss. The key idea is to use clean 'prior samples' from the pre-training corpus as the prosody prompt during inference and to add a prior-preservation loss during fine-tuning to prevent overfitting to noisy or limited target-speaker samples. The authors evaluate Stable-TTS on LibriTTS, VCTK, and VoxCeleb, reporting MOS, SMOS, WER, and SECS, and find large WER reductions relative to Grad-StyleSpeech and UnitSpeech while roughly preserving speaker similarity. They also report a fine-tuning data-scale experiment showing stable WER below 1% even with one target sample.

Significance. If the claimed results hold, Stable-TTS addresses a practically important problem: robust speaker adaptation under limited and noisy target speech. The reported WER improvements over strong baselines are substantial (81–92% relative reductions), and the architecture is a reasonable combination of existing components (Mega-TTS-style PLM, Grad-StyleSpeech diffusion backbone, DreamBooth-style prior-preservation loss). The paper includes a demo page, which is a strength. However, the central causal attribution—that prior-prosody prompting is what drives the robustness—is not cleanly supported by the paper's own ablation results, and the sensitivity of the method to the hand-picked prior sample is not examined. These issues must be addressed before the central claim can be accepted.

major comments (3)

[Section III-C, Table I] The text states that 'removing either component results in a degradation of performance in terms of MOS, SMOS, and WER across all scenarios,' but Table I contradicts this for the prior-prompt ablation. Stable-TTS(w/o Prior Prompt) achieves lower (better) WER than full Stable-TTS on LibriTTS (0.83 vs 1.13) and on VoxCeleb (1.02 vs 1.32), and higher MOS on LibriTTS (3.65 vs 3.37). Also, Stable-TTS(w/o PLM) gives higher SMOS on VoxCeleb (2.93 vs 2.65). Thus the ablation does not demonstrate that the prior-prosody-prompting component is necessary for the headline WER gains; the measured effect of that component is mixed and dataset-dependent. The authors should correct this claim, re-analyze the ablations, and discuss which component actually drives the improvement over baselines.
[Section III-A, implementation details] The prior sample used for the PLM prompt is described as 'carefully choose one speech sample for each male and female,' with no selection criteria, no description of how representative these samples are, and no sensitivity analysis. Since the entire prosody-prompting mechanism depends on the assumption that these two fixed samples provide prosody that transfers across arbitrary input texts and target speakers, the paper must vary the prior sample (e.g., multiple candidates per gender, different lengths, different prosodic content) and report WER/MOS/SMOS across those variations. Without this, the claimed robustness of prior-prosody prompting is not established, and the results could be artifacts of the specific chosen utterances.
[Table I and Table III] No statistical significance tests are reported for any of the subjective or objective metrics. Many of the differences that support the ablations are within the reported 95% confidence intervals; for example, on LibriTTS the MOS gap between Stable-TTS and Stable-TTS(w/o Prior Prompt) is 3.37±0.20 vs 3.65±0.21, and on VoxCeleb the SMOS gap between Stable-TTS and Stable-TTS(w/o PLM) is 2.65±0.23 vs 2.93±0.22. The authors should provide significance tests (e.g., paired bootstrap or Wilcoxon signed-rank) for the key comparisons, at least for the main result versus baselines and for the ablations, so that the reader can judge whether the claimed improvements are meaningful.

minor comments (5)

[Section II-B, Eq. (4)] The notation in Eq. (4) is slightly underspecified: the target prosody sequence p_t and the conditioning p_<t are not explicitly defined as the prosody codes aligned with the input phonemes µ_≤t; please clarify the indexing and the training objective for the PLM.
[Section III-A, evaluation metrics] The paper reports SECS using Resemblyzer, which is a relatively weak speaker-verification model; please justify this choice or supplement it with a more standard speaker-embedding metric (e.g., ECAPA-TDNN cosine similarity).
[Fig. 4] The t-SNE visualization in the left panel is only qualitative and is not directly linked to WER or SECS; consider either removing it or adding a quantitative analysis of the representation space.
[Section III-A, implementation details] The phrase 'a low band with rich prosodic contents of size 15' is vague; specify which mel bins are used and how the low band was selected.
[Section III-A, subjective evaluation] The paper states that 20 evaluators rated the samples, but it does not say whether they were expert or crowd-sourced, nor how they were recruited; please provide this information for reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: Stable-TTS is an empirical system paper whose central claims are validated by external baselines and ablations, not reduced to its inputs by construction; the score reflects only minor self-citations that are not load-bearing.

full rationale

Stable-TTS does not present a derivation chain in which a quantity is predicted from a fitted parameter or defined in terms of its own output. The Prosody Language Model is adopted from Mega-TTS [4] and the prior-preservation loss from DreamBooth [11], both external to the authors; the self-citations to Grad-StyleSpeech [3] and Meta-StyleSpeech [15] attribute the base diffusion architecture and style encoder and are not used to justify the paper's central claim that clean prior prosody prompting plus prior-preservation loss stabilizes few-shot noisy TTS. That claim is assessed empirically against non-author baselines and through ablations, so no fitted value is relabeled as a prediction and no equation is equivalent to its input by construction. Two robustness concerns should be flagged, but they are not circularity: Section III-A's 'carefully choose one speech sample for each male and female' prior prompt is never varied in a sensitivity analysis, weakening the causal attribution of the benefit to that prompt; and Section III-C's assertion that removing any component degrades MOS, SMOS, and WER 'across all scenarios' is contradicted by Table I, where Stable-TTS(w/o Prior Prompt) achieves lower WER than full Stable-TTS on LibriTTS (0.83 vs 1.13) and VoxCeleb (1.02 vs 1.32) and higher MOS on LibriTTS (3.65 vs 3.37). These are empirical-support issues, not self-referential reductions, so the circularity score is low.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several hand-chosen hyperparameters (prior samples, low band size, k, fine-tuning steps) and on domain assumptions about data quality and prosody transfer. No new physical or theoretical entities are introduced.

free parameters (5)

Prior sample selection = One male and one female sample from LibriTTS-R, used in most cases
Prior samples are hand-picked as prosody prompts; selection criteria are not specified and could bias results.
Prosody low band size = 15 mel bins
The prosody encoder input is a low band of size 15; no ablation is shown for this choice.
Number of timbre mel-spectrograms k = 3
Equation (1) concatenates k=3 mel-spectrograms for the timbre encoder; no ablation is shown for k.
Fine-tuning steps = 500
Figure 4 identifies 500 steps as the sweet spot; steps are held constant across data scales in Table III, which may disadvantage baselines that need fewer steps.
Prior sample mel length = 100 to 150 frames
Prior samples are randomly chosen with mel length 100 to 150; this constraint is ad hoc and not justified.

assumptions (4)

domain assumption LibriTTS-R clean-100 and clean-360 subsets are high-quality and representative of clean speech
These subsets are used as pretraining data and as the source of prior samples; if they are not clean, the prosody prompt may not be reliable.
domain assumption Whisper ASR transcripts for VoxCeleb are accurate enough for WER evaluation
VoxCeleb speech is noisy and transcripts are generated by Whisper; ASR errors could inflate or deflate WER measurements.
domain assumption Prosody codes from Mega-TTS and the Prosody Language Model transfer to arbitrary text and speakers
Section II-B assumes that prior prosody prompts produce stable prosody codes for any input text, which is the load-bearing generalization of the method.
domain assumption The diffusion model setup and pretraining follow Grad-StyleSpeech without modification
The paper states it 'closely follows' Grad-StyleSpeech's pretraining setup and relies on that prior implementation for the base architecture.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting." pith.science (2026). https://pith.science/paper/W6RFKARU

@misc{pith2026241220155,
  author       = {Pith},
  title        = {Pith review of: Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W6RFKARU}},
  note         = {Machine review of arXiv:2412.20155}
}

read the original abstract

Speaker-adaptive Text-to-Speech (TTS) synthesis has attracted considerable attention due to its broad range of applications, such as personalized voice assistant services. While several approaches have been proposed, they often exhibit high sensitivity to either the quantity or the quality of target speech samples. To address these limitations, we introduce Stable-TTS, a novel speaker-adaptive TTS framework that leverages a small subset of a high-quality pre-training dataset, referred to as prior samples. Specifically, Stable-TTS achieves prosody consistency by leveraging the high-quality prosody of prior samples, while effectively capturing the timbre of the target speaker. Additionally, it employs a prior-preservation loss during fine-tuning to maintain the synthesis ability for prior samples to prevent overfitting on target samples. Extensive experiments demonstrate the effectiveness of Stable-TTS even under limited amounts of and noisy target speech samples.

Figures

Figures reproduced from arXiv: 2412.20155 by the authors.

**Figure 1.** Concept. Our objective is to build a speaker-adaptive TTS model that utilizes prosody prompt and prior-preservation with prior samples (blue box) to generate a high-quality voice even when fine-tuning with noisy target samples (green box). challenging due to unclear pronunciation and severe background noise, unlike the clear speech data in pre-training [10]. In such cases, fewer fine-tuning steps may reduce voice s… view at source ↗

**Figure 2.** Overview of Stable-TTS. (a) During training, we utilize both the prosody encoder and timbre encoder to enhance timbre and ensure prosody consistency. All representations are utilized as a condition for the diffusion model. (b) During the inference phase, we leverage a Prosody Language Model (PLM) to predict the prosody code utilizing the prompt from a prior sample and generate a timbre vector by concatenating multip… view at source ↗

**Figure 3.** Details of Fine-tuning. We train the diffusion models only using both diffusion and prior-preservation loss. of the synthesized speech remains consistent with the pretraining samples, even after fine-tuning. Moreover, instead of using target speech as the prompt speech X˜ [4], we propose leveraging prior samples—clean speech from the pre-training dataset as a prompt. This allows the PLM to generate a more stable se… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: (left), Stable-TTS already exhibits commendable performance in zero-shot scenarios. However, fine-tuning enhances both naturalness and similarity. Further analysis of fine-tuning steps in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

[1]

NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers,

K. Shen, Z. Ju, X. Tan, Y . Liu, Y . Leng, L. He, T. Qin, S. Zhao, and J. Bian, “NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers,” International Conference on Learning Representations (ICLR) , 2024

work page 2024
[2]

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,

C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li et al. , “Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,” arXiv preprint arXiv:2301.02111 , 2023

arXiv 2023
[3]

Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models,

M. Kang, D. Min, and S. J. Hwang, “Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023

work page 2023
[4]

Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias,

Z. Jiang, Y . Ren, Z. Ye, J. Liu, C. Zhang, Q. Yang, S. Ji, R. Huang, C. Wang, X. Yin et al., “Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias,” arXiv preprint arXiv:2306.03509 , 2023

arXiv 2023
[5]

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone,

E. Casanova, J. Weber, C. D. Shulby, A. C. J ´unior, E. G ¨olge, and M. A. Ponti, “YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone,” inInternational Conference on Machine Learning (ICML) , 2022

work page 2022
[6]

UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data,

H. Kim, S. Kim, J. Yeom, and S. Yoon, “UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data,” in Conference of the Inter- national Speech Communication Association (Interspeech) , 2023

work page 2023
[7]

AdaSpeech: Adaptive Text to Speech for Custom V oice,

M. Chen, X. Tan, B. Li, Y . Liu, T. Qin, S. Zhao, and T. Liu, “AdaSpeech: Adaptive Text to Speech for Custom V oice,” inInternational Conference on Learning Representations (ICLR) , 2021

work page 2021
[8]

Adapting TTS models For New Speakers using Transfer Learning,

P. Neekhara, J. Li, and B. Ginsburg, “Adapting TTS models For New Speakers using Transfer Learning,” arXiv preprint arXiv:2110.05798 , 2021

arXiv 2021

Show all 25 references

[9]

V oxCeleb: A Large- Scale Speaker Identification Dataset,

A. Nagrani, J. S. Chung, and A. Zisserman, “V oxCeleb: A Large- Scale Speaker Identification Dataset,” in Conference of the International Speech Communication Association (Interspeech) , 2017

2017
[10]

LibriTTS: A Corpus Derived from LibriSpeech for Text- to-Speech,

H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “LibriTTS: A Corpus Derived from LibriSpeech for Text- to-Speech,” in Conference of the International Speech Communication Association (Interspeech), 2019

2019
[11]

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject- Driven Generation,

N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject- Driven Generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

2023
[12]

Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech,

V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. A. Kudinov, “Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech,” in International Conference on Machine Learning (ICML) , 2021

2021
[13]

Fast- Speech: Fast, Robust and Controllable Text to Speech,

Y . Ren, Y . Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fast- Speech: Fast, Robust and Controllable Text to Speech,” in Conference on Neural Information Processing Systems (NeurIPS) , 2019

2019
[14]

One TTS alignment to rule them all,

R. Badlani, A. Lancucki, K. J. Shih, R. Valle, W. Ping, and B. Catanzaro, “One TTS alignment to rule them all,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2022

2022
[15]

Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation,

D. Min, D. B. Lee, E. Yang, and S. J. Hwang, “Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation,” in International Conference on Machine Learning (ICML) , 2021

2021
[16]

Neural Discrete Representation Learning,

A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural Discrete Representation Learning,” in Conference on Neural Information Pro- cessing Systems (NeurIPS) , 2017

2017
[17]

Denoising Diffusion Probabilistic Models,

J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models,” in Conference on Neural Information Processing Systems (NeurIPS), 2020

2020
[18]

Score-Based Generative Modeling through Stochastic Differential Equations,

Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” in International Conference on Learning Rep- resentations (ICLR), 2021

2021
[19]

ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech,

Y . Ren, M. Lei, Z. Huang, S. Zhang, Q. Chen, Z. Yan, and Z. Zhao, “ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2022

2022
[20]

Language Models are Unsupervised Multitask Learners,

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language Models are Unsupervised Multitask Learners,” OpenAI blog, 2019

2019
[21]

CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR V oice Cloning Toolkit (version 0.92),

J. Yamagishi, C. Veaux, and K. MacDonald, “CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR V oice Cloning Toolkit (version 0.92),” University of Edinburgh. The Centre for Speech Technology Research (CSTR), Tech. Rep., 2019

2019
[22]

LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus,

Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, and A. Bapna, “LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus,” in Conference of the International Speech Communication Association (Interspeech) , 2023

2023
[23]

Robust Speech Recognition via Large-Scale Weak Su- pervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust Speech Recognition via Large-Scale Weak Su- pervision,” in International Conference on Machine Learning (ICML) , 2023

2023
[24]

Resemblyzer,

Resemble.AI, “Resemblyzer,” https://github.com/resemble-ai/ Resemblyzer, 2020

2020
[25]

HiFi-GAN: Generative Adversarial Net- works for Efficient and High Fidelity Speech Synthesis,

J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative Adversarial Net- works for Efficient and High Fidelity Speech Synthesis,” in Conference on Neural Information Processing Systems (NeurIPS) , 2020

2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers,

K. Shen, Z. Ju, X. Tan, Y . Liu, Y . Leng, L. He, T. Qin, S. Zhao, and J. Bian, “NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers,” International Conference on Learning Representations (ICLR) , 2024

work page 2024

[2] [2]

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,

C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li et al. , “Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,” arXiv preprint arXiv:2301.02111 , 2023

arXiv 2023

[3] [3]

Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models,

M. Kang, D. Min, and S. J. Hwang, “Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023

work page 2023

[4] [4]

Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias,

Z. Jiang, Y . Ren, Z. Ye, J. Liu, C. Zhang, Q. Yang, S. Ji, R. Huang, C. Wang, X. Yin et al., “Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias,” arXiv preprint arXiv:2306.03509 , 2023

arXiv 2023

[5] [5]

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone,

E. Casanova, J. Weber, C. D. Shulby, A. C. J ´unior, E. G ¨olge, and M. A. Ponti, “YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone,” inInternational Conference on Machine Learning (ICML) , 2022

work page 2022

[6] [6]

UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data,

H. Kim, S. Kim, J. Yeom, and S. Yoon, “UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data,” in Conference of the Inter- national Speech Communication Association (Interspeech) , 2023

work page 2023

[7] [7]

AdaSpeech: Adaptive Text to Speech for Custom V oice,

M. Chen, X. Tan, B. Li, Y . Liu, T. Qin, S. Zhao, and T. Liu, “AdaSpeech: Adaptive Text to Speech for Custom V oice,” inInternational Conference on Learning Representations (ICLR) , 2021

work page 2021

[8] [8]

Adapting TTS models For New Speakers using Transfer Learning,

P. Neekhara, J. Li, and B. Ginsburg, “Adapting TTS models For New Speakers using Transfer Learning,” arXiv preprint arXiv:2110.05798 , 2021

arXiv 2021