Pith. sign in

REVIEW 4 major objections 5 minor 83 references

Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that conditioning a phrasing model on speaker embeddings and phoneme-level pre-trained language models improves respiratory pause prediction for multi-speaker text-to-speech, and that a small adapter extends this to new spe

desk verdict Solid, well-scoped empirical work on speaker-conditioned phrasing with a genuinely useful few-shot adapter, but the headline phoneme-PLM advantage is confounded by unequal pre-training and should be softened or re-run. read the letter →

arxiv 2509.00675 v1 pith:6AYPT3RX submitted 2025-08-31 eess.AS

Dong Yang , Yuki Saito , Takaaki Saeki , Tomoki Koriyama , Wataru Nakata , Detai Xin , Hiroshi Saruwatari This is my paper · ORCID
classification eess.AS
keywords phrasebreakpredictionrespiratorypauseinsertionmulti-speakertext-to-speechspeakerembeddingphoneme-levelpre-trainedlanguagemodelfew-shotadaptationMPBERTprosodymodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attacks the pause-insertion problem in multi-speaker text-to-speech: when a TTS system reads long text, it must decide where to put breaths and phrase breaks that are not marked by punctuation. It argues that current phrasing models fail in two ways—they ignore who is speaking, and they rely on subword text representations that lack acoustic information. Its central claim is that conditioning the phrasing model on a speaker embedding and on phoneme-level pre-trained representations removes both limits, and it reports large F0.5 gains on a large audiobook corpus. The same setup, with a small adapter that maps pre-trained speaker embeddings into the model's embedding space, lets a pre-trained phrasing model handle new speakers from a few dozen utterances without fine-tuning. If correct, this gives long-form TTS systems a practical way to match pause placement to individual voices.

What carries the argument

The load-bearing object is the speaker-conditioned encoder-decoder phrasing model: a PLM encoder (subword or phoneme-level) followed by two BiLSTM layers with dropout and layer normalization, plus a speaker embedding layer whose output is linearly projected with GELU and added at the encoder–decoder interface. For unseen speakers, the paper adds an embedding adapter: a two-layer MLP with ReLU trained on seen-speaker pairs (pre-trained speaker-verification embedding to trained embedding) under mean-squared error, then applied to embeddings of new speakers to replace the embedding layer. The paper also invokes a mutual-information argument: RP labels are word-level, and phoneme representations

What would settle it

For a held-out set of unseen speakers whose RP insertion frequency matches the training distribution, compare three conditions on the same test text: no adaptation, frozen raw pre-trained speaker embeddings, and adapter-mapped embeddings. If the adapter condition does not beat the frozen condition at roughly 30–40 utterances per speaker, the injective-mapping assumption fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that speaker identity and phoneme-level representation are both usable signals for predicting respiratory pauses. A speaker-conditioned phrasing model—a PLM encoder plus a two-layer BiLSTM decoder with a speaker embedding injected at the boundary—raises F0.5 on seen speakers from 0.3719 for a subword BERT baseline to 0.4991 with mixed-phoneme BERT and a trainable random-initialized speaker embedding. Phoneme-level models beat subword-level models, and a few-shot adapter reaches 0.4041 on unseen speakers with about 40 utterances per speaker, above the baseline's 0.3188. The paper reads these numbers as confirmation that RP insertion style varies by s

Load-bearing premise

The few-shot claim would collapse if the learned mapping from pre-trained speaker embeddings to the trained speaker-embedding layer, fit on seen speakers, does not transfer to unseen speakers.

Editorial extensions

If this is right

  • If correct, multi-speaker TTS front-ends can condition phrasing on speaker identity at negligible inference cost, since only speaker IDs are needed.
  • Phoneme-level PLMs become a candidate default encoder for phrasing and other prosodic front-end tasks, not just for acoustic model encoders.
  • Pre-trained speaker embeddings plus a small adapter give a fine-tuning-free route to adapting phrasing to new voices, which matters when full fine-tuning is too costly.
  • The finding that phrasing-trained embeddings encode gender, age, and fluency suggests RP placement can serve as a low-cost signal for speaker profiling.
  • Improved phrasing should translate into more natural synthetic speech in long-form reading; the paper's MOS results support this direction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test this, but the same speaker-embedding injection could plausibly extend to other pause- or prosody-related front-end modules, such as duration prediction, where speaker style also matters.
  • The reported mismatch between F0.5 and MOS for MP BERT suggests the objective metric may undervalue pauses that listeners find natural; a listening-oriented training objective could close that gap more than further F0.5 tuning.
  • If the adapter mechanism generalizes, it provides a template for making other speaker-conditioned front-end components few-shot adaptable without retraining the bulk of the model.
  • The correlations between fluency-related annotations and phrasing embeddings hint that RP prediction could be used as an unsupervised probe for fluency-related speaker traits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper addresses phrase break prediction (phrasing) for multi-speaker text-to-speech. It proposes a speaker-conditioned phrasing model that inserts a speaker embedding layer (randomly initialized or from a pre-trained speaker verification model, PSVM) between a PLM encoder and a BiLSTM decoder, and evaluates a range of subword- and phoneme-level PLMs. For unseen speakers it introduces an embedding adapter that maps PSVM embeddings to the trained speaker-embedding layer, enabling few-shot adaptation without fine-tuning. Experiments on a LibriTTS-R-derived dataset report F0.5 gains from 0.3719 for the baseline to 0.4991 with the best seen-speaker model (MP BERT + trainable random speaker embedding), and from 0.3188 to 0.4041 with few-shot adaptation on unseen speakers. MOS tests using VITS and Matcha-TTS are also reported, together with a clustering/chi-square analysis of speaker embeddings against LibriTTS-P annotations.

Significance. If the results hold, the paper makes useful contributions: speaker conditioning improves multi-speaker phrasing, and the proposed adapter offers a practical way to adapt to unseen speakers with a handful of utterances. The paper is also the first to apply phoneme-level PLMs to this TTS front-end task. Strengths include the large-scale dataset construction, the two-stage training recipe, evaluation with two TTS backbones, subjective listening tests, and public release of model weights and a demo page. The central phoneme-level claim, however, is currently confounded by mismatched pre-training settings, and the objective F0.5 comparisons are single point estimates without uncertainty, so the headline claims outrun the evidence as presented.

major comments (4)
  1. [§5.1.3, Table 4; §8] The claim that phoneme-level PLMs 'significantly boost' phrasing accuracy is not established. MP BERT and PL BERT were implemented and pre-trained by the authors for only 10 epochs on BookCorpus + English Wikipedia, whereas all subword PLMs are released checkpoints pre-trained on much larger and differently distributed corpora. The F0.5 differences (MP BERT 0.4991 vs. RoBERTa_LARGE 0.4865; PL BERT 0.4858) could reflect pre-training data, compute, tokenization, or implementation choices, not the phoneme-level representation itself. Eq. (2) is an asserted inequality; no mutual information values are computed. Section 8 explicitly states the authors cannot disentangle the effects of phonemes and subwords. A controlled comparison (e.g., a subword-level PLM pre-trained with the same data and epochs, or an equivalently pre-trained phoneme-level checkpoint) is needed before the abstract's causa
  2. [§5.1, Tables 3–4; §5.2.2, Table 7] All F0.5 scores are point estimates from a single training run, with no variance, confidence intervals, or significance tests. For example, BERTBASE 0.4755 vs. RoBERTaBASE 0.4776 is a difference of 0.002, and PL BERT 0.4858 vs. RoBERTa_LARGE 0.4865 is 0.0007; without multiple runs or a paired test, these rankings are within plausible noise. The abstract uses 'significantly,' but only the MOS results have t-tests. Please provide seed variance or paired significance tests for the load-bearing objective comparisons.
  3. [§5.2.2, Tables 7–8] The few-shot claim is partly selected on the test-unseen set. The underlined models chosen for MOS correspond to the best F0.5 at particular sample counts (e.g., BERTBASE+ResNet-TDNN trainable at 40 samples, 0.4041; MP BERT+ResNet-TDNN trainable at 30, 0.3723) rather than to a pre-specified validation policy, and neighboring sample counts differ substantially (0.4041 vs. 0.4007 at 50 for the first example). This selection, combined with the absence of uncertainty estimates, may overstate the few-shot gain. A validation-based selection rule or an explicit multiple-comparison handling should be reported.
  4. [§3.4, §5.2.2] The few-shot adapter relies on the assumption that an injective mapping exists from PSVM embeddings to the trained speaker-embedding layer and that a two-layer MLP trained on seen speakers approximates it for unseen speakers. This assumption is asserted, not validated. If the mapping fails to transfer, the Table 7 results are specific to the chosen validation-unseen speakers. I recommend reporting, at minimum, the adapter's performance on held-out seen speakers, or per-speaker variance on unseen speakers, to support the generalization claim.
minor comments (5)
  1. [§5.2.1] The sentence 'The phrasing results are presented in Table 2' should refer to Table 6, which reports the baseline results on unseen speakers.
  2. [§3.3, Eq. (2)] The random variables X_phoneme, X_subword, and Y_word are not formally defined, and no estimates of I(X;Y) are provided. Please either define the notation precisely or present the mutual-information argument as a motivating intuition rather than a formal derivation.
  3. [§6.2.2, Tables 10–12] The chi-square tests are performed for many characteristics and multiple k values without correction for multiple comparisons. Some significant results are expected by chance; reporting adjusted p-values or clearly labeling this as exploratory would strengthen the analysis.
  4. [Appendix D] The results in Table D.15/D.16 are interesting but raise more questions than they answer; e.g., ECAPA-TDNN with BERTBASE drops to 0.2737 on seen speakers when trained on LibriTTS-R, while ResNet-TDNN improves. A brief discussion of why the PSVM training-data mismatch affects different PSVMs so differently would be helpful.
  5. [§4.4.2] The MOS section reports 48–50 valid responses per test but does not specify whether subjects were screened for reliability, how many utterances each subject rated, or how the t-test was paired. Please add these details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central claims rest on held-out evaluations; only minor self-citation to prior architecture, not load-bearing.

full rationale

The paper's central claims—speaker conditioning improves phrasing, phoneme-level PLMs improve F0.5, and a few-shot adapter generalizes to unseen speakers—are supported by held-out test sets (test-seen/test-unseen) and external annotations (LibriTTS-P). The few-shot adapter is trained on seen-speaker pairs (PSVM embeddings vs. trained embedding layer) and evaluated on unseen speakers; thus the evaluation is not a fit to the target. Section 3.3's mutual-information argument is motivational and not used as a derivation; no I(X;Y) values are computed. The only self-citation is to the authors' previous phrasing model [30], which supplies the architecture; this is prior work, not an unverified premise that forces the results. Section 8 explicitly acknowledges the phoneme/subword entanglement as a limitation, which is a correctness/interpretability concern, not circularity. No equation in the paper reduces a prediction to its fitted input, and no load-bearing claim is justified solely by a self-citation chain.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The empirical claims mainly rest on labeling choices and hyperparameters. The 50 ms RP definition is the most consequential: it fully determines the labels and hence all F0.5 and MOS results. The remaining entries are standard training/evaluation parameters; none are fitted to the test-set outcome in a physics sense, though the post-hoc choice of sample counts for MOS introduces selection risk.

free parameters (5)
  • RP duration threshold = 50 ms
    Pauses above 50 ms at non-punctuation word boundaries are labeled as respiratory pauses (Section 4.1); this hand-chosen threshold defines the ground truth for all experiments.
  • F0.5 decision threshold = selected per model on validation-seen set
    A probability threshold is tuned on validation-seen to maximize F0.5 (Section 4.4.1); this is a fitted parameter, though standard.
  • Number of k-means clusters k = 4, 16, 64
    Cluster count for the speaker-embedding analysis is chosen by hand; results vary with k (Section 6.2.2).
  • Few-shot sample count for MOS selection = 30-50 samples per speaker, best underlined per model
    Subjective evaluation uses the best performing sample sizes chosen post hoc from Table 7 (Section 5.2.2).
  • Embedding adapter architecture/training = two linear layers, hidden 1024, 100k steps, lr 1e-5
    Adapter hyperparameters chosen without systematic search (Sections 3.4, 4.5.3).
assumptions (4)
  • domain assumption MFA alignment plus a 50 ms threshold yields reliable ground-truth RP labels
    All labels, metrics, and MOS samples derive from this pipeline (Section 4.1); errors here propagate everywhere.
  • domain assumption An injective mapping exists from PSVM embeddings to the trained speaker-embedding layer, approximated by a two-layer MLP
    The few-shot adapter in Section 3.4 relies on this transfer; no evidence beyond empirical test performance.
  • domain assumption Masking non-final word tokens makes evaluations fair across different tokenizers
    Section 4.1 and Fig. 4: labels are attached only to word-final tokens; assumes tokenization differences do not bias model comparisons.
  • ad hoc to paper The mutual-information inequality I(X_phoneme;Y_word)>I(X_subword;Y_word) in Eq. (2)
    Section 3.3 states this as motivation but never measures MI or entropy; it is an assertion, not a derived theorem.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model." pith.science (2026). https://pith.science/paper/6AYPT3RX

@misc{pith2026250900675,
  author       = {Pith},
  title        = {Pith review of: Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AYPT3RX}},
  note         = {Machine review of arXiv:2509.00675}
}
read the original abstract

This paper advances phrase break prediction (also known as phrasing) in multi-speaker text-to-speech (TTS) systems. We integrate speaker-specific features by leveraging speaker embeddings to enhance the performance of the phrasing model. We further demonstrate that these speaker embeddings can capture speaker-related characteristics solely from the phrasing task. Besides, we explore the potential of pre-trained speaker embeddings for unseen speakers through a few-shot adaptation method. Furthermore, we pioneer the application of phoneme-level pre-trained language models to this TTS front-end task, which significantly boosts the accuracy of the phrasing model. Our methods are rigorously assessed through both objective and subjective evaluations, demonstrating their effectiveness.

Figures

Figures reproduced from arXiv: 2509.00675 by the authors.

Figure 1
Figure 1. Architecture of phrasing models. PSVM Utterance 0 Utterance 1 … Utterance N-1 Embedding 0 Embedding 1 … Embedding N-1 PSVM Embedding Average [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. PSVM embedding extraction pipeline for a single speaker. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Architecture, training, and inference of the embedding adapter. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Example of text pre-processing, tokenization, and label generation. The text in the example was not used in the experiments. (a) Text [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Distribution of RP insertion frequency across speakers. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Distribution of average RP duration across speakers. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Visualization of speaker embeddings with t-SNE. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 74 canonical work pages

  1. [1]

    X. Tan, T. Qin, F. Soong, T.-Y . Liu, A survey on neural speech synthesis, arXiv preprint arXiv:2106.15561 (2021)

    arXiv 2021

  2. [2]

    Khanam, F

    F. Khanam, F. A. Munmun, N. A. Ritu, A. K. Saha, M. F. Mridha, Text to speech synthesis: A systematic review and deep learning based architecture and future research direction, Journal of Advances in Information Technology 13 (5) (2022) 1–22

    2022

  3. [3]

    W. Ping, K. Peng, A. Gibiansky, S. . Arik, A. Kannan, S. Narang, J. Raiman, J. Miller, Deep V oice 3: Scaling text-to-speech with convolutional sequence learning, in: Proc. ICLR, 2018

    2018

  4. [4]

    J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y . Zhang, Y . Wang, R. Skerry-Ryan, R. A. Saurous, Y . Agiomyrgian- nakis, Y . Wu, Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions, in: Proc. ICASSP, 2018, pp. 4779–4783

    2018

  5. [5]

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, T.-Y . Liu, FastSpeech 2: Fast and high-quality end-to-end text to speech, in: Proc. ICLR, 2021

    work page 2021

  6. [6]

    Popov, I

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, M. A. Kudinov, Grad-TTS: A diffusion probabilistic model for text-to-speech, in: Proc. ICML, V ol. 139, 2021, pp. 8599–8608

    work page 2021

  7. [7]

    J. Kim, J. Kong, J. Son, Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech, in: Proc. ICML, V ol. 139, 2021, pp. 5530–5540

    work page 2021

  8. [8]

    J. Kong, J. Park, B. Kim, J. Kim, D. Kong, S. Kim, VITS2: improving quality and e fficiency of single-stage text-to-speech with adversarial learning and architecture design, in: Proc. Interspeech, 2023

    work page 2023

Show all 83 references
  1. [9]

    Mehta, R

    S. Mehta, R. Tu, J. Beskow, ´E. Sz´ekely, G. E. Henter, Matcha-TTS: A fast TTS architecture with conditional flow matching, in: Proc. ICASSP, 2024, pp. 11341–11345

    2024

  2. [10]

    van den Oord, S

    A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, K. Kavukcuoglu, WaveNet: A Generative Model for Raw Audio, in: Proc. ISCA Speech Synthesis Workshop (SSW), 2016

    2016

  3. [11]

    Kumar, R

    K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. de Brbisson, Y . Bengio, A. C. Courville, MelGAN: Generative adversarial networks for conditional waveform synthesis, in: Proc. NeurIPS, 2019, pp. 14881–14892

    2019

  4. [12]

    J. Kong, J. Kim, J. Bae, HiFi-GAN: Generative adversarial networks for e fficient and high fidelity speech synthesis, in: Proc. NeurIPS, 2020

    2020

  5. [13]

    Z. Kong, W. Ping, J. Huang, K. Zhao, B. Catanzaro, Di ffWave: A versatile diffusion model for audio synthesis, in: Proc. ICLR, 2021

    2021

  6. [14]

    Siuzdak, V ocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis, in: Proc.ICLR, 2024

    H. Siuzdak, V ocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis, in: Proc.ICLR, 2024

    2024

  7. [15]

    Klimkov, A

    V . Klimkov, A. Nadolski, A. Moinet, B. Putrycz, R. Barra-Chicote, T. Merritt, T. Drugman, Phrase break prediction for long-form reading TTS: Exploiting text structure information, in: Proc. Interspeech, 2017, pp. 1064–1068

    2017

  8. [16]

    Bailly, C

    G. Bailly, C. Gouvernayre, Pauses and respiratory markers of the structure of book reading, in: Proc. Interspeech, 2012

    2012

  9. [17]

    Abbas, T

    A. Abbas, T. Merritt, A. Moinet, S. Karlapati, E. Muszynska, S. Slangen, E. Gatti, T. Drugman, Expressive and variable and controllable duration modelling in tts, in: Proc. Interspeech, 2022, pp. 4546–4550

    2022

  10. [18]

    L. Xue, F. K. Soong, S. Zhang, L. Xie, ParaTTS: Learning linguistic and prosodic cross-sentence information in paragraph-based TTS, TASLP 30 (2022) 2854–2864

    2022

  11. [19]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Proc. NeurIPS, 2017, pp. 5998–6008

    2017

  12. [20]

    Devlin, M

    J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proc. NAACL-HLT, 2019, pp. 4171–4186

    2019

  13. [21]

    Nakata, T

    W. Nakata, T. Koriyama, S. Takamichi, N. Tanji, Y . Ijima, R. Masumura, H. Saruwatari, Audiobook speech synthesis conditioned by cross- sentence context-aware word embeddings, in: Proc. ISCA Speech Synthesis Workshop (SSW), 2021, pp. 211–215

    2021

  14. [22]

    Zhang, C

    W. Zhang, C. Yeh, W. Beckman, T. Raitio, R. Rasipuram, D. W. L. Golipour, Audiobook synthesis with long-form neural text-to-speech, in: Proc. ISCA Speech Synthesis Workshop (SSW), 2023, pp. 139–143

    2023

  15. [23]

    Saeki, S

    T. Saeki, S. Maiti, X. Li, S. Watanabe, S. Takamichi, H. Saruwatari, Learning to speak from text: Zero-shot multilingual text-to-speech with unsupervised text pretraining, in: Proc. IJCAI, 2023, pp. 5179–5187

    2023

  16. [24]

    Saeki, S

    T. Saeki, S. Maiti, X. Li, S. Watanabe, S. Takamichi, H. Saruwatari, Text-inductive graphone-based language adaptation for low-resource speech synthesis, TASLP 32 (2024) 1829–1844

    2024

  17. [25]

    Futamata, B

    K. Futamata, B. Park, R. Yamamoto, K. Tachibana, Phrase break prediction with bidirectional encoder representations in Japanese text-to- speech synthesis, in: Proc. Interspeech, 2021, pp. 3126–3120

    2021

  18. [26]

    H. Lee, H. Yoon, J. Kim, J. Kim, Cross-lingual transfer learning for phrase break prediction with multilingual language model, in: Proc. Interspeech, 2023

    2023

  19. [27]

    R. Liu, B. Liu, H. Li, Emotion-aware prosodic phrasing for expressive text-to-speech, in: Proceedings of NCMMSC, 2024, pp. 326–337

    2024

  20. [28]

    Zhang, K

    G. Zhang, K. Song, X. Tan, D. Tan, Y . Yan, Y . Liu, G. Wang, W. Zhou, T. Qin, T. Lee, S. Zhao, Mixed-phoneme BERT: Improving BERT with mixed phoneme and sup-phoneme representations for text to speech, in: Proc. Interspeech, 2022, pp. 456–460

    2022

  21. [29]

    Y . A. Li, C. Han, X. Jiang, N. Mesgarani, Phoneme-level BERT for enhanced prosody of text-to-speech with grapheme predictions, in: Proc. ICASSP, 2023

    2023

  22. [30]

    D. Yang, T. Koriyama, Y . Saito, T. Saeki, D. Xin, H. Saruwatari, Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speech, in: Proc. ICASSP, 2023

    2023

  23. [31]

    Koizumi, H

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, A. Bapna, LibriTTS-R: A restored multi-speaker text-to-speech corpus, in: Proc. Interspeech, 2023

    2023

  24. [32]

    Chiche, B

    A. Chiche, B. Yitagesu, Part of speech tagging: a systematic review of deep learning and machine learning approaches, Big Data 9 (10) (2022)

    2022

  25. [33]

    Q. Shi, J. DanNing, M. FanPing, Q. Yong, Combining length distribution model with decision tree in prosodic phrase prediction, in: Proc. INTERSPEECH, 2007, pp. 454–457

    2007

  26. [34]

    Parlikar, A

    A. Parlikar, A. W. Black, A grammar based approach to style specific phrase prediction, in: Proc. INTERSPEECH, 2011, pp. 2149–2152. 28

    2011

  27. [35]

    Watts, J

    O. Watts, J. Yamagishi, S. King, Unsupervised continuous-valued word features for phrase-break prediction without a part-of-speech tagger, in: Proc. INTERSPEECH, 2011, pp. 2157–2160

    2011

  28. [36]

    Braunschweiler, R

    N. Braunschweiler, R. Maia, Pause prediction from text for speech synthesis with user-definable pause insertion likelihood threshold, in: Proc. INTERSPEECH, 2016, pp. 3191–3195

    2016

  29. [37]

    Zhang, F

    Z. Zhang, F. Wu, C. Yang, M. Dong, F. Zhou, Mandarin prosodic phrase prediction based on syntactic trees, in: Proc. ISCA Speech Synthesis Workshop (SSW), 2016, pp. 175–180

    2016

  30. [38]

    P. Bell, T. Burrows, P. Taylor, Adaptation of prosodic phrasing models, in: Proc. Speech Prosody, 2006

    2006

  31. [39]

    I. Read, S. Cox, Stochastic and syntactic techniques for predicting phrase breaks, Computer Speech and Language 21 (3) (2007) 519–542

    2007

  32. [40]

    Chen, Z.-H

    Q. Chen, Z.-H. Ling, C.-Y . Yang, L.-R. Dai, Automatic phrase boundary labeling of speech synthesis database using context-dependent HMMs and n-gram prior distributions, in: Proc. INTERSPEECH, 2015, pp. 1581–1585

    2015

  33. [41]

    Mikolov, K

    T. Mikolov, K. Chen, G. Corrad, J. Dean, E fficient estimation of word representations in vector space, in: Proc. ICLR Workshop, 2013

    2013

  34. [42]

    Hochreiter, J

    S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation 9 (8) (1997) 1735–1780

    1997

  35. [43]

    H. Sak, A. W. Senior, F. Beaufays, Long short-term memory recurrent neural network architectures for large scale acoustic modeling, in: Proc. Interspeech, 2014, pp. 338–342

    2014

  36. [44]

    Graves, J

    A. Graves, J. Schmidhuber, Framewise phoneme classification with bidirectional lstm and other neural network architectures, in: Proc. IJCNN, 2005

    2005

  37. [45]

    Stan, Residual information in deep speaker embedding architectures, Mathematics 10 (21) (2022)

    A. Stan, Residual information in deep speaker embedding architectures, Mathematics 10 (21) (2022)

    2022

  38. [46]

    Jakubec, R

    M. Jakubec, R. Jarina, E. Lieskovska, P. Kasak, Deep speaker embeddings for speaker verification: Review and experimental comparison, Engineering Applications of Artificial Intelligence 127 (Part A) (2024) 107232

    2024

  39. [47]

    Z. Bai, X. Zhang, Speaker recognition based on deep learning: An overview, Neural Networks 140 (2021) 65–99

    2021

  40. [48]

    J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia, S. Zafeiriou, ArcFace: Additive angular margin loss for deep face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (10) (2022) 5962–5979

    2022

  41. [49]

    Nagrani, J

    A. Nagrani, J. S. Chung, A. Zisserman, V oxCeleb: A large-scale speaker identification dataset, in: Proc. Interspeech, 2017, pp. 2616–2620

    2017

  42. [50]

    J. S. Chung, A. Nagrani, A. Zisserman, V oxCeleb2: Deep speaker recognition, in: Proc. Interspeech, 2018, pp. 1086–1090

    2018

  43. [51]

    S. Arik, G. Diamos, A. Gibiansky, J. Miller, K. Peng, W. Ping, J. Raiman, Y . Zhou, Deep V oice 2: Multi-speaker neural text-to-speech, in: Proc. NeurIPS, 2017, pp. 2962–2970

    2017

  44. [52]

    Y . Chen, Y . Assael, B. Shillingford, D. Budden, S. Reed, H. Zen, Q. Wang, L. C. Cobo, A. Trask, B. Laurie, C. Gulcehre, A. van den Oord, O. Vinyals, N. de Freitas, Sample efficient adaptive text-to-speech, in: Proc. ICLR, 2019

    2019

  45. [53]

    Y . Jia, Y . Zhang, R. J. Weiss, Q. Wang, J. Shen, F. Ren, Z. Chen, P. Nguyen, R. Pang, I. Lpez-Moreno, Y . Wu, Transfer learning from speaker verification to multispeaker text-to-speech synthesis, in: Proc. NeurIPS, 2018, pp. 4485–4495

    2018

  46. [54]

    S. Ding, A. Renduchintala, K. Duh, A call for prudent choice of subword merge operations in neural machine translation, in: Proc. Machine Translation Summit XVII, V ol. 1, 2019, pp. 204–213

    2019

  47. [55]

    Kharitonov, M

    E. Kharitonov, M. Baroni, D. Hupkes, How BPE a ffects memorization in transformers, arXiv preprint arXiv:2110.02782 (2021)

    2021 arXiv

  48. [56]

    S. J. Mielke, Z. Alyafeai, E. Salesky, C. Ra ffel, M. Dey, M. Gall, A. Raja, C. Si, W. Y . Lee, B. Sagot, S. Tan, Between words and characters: A brief history of open-vocabulary modeling and tokenization in NLP, arXiv preprint arXiv:2112.10508 (2021)

    2021 arXiv

  49. [57]

    Sennrich, B

    R. Sennrich, B. Haddow, A. Birchx, Neural machine translation of rare words with subword units, in: Proc. ACL, 2016, pp. 5149–5152

    2016

  50. [58]

    Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, R. Soricut, ALBERT: A lite BERT for self-supervised learning of language represen- tations, in: Proc. ICLR, 2020

    2020

  51. [59]

    Hendrycks, K

    D. Hendrycks, K. Gimpel, Gaussian error linear units (GELUs), arXiv preprint arXiv:1606.08415 (2016)

    2016 arXiv

  52. [60]

    D. Yang, T. Koriyama, Y . Saito, Frame-wise breath detection with self-training: An exploration of enhancing breath naturalness in text-to- speech, in: Proc. Interspeech, 2024

    2024

  53. [61]

    Schuster, K

    M. Schuster, K. Nakajima, Japanese and korean voice search, in: Proc. ICASSP, 2012, pp. 5149–5152

    2012

  54. [62]

    https: //github.com/ydqmkkx/phonemetokenizer

  55. [63]

    McAuli ffe, M

    M. McAuli ffe, M. Socolof, S. Mihuc, M. Wagner, M. Sonderegger, Montreal Forced Aligner: Trainable text-speech alignment using Kaldi, in: Proc. Interspeech, 2017, pp. 498–502

    2017

  56. [64]

    Z. Yang, Z. Dai, Y . Yang, J. G. Carbonell, R. Salakhutdinov, Q. V . Le, XLNet: Generalized autoregressive pretraining for language under- standing, in: Proc. NeurIPS, 2019, pp. 5754–5764

    2019

  57. [65]

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V . Stoyanov, RoBERTa: A robustly optimized BERT pretraining approach, arXiv preprint arXiv:1907.11692 (2019)

    1907 arXiv

  58. [66]

    P. He, J. Gao, W. Chen, DeBERTaV3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing, in: Proc. ICLR, 2023

    2023

  59. [67]

    Desplanques, J

    B. Desplanques, J. Thienpondt, K. Demuynck, ECAPA-TDNN: emphasized channel attention and propagation and aggregation in TDNN based speaker verification, in: Proc. Interspeech, 2020, pp. 3830–3834

    2020

  60. [68]

    Villalba, N

    J. Villalba, N. Chen, D. Snyder, D. Garcia-Romero, A. McCree, G. Sell, J. Borgstrom, L. P. Garca-Perera, F. Richardson, R. Dehak, P. A. Torres-Carrasquillo, N. Dehak, State-of-the-art speaker recognition with neural network embeddings in nist sre18 and speakers in the wild eva...

    2020

  61. [69]

    N. R. Koluguri, J. Li, V . Lavrukhin, B. Ginsburg, SpeakerNet: 1d depth-wise separable convolutional network for text-independent speaker recognition and verification, arXiv preprint arXiv:2010.12653 (2020)

    2010 arXiv

  62. [70]

    N. R. Koluguri, T. Park, B. Ginsburg, TitaNet: Neural model for speaker representation with 1d depth-wise separable convolutions and global context, in: Proc. ICASSP, 2022, pp. 8102–8106

    2022

  63. [71]

    Glorot, Y

    X. Glorot, Y . Bengio, Understanding the di fficulty of training deep feedforward neural networks, in: Proc. AISTATS, V ol. 9, 2010, pp. 249–256

    2010

  64. [72]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Decoupled weight decay regularization, in: Proc. ICLR, 2019

    2019

  65. [73]

    Cieri, D

    C. Cieri, D. Gra ff, O. Kimball, D. Miller, K. Walker, Fisher english training speech part 1 transcripts, Linguistic Data Consortium (2004)

    2004

  66. [74]

    Godfrey, E

    J. Godfrey, E. Holliman, Switchboard-1 release 2 ldc97s62, Linguistic Data Consortium (1993). 29

    1993

  67. [75]

    Panayotov, G

    V . Panayotov, G. Chen, D. Povey, S. Khudanpur, LibriSpeech: An ASR corpus based on public domain audio books, in: Proc. ICASSP, 2015, pp. 5206–5210

    2015

  68. [76]

    Y . Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, S. Fidler, Aligning books and movies: Towards story-like visual explanations by watching movies and reading books, in: Proc. ICCV , 2015, pp. 19–27

    2015

  69. [77]

    Kocmi, O

    T. Kocmi, O. Bojar, An exploration of word embedding initialization in deep-learning tasks, in: Proc. ICON, 2017, pp. 56–64

    2017

  70. [78]

    J. MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, V ol. 1, 1967, pp. 281–297

    1967

  71. [79]

    van der Maaten, G

    L. van der Maaten, G. Hinton, Visualizing data using t-sne, Journal of Machine Learning Research 9 (2008) 2579–2605

    2008

  72. [80]

    Kawamura, R

    M. Kawamura, R. Yamamoto, Y . Shirahata, T. Hasumi, K. Tachibana, LibriTTS-P: A corpus with speaking style and speaker identity prompts for text-to-speech and style captioning, in: Proc.INTERSPEECH, 2024

    2024

  73. [81]

    K. Ito, L. Johnson, The LJ Speech dataset, https://keithito.com/LJ-Speech-Dataset/ (2017)

    2017

  74. [82]

    Veaux, J

    C. Veaux, J. Yamagishi, K. MacDonald, CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit, University of Edinburgh. The Centre for Speech Technology Research (CSTR) (2017)

    2017

  75. [83]

    Ravanelli, T

    M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong, J.-C. Chou, S.-L. Yeh, S.-W. Fu, C.-F. Liao, E. Rastorgueva, F. Grondin, W. Aris, H. Na, Y . Gao, R. D. Mori, Y . Bengio, SpeechBrain: A general-purpose s...

    2021 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.