Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SpeakStream: Streaming Text-to-Speech with Interleaved Data

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SpeakStream claims a streaming TTS can match non-streaming quality while producing its first phoneme in under 50 ms.

desk verdict Solid streaming-TTS recipe with real WER evidence, but the sub-50ms quality claim is untested because quality uses a 150ms vocoder and latency uses a different one. read the letter →

arxiv 2505.19206 v1 pith:4TCX6I3D submitted 2025-05-25 cs.CL cs.LGcs.SDeess.AS

Richard He Bai , Zijin Gu , Tatiana Likhomanenko , Navdeep Jaitly This is my paper · ORCID
classification cs.CLcs.LGcs.SDeess.AS
keywords streamingtext-to-speechdecoder-onlytransformerinterleavedtextandspeechfirst-tokenlatencydMeltokensvocoderforcedalignmentworderrorrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that streaming text-to-speech need not sacrifice quality for speed. SpeakStream trains a single decoder-only transformer on text and speech sequences interleaved word-by-word, using an external force aligner to decide where each speech segment belongs. At inference, the model reads a few words of incoming text, synthesizes the corresponding audio immediately, and continues chunk by chunk while reusing its cached context. On a standard single-speaker benchmark it reports word error rates close to full-sentence synthesis while producing the first phoneme tens of milliseconds after the first text token arrives. The result matters for conversational agents because it removes the two delays that normally force users to wait for a complete sentence.

What carries the argument

The load-bearing mechanism is the interleaved sequence representation $[T_1,A_1,T_2,A_2,\ldots]$, where each $T_i$ is a window of character-level text tokens, each $A_i$ is a speech segment of discretized mel-spectrogram tokens (dMel tokens) wrapped in beginning-of-speech and end-of-speech tokens, and the windows are fixed offline by a force aligner. The model is trained with a next-step prediction loss on speech tokens only, so it learns to emit an end-of-speech token as a natural chunk boundary. At inference, that end-of-speech token tells the decoder when to accept the next incoming text window, while the key-value cache preserves all prior text and speech context and a streaming causal vocoder converts each generated mel frame into a waveform with one-frame latency.

What would settle it

Create training data from the same speech with alignments deliberately shifted by 50 to 100 ms and retrain; if word error rate climbs well above the reported 3.38% or the end-of-speech tokens drift from audible word boundaries, the central claim that alignment-free streaming inference preserves quality would be refuted. A second check is to run SpeakStream on utterances rich in heteronyms and cross-word coarticulation, where the extra future-text words are supposed to matter; if quality does not drop when those words are removed, the lookahead mechanism is not doing the claimed work.

Watch

Extended reading notes

Core claim

SpeakStream's central discovery is that a decoder-only language model trained on interleaved text-speech chunks learns to generate speech incrementally without an explicit alignment mechanism at inference. The training sequence alternates text windows and speech segments, with a beginning-of-speech and end-of-speech token wrapping each audio chunk, and the loss is applied only to speech tokens. Each speech segment is predicted conditioned on the full preceding text and speech history, so short text windows do not degrade intelligibility. Text tokens are repeated across windows so the model always sees future context beyond the words it is currently speaking. The paper demonstrates that with a text window of five words and a speech hop of one word, word error rate matches the non-streaming baseline, and that replacing the vocoder with a fully causal streaming vocoder brings total first-phoneme latency below 50 ms.

Load-bearing premise

The whole streaming scheme depends on the force aligner's word-level boundaries being accurate enough that each text window and its speech chunk really correspond; if those alignments are noisy, the model trains on mismatched pairs and the learned end-of-speech boundaries at inference may not match the audio.

Editorial extensions

If this is right

  • A conversational agent can start playback while the language model is still composing the rest of the sentence, because SpeakStream only needs a five-word text window before generating audio.
  • The reported word error rate of 3.38% at a five-word window makes streaming quality comparable to the non-streaming full-sentence baseline in automatic evaluation.
  • The end-of-speech token learned during training removes the need for forced alignment or CTC-based chunking at inference time.
  • With a streaming vocoder, the TTS and vocoder together contribute roughly 40 to 45 ms of total latency, leaving room for the text-generation stage in an interactive loop.
  • The same interleaved representation can be reused for any decoder-only backbone, so the approach is not tied to one specific model architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, one testable extension is to apply the same interleaving recipe to multi-speaker dialogue, where overlapping turns would likely make the force-alignment boundaries harder to learn and would stress the model's reliance on chunk history.
  • An implicit consequence is that the repeated-text window acts as a tunable lookahead: window size $m$ and hop $n$ control not only latency but also how much future context the model can use, so applications with predictable sentence structure could shrink $m$ further.
  • If the end-of-speech token generalizes to punctuation and sentence boundaries, the model could be adapted to prosody-aware streaming, pausing at commas and periods without explicit rules.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. SpeakStream proposes a decoder-only streaming text-to-speech system that interleaves text and speech tokens using A3T force-alignments during training, treats the sequence as a next-token prediction task with loss only on speech tokens, and at inference streams text in while generating speech segments bounded by predicted EOS tokens. The paper reports WER as low as 3.38% (m=5, n=1) with the ParallelWaveGAN vocoder, human naturalness/coherence ratings comparable to non-streaming RichTTS, and a combined TTS+vocoder latency below 50 ms using a newly proposed streaming vocoder, VocStream. The central claim is that SpeakStream achieves state-of-the-art first-token latency while maintaining the quality of non-streaming TTS.

Significance. If the claims are fully supported, this is a valuable contribution: it shows a single decoder can perform streaming synthesis with quality close to non-streaming models, and the interleaving schemes with future context are a natural way to handle prosodic and polyphonic context. The paper includes extensive ablations over window and hop sizes, and the quality evaluations use an external ASR (WhisperX) plus human raters, which is good empirical practice. The demo website and the use of a Mac Mini for latency measurement are also useful. However, the headline latency-quality combination is not measured in any single configuration, and several important details about the low-latency vocoder and the latency metric are missing.

major comments (4)
  1. [§IV.B, §IV.D, Table III] The central claim that SpeakStream achieves low latency while maintaining non-streaming quality is not supported by any single configuration. All WER results (Table I) and human evaluations (Table II) are for SpeakStream decoded with ParallelWaveGAN, whose vocoder latency is about 150 ms (Table III). The low-latency numbers in Table III use VocStream, but no WER or human rating is reported for SpeakStream+VocStream. Moreover, Section IV.E says VocStream is trained on LibriTTS-R at 24 kHz, whereas SpeakStream and ParallelWaveGAN are trained on LJSpeech at 22 kHz, and the text does not describe how the 80-channel, 25 ms, 22 kHz dMel features are converted for the VocStream upsampler (which outputs 120-channel, 6.25 ms features) or how the 24 kHz output is reconciled with the 22 kHz training data. Please add quality evaluations for the SpeakStream+VocStream configuration (including the m=5, n=1 setting) or explicitly limit the claim to the ParallelWaveGAN-based system.
  2. [§IV.D and footnote 1] The latency metric is redefined in a way that may make the comparison with baselines unfair. The paper measures 'first phoneme' latency rather than the 'first byte of speech' used by tts-latency-benchmark, and it prompts the model with 200–400 ms of silence so that the first generated dMel frame already contains a speech pattern. This prompt is not described in the inference section (III.D), and its effect on the reported 40–45 ms total latency is unclear. Please report the unmodified first-byte latency as an additional metric, and specify the exact hardware, software, and metric definitions for the baseline numbers cited in Figure 1 so that the state-of-the-art claim can be verified.
  3. [§IV.B, Table I] The n-gram RichTTS and XTTS baselines are not described in enough detail to assess whether the comparison is fair. It is unclear whether these baselines receive the same future-context words (the (m−n) extra words) as SpeakStream, or whether they are evaluated with the same chunking and EOS-boundary protocol. Since the paper's claim of 'lowest error rate across all latency configurations' depends on this comparison, please specify the exact input construction for each baseline and, ideally, control for the amount of future context seen by each model.
  4. [§III.C, §III.D, Figure 3] The paper relies on A3T alignments to create the interleaved training data and on predicted EOS tokens to determine chunk boundaries at inference, but it provides no quantitative analysis of alignment or EOS prediction quality. While the strong WER numbers indirectly suggest that the alignments are reasonable, a direct evaluation (e.g., comparing predicted speech-chunk boundaries to A3T alignments on a held-out set) would strengthen the claim that the approach is robust to alignment noise and that the streaming inference scheme does not introduce misalignment artifacts.
minor comments (5)
  1. [Table I caption] The caption contains a typo: 'Groudtruth' should be 'Ground truth'.
  2. [§IV.E] The name 'V ocStream' has an inconsistent space in several places; it should be 'VocStream' throughout the text.
  3. [§IV.C] The human evaluation only covers the n=2 configurations (m=4 and m=6); it would be helpful to also report human ratings for the best WER configuration (m=5, n=1).
  4. [Figure 1 and references] The baseline latency values are attributed to 'tts-latency-benchmark', but this benchmark is not included in the reference list; please add a reference or a URL.
  5. [§IV.B] The paper observes that WER degrades for m>5 but does not explain why; a brief discussion of the trade-off between context length and sequence-length effects would be useful.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: external ASR and human evaluation anchor the quality claims, and no equation reduces a prediction to a fitted parameter. Minor self-citations to dMel/A3T are not load-bearing.

full rationale

SpeakStream's derivation chain is largely self-contained and empirically grounded. The WER numbers (Table I) are obtained by transcribing with external WhisperX against ground-truth WER 2.09, and human naturalness/coherence (Table II) uses 284 annotators with 95% confidence intervals; these are external benchmarks, not fitted quantities. The latency measurements (Table III) are direct measurements on Apple Silicon, not derived from an equation, and the comparison baselines come from an external benchmark. The main self-citations—dMel [9] and A3T [10]—are used as tokenizer and force-alignment tooling; they define the training representation but are not used as proof of the quality/latency claim, and the claim is validated independently. No equation in the paper reduces a predicted quantity to a fitted parameter, and no 'uniqueness theorem' from prior work is invoked to force the architecture. The one substantive gap is not circularity: the sub-50ms SpeakStream+VocStream configuration in Table III is never quality-tested, since all WER and human scores use ParallelWaveGAN (150ms), so the abstract's conjunction 'low latency while maintaining quality' is missing direct support. This is a correctness/completeness concern, not a circular derivation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper depends on its own prior pipeline (dMel, RichTTS architecture, A3T aligner) and on unverified assumptions about the streaming vocoder and ASR proxy. The central result rests on empirical measurements rather than derivation; the main ledger entries are the chosen interleaving hyperparameters (m, n) and the silence-prompting procedure used in latency measurement.

free parameters (3)
  • text window size m = 5
    Number of text words used as input before generating each speech chunk. The configuration (m=5,n=1) is selected from Table I as the best WER-latency trade-off; this is a post-hoc hyperparameter choice.
  • speech hop size n = 1
    Number of words covered by each generated speech segment. n=1 with m=5 gives the reported 3.38% WER.
  • silence prompt length = 200-400 ms
    In the latency benchmark, the model is prompted with synthetic silence so the first generated dMel frame contains an actual phoneme (Section IV.D). This is a measurement-time choice that affects the reported latency.
assumptions (5)
  • domain assumption A3T force aligner produces accurate word-to-speech alignments for creating interleaved training data.
    Invoked in Section III.B; alignment errors would corrupt the correspondence between text and speech segments and degrade the learned chunk boundaries.
  • domain assumption dMel discretization preserves enough acoustic information for high-quality speech synthesis.
    Speech tokens are dMel codes from the authors' prior work [9]; the quality ceiling of the system depends on this representation.
  • domain assumption WhisperX ASR word-error-rate is a valid proxy for TTS quality.
    All automatic evaluations rely on WhisperX transcriptions (Section IV.A); ASR errors can add noise to the WER comparisons.
  • domain assumption The decoder can learn to predict chunk boundaries via an EOS token and generalize to streaming text without an aligner at inference.
    Core to the inference scheme (Section III.D); unreliable EOS predictions would break the text-audio synchronization.
  • domain assumption VocStream's causal upsampler and vocoder reconstruct high-fidelity audio from a single Mel frame.
    Underlies the 13ms vocoder latency (Section IV.E); the paper does not report any quality metric for VocStream output.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SpeakStream: Streaming Text-to-Speech with Interleaved Data." pith.science (2026). https://pith.science/paper/4TCX6I3D

@misc{pith2026250519206,
  author       = {Pith},
  title        = {Pith review of: SpeakStream: Streaming Text-to-Speech with Interleaved Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TCX6I3D}},
  note         = {Machine review of arXiv:2505.19206}
}
read the original abstract

The latency bottleneck of traditional text-to-speech (TTS) systems fundamentally hinders the potential of streaming large language models (LLMs) in conversational AI. These TTS systems, typically trained and inferenced on complete utterances, introduce unacceptable delays, even with optimized inference speeds, when coupled with streaming LLM outputs. This is particularly problematic for creating responsive conversational agents where low first-token latency is critical. In this paper, we present SpeakStream, a streaming TTS system that generates audio incrementally from streaming text using a decoder-only architecture. SpeakStream is trained using a next-step prediction loss on interleaved text-speech data. During inference, it generates speech incrementally while absorbing streaming input text, making it particularly suitable for cascaded conversational AI agents where an LLM streams text to a TTS system. Our experiments demonstrate that SpeakStream achieves state-of-the-art latency results in terms of first-token latency while maintaining the quality of non-streaming TTS systems.

Figures

Figures reproduced from arXiv: 2505.19206 by the authors.

Figure 1
Figure 1. First-token to speech (time taken from the moment the LLM produces [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. SpeakStream transformer decoder architecture. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A 30-second Mel-spectrogram generated from the SpeakStream model. It corresponds to the audio in our demo, whose transcription is the abstract [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ChipChat: Low-Latency Cascaded Conversational Agent in MLX

    eess.AS 2025-08 conditional novelty 5.0 of 10

    ChipChat is a fully on-device cascaded voice agent that reports about 920 ms total latency using streaming ASR, a state-action LLM, streaming TTS, and a vocoder.

Reference graph

Works this paper leans on

35 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [8]

    Interleaved speech-text language models are simple streaming text to speech synthesizers,

    Y . Yang, Z. Ma, S. Liu, J. Li, H. Wang, L. Meng, H. Sun, Y . Liang, R. Xu, Y . Huet al., “Interleaved speech-text language models are simple streaming text to speech synthesizers,” arXiv preprint arXiv:2412.16102, 2024

    arXiv 2024

  2. [1]

    Audi- olm: a language modeling approach to audio generation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Shar- ifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi et al. , “Audi- olm: a language modeling approach to audio generation,” IEEE/ACM transactions on audio, speech, and language processing , vol. 31, pp. 2523–2533, 2023

    work page 2023

  3. [3]

    Qwen2.5-omni technical report,

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Dang, B. Zhang, X. Wang, Y . Chu, and J. Lin, “Qwen2.5-omni technical report,” arXiv preprint arXiv:2503.20215 , 2025

    arXiv 2025

  4. [4]

    Spirit-lm: Interleaved spoken and written language model,

    T. A. Nguyen, B. Muller, B. Yu, M. R. Costa-Jussa, M. Elbayad, S. Popuri, C. Ropers, P.-A. Duquenne, R. Algayres, R. Mavlyutov et al. , “Spirit-lm: Interleaved spoken and written language model,” Transactions of the Association for Computational Linguistics , vol. 13, pp. 30–52, 2025

    2025

  5. [5]

    Mmau: A massive multi- task audio understanding and reasoning benchmark,

    S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha, “Mmau: A massive multi- task audio understanding and reasoning benchmark,” arXiv preprint arXiv:2410.19168, 2024

    arXiv 2024

  6. [6]

    Zero- shot text-to-speech from continuous text streams,

    T. Dang, D. Aponte, D. Tran, T. Chen, and K. Koishida, “Zero- shot text-to-speech from continuous text streams,” arXiv preprint arXiv:2410.00767, 2024

    arXiv 2024

  7. [7]

    Speak while you think: Streaming speech synthesis during text generation,

    A. Dekel, S. Shechtman, R. Fernandez, D. Haws, Z. Kons, and R. Hoory, “Speak while you think: Streaming speech synthesis during text generation,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 11 931–11 935

    work page 2024

  8. [9]

    dmel: Speech tokenization made simple,

    H. Bai, T. Likhomanenko, R. Zhang, Z. Gu, Z. Aldeneh, and N. Jaitly, “dmel: Speech tokenization made simple,” arXiv preprint arXiv:2407.15835, 2024

    arXiv 2024

Show all 35 references
  1. [10]

    A 3T: Alignment-aware acoustic and text pretraining for speech synthesis and editing,

    H. Bai, R. Zheng, J. Chen, M. Ma, X. Li, and L. Huang, “A 3T: Alignment-aware acoustic and text pretraining for speech synthesis and editing,” in Proceedings of the 39th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri,...

    2022

  2. [11]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,

    E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. G ¨olge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,” in International Conference on Machine Learning. PMLR, 2022, pp. 2709–2720

    2022

  3. [12]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Ma et al. , “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  4. [13]

    E3 tts: Easy end-to- end diffusion-based text to speech,

    Y . Gao, N. Morioka, Y . Zhang, and N. Chen, “E3 tts: Easy end-to- end diffusion-based text to speech,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2023, pp. 1–8

    2023

  5. [14]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  6. [15]

    Tacotron: Towards end- to-end speech synthesis,

    Y . Wang, R. Skerry-Ryan, D. Stanton, Y . Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y . Xiao, Z. Chen, S. Bengio et al. , “Tacotron: Towards end- to-end speech synthesis,” arXiv preprint arXiv:1703.10135 , 2017

    2017 arXiv

  7. [16]

    (2024) Text-to-speech guide

    OpenAI. (2024) Text-to-speech guide. [Online]. Available: https: //platform.openai.com/docs/guides/text-to-speech

    2024

  8. [17]

    Streamspeech: Low-latency neural architecture for high-quality on-device speech synthesis,

    G. Shopov, S. Gerdjikov, and S. Mihov, “Streamspeech: Low-latency neural architecture for high-quality on-device speech synthesis,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023, pp. 1–5

    2023

  9. [18]

    MLX: Efficient and flexible machine learning on apple silicon,

    A. Hannun, J. Digani, A. Katharopoulos, and R. Collobert, “MLX: Efficient and flexible machine learning on apple silicon,” 2023. [Online]. Available: https://github.com/ml-explore

    2023

  10. [19]

    Vall-t: Decoder-only generative transducer for robust and decoding-controllable text-to-speech,

    C. Du, Y . Guo, H. Wang, Y . Yang, Z. Niu, S. Wang, H. Zhang, X. Chen, and K. Yu, “Vall-t: Decoder-only generative transducer for robust and decoding-controllable text-to-speech,” arXiv preprint arXiv:2401.14321, 2024

    2024 arXiv

  11. [20]

    Bert: a review of applications in natural language processing and understanding,

    M. V . Koroteev, “Bert: a review of applications in natural language processing and understanding,” arXiv preprint arXiv:2103.11943, 2021

    2021 arXiv

  12. [21]

    Non-attentive tacotron: Robust and controllable neural tts synthesis including unsupervised duration modeling,

    J. Shen, Y . Jia, M. Chrzanowski, Y . Zhang, I. Elias, H. Zen, and Y . Wu, “Non-attentive tacotron: Robust and controllable neural tts synthesis including unsupervised duration modeling,” arXiv preprint arXiv:2010.04301, 2020

    2010 arXiv

  13. [22]

    Livespeech: Low- latency zero-shot text-to-speech via autoregressive modeling of audio discrete codes,

    T. Dang, D. Aponte, D. Tran, and K. Koishida, “Livespeech: Low- latency zero-shot text-to-speech via autoregressive modeling of audio discrete codes,” arXiv preprint arXiv:2406.02897 , 2024

    2024 arXiv

  14. [23]

    Transduce and speak: Neural transducer for text-to-speech with semantic token predic- tion,

    M. Kim, M. Jeong, B. J. Choi, D. Lee, and N. S. Kim, “Transduce and speak: Neural transducer for text-to-speech with semantic token predic- tion,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–7

    2023

  15. [24]

    Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,

    R. Yamamoto, E. Song, and J.-M. Kim, “Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,” in ICASSP 2020-2020 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, ...

    2020

  16. [25]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in neural information processing systems , vol. 33, pp. 17 022–17 033, 2020

    2020

  17. [26]

    Bigvgan: A universal neural vocoder with large-scale training,

    S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigvgan: A universal neural vocoder with large-scale training,” in The Eleventh International Conference on Learning Representations

  18. [27]

    V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,

    H. Siuzdak, “V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,” in The Twelfth International Conference on Learning Representations

  19. [28]

    Non-causal to causal ssl-supported transfer learning: Towards a high-performance low-latency speech vocoder,

    R. Shi, A. B ¨ar, M. Sach, W. Tirry, and T. Fingscheidt, “Non-causal to causal ssl-supported transfer learning: Towards a high-performance low-latency speech vocoder,” in 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC) . IEEE, 2024, pp. 359–363

    2024

  20. [29]

    Coqui TTS: A deep learning toolkit for Text-to-Speech, battle-tested in research and production,

    E. G ¨olge and The Coqui TTS Team, “Coqui TTS: A deep learning toolkit for Text-to-Speech, battle-tested in research and production,” 1

  21. [30]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” https://keithito.com/ LJ-Speech-Dataset/, 2017

    2017

  22. [31]

    Whisperx: Time-accurate speech transcription of long-form audio,

    M. Bain, J. Huh, T. Han, and A. Zisserman, “Whisperx: Time-accurate speech transcription of long-form audio,” INTERSPEECH 2023, 2023

    2023

  23. [32]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” in International Conference on Machine Learning . PMLR, 2023, pp. 28 492–28 518

    2023

  24. [33]

    Libritts-r: A restored multi-speaker text-to-speech corpus,

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, and A. Bapna, “Libritts-r: A restored multi-speaker text-to-speech corpus,” in Proc. Interspeech 2023 , 2023, pp. 5496–5500

    2023

  25. [34]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

    2015

  26. [35]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar ´e, M. Orsini, A. Royer, P. P ´erez, H. J ´egou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” 2024. [Online]. Available: https: //arxiv.org/abs/2410.00037

    2024 arXiv

  27. [2021]

    Available: https://www.coqui.ai

    [Online]. Available: https://www.coqui.ai

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.