Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VoiceStar trains on 30-second clips and speaks clearly for 50 seconds by interpolating positions instead of extrapolating lengths.

desk verdict Strong empirical paper on duration-controllable, extrapolating zero-shot TTS; the 'first' claim needs checking against VALL-T before it can be taken at face value. read the letter →

arxiv 2505.19462 v2 pith:NI2WW3YT submitted 2025-05-26 eess.AS cs.SD

Puyuan Peng , Shang-Wen Li , Abdelrahman Mohamed , David Harwath This is my paper
classification eess.AScs.SD
keywords zero-shottext-to-speechneuralcodeclanguagemodeldurationcontrollengthextrapolationrotarypositionembeddingautoregressiveTTSspeechsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to give zero-shot text-to-speech two capabilities that neural codec language models have lacked together: precise control over how long the generated audio will be, and the ability to keep generating intelligible speech well beyond the durations seen in training. Its central claim is that both follow from one design change, a Progress-Monitoring Rotary Position Embedding (PM-RoPE) that replaces absolute token positions with fractional progress toward a target length. A second training scheme, continuation-prompt mixed (CPM) training, mixes real speech continuation with same-speaker different-utterance prompts to close the training/inference gap and improve intelligibility and speaker similarity. On long-form benchmarks from 20 to 50 seconds, the trained model reports word error rates near 12 percent at 40-50 seconds, where flow-matching baselines F5-TTS and MaskGCT reach 52 and 82 percent. The paper argues this makes it the first zero-shot TTS model with both duration control and extrapolation.

What carries the argument

Progress-Monitoring Rotary Position Embedding (PM-RoPE): a rotary position embedding in which the rotation angle of a token at position s in a sequence of length S is R((s/S) N theta) rather than R(s theta), so attention depends on the difference of fractional progress t/T - s/S. It carries the argument by (1) making cross-attention initially concentrate on tokens at the same relative position, giving a flat-start text-speech alignment; (2) encoding the desired target duration T into every timestep, so the model can learn to stop when progress reaches 100 percent; and (3) fixing all sequences into the same interval [0,N], so test-time length extrapolation becomes denser sampling and interpolation of that interval. The paper applies it to both the phoneme encoder and the acoustic decoder and reports that both are necessary for extrapolation.

What would settle it

Train the model with a hard 20-second context cap and ask it to generate 40-50 seconds of speech from the same prompts: if word error rate rises into the 35-82 percent range of the baselines rather than staying near the reported single digits, the extrapolation mechanism is not doing the work. A more direct check is to plot the cross-attention maps (as the paper's Figure 6 does) at inference-time angular positions N/T_test and test whether they lie within the distribution of training positions; large discontinuities would falsify the interpolation claim.

Watch

Extended reading notes

Core claim

VoiceStar is an autoregressive encoder-decoder neural codec language model. Its claim is that PM-RoPE turns length extrapolation into interpolation: instead of indexing positions by absolute token index, the encoder and decoder place every token at s/S times N and t/T times N, where S and T are the total text and speech sequence lengths and N is a fixed pseudo-length (2000 in the experiments). Because all sequences, however long, occupy the same fixed angular interval [0,N], a model trained on 20-30 second contexts can be asked at inference to produce 40-50 second speech by sampling more points inside that interval. The same progress signal also gives the model flat-start text-speech alignment and tells it when to emit the end-of-generation token, which is what makes requested output duration a conditioning variable. CPM training and prompt repetition then address intelligibility and speaker similarity. If the claims hold, the paper establishes that a single autoregressive codec language model can be both duration-controllable and extrapolative for zero-shot speech synthesis.

Load-bearing premise

The extrapolation claim rests on the untested assumption that the model's attention behavior interpolates smoothly to angular step sizes N/T_test finer than anything seen in training; if the learned attention is not smooth enough, generating beyond the training length will degrade.

Editorial extensions

If this is right

  • A user can specify output length directly and expect the model to stop near that target, with duration error at codec resolution (about 0.02 seconds) in the ablations.
  • Long-form zero-shot synthesis up to 50 seconds becomes feasible without training on long-form data: the model reports 7.27 percent WER at 30-40 seconds and 11.91 percent at 40-50 seconds, compared with 13.81 and 82.29 percent for MaskGCT.
  • Prompt repetition can raise speaker similarity without harming intelligibility, provided CPM training separates reference and target utterances with dedicated tokens.
  • An encoder-decoder NCLM with progress-based positions outperforms decoder-only continuation models on word error rate, naturalness, and duration control in the paper's ablations.
  • The model reaches parity with current best systems on short-form benchmarks (LibriSpeech-PC, Seed-TTS) while adding capabilities the baselines lack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes the pseudo-length N at 2000 but does not explore the trade-off: a smaller N compresses training positions and may hurt short-form alignment, while a larger N stretches the interval and may weaken the recency bias that makes the attention mechanism work.
  • The same progress-interpolation idea could apply to other autoregressive generation tasks where a source sequence and a target sequence have an unknown monotonic alignment, such as machine translation or speech-to-text, where a fixed interval [0,N] would turn length generalization into interpolation.
  • A testable extension is to vary N during training instead of fixing it, to see whether extrapolation robustness improves when the model sees multiple angular step sizes.
  • The paper's long-form numbers rely partly on ground-truth durations; its own appendix shows that an automatic duration estimator degrades WER (for example, from 11.91 to 17.33 at 40-50 seconds), so the practical strength of the extrapolation claim depends on having a good duration predictor.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. VoiceStar is an autoregressive encoder-decoder neural codec language model for zero-shot TTS. The paper introduces Progress-Monitoring Rotary Position Embedding (PM-RoPE), which encodes the fractional progress of source and target positions over a fixed pseudo-length N, and Continuation-Prompt Mixed (CPM) training, which mixes same-speaker different-utterance prompts with continuation-style prompting. The authors claim this is the first zero-shot TTS system with both output duration control and length extrapolation, report near-exact duration control (DurDiff 0.009 at codec resolution), and show large long-form WER gains over F5-TTS and MaskGCT at 30-50 s, supplemented by human MOS. The evaluation includes ablations, short-form comparisons on LibriSpeech-PC and Seed-TTS, and appendices on prompt repetition, estimated durations, and human-test instructions.

Significance. If the results hold, the paper makes a valuable engineering contribution: it demonstrates exact duration control through a positional-encoding mechanism, extrapolation from 30 s training data to 40-50 s generation, and improved long-form intelligibility while remaining competitive on short-form benchmarks. The ablation evidence is internally consistent, the human evaluation is reasonably thorough, and the authors explicitly disclose limitations (speaker similarity gap on long-form, slow generation). The duration-control evaluation is not circular: the model is conditioned on a target length T and could fail to follow it, so the measured DurDiff is informative. The main significance risk is the unverified 'first' claim, because the manuscript cites VALL-T as a decoding-controllable transducer TTS and never positions that system on the duration-control/extrapolation axes.

major comments (3)
  1. [Abstract; Section 2; Table 1] The central claim that VoiceStar is 'the first zero-shot TTS model that achieves both output duration control and extrapolation' is not supported by the prior-art discussion. Section 2 describes VALL-T [21] as a transducer-based TTS that 'implicitly learn[s] text-speech alignment' and is 'decoding-controllable', but VALL-T is absent from Table 1 and is never evaluated or positioned on either the duration-control or extrapolation axis. Because a transducer can in principle emit a variable number of acoustic tokens and can preserve monotonic alignment for long inputs, the cited description is consistent with both capabilities. The authors need to either include VALL-T in the conceptual comparison and in the duration/long-form experiments, or provide a concrete, cited argument for why VALL-T lacks one of the two capabilities. This is necessary to substantiate the headline novelty claim.
  2. [Section 3.1; Appendix A.1; Fig. 3] The extrapolation mechanism is presented as 'interpolation' over a fixed pseudo-interval [0, N], but the mathematical argument is incomplete. In Eq. (2) and Fig. 3, inference at T_test > T_train evaluates the attention function at angular positions t/T_test * N that are denser than any training step, so the mechanism works only if the learned attention function is smooth enough to interpolate at these unseen positions. No proof or analysis of this smoothness is provided. Given that extrapolation is a central contribution, the paper should add a concrete analysis, such as attention-logit plots across interpolated positions, a WER-vs-T sweep, or an explicit discussion of why the empirical results in Table 3 and Table 5 support the interpolation claim. The empirical evidence is strong, but the mechanism claim currently rests on an unexamined assumption.
  3. [Section 4.1; Section 4.4; Table 5] The long-form comparison protocol needs clarification. The paper states that 'we only compare against models that can also control the duration of their generations' and uses ground-truth durations, but it does not specify how the target duration T was supplied to F5-TTS and MaskGCT, whether the same T value was used for all models, or whether the baselines' duration-control mechanisms are comparable to VoiceStar's token-level control. This matters because the headline long-form gains could be affected by how the duration constraint is imposed. Please state the exact procedure (e.g., frames or seconds given to each model) and, if possible, report DurDiff for all models on the long-form sets, not only for the short-form ablation in Table 2.
minor comments (6)
  1. [Abstract; Section 3.1; Table 2] The abstract says 'much longer in duration than those seen during' and is missing the word 'training'; Section 3.1 contains the typo 'attention machenism', and Table 2 captions contain 'architecutre' and 'contiuation'.
  2. [Section 4.4; Conclusion] The conclusion that VoiceStar 'sets new state-of-the-art results on both short-form and long-form TTS benchmarks' is stronger than Tables 4 and 5 support. On Seed-TTS (en), VoiceStar has WER 2.15 vs F5-TTS 1.78 and SpkSim 0.63 vs 0.66, and on long-form SpkSim is often second. Please qualify the state-of-the-art claim by metric and benchmark.
  3. [Appendix C.3; Table 8] The estimated-duration results should be referenced in the main text, because at 40-50 s VoiceStar's WER rises from 11.91 with ground-truth duration to 17.33 with estimated duration. A reader of Table 5 alone may overestimate robustness to automatic duration estimation.
  4. [Section 4.1; Appendix C.1] The F5-TTS prompt-repetition comparison is not a controlled ablation because F5-TTS uses a different training recipe; the paper should state this more prominently and avoid implying a direct mechanistic comparison.
  5. [Section 4.1] WER numbers are reported without confidence intervals or significance tests; given that several short-form differences are within 0.1-0.5 WER, a bootstrap confidence interval or paired test would help the reader judge whether differences are meaningful.
  6. [Section 1; Abstract] The paper says 'Code and model weights will be open-sourced' but gives a GitHub link in the abstract; please clarify the license and the exact release plan for model weights.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: duration control and extrapolation are empirical conditioning results; the 'first' claim's prior-art gap is a novelty risk, not circularity.

full rationale

The paper's central claims are not equivalent to their inputs by construction. Duration control is a conditional generation task: PM-RoPE injects the target length T into the position angles (t/T and s/S), but the model must still learn to emit an end-of-generation token at t/T = 1. The ablation in Table 2 shows this is learned rather than forced, since the RoPE baseline has DurDiff 1.812 while PM-RoPE reaches 0.009, at the codec resolution floor. Extrapolation to 30-50s is an empirical claim tested against F5-TTS and MaskGCT on Libriheavy long-form data in Table 5; the paper openly states the underlying smoothness assumption ('we can extrapolate ... by sampling a larger number of points in the interval [0,N] and interpolating between them'), which is a correctness risk rather than a circular step. The self-citation to VoiceCraft [7] supplies the Encodec tokenizer and codebook training details; it is code-reproduced and not load-bearing for PM-RoPE or CPM training. The main non-circular weakness is the Abstract and Conclusion novelty assertion that VoiceStar is 'the first zero-shot TTS model' with both duration control and extrapolation: the paper cites VALL-T [21] as decoding-controllable but never evaluates it on the same axes, so the 'first' claim is under-supported by prior-art checking. That is a factual/novelty risk, not a reduction of a prediction to its inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central new capability rests on the hand-chosen pseudo-length N and on the untested smoothness of attention over fractional positions. CPM quality gains rest on the reliability of speaker labels and on stochastic training hyperparameters whose final values are not fully reported.

free parameters (4)
  • Pseudo sequence length N = 2000
    In PM-RoPE, both encoder and decoder positions are rescaled by N (Section 3.1). The extrapolation and duration control behavior depends on this hand-chosen value; no sensitivity study is reported.
  • CPM prompt probability p
    Probability of using a different same-speaker utterance instead of continuation (Section 3.2). Figure 4 shows WER varies with p, but the chosen value is not stated in the paper.
  • Speed augmentation probability p'
    Probability of applying speed perturbation to the reference prompt (Section 3.2). Figure 4 shows its effect; the final value is not stated.
  • Speed perturbation factor delta = 0.25
    Speed changes within 1 +/- 0.25 during prompt speed augmentation (Section 3.2).
assumptions (5)
  • domain assumption Sampling more points in the fixed interval [0,N] produces valid longer sequences at inference.
    Section 3.1 and Figure 3 state extrapolation is achieved by interpolating within [0,N]; no proof is given that the model generalizes to finer angular steps.
  • domain assumption The target token length T can be specified in advance and the model can learn to emit EOS when the fraction reaches 1.
    Duration control relies on conditioning on T during training; described in Section 3.1.
  • domain assumption Speech and phoneme sequences are monotonically aligned by relative progress.
    PM-RoPE's cross-attention flat-start alignment assumes an acoustic token at progress t/T should attend to the phoneme at progress s/S; Section 3.1 and Appendix A.2.
  • domain assumption Emilia's automatic speaker diarization labels are reliable enough for same-speaker pair sampling.
    Section 4.1 states the labels were found to be very reliable but provides no quantitative verification; CPM depends on pairing same-speaker utterances.
  • domain assumption Encodec 4-codebook 50Hz tokens are a sufficient speech representation for high-quality synthesis.
    VoiceStar tokenizes speech with VoiceCraft's Encodec model (Section 4.1); this is an inherited modeling choice, not a contribution of this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation." pith.science (2026). https://pith.science/paper/NI2WW3YT

@misc{pith2026250519462,
  author       = {Pith},
  title        = {Pith review of: VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NI2WW3YT}},
  note         = {Machine review of arXiv:2505.19462}
}
read the original abstract

We present VoiceStar, the first zero-shot TTS model that achieves both output duration control and extrapolation. VoiceStar is an autoregressive encoder-decoder neural codec language model, that leverages a novel Progress-Monitoring Rotary Position Embedding (PM-RoPE) and is trained with Continuation-Prompt Mixed (CPM) training. PM-RoPE enables the model to better align text and speech tokens, indicates the target duration for the generated speech, and also allows the model to generate speech waveforms much longer in duration than those seen during. CPM training also helps to mitigate the training/inference mismatch, and significantly improves the quality of the generated speech in terms of speaker similarity and intelligibility. VoiceStar outperforms or is on par with current state-of-the-art models on short-form benchmarks such as Librispeech and Seed-TTS, and significantly outperforms these models on long-form/extrapolation benchmarks (20-50s) in terms of intelligibility and naturalness. Code and models: https://github.com/jasonppy/VoiceStar. Audio samples: https://jasonppy.github.io/VoiceStar_web

Figures

Figures reproduced from arXiv: 2505.19462 by the authors.

Figure 1
Figure 1. WER comparison between our VOICESTAR and F5-TTS [3] under dif￾ferent context lengths. Both models are trained with maximal context length of 30 seconds. Next, a Transformer [4] language model is used to autore￾gressively model these token sequences. The success of this approach is due to combination of the modeling power of Transformer language models and the ease of recon￾structing high-fidelity waveforms from the … view at source ↗
Figure 2
Figure 2. Left: The architecture of VOICESTAR. Right: the common general architecture for zero-shot TTS models, such as VALL-E (AR part), VoiceCraft, CosyVoice, FireRedTTS, Llasa etc. VOICESTAR differs from them in three aspects: 1) it uses an encoder-decoder architecture with PM-RoPE to provide text-speech alignment, duration control, and extrapolation capability; 2) it uses prompt-continuation mixed training to mitigate the… view at source ↗
Figure 3
Figure 3. An example on how PM-ROPE turns extrapolation into interpolation: during training, the maximal training sequence length is 4, and dur￾ing inference the target length is 7. The positional encodings for both can be expressed as sampling points inside the same interval [0, N]. We found it to be necessary to apply PM￾ROPE to the encoder’s phonetic input sequence to achieve extrapolation, which we believe is due to the f… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Effects of prompt-prob and speed augmenta [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: shows the impact of prompt repetition on generation quality, measured by WER (left in red) and SpkSim (right in blue). The red and blue dotted horizontal lines indi￾cate the performance when we repeat prompt for each sample so that the context length (i.e. prompt + gen…
Figure 6
Figure 6. Figure 6: attention maps A Model Details A.1 RoPE and PM-RoPE for higher dimensions For higher dimensions, rotation matrix is defined by dividing the space into D/2 2-dimensional spaces, and applying a 2-dimensional rotation matrix (with different θs) to each space. Mathematical…
Figure 7
Figure 7. Figure 7: Comparison of the impact of prompt repetition on [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: screenshots for comparative speech naturalness human evaluation. There are seven options: [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: screenshots for comparative speaker similarity human evaluation. There are also seven [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: screenshots for speech naturalness human evaluation. There are five options, Excellent, [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: screenshots for speech intelligibility human evaluation. There are five options, Excellent, [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: screenshots for speech intelligibility human evaluation. There are five options, Excellent, [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models

    cs.CL 2026-04 conditional novelty 6.0 of 10

    Synthetic data for low-resource spoken language models creates a Stability-Expressivity Gap that DGSA and TDSC self-alignment close, enabling SOTA Thai TTS and first Lao zero-shot voice cloning.

Reference graph

Works this paper leans on

80 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [21]

    Vall-t: Decoder-only generative transducer for robust and decoding- controllable text-to-speech

    Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen, and Kai Yu. Vall-t: Decoder-only generative transducer for robust and decoding- controllable text-to-speech. 2024

    work page 2024

  2. [1]

    Soundstream: An end-to-end neural audio codec.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507, 2021

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. Soundstream: An end-to-end neural audio codec.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507, 2021

    2021

  3. [2]

    High fidelity neural audio compression.ArXiv, abs/2210.13438, 2022

    Alexandre Defossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression.ArXiv, abs/2210.13438, 2022

    arXiv 2022

  4. [3]

    F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching.ArXiv, abs/2410.06885, 2024

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching.ArXiv, abs/2410.06885, 2024

    arXiv 2024

  5. [4]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeural Information Processing Systems, 2017

    2017

  6. [5]

    Neural codec language models are zero-shot text to speech synthesizers.ArXiv, abs/2301.02111, 2023

    Chengyi Wang, Sanyuan Chen, Yu Wu, Zi-Hua Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Neural codec language models are zero-shot text to speech synthesizers.ArXiv, abs/2301.02111, 2023

    arXiv 2023

  7. [6]

    Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers, 2024

    Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers, 2024

    work page 2024

  8. [7]

    V oicecraft: Zero-shot speech editing and text-to-speech in the wild

    Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, and David Harwath. V oicecraft: Zero-shot speech editing and text-to-speech in the wild. InAnnual Meeting of the Association for Computational Linguistics, 2024

    work page 2024

Show all 80 references
  1. [8]

    On generative spoken language modeling from raw audio.Transactions of the Association for Computational Linguistics, 9:1336–1354, 2021

    Kushal Lakhotia, Evgeny Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu Nguyen, Jade Copet, Alexei Baevski, Adel Ben Mohamed, and Emmanuel Dupoux. On generative spoken language modeling from raw audio.Transactions of the Association for Computational Lingu...

    2021

  2. [9]

    Audiolm: A language modeling approach to audio generation.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2523–2533, 2022

    Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matthew Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. Audiolm: A language modeling approach to audio generation.IEEE/ACM Transactions on Audi...

    2022

  3. [10]

    Audiogen: Textually guided audio generation

    Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Defossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. Audiogen: Textually guided audio generation. ArXiv, abs/2209.15352, 2022

    2022 arXiv

  4. [11]

    Speak, read and prompt: High-fidelity text-to-speech with minimal supervision.Transactions of the Association for Computational Linguistics, 11:1703–1718, 2023

    Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matthew Sharifi, Marco Tagliasacchi, and Neil Zeghidour. Speak, read and prompt: High-fidelity text-to-speech with minimal supervision.Transactions of the Association for Comput...

    2023

  5. [12]

    Soundstorm: Efficient parallel audio generation.ArXiv, abs/2305.09636, 2023

    Zalán Borsos, Matthew Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. Soundstorm: Efficient parallel audio generation.ArXiv, abs/2305.09636, 2023

    2023 arXiv

  6. [13]

    V oicebox: Text-guided multilingual universal speech generation at scale.ArXiv, abs/2306.15687, 2023

    Matt Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. V oicebox: Text-guided multilingual universal speech generation at scale.ArXiv, abs/2306.15687, 2023

    2023 arXiv

  7. [14]

    Maskgct: Zero-shot text-to-speech with masked generative codec transformer.ArXiv, abs/2409.00750, 2024

    Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. Maskgct: Zero-shot text-to-speech with masked generative codec transformer.ArXiv, abs/2409.00750, 2024

    2024 arXiv

  8. [15]

    Eric Battenberg, R. J. Skerry-Ryan, Daisy Stanton, Soroosh Mariooryad, Matt Shannon, Julian Salazar, and David Kao. Very attentive tacotron: Robust and unbounded length generalization in autoregressive transformer-based text-to-speech.ArXiv, abs/2410.22179, 2024. 10

    arXiv

  9. [16]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens.ArXiv, abs/2407.05407, 2024

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens.ArXiv, abs/2407.05407, 2024

    2024 arXiv

  10. [17]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models.ArXiv, abs/2412.10117, 2024

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jing-Ru Zhou. Cosyvoice 2: Scalable streaming speech synthesis ...

    2024 arXiv

  11. [18]

    Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis

    Zhen Ye, Xinfa Zhu, Chi min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, Hongzhan Lin, Jianyi Chen, Xingjian Du, Liumeng Xue, Yunlin Chen, Zhifei Li, Lei Xie, Qiuqiang Kong, Yi-Ting Guo, and Wei Xue. Llasa: Scaling train-time and inference-...

    2025

  12. [19]

    Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering

    Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering. 2024

    2024

  13. [20]

    Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment.ArXiv, abs/2406.07855, 2024

    Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanmin Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment.ArXiv, abs/2406.07855, 2024

    2024 arXiv

  14. [22]

    Attention- constrained inference for robust decoder-only text-to-speech.2024 IEEE Spoken Language Technology Workshop (SLT), pages 630–637, 2024

    Hankun Wang, Chenpeng Du, Yiwei Guo, Shuai Wang, Xie Chen, and Kai Yu. Attention- constrained inference for robust decoder-only text-to-speech.2024 IEEE Spoken Language Technology Workshop (SLT), pages 630–637, 2024

    2024

  15. [23]

    Improving robustness of llm-based speech synthesis by learning monotonic alignment.ArXiv, abs/2406.17957, 2024

    Paarth Neekhara, Shehzeen Samarah Hussain, Subhankar Ghosh, Jason Li, Rafael Valle, Rohan Badlani, and Boris Ginsburg. Improving robustness of llm-based speech synthesis by learning monotonic alignment.ArXiv, abs/2406.17957, 2024

    2024 arXiv

  16. [24]

    Haohan Guo, Fenglong Xie, Dongchao Yang, Xixin Wu, and Helen M. Meng. Speaking from coarse to fine: Improving neural codec language model via multi-scale speech coding and generation.ArXiv, abs/2409.11630, 2024

    2024 arXiv

  17. [25]

    Mega-TTS 2: Boosting prompting mechanisms for zero-shot speech synthesis

    Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, Xiang Yin, Zejun MA, and Zhou Zhao. Mega-TTS 2: Boosting prompting mechanisms for zero-shot speech synthesis. InThe Twelfth International Conference on L...

    2024

  18. [26]

    Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis.ArXiv, abs/2409.07556, 2024

    Helin Wang, Meng Yu, Jiarui Hai, Chen Chen, Yuchen Hu, Rilin Chen, Najim Dehak, and Dong Yu. Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis.ArXiv, abs/2409.07556, 2024

    2024 arXiv

  19. [27]

    Rall-e: Robust codec language modeling with chain-of-thought prompting for text-to-speech synthesis.ArXiv, abs/2404.03204, 2024

    Detai Xin, Xu Tan, Kai Shen, Zeqian Ju, Dongchao Yang, Yuancheng Wang, Shinnosuke Takamichi, Hiroshi Saruwatari, Shujie Liu, Jinyu Li, and Sheng Zhao. Rall-e: Robust codec language modeling with chain-of-thought prompting for text-to-speech synthesis.ArXiv, abs/2404.03204, 2024

    2024 arXiv

  20. [28]

    Enhancing zero-shot text-to-speech synthesis with human feedback.ArXiv, abs/2406.00654, 2024

    Chen Chen, Yuchen Hu, Wen Wu, Helin Wang, Chng Eng Siong, and Chao Zhang. Enhancing zero-shot text-to-speech synthesis with human feedback.ArXiv, abs/2406.00654, 2024

    2024 arXiv

  21. [29]

    Robust zero-shot text-to-speech synthesis with reverse inference optimization.ArXiv, abs/2407.02243, 2024

    Yuchen Hu, Chen Chen, Siyin Wang, Chng Eng Siong, and Chao Zhang. Robust zero-shot text-to-speech synthesis with reverse inference optimization.ArXiv, abs/2407.02243, 2024

    2024 arXiv

  22. [30]

    Desta, Roy Fejgin, Rafael Valle, and Jason Li

    Shehzeen Samarah Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Mikyas T. Desta, Roy Fejgin, Rafael Valle, and Jason Li. Koel-tts: Enhancing llm based speech generation with preference alignment and classifier free guidance. 2025

    2025

  23. [31]

    Speak foreign languages with your own voice: Cross-lingual neural codec language modeling.ArXiv, abs/2303.03926, 2023

    Zi-Hua Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yan- qing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling.ArXiv, abs/2303.03926, 2023. 11

    2023 arXiv

  24. [32]

    Generative pre-trained speech language model with efficient hierarchical transformer.ArXiv, abs/2406.00976, 2024

    Yongxin Zhu, Dan Su, Liqiang He, Linli Xu, and Dong Yu. Generative pre-trained speech language model with efficient hierarchical transformer.ArXiv, abs/2406.00976, 2024

    2024 arXiv

  25. [33]

    Viola: Unified codec language models for speech recognition, synthesis, and translation.ArXiv, abs/2305.16107, 2023

    Tianrui Wang, Long Zhou, Zi-Hua Zhang, Yu Wu, Shujie Liu, Yashesh Gaur, Zhuo Chen, Jinyu Li, and Furu Wei. Viola: Unified codec language models for speech recognition, synthesis, and translation.ArXiv, abs/2305.16107, 2023

    2023 arXiv

  26. [34]

    Lauragpt: Listen, attend, understand, and regenerate audio with gpt.ArXiv, abs/2310.04673, 2023

    Jiaming Wang, Zhihao Du, Qian Chen, Yunfei Chu, Zhifu Gao, Zerui Li, Kai Hu, Xiaohuan Zhou, Jin Xu, Ziyang Ma, Wen Wang, Siqi Zheng, Chang Zhou, Zhijie Yan, and Shiliang Zhang. Lauragpt: Listen, attend, understand, and regenerate audio with gpt.ArXiv, abs/2310.04673, 2023

    arXiv

  27. [35]

    Soumi Maiti, Yifan Peng, Shukjae Choi, Jee weon Jung, Xuankai Chang, and Shinji Watanabe. V oxtlm: Unified decoder-only models for consolidating speech recognition, synthesis and speech, text continuation tasks.ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Spe...

    2024

  28. [36]

    Speechcomposer: Unifying multiple speech tasks with prompt composition.ArXiv, abs/2401.18045, 2024

    Yihan Wu, Soumi Maiti, Yifan Peng, Wangyou Zhang, Chenda Li, Yuyue Wang, Xihua Wang, Shinji Watanabe, and Ruihua Song. Speechcomposer: Unifying multiple speech tasks with prompt composition.ArXiv, abs/2401.18045, 2024

    2024 arXiv

  29. [37]

    Metis: A foundation speech generation model with masked generative pre-training

    Yuancheng Wang, Jiachen Zheng, Junan Zhang, Xueyao Zhang, Huan Liao, and Zhizheng Wu. Metis: A foundation speech generation model with masked generative pre-training. 2025

    2025

  30. [38]

    Prompttts: Controllable text-to-speech with text descriptions.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2022

    Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xuejiao Tan. Prompttts: Controllable text-to-speech with text descriptions.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2022

    2023

  31. [39]

    Meng, and Dong Yu

    Dongchao Yang, Songxiang Liu, Rongjie Huang, Guangzhi Lei, Chao Weng, Helen M. Meng, and Dong Yu. Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt.ArXiv, abs/2301.13662, 2023

    2023 arXiv

  32. [40]

    Promptstyle: Controllable style transfer for text-to-speech with natural language descriptions

    Guanghou Liu, Yongmao Zhang, Yinjiao Lei, Yunlin Chen, Rui Wang, Zhifei Li, and Linfu Xie. Promptstyle: Controllable style transfer for text-to-speech with natural language descriptions. ArXiv, abs/2305.19522, 2023

    2023 arXiv

  33. [41]

    Textrolspeech: A text style control speech corpus with codec language text-to-speech models.ArXiv, abs/2308.14430, 2023

    Shengpeng Ji, Jia li Zuo, Minghui Fang, Ziyue Jiang, Feiyang Chen, Xinyu Duan, Baoxing Huai, and Zhou Zhao. Textrolspeech: A text style control speech corpus with codec language text-to-speech models.ArXiv, abs/2308.14430, 2023

    2023 arXiv

  34. [42]

    Prompttts 2: Describing and generating voices with text prompt.ArXiv, abs/2309.02285, 2023

    Yichong Leng, Zhifang Guo, Kai Shen, Xu Tan, Zeqian Ju, Yanqing Liu, Yufei Liu, Dongchao Yang, Leying Zhang, Kaitao Song, Lei He, Xiang-Yang Li, Sheng Zhao, Tao Qin, and Jiang Bian. Prompttts 2: Describing and generating voices with text prompt.ArXiv, abs/2309.02285, 2023

    2023 arXiv

  35. [43]

    Natural language guidance of high-fidelity text-to-speech with synthetic annotations.ArXiv, abs/2402.01912, 2024

    Daniel Lyth and Simon King. Natural language guidance of high-fidelity text-to-speech with synthetic annotations.ArXiv, abs/2402.01912, 2024

    2024 arXiv

  36. [44]

    Scaling rich style-prompted text-to-speech datasets

    Anuj Diwan, Zhisheng Zheng, David Harwath, and Eunsol Choi. Scaling rich style-prompted text-to-speech datasets. 2025

    2025

  37. [45]

    Moshi: a speech-text foundation model for real-time dialogue.ArXiv, abs/2410.00037, 2024

    Alexandre D’efossez, Laurent Mazar’e, Manu Orsini, Am’elie Royer, Patrick P’erez, Herv’e J’egou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dialogue.ArXiv, abs/2410.00037, 2024

    2024 arXiv

  38. [46]

    Language model can listen while speaking.ArXiv, abs/2408.02622, 2024

    Ziyang Ma, Ya-Zhen Song, Chenpeng Du, Jian Cong, Zhuo Chen, Yuping Wang, Yuxuan Wang, and Xie Chen. Language model can listen while speaking.ArXiv, abs/2408.02622, 2024

    2024 arXiv

  39. [47]

    Llama- omni: Seamless speech interaction with large language models.ArXiv, abs/2409.06666, 2024

    Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. Llama- omni: Seamless speech interaction with large language models.ArXiv, abs/2409.06666, 2024

    2024 arXiv

  40. [48]

    Salmonn-omni: A codec-free llm for full-duplex speech understanding and generation.ArXiv, abs/2411.18138, 2024

    Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, and Chao Zhang. Salmonn-omni: A codec-free llm for full-duplex speech understanding and generation.ArXiv, abs/2411.18138, 2024. 12

    2024 arXiv

  41. [49]

    Enabling real-time conversations with minimal training costs.ArXiv, abs/2409.11727, 2024

    Wang Xu, Shuo Wang, Weilin Zhao, Xu Han, Yukun Yan, Yudi Zhang, Zhe Tao, Zhiyuan Liu, and Wanxiang Che. Enabling real-time conversations with minimal training costs.ArXiv, abs/2409.11727, 2024

    2024 arXiv

  42. [50]

    Intrinsicvoice: Empowering llms with intrinsic real-time voice interaction abilities.ArXiv, abs/2410.08035, 2024

    Xin Zhang, Xiang Lyu, Zhihao Du, Qian Chen, Dong Zhang, Hangrui Hu, Chaohong Tan, Tianyu Zhao, Yuxuan Wang, Bin Zhang, Heng Lu, Yaqian Zhou, and Xipeng Qiu. Intrinsicvoice: Empowering llms with intrinsic real-time voice interaction abilities.ArXiv, abs/2410.08035, 2024

    arXiv

  43. [51]

    Llm-enhanced dialogue management for full-duplex spoken dialogue systems

    Hao Zhang, Weiwei Li, Rilin Chen, Vinay Kothapally, Meng Yu, and Dong Yu. Llm-enhanced dialogue management for full-duplex spoken dialogue systems. 2025

    2025

  44. [52]

    Base tts: Lessons from building a billion-parameter text-to-speech model on 100k hours of data.ArXiv, abs/2402.08093, 2024

    Mateusz Lajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszy’nska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokol...

    2024 arXiv

  45. [53]

    Hall-e: Hierarchical neural codec language model for minute-long zero-shot text-to-speech synthesis

    Yuto Nishimura, Takumi Hirose, Masanari Ohi, Hideki Nakayama, and Nakamasa Inoue. Hall-e: Hierarchical neural codec language model for minute-long zero-shot text-to-speech synthesis. ArXiv, abs/2410.04380, 2024

    2024 arXiv

  46. [54]

    Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers.ArXiv, abs/2304.09116, 2023

    Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers.ArXiv, abs/2304.09116, 2023

    2023 arXiv

  47. [55]

    Shih, Rohan Badlani, João Felipe Santos, Evelina Bakhturina, Mikyas T

    Sungwon Kim, Kevin J. Shih, Rohan Badlani, João Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, and Bryan Catanzaro. P-flow: A fast and data-efficient zero-shot tts through speech prompting. InNeural Information Processing Systems, 2023

    2023

  48. [56]

    Ditto-tts: Diffusion transformers for scalable text-to-speech without domain-specific factors

    Keon Lee, Dong Won Kim, Jaehyeon Kim, Seungjun Chung, and Jaewoong Cho. Ditto-tts: Diffusion transformers for scalable text-to-speech without domain-specific factors. 2024

    2024

  49. [57]

    Peebles and Saining Xie

    William S. Peebles and Saining Xie. Scalable diffusion models with transformers.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4172–4182, 2022

    2023

  50. [58]

    Dmospeech: Direct metric optimization via distilled diffusion model in zero-shot speech synthesis

    Yingahao Aaron Li, Rithesh Kumar, and Zeyu Jin. Dmospeech: Direct metric optimization via distilled diffusion model in zero-shot speech synthesis. 2024

    2024

  51. [59]

    E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts.2024 IEEE Spoken Language Technology Workshop (SLT), pages 682–689, 2024

    Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, Yanqing Liu, Sheng Zhao, and Naoyuki Kanda. E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts.2024 IEEE Spoken Language Technolo...

    2024

  52. [60]

    Apoorv Vyas, Bowen Shi, Matt Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, W.K.F. Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Tunde Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, Chris Summers...

    2023 arXiv

  53. [61]

    Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications.ArXiv, abs/2409.03283, 2024

    Hao-Han Guo, Kun Liu, Fei-Yu Shen, Yi-Chen Wu, Fenglong Xie, Kun Xie, and Kai-Tuo Xu. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications.ArXiv, abs/2409.03283, 2024

    2024 arXiv

  54. [62]

    Simple and controllable music generation.ArXiv, abs/2306.05284, 2023

    Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Defossez. Simple and controllable music generation.ArXiv, abs/2306.05284, 2023

    2023 arXiv

  55. [63]

    Neural machine translation by jointly learning to align and translate, 2016

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate, 2016

    2016

  56. [64]

    Roformer: Enhanced transformer with rotary position embedding, 2023

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2023. 13

    2023

  57. [65]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023

    2023

  58. [66]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023

    2023

  59. [67]

    Smith, and Mike Lewis

    Ofir Press, Noah A. Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation, 2022

    2022

  60. [68]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation.2024 IEEE Spoken Language Technology Workshop (SLT), pages 885–890, 2024

    Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation.20...

    2024

  61. [69]

    Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey. Libriheavy: A 50,000 hours asr corpus with punctuation casing and context.ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)...

    2024

  62. [70]

    eSpeak NG: Speech synthesiser.https://github.com/espeak-ng/espeak-ng

    Jonathan Duddington, Reece Dunn, and the eSpeak NG Community. eSpeak NG: Speech synthesiser.https://github.com/espeak-ng/espeak-ng. Accessed: March 15, 2025

    2025

  63. [71]

    Seed-tts: A family of high-quality versatile speech generation models.ArXiv, abs/2406.02430, 2024

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, Jiaxin Li, Xiaoyang Li, Xingxing Li...

    2024 arXiv

  64. [72]

    Zipformer: A faster and better encoder for automatic speech recognition

    Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. Zipformer: A faster and better encoder for automatic speech recognition. InICLR, 2024

    2024

  65. [73]

    Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens

    Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, Weizhen Bian, Zhen Ye, Sitong Cheng, Ruibin Yuan, Zhixian Zhao, Xinfa Zhu, Jiahao Pan, Liumeng Xue, Pengcheng Zhu, Yunlin Chen, Zhifei Li, Xie Chen, L...

    2025

  66. [74]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022, 2022

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. Utmos: Utokyo-sarulab system for voicemos challenge 2022, 2022

    2022

  67. [75]

    Robust speech recognition via large-scale weak supervision.ArXiv, abs/2212.04356, 2022

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision.ArXiv, abs/2212.04356, 2022

    2022 arXiv

  68. [76]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16:1505–1518, 2021

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Micheal Zeng, and Furu Wei. Wavlm: Large-scale self-supervised pre-training for full stack speec...

    2021

  69. [77]

    Speech quality assessment

    Philipos C Loizou. Speech quality assessment. InMultimedia analysis, processing and communications, pages 623–654. Springer, 2011

    2011

  70. [78]

    Llm.int8(): 8-bit matrix multiplication for transformers at scale, 2022

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int8(): 8-bit matrix multiplication for transformers at scale, 2022

    2022

  71. [79]

    Fast and high-quality auto- regressive speech synthesis via speculative decoding

    Bohan Li, Hankun Wang, Situo Zhang, Yiwei Guo, and Kai Yu. Fast and high-quality auto- regressive speech synthesis via speculative decoding. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025. 14

    2025

  72. [80]

    natural” in comparative naturalness task with “similar

    Tan Dat Nguyen, Ji-Hoon Kim, Jeongsoo Choi, Shukjae Choi, Jinseok Park, Younglo Lee, and Joon Son Chung. Accelerating codec-based speech synthesis with multi-token prediction and speculative decoding. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Si...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.