Pith. sign in

REVIEW 5 major objections 5 minor 37 references

Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Speech Speculative Decoding (SSD) accelerates autoregressive speech synthesis by having a small draft model propose token sequences that a large frozen target model verifies in parallel, yielding a 1.4x speedup on CosyVoice 2 with matched…

desk verdict Useful speedup, but the beta=0 control contradicts the paper's own theory and the fidelity claim needs a much closer look. read the letter →

arxiv 2505.15380 v2 pith:UMHCAPBW submitted 2025-05-21 cs.SD cs.AIeess.AS

Zijian Lin , Yang Zhang , Yougen Yuan , Yuming Yan , Jinjiang Liu , Zhiyong Wu , Pengfei Hu , Qun Yu This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords speechsynthesisspeculativedecodingautoregressiveTTSinferenceaccelerationdraftmodeltolerancefactorCosyVoice2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speech Speculative Decoding (SSD) aims to cut the latency of autoregressive text-to-speech by replacing strict one-token-at-a-time generation with a two-model pipeline: a small draft model predicts several speech tokens, and the large target model scores them all in a single forward pass. The paper's central claim is that this yields a 1.4x speedup in the language-model stage of CosyVoice 2 while keeping naturalness, speaker similarity, and subjective ratings effectively unchanged. The key adaptation to speech is a tolerance factor inside the acceptance test, which lets the system keep draft tokens that the target model only mildly disagrees with; the paper justifies this by arguing that speech tokens are not one-to-one with linguistic units, so several different token strings decode to nearly the same audio. A reader should care because it suggests autoregressive TTS can be accelerated without retraining or restructuring the large model, and without replacing it with a non-autoregressive architecture.

What carries the argument

The load-bearing object is the modified acceptance criterion of Eq. (5), accept $x_i$ when $r_i < \min(1, q_i^s/p_i^d)+\beta$, together with the resampling rule of Eq. (6), $x_i' \sim \mathrm{normalize}(\max(0, q_i^s - p_i^d))$. What it does is convert the draft model's speculative guesses into verified output: the target model's parallel pass produces a score for every proposed position, and $\beta$ relaxes the strict probability-ratio test so that acoustically plausible alternatives are not rejected. The second load-bearing piece is the draft model itself, a shallower copy of the target that inherits most of its layers, which lets a small amount of training align it with the target's token vocabulary.

What would settle it

Take the positions where beta=0.4 leads to acceptance of a token the target model gave lower probability, resynthesize the audio with and without those tokens (i.e., SSD output versus target-model-only output on the same prompt), and run a forced-choice listening test or measure a spectral distance such as mel-cepstral distance between the two waveform segments; a consistent audible or measurable difference at those positions would show the tolerance factor is buying speed with quality.

Watch

Extended reading notes

Core claim

SSD keeps CosyVoice 2 frozen and adds an independently trained 8-layer draft model whose parameters come from the target's own layers (lower two plus upper six), with only the first two layers and the classification head updated on LibriTTS. At inference the draft model autoregressively emits $L_d=3$ candidate tokens; the target model processes the whole sequence in parallel and accepts token $x_i$ when a uniform draw falls below $\min(1, q_i/p_i)+\beta$ with $\beta=0.4$, resampling rejected tokens from the distribution proportional to $\max(0, q_i-p_i)$. On the LibriTTS test set the language-model real-time factor drops from 0.504 to 0.360 (about 1.4x), word error rate rises from 3.67% to 5.70%, and NMOS (3.96 to 3.94) and speaker similarity (0.62 to 0.63) stay effectively flat; subjective similarity and naturalness MOS values land within the confidence bounds of the target model. The paper's central assertion is that the tolerance factor is safe precisely because speech tokens are perceptually many-to-one: the extra accepted tokens still reconstruct nearly the same waveform.

Load-bearing premise

The claim stands on the assumption that the extra draft tokens admitted by the relaxed acceptance rule are perceptually interchangeable with the target model's own token choices; if those tokens are audible differences, the quality-preservation claim collapses.

Editorial extensions

If this is right

  • A frozen, off-the-shelf autoregressive TTS model can be accelerated by training only a small auxiliary model on a small dataset; no target-model fine-tuning is required.
  • The tolerance factor beta acts as a speed-quality dial: increasing it from 0 to 0.4 lowers the language-model real-time factor from 0.509 to 0.360 while reported objective quality metrics stay nearly constant.
  • Draft model quality does not need to be high for the final output to be good; even a draft model with 16.13% WER yields 5.70% WER after SSD verification, because errors are caught in parallel.
  • The 1.4x language-model speedup, combined with the paper's measurement that the LM is about 70% of total synthesis time, implies a smaller but still meaningful end-to-end latency reduction for LM-based TTS systems.
  • The layer-inheritance recipe is presented as a general way to build draft models for other autoregressive speech models that share the target architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper does not quantify the distributional shift that beta causes; a direct measure of per-token divergence between the target distribution and the accepted draft distribution would show where the WER increase of 3.67% to 5.70% actually comes from.
  • Editorial inference: beta relaxes the acceptance threshold uniformly, so tokens the target strongly disagrees with can still be accepted whenever the uniform draw is small but below the inflated threshold; testing on minimal pairs or prosody-critical sentences would reveal whether the added WER is concentrated on semantically important tokens.
  • Editorial inference: the paper's explanation for the WER increase, limited draft-model training data and extrapolation, is asserted rather than demonstrated; an ablation with beta=0 and a better-trained draft would separate the effect of the tolerance factor from the effect of draft-model coverage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Speech Speculative Decoding (SSD), an adaptation of speculative decoding for autoregressive speech synthesis. A lightweight 8-layer draft model, initialized from the upper layers of the target CosyVoice 2 model and trained on LibriTTS, proposes a block of L_d candidate speech tokens; the target model verifies the block in parallel, and tokens are accepted using a modified criterion that includes a tolerance factor beta. Experiments on a 500-utterance LibriTTS test set report an LM-RTF improvement from 0.504 to 0.360 (1.4x) with WER increasing from 3.67% to 5.70%, while SS and NMOS remain similar and subjective MOS overlap with the baseline. The paper claims that SSD maintains high fidelity and naturalness while accelerating inference.

Significance. If the quality-preservation claim held, SSD would be a practical and simple acceleration method for codec-based autoregressive TTS that avoids fine-tuning the target model, and the speedup measurement itself is straightforward and reproducible in spirit because it uses the open-source CosyVoice 2 and LibriTTS. The independent-draft-model design and the layer-inheritance initialization are reasonable engineering contributions. However, the current evidence does not support the fidelity claim: the 55% relative WER increase in Table 1 is substantial, and more importantly, the beta=0 control in Table 3 contradicts textbook speculative-decoding guarantees, which casts doubt on whether the verification loop actually samples from the target model's distribution. These issues are load-bearing for the paper's main claim, so the manuscript needs substantial revision before it can be accepted.

major comments (5)
  1. [§4.3, Table 3] The beta=0 row in Table 3 reports WER 6.34% versus 3.67% for CosyVoice 2, but with beta=0 Algorithm 1 is exactly the standard speculative-decoding rejection sampler (accept with probability min(1, q/p); on rejection sample from normalized max(0, q-p)), which is guaranteed to reproduce the target model's autoregressive distribution. Unless there is an implementation difference between the parallel verification distributions q^s_i and the autoregressive next-token distribution, the beta=0 output should be statistically indistinguishable from CosyVoice 2, not nearly double in WER. This discrepancy is not addressed in the paper and undercuts the fidelity claim; the authors should identify the cause (e.g., decoding-parameter mismatch, parallel-versus-autoregressive distribution mismatch, or an implementation bug) or re-run the experiment.
  2. [§2.2, Eq. (5)] The tolerance factor beta>0 is not a benign relaxation: for beta=0.4, a draft token is accepted with probability at least 0.4 even when the target model assigns it probability zero, and the acceptance probability min(1, q/p)+beta can exceed 1, making the rule an invalid rejection sampler. Consequently, the output distribution of SSD is no longer the target model's distribution, and no analysis or correction is provided. The paper's justification in Section 4.3 (that different token sequences can map to similar audio) is plausible but qualitative; it needs quantitative support, such as measuring the induced distribution shift (e.g., KL divergence between SSD and target token distributions) or ablating with an unbiased sampler.
  3. [§2.2, Eq. (6)] The resampling step on rejection uses p_new = normalize(max(0, q - p)), which is the correct residual for standard speculative decoding only when the acceptance rule is min(1, q/p). With beta>0, the composition of accepted and rejected tokens changes, so this normalized residual no longer yields samples from the target conditional distribution conditioned on the draft prefix. The paper does not state what distribution SSD actually targets, and the claim in Section 4.1 that the verification mechanism keeps quality close to CosyVoice 2 is therefore unsupported. A formal derivation or at least an empirical comparison against the target distribution is needed.
  4. [§3.1 and §4.3–4.4] The hyperparameters beta=0.4 and L_d=3 are set and then analyzed using the same 500-utterance test set (Table 3 and Figure 3), with no separate validation set for these choices. Since the reported 1.4x speedup and quality metrics are the result of this selection, the results are susceptible to test-set overfitting; the authors should report the selection procedure or validate on a held-out set.
  5. [§4.1, Table 1] The abstract and Section 4.1 claim SSD 'maintains high fidelity' while Table 1 shows WER increasing by 55% relative (3.67% to 5.70%), a larger degradation than the NMOS/SS differences. No confidence intervals or significance tests are provided for Table 1, and the subjective tests in Table 2 have overlapping confidence intervals but may lack power to detect the WER difference. The authors should either reinterpret the result as a quality-speed tradeoff or provide evidence that the WER increase is perceptually irrelevant.
minor comments (5)
  1. [Algorithm 1 and Eq. (6)] The word 'normlize' appears in Algorithm 1 and Eq. (6); it should be 'normalize'.
  2. [Table 2] The column header 'NA T-MOS' is a typo; it should be 'NAT-MOS'.
  3. [§3.2] The claim that SSD is trained on approximately 1/300 the size of CosyVoice 2's proprietary dataset is presented without a citation or a precise computation; please justify or qualify this statement.
  4. [§4.3] The statement that 'varying tolerance factors do not lead to significant differences in speech quality' is not supported by statistical tests; WER varies from 5.70% to 6.34% across beta values, which may be meaningful.
  5. [Figure 3] The LM-RTF values in Figure 3 are plotted without error bars or repetition information, so it is unclear whether the differences across L_d values are reliable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: speedup and quality claims are empirical measurements on external benchmarks.

full rationale

The paper's central claims are empirical. The 1.4x speedup is computed from measured LM-RTF values (Table 1: 0.504 vs 0.360) on a fixed 500-utterance test set, and quality is assessed with external metrics (WER via Whisper-large V3, SS via ERes2Net, NMOS, and subjective MOS). The draft model is trained on LibriTTS and evaluated on a disjoint test-clean subset, so no parameter is defined in terms of the target result. The tolerance factor β and draft length L_d are hyperparameters chosen from sweeps (Table 3, Figure 3); although selecting them on the same test set raises overfitting concerns, the reported speedup is a measurement, not a quantity forced by construction, and no equation reduces to its own input. The assumption that multiple speech-token sequences can reconstruct similar audio (Section 4.3) is an empirical premise, not a circular derivation. The β=0 row in Table 3 (WER 6.34% vs CosyVoice 2's 3.67%) is inconsistent with the unbiasedness guarantee of standard speculative decoding, which flags a correctness/fidelity issue outside circularity; the paper does not address this discrepancy, but the discrepancy does not make the derivation circular.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard speculative decoding assumptions, the perceptual equivalence of token substitutions, and the hand-chosen hyperparameters beta and L_d. No new physical or conceptual entities are introduced.

free parameters (2)
  • tolerance factor beta = 0.4
    Chosen for the main result; Table 3 shows beta=0.4 gives the best LM-RTF and lowest WER among tested values, suggesting selection on the same test set.
  • draft length L_d = 3
    Figure 3 shows L_d=3 gives the lowest LM-RTF; appears selected after inspecting test-set results.
assumptions (3)
  • domain assumption Speech tokens can be substituted with acoustically plausible alternatives without perceptible quality loss, justifying the relaxed acceptance criterion with beta>0.
    Sections 2.2 and 4.3 assert that multiple token combinations can reconstruct similar speech through the decoder; this is assumed, not proven.
  • domain assumption The draft model's distribution is sufficiently close to the target model's distribution for high acceptance rates.
    Speculative decoding efficiency relies on draft-target agreement; the paper does not quantify acceptance rates or distribution divergence.
  • standard math Standard speculative decoding theory (Leviathan et al., Chen et al.) applies to speech token language models.
    Section 2.2 builds directly on the acceptance/rejection framework; the beta=0 case inherits distribution-preserving properties, but the beta>0 case does not.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding." pith.science (2026). https://pith.science/paper/UMHCAPBW

@misc{pith2026250515380,
  author       = {Pith},
  title        = {Pith review of: Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UMHCAPBW}},
  note         = {Machine review of arXiv:2505.15380}
}
read the original abstract

Modern autoregressive speech synthesis models leveraging language models have demonstrated remarkable performance. However, the sequential nature of next token prediction in these models leads to significant latency, hindering their deployment in scenarios where inference speed is critical. In this work, we propose Speech Speculative Decoding (SSD), a novel framework for autoregressive speech synthesis acceleration. Specifically, our method employs a lightweight draft model to generate candidate token sequences, which are subsequently verified in parallel by the target model using the proposed SSD framework. Experimental results demonstrate that SSD achieves a significant speedup of 1.4x compared with conventional autoregressive decoding, while maintaining high fidelity and naturalness. Subjective evaluations further validate the effectiveness of SSD in preserving the perceptual quality of the target model while accelerating inference.

Figures

Figures reproduced from arXiv: 2505.15380 by the authors.

Figure 1
Figure 1. Visualization of autoregressive speech synthesis sys￾tem and inference time composition of CosyVoice 2. sequence length [15], while the computational complexity of the Transformer backbone grows quadratically, further exacer￾bating inefficiency. For instance, in CosyVoice 2 [5], a state￾of-the-art LM-based speech synthesis model, the autoregressive inference process of the LM takes up approximately 70% of the total … view at source ↗
Figure 2
Figure 2. The Inference Architecture of proposed Speech Speculative Decoding (SSD) framework. The dashed line depicts autore￾gressive decoding in the draft model, followed by parallel verification by the target model. The prefix comprises all prompt tokens and previously generated tokens during inference. Green indicates accepted tokens, while red represents rejected tokens. criterion tailored to speech tokens, enabling more … view at source ↗
Figure 3
Figure 3. LM-RTF results of SSD with different Ld. 5. Conclusions In this work, we introduce Speculative Speech Decoding (SSD), a novel approach to accelerate autoregressive speech synthe￾sis. We propose a lightweight draft model constructed by fine￾tuning a few parameters from the target model. Experiments on CosyVoice 2 show that our proposed SSD achieves 1.4x acceler￾ation while maintaining high synthesis quality. Subjecti… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 11 canonical work pages

  1. [24]

    Vits2: Improving quality and efficiency of single-stage text-to-speech with adversarial learning and architecture design,

    J. Kong, J. Park, B. Kim, J. Kim, D. Kong, and S. Kim, “Vits2: Improving quality and efficiency of single-stage text-to-speech with adversarial learning and architecture design,” inInterspeech 2023, 2023, pp. 4374–4378

    2023

  2. [25]

    F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,

    Y . Chen, Z. Niu, Z. Ma, K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen, “F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,”arXiv preprint arXiv:2410.06885, 2024

    arXiv 2024

  3. [1]

    The text encoder tokenizes input text into fine-grained sequences, and in some cases, further transforms them into high-dimensional se- mantic representations

    Introduction Modern autoregressive speech synthesis systems [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] often employ four core components to achieve human-like speech generation, as shown in Figure 1a. The text encoder tokenizes input text into fine-grained sequences, and in some cases, further transforms them into high-dimensional se- mantic representations. The...

    arXiv 2025

  4. [2]

    Methodology In this section, we demonstrate the proposed SSD accelera- tion framework, which comprises two core components: a tar- get model (the large model requiring acceleration) and a draft model (a lightweight auxiliary model). As illustrated in Fig- ure 2, the process unfolds in three phases: 1) Autoregressively generatingmcandidate tokens using the...

    work page

  5. [3]

    Implementation Details We use CosyV oice 2 [5] as the target model, which is a highly efficient TTS model consisting of 24 layers of Transformer adapted from Qwen2.5 (0.5B) [28]

    Experiments 3.1. Implementation Details We use CosyV oice 2 [5] as the target model, which is a highly efficient TTS model consisting of 24 layers of Transformer adapted from Qwen2.5 (0.5B) [28]. Based on this model, we design a draft model consisting of 8 layers of Transformer. We follow the setting of CosyV oice 2: each Transformer layer typically emplo...

    work page

  6. [4]

    train-clean-100

    During the training of the draft model, we only train the first two layers of the Transformer and the classification layer. The length of draft tokens asL d is set to 3 and the tolerance fac- torβis set to 0.4. We conduct all the experiments on NVIDIA A100-SXM4-40GB GPU. 3.2. Dataset LibriTTS, introduced by [29], is a well-known multi-speaker English spee...

    work page

  7. [5]

    Results and Analysis We apply SSD to CosyV oice 2 and compare the result with draft model and CosyV oice 2 using autoregressive decoding to syn- thesize speech. 4.1. Objective Evaluation As shown in Table 1, our Speech Speculative Decoding (SSD) framework achieves 1.4× faster synthesis than the conven- tional autoregressive decoding used by CosyV oice 2. ...

    work page

  8. [6]

    We propose a lightweight draft model constructed by fine- tuning a few parameters from the target model

    Conclusions In this work, we introduce Speculative Speech Decoding (SSD), a novel approach to accelerate autoregressive speech synthe- sis. We propose a lightweight draft model constructed by fine- tuning a few parameters from the target model. Experiments on CosyV oice 2 show that our proposed SSD achieves 1.4x acceler- ation while maintaining high synth...

    work page

Show all 37 references
  1. [7]

    Acknowledgements This work is supported by National Natural Science Founda- tion of China (62076144), National Social Science Foundation of China (13&ZD189) and Shenzhen Science and Technology Program (JCYJ20220818101014030)

  2. [8]

    Neural codec language mod- els are zero-shot text to speech synthesizers,

    C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Liet al., “Neural codec language mod- els are zero-shot text to speech synthesizers,”arXiv preprint arXiv:2301.02111, 2023

    2023 arXiv

  3. [9]

    Speak foreign languages with your own voice: Cross-lingual neural codec language modeling,

    Z. Zhang, L. Zhou, C. Wang, S. Chen, Y . Wu, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Liet al., “Speak foreign languages with your own voice: Cross-lingual neural codec language modeling,”arXiv preprint arXiv:2303.03926, 2023

    2023 arXiv

  4. [10]

    Seed-tts: A family of high-quality versatile speech generation models,

    P. Anastassiou, J. Chen, J. Chen, Y . Chen, Z. Chen, Z. Chen, J. Cong, L. Deng, C. Ding, L. Gaoet al., “Seed-tts: A family of high-quality versatile speech generation models,”arXiv preprint arXiv:2406.02430, 2024

    2024 arXiv

  5. [11]

    Cosyvoice: A scalable multi- lingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Maet al., “Cosyvoice: A scalable multi- lingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,”arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  6. [12]

    Cosyvoice 2: Scalable stream- ing speech synthesis with large language models,

    Z. Du, Y . Wang, Q. Chen, X. Shi, X. Lv, T. Zhao, Z. Gao, Y . Yang, C. Gao, H. Wanget al., “Cosyvoice 2: Scalable stream- ing speech synthesis with large language models,”arXiv preprint arXiv:2412.10117, 2024

    2024 arXiv

  7. [13]

    Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis,

    S. Liao, Y . Wang, T. Li, Y . Cheng, R. Zhang, R. Zhou, and Y . Xing, “Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis,”arXiv preprint arXiv:2411.01156, 2024

    2024 arXiv

  8. [14]

    Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications,

    H.-H. Guo, K. Liu, F.-Y . Shen, Y .-C. Wu, F.-L. Xie, K. Xie, and K.-T. Xu, “Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications,”arXiv preprint arXiv:2409.03283, 2024

    2024 arXiv

  9. [15]

    Speak, read and prompt: High-fidelity text-to-speech with min- imal supervision,

    E. Kharitonov, D. Vincent, Z. Borsos, R. Marinier, S. Girgin, O. Pietquin, M. Sharifi, M. Tagliasacchi, and N. Zeghidour, “Speak, read and prompt: High-fidelity text-to-speech with min- imal supervision,”Transactions of the Association for Computa- tional Linguistics, vol. 11,...

    2023

  10. [16]

    Base tts: Lessons from building a billion-parameter text-to-speech model on 100k hours of data,

    M. Łajszczak, G. C ´ambara, Y . Li, F. Beyhan, A. van Korlaar, F. Yang, A. Joly, ´A. Mart ´ın-Cortinas, A. Abbas, A. Michal- skiet al., “Base tts: Lessons from building a billion-parameter text-to-speech model on 100k hours of data,”arXiv preprint arXiv:2402.08093, 2024

    2024 arXiv

  11. [17]

    VoiceCraft: Zero-shot speech editing and text-to-speech in the wild,

    P. Peng, P.-Y . Huang, S.-W. Li, A. Mohamed, and D. Harwath, “VoiceCraft: Zero-shot speech editing and text-to-speech in the wild,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L.-W. Ku, A. Martins, and V . ...

    2024

  12. [18]

    Uniaudio: An audio founda- tion model toward universal audio generation,

    D. Yang, J. Tian, X. Tan, R. Huang, S. Liu, X. Chang, J. Shi, S. Zhao, J. Bian, X. Wuet al., “Uniaudio: An audio founda- tion model toward universal audio generation,”arXiv preprint arXiv:2310.00704, 2023

    2023 arXiv

  13. [19]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

    2021

  14. [20]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,”IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 30, pp. 495–507, 2022

    2022

  15. [21]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Transactions on Machine Learning Research

  16. [22]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural In- formation Processing Systems, 2017

    2017

  17. [23]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” inInter- national Conference on Machine Learning. PMLR, 2021, pp. 5530–5540

    2021

  18. [26]

    Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tanget al., “Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,”arXiv preprint arXiv:2403.03100, 2024

    2024 arXiv

  19. [27]

    E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts,

    S. E. Eskimez, X. Wang, M. Thakker, C. Li, C.-H. Tsai, Z. Xiao, H. Yang, Z. Zhu, M. Tang, X. Tanet al., “E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts,” in2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 682– 689

    2024

  20. [28]

    V oicebox: Text-guided multilingual universal speech generation at scale,

    M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V . Manohar, Y . Adi, J. Mahadeokaret al., “V oicebox: Text-guided multilingual universal speech generation at scale,”Advances in neural information processing systems, vol. 36, 2024

    2024

  21. [29]

    Accelerating large language model decoding with speculative sampling,

    C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,”arXiv preprint arXiv:2302.01318, 2023

    2023 arXiv

  22. [30]

    Fast inference from transformers via speculative decoding,

    Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from transformers via speculative decoding,” inInternational Confer- ence on Machine Learning. PMLR, 2023, pp. 19 274–19 286

    2023

  23. [31]

    Accelerating codec-based speech synthesis with multi- token prediction and speculative decoding,

    T. D. Nguyen, J.-H. Kim, J. Choi, S. Choi, J. Park, Y . Lee, and J. S. Chung, “Accelerating codec-based speech synthesis with multi- token prediction and speculative decoding,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

    2025

  24. [32]

    Fast and high- quality auto-regressive speech synthesis via speculative decod- ing,

    B. Li, H. Wang, S. Zhang, Y . Guo, and K. Yu, “Fast and high- quality auto-regressive speech synthesis via speculative decod- ing,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1– 5

    2025

  25. [33]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” inInternational conference on machine learning. PMLR, 2019, pp. 2790–2799

    2019

  26. [34]

    Efficient adapter transfer of self-supervised speech models for automatic speech recogni- tion,

    B. Thomas, S. Kessler, and S. Karout, “Efficient adapter transfer of self-supervised speech models for automatic speech recogni- tion,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 7102–7106

    2022

  27. [35]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  28. [36]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,”arXiv preprint arXiv:1904.02882, 2019

    1904 arXiv

  29. [37]

    Unicats: A unified context-aware text- to-speech framework with contextual vq-diffusion and vocoding,

    C. Du, Y . Guo, F. Shen, Z. Liu, Z. Liang, X. Chen, S. Wang, H. Zhang, and K. Yu, “Unicats: A unified context-aware text- to-speech framework with contextual vq-diffusion and vocoding,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp....

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.