Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vevo claims that a single self-supervised tokenizer pair separates timbre, style, and content, enabling zero-shot voice imitation.

desk verdict A genuinely useful self-supervised disentanglement recipe for zero-shot voice imitation, but the style-imitation headline rests on CMOS comparisons that are effectively unmeasured. read the letter →

arxiv 2502.07243 v1 pith:SD2UXZEW submitted 2025-02-11 cs.SD cs.AIeess.AS

Xueyao Zhang , Xiaohui Zhang , Kainan Peng , Zhenyu Tang , Vimal Manohar , Yingru Liu , Jeff Hwang , Dangna Li
show 5 more authors
Yuhao Wang Julian Chan Yuan Huang Zhizheng Wu Mingbo Ma
This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords zero-shotvoiceimitationconversiontext-to-speechself-superviseddisentanglementVQ-VAEinformationbottleneckHuBERTflowmatchingaccentandemotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that speech can be split into linguistic content, speaking style, and timbre using only self-supervised training, by treating the vocabulary size of a VQ-VAE codebook as an adjustable information bottleneck on HuBERT speech features. With 4096 codes for what it calls content-style tokens and 32 codes for content tokens, timbre is filtered out first and most style information second, leaving linguistic content. Two prompted transformers then recombine these pieces: an autoregressive model writes style into the content tokens from a style reference, and a flow-matching model renders the acoustic signal with the timbre of a reference voice. Trained only on 60K hours of audiobook speech, with no fine-tuning on accented or emotional corpora, Vevo matches or surpasses dedicated accent- and emotion-conversion systems in zero-shot settings, and it is competitive on voice conversion and zero-shot text-to-speech. The significance is that controllable timbre and style imitation may not require annotated style data at all.

What carries the argument

The central mechanism is the VQ-VAE codebook vocabulary size acting as an information bottleneck on HuBERT hidden features: the paper posits that reducing the vocabulary $K$ filters timbre first, then style, then content, so one quantizer with $K=4096$ yields content-style tokens and one with $K=32$ yields content tokens; a duration-reduction step that merges consecutive duplicate content tokens removes unit-level duration patterns. Style is then written in by a decoder-only autoregressive transformer that continues from the source's reduced content tokens plus a style reference (encoded by a global style encoder built from self-supervised representation and TDNN layers, and optionally the reference's own content-style tokens), and the acoustic signal is produced by a flow-matching transformer conditioned on those content-style tokens and a temporally masked timbre-reference mel spectrogram, following the conditional flow matching objective used in [27].

What would settle it

Compute speaker similarity and pitch correlation for each individual evaluation speaker as the token vocabulary is swept from 16384 down to 8; any speaker who loses style before timbre, or who still carries timbre at 32 codes, would falsify the claimed bottleneck ordering.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that vector quantization of HuBERT's continuous hidden features, with the VQ-VAE codebook size used as the width of an information bottleneck, yields a progressive and useful disentanglement of speech attributes: as the vocabulary shrinks, timbre information is the first to drop out, leaving content-style tokens, and further shrinkage removes most style information, leaving content tokens. The paper selects $K_s = 4096$ for the content-style tokenizer and $K_c = 32$ for the content tokenizer, and verifies this ordering with intelligibility (WER), speaker similarity (S-SIM), and F0-correlation (FPC) measurements on a 700-sample evaluation set. Vevo-Timbre, Vevo-Style, Vevo-Voice, and Vevo-TTS are the same two models invoked in different inference pipelines, so that content comes from the source, style from a style reference prompt, and timbre from a timbre reference prompt. The headline empirical claim is that this fully self-supervised recipe outperforms existing accent and emotion conversion systems, which rely on parallel corpora or style labels, without ever being trained on style-specific data.

Load-bearing premise

The method assumes that shrinking the token vocabulary from 4096 to 32 removes timbre before style, and that this ordering holds for every unseen speaker, accent, and emotion.

Editorial extensions

If this is right

  • Accent and emotion conversion can be performed zero-shot from a few seconds of reference speech, with no parallel corpus, style labels, or fine-tuning on style-specific data.
  • One pair of self-supervised tokenizers supports four tasks—timbre imitation, style imitation, voice conversion, and zero-shot TTS—by changing only the inference pipeline.
  • Content tokens with a vocabulary of 32 preserve intelligibility well enough in the appendix's ABX analysis that a small vocabulary suffices for linguistic content, suggesting cheaper content representation for downstream tasks.
  • The reference-global-guided continuation mode cuts inference input length to 42% with only a slight performance drop, which matters for memory and latency in deployment.
  • Both stages train with self-supervision and in-context learning, so scaling up to more data is straightforward and is argued to improve results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper itself notes in Section 4.1 that $K_c$ and $K_s$ 'may not be optimal,' which invites a sensitivity analysis across diverse speakers (breathy, whispered, very fast, or code-switched speech); if the timbre-then-style ordering is not speaker-universal, per-speaker or per-utterance codebook selection would be needed.
  • Editorial inference: because the bottleneck operates on whatever self-supervised features feed it, the recipe should transfer to other pretrained speech encoders and to multilingual or in-the-wild data, with vocabulary sizes then needing re-tuning.
  • Editorial inference: duration reduction removes unit-level duration as style, but duration is a major prosody carrier; the DDUR improvements in the ablations suggest that restoring duration at the acoustic stage could become a separate, controllable style axis, such as speech-rate cloning.
  • Editorial inference: Vevo-TTS's emotion-similarity MOS on the EMOTION subset is numerically the best among the compared synthesizers; a direct replication on a larger emotion corpus with multiple listener groups would determine whether content-style tokens genuinely encode emotion better than semantic tokens obtained from in-the-wild data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Vevo, a zero-shot voice imitation framework with two stages: an autoregressive content-style model that generates content-style tokens from content tokens or text under the guidance of a style reference, and a flow-matching acoustic model that converts content-style tokens into mel spectrograms under the guidance of a timbre reference. The key representational contribution is a fully self-supervised disentanglement scheme in which VQ-VAE tokenizers with different codebook sizes are applied to HuBERT features: a large codebook (Ks=4096) yields content-style tokens that preserve content and style but discard timbre, while a small codebook (Kc=32) yields content tokens that preserve mainly linguistic content. The models are trained on 60K hours of audiobook speech without style-specific fine-tuning, and evaluated on timbre imitation, style imitation (accent and emotion), zero-shot voice conversion, and zero-shot TTS. The paper claims that Vevo matches or surpasses existing methods in accent and emotion conversion tasks while also performing competitively on voice conversion and TTS.

Significance. If the claims hold, the paper is significant: it offers a scalable, annotation-free route to disentangling timbre, style, and content, and unifies several zero-shot speech generation tasks in one framework. The core idea of using VQ-VAE codebook size as an information bottleneck is simple and empirically actionable, and the paper provides extensive experiments, ablations, and audio samples. The use of only self-supervised tokenizers and in-context learned models on 60K hours is a meaningful step beyond methods that require parallel corpora, style labels, or fine-tuning on style-specific data. The paper also reports detailed training and inference configurations, which aids reproducibility, although no code is released.

major comments (3)
  1. [Table 4, Section 4.3] The headline claim that Vevo 'matches or surpasses existing methods in accent and emotion conversion tasks' rests on CMOS values for baselines that are listed as 0.00±0.00. The paper does not describe any measurement of baseline CMOS; if these values are arbitrary anchors rather than measured pairwise scores, then the reported N-CMOS and A-/E-CMOS for Vevo-Style are not comparable to the baselines. The authors must either run proper paired CMOS tests between Vevo and each baseline (with the baseline output as the reference), or explicitly state that the 0.00 values are references and remove them from the results table. Without this, the 'surpasses' wording in the abstract and conclusion is not verifiable from the paper alone.
  2. [Section 4.1, Tables 2-5] The hyperparameters Kc=32 and Ks=4096 are selected by inspecting Table 2 metrics (WER, S-SIM, FPC) on the zero-shot timbre imitation task, and the same task family is then used in Table 3 to claim superiority over baselines. The paper does not describe a separate validation set or a clear development/test split for hyperparameter selection. This introduces a risk of selection on the evaluation metric. The authors should clarify which samples were used for model selection versus final evaluation and, ideally, report results for a range of (Kc, Ks) on the final test sets to demonstrate that the main conclusions are not an artifact of this selection.
  3. [Section 4.1, Table 2] The claim that the vocabulary-size bottleneck filters timbre first, then style, and finally content is supported only by aggregate proxy metrics (S-SIM, FPC, WER) on a 700-sample evaluation set. The paper provides no analysis of variance across speakers or styles, and it is possible that the ordering of information loss is speaker-dependent or style-dependent. Since the zero-shot claims rely on the universality of this ordering, the authors should add per-speaker or per-style breakdowns, or at least explicitly acknowledge this limitation in the main text rather than only stating that the selected K values 'may not be optimal'.
minor comments (5)
  1. [Abstract and Section 5] The wording 'matches or surpasses' is stronger than the evidence in Table 4 supports, given the CMOS anchoring issue. Weaker phrasing such as 'is competitive with' or 'shows promising performance relative to' would be more appropriate until the CMOS comparisons are properly executed.
  2. [Table 4] The column header 'N-COMS' appears to be a typo for 'N-CMOS'.
  3. [Appendix D.1.1] The text contains a typo: 'obatined' should be 'obtained'.
  4. [Section 4.1] The paper states that the 18th layer of HuBERT-Large is used for the tokenizer reconstruction objective, but it does not explain why the 18th layer is chosen over other layers, beyond Table 2 showing a tradeoff. A brief justification would help readers understand the choice.
  5. [Tables 3-5 and Appendix D.2] The objective metrics (WER, S-SIM, A-SIM, E-SIM) are reported without confidence intervals or significance tests. Given that the evaluation set is 700 samples and the differences are sometimes small, reporting bootstrap confidence intervals or pairwise significance tests would strengthen the claims and allow readers to gauge the reliability of the differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the tokenizer bottleneck sizes are empirically chosen hyperparameters, and the downstream evaluations are external to the training objectives.

full rationale

The paper's disentanglement claim rests on an experimental sweep of VQ-VAE codebook sizes (Section 4.1, Table 2), not on a parameter fitted to the final evaluation metrics. Kc=32 and Ks=4096 are selected from development observations about WER, S-SIM, and FPC; the actual claims about accent, emotion, timbre, and voice imitation are then produced by independently trained Mstyle and Macoustic models and measured with external classifiers (WavLM TDNN, CommonAccent, emotion2vec), ASR (Whisper), and human CMOS/MOS ratings. None of these metrics appears in the tokenizer or generation losses, and the evaluation samples are drawn from demo websites, Common Voice, and a reserved internal test corpus, not from the tokenizer training data. The Table 4 '0.00' entries are a standard CMOS anchoring convention in which the baseline serves as the reference system; they do not make Vevo's score equivalent to its inputs. Self-citations (e.g., MaskGCT, Amphion, the FPC metric) are used as baselines or tooling and do not justify the central disentanglement or generation claims. No equation in the paper reduces a claimed prediction to a fitted quantity or to an author-supplied uniqueness theorem.

Assumptions & free parameters 3 free parameters · 4 assumptions · 3 invented entities

The framework depends on the empirical hypothesis that HuBERT features contain separable timbre, style, and content, and that codebook size filters them in a fixed order. These assumptions are validated on the same 700-sample evaluation set used to select the hyperparameters, so the contribution rests on in-paper evidence with no external replication.

free parameters (3)
  • Kc (content tokenizer vocabulary size) = 32
    Selected in Section 4.1 as the point where most style is filtered out while content is retained; used to build content tokens for all main experiments.
  • Ks (content-style tokenizer vocabulary size) = 4096
    Selected as the point where timbre is filtered but style is retained; used to build content-style tokens for all main experiments.
  • HuBERT layer used for tokenizer reconstruction = 18
    Chosen in Section 4.1 and Table 2 as the initial stage of information filtering; all VQ-VAE tokenizers are trained to reconstruct the 18th layer features.
assumptions (4)
  • domain assumption HuBERT hidden features contain rich information about timbre, style, and linguistic content.
    Section 3.1 Motivation; verified indirectly in Table 2 but assumed as starting point.
  • ad hoc to paper Reducing VQ-VAE vocabulary size filters speech information in a fixed order: timbre first, then style, then content.
    Section 3.1 Analysis and Table 2; this monotonic bottleneck hypothesis is the core mechanism and is validated only on the paper's evaluation set.
  • domain assumption The same (Kc, Ks) generalize across unseen speakers, accents, and emotions.
    Section 4.1; the paper acknowledges Kc and Ks may not be optimal, yet the entire framework depends on this choice.
  • domain assumption Subjective MOS and CMOS ratings by paid experienced raters are reliable indicators of voice, emotion, and accent similarity.
    Appendix E; raters are experienced, but no inter-rater agreement or statistical tests are reported.
invented entities (3)
  • Content tokens Qc
    purpose: Discrete representation of linguistic content with timbre and most style removed
    Produced by a VQ-VAE with K=32; evaluated only within this paper; no tokenizer checkpoint or external benchmark released.
  • Content-style tokens Qs
    purpose: Discrete representation retaining content plus style while filtering timbre
    Produced by a VQ-VAE with K=4096; same internal evaluation situation.
  • Global style embedding g(u)
    purpose: Summary embedding of speaking style used by the autoregressive transformer
    WavLM plus TDNN based encoder; no ablated analysis of its standalone quality beyond overall system performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement." pith.science (2026). https://pith.science/paper/SD2UXZEW

@misc{pith2026250207243,
  author       = {Pith},
  title        = {Pith review of: Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SD2UXZEW}},
  note         = {Machine review of arXiv:2502.07243}
}
read the original abstract

The imitation of voice, targeted on specific speech attributes such as timbre and speaking style, is crucial in speech generation. However, existing methods rely heavily on annotated data, and struggle with effectively disentangling timbre and style, leading to challenges in achieving controllable generation, especially in zero-shot scenarios. To address these issues, we propose Vevo, a versatile zero-shot voice imitation framework with controllable timbre and style. Vevo operates in two core stages: (1) Content-Style Modeling: Given either text or speech's content tokens as input, we utilize an autoregressive transformer to generate the content-style tokens, which is prompted by a style reference; (2) Acoustic Modeling: Given the content-style tokens as input, we employ a flow-matching transformer to produce acoustic representations, which is prompted by a timbre reference. To obtain the content and content-style tokens of speech, we design a fully self-supervised approach that progressively decouples the timbre, style, and linguistic content of speech. Specifically, we adopt VQ-VAE as the tokenizer for the continuous hidden features of HuBERT. We treat the vocabulary size of the VQ-VAE codebook as the information bottleneck, and adjust it carefully to obtain the disentangled speech representations. Solely self-supervised trained on 60K hours of audiobook speech data, without any fine-tuning on style-specific corpora, Vevo matches or surpasses existing methods in accent and emotion conversion tasks. Additionally, Vevo's effectiveness in zero-shot voice conversion and text-to-speech tasks further demonstrates its strong generalization and versatility. Audio samples are available at https://versavoice.github.io.

Figures

Figures reproduced from arXiv: 2502.07243 by the authors.

Figure 1
Figure 1. Vevo inference pipeline. Notably, it can take [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Content-style modeling based on autoregressive transformer. During inference, we employ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Acoustic modeling based on a flow-matching transformer. During inference, we append [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Encoder and decoder architecture of our VQ-VAE tokenizer. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Reference-global-guided continuation of Mstyle for inference. B.3 CONTENT-STYLE MODELING (TEXT AS INPUT) Compared to Mstyle, the only difference of Mfstyle is that its input becomes text tokens, rather than the duration reduced content tokens. Specifically, we adopt th…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Entropy-based Coarse and Compressed Semantic Speech Representation Learning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    Predictive entropy from a token-level speech language model finds merge boundaries, producing compressed semantic tokens that keep ASR and translation accuracy at 15 Hz while lowering latency.

  2. NouveauVoice: Generating Novel Pseudo Speakers for Voice Anonymization

    eess.AS 2026-07 conditional novelty 5.5 of 10

    A hierarchical NVAE plug-in generates diverse pseudo-speaker embeddings that raise ASV EER above 38% on FACodec/CosyVoice2 with a controllable privacy-utility trade-off.

  3. IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech

    cs.CL 2025-06 conditional novelty 5.0 of 10

    IndexTTS2 achieves precise token-count-based duration control and emotion/speaker disentanglement in an autoregressive zero-shot TTS, reporting SOTA WER, speaker similarity, and emotional fidelity.

Reference graph

Works this paper leans on

93 extracted references · 58 canonical work pages · cited by 3 Pith papers

  1. [1]

    Neural discrete representation learning

    A ¨aron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In NIPS, pages 6306–6315, 2017

    2017

  2. [2]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhut- dinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE ACM Trans. Audio Speech Lang. Process. , 29: 3451–3460, 2021

    2021

  3. [3]

    An overview of voice conversion systems

    Seyed Hamidreza Mohammadi and Alexander Kain. An overview of voice conversion systems. Speech Commun., 88:65–82, 2017

    2017

  4. [4]

    An overview of voice con- version and its challenges: From statistical modeling to deep learning

    Berrak Sisman, Junichi Yamagishi, Simon King, and Haizhou Li. An overview of voice con- version and its challenges: From statistical modeling to deep learning. IEEE ACM Trans. Audio Speech Lang. Process., 29:132–157, 2021

    2021

  5. [5]

    Foreign accent conversion in computer assisted pronunciation training

    Daniel Felps, Heather Bortfeld, and Ricardo Gutierrez-Osuna. Foreign accent conversion in computer assisted pronunciation training. Speech Commun., 51(10):920–932, 2009

    2009

  6. [6]

    L2-ARCTIC: A non-native english speech corpus

    Guanlong Zhao, Sinem Sonsaat, Alif Silpachai, Ivana Lucic, Evgeny Chukharev-Hudilainen, John Levis, and Ricardo Gutierrez-Osuna. L2-ARCTIC: A non-native english speech corpus. In INTERSPEECH, pages 2783–2787. ISCA, 2018

    2018

  7. [7]

    Emotional voice conversion: Theory, databases and ESD

    Kun Zhou, Berrak Sisman, Rui Liu, and Haizhou Li. Emotional voice conversion: Theory, databases and ESD. Speech Commun., 137:1–18, 2022

    2022

  8. [8]

    Neural Text-to-Speech Synthesis

    Xu Tan. Neural Text-to-Speech Synthesis. Springer, 2023

    2023

Show all 93 references
  1. [9]

    Converting foreign accent speech without a reference

    Guanlong Zhao, Shaojin Ding, and Ricardo Gutierrez-Osuna. Converting foreign accent speech without a reference. IEEE ACM Trans. Audio Speech Lang. Process. , 29:2367–2381, 2021

    2021

  2. [10]

    Sahidullah, Aur ´elien Bellet, Marc Tom- masi, and Emmanuel Vincent

    Brij Mohan Lal Srivastava, Nathalie Vauquier, Md. Sahidullah, Aur ´elien Bellet, Marc Tom- masi, and Emmanuel Vincent. Evaluating voice conversion-based privacy protection against informed attackers. In ICASSP, pages 2802–2806. IEEE, 2020

    2020

  3. [11]

    Seed-tts: A family of high-quality versatile speech generation models

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, Jiaxin Li, Xiaoyang Li, Xingxing Li...

    2024 arXiv

  4. [12]

    Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications

    Hao-Han Guo, Kun Liu, Fei-Yu Shen, Yi-Chen Wu, Feng-Long Xie, Kun Xie, and Kai-Tuo Xu. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. arXiv preprint, abs/2409.03283, 2024

    2024 arXiv

  5. [13]

    Maskgct: Zero-shot text-to- speech with masked generative codec transformer

    Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. Maskgct: Zero-shot text-to- speech with masked generative codec transformer. In ICLR. OpenReview.net, 2025. 11 Published as a conference ...

    2025

  6. [14]

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson, and David D. Cox. Un- supervised speech decomposition via triple information bottleneck. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 7836–7846. PMLR, 2020

    2020

  7. [15]

    Speech resynthesis from discrete disentan- gled self-supervised representations

    Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, Wei-Ning Hsu, Abdelrahman Mohamed, and Emmanuel Dupoux. Speech resynthesis from discrete disentan- gled self-supervised representations. In INTERSPEECH, pages 3615–3619. ISCA, 2021

    2021

  8. [16]

    Mega-tts: Zero-shot text-to- speech at scale with intrinsic inductive bias

    Ziyue Jiang, Yi Ren, Zhenhui Ye, Jinglin Liu, Chen Zhang, Qian Yang, Shengpeng Ji, Rongjie Huang, Chunfeng Wang, Xiang Yin, Zejun Ma, and Zhou Zhao. Mega-tts: Zero-shot text-to- speech at scale with intrinsic inductive bias. arXiv preprint, abs/2306.03509, 2023

    2023 arXiv

  9. [17]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Eric Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiangyang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, and Sheng Zhao. Naturalspeech 3: Zero-shot speech synthesis with fa...

    2024

  10. [18]

    Lifa Sun, Shiyin Kang, Kun Li, and Helen M. Meng. V oice conversion using deep bidirectional long short-term memory based recurrent neural networks. InICASSP, pages 4869–4873. IEEE, 2015

    2015

  11. [19]

    Deep bidirectional LSTM modeling of timbre and prosody for emotional voice conversion

    Huaiping Ming, Dong-Yan Huang, Lei Xie, Jie Wu, Minghui Dong, and Haizhou Li. Deep bidirectional LSTM modeling of timbre and prosody for emotional voice conversion. In IN- TERSPEECH, pages 2453–2457. ISCA, 2016

    2016

  12. [20]

    V oiceshop: A unified speech-to-speech framework for identity-preserving zero-shot voice editing

    Philip Anastassiou, Zhenyu Tang, Kainan Peng, Dongya Jia, Jiaxin Li, Ming Tu, Yuping Wang, Yuxuan Wang, and Mingbo Ma. V oiceshop: A unified speech-to-speech framework for identity-preserving zero-shot voice editing. arXiv preprint, abs/2404.06674, 2024

    2024 arXiv

  13. [21]

    Convert and speak: Zero-shot accent conversion with minimum supervision

    Huaying Xue, Xiulian Peng, Yan Lu, et al. Convert and speak: Zero-shot accent conversion with minimum supervision. In ACM Multimedia. ACM, 2024

    2024

  14. [22]

    Au- tovc: Zero-shot voice style transfer with only autoencoder loss

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. Au- tovc: Zero-shot voice style transfer with only autoencoder loss. In ICML, volume 97 of Pro- ceedings of Machine Learning Research , pages 5210–5219. PMLR, 2019

    2019

  15. [23]

    BASE TTS: lessons from building a billion- parameter text-to-speech model on 100k hours of data

    Mateusz Lajszczak, Guillermo C ´ambara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, ´Alvaro Mart´ın-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszynska, Haohan Guo, Bartosz Putrycz, Soledad L ´opez Gambino, Kayeon Yoo, Elena ...

    2024 arXiv

  16. [24]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint , abs/2407.05407, 2024

    2024 arXiv

  17. [25]

    Better speech synthesis through scaling

    James Betker. Better speech synthesis through scaling. arXiv preprint, abs/2305.07243, 2023

    2023 arXiv

  18. [26]

    Neural codec language models are zero-shot text to speech synthesizers

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint, abs/2301.02111, 2023

    2023 arXiv

  19. [27]

    V oicebox: Text- guided multilingual universal speech generation at scale

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. V oicebox: Text- guided multilingual universal speech generation at scale. In NeurIPS, 2023

    2023

  20. [28]

    Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Haohan Guo, Xu- ankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Zhou Zhao, Xixin Wu, and Helen M. Meng. Uniaudio: Towards universal audio generation with large language models. In ICML. OpenReview.net, 2024....

    2024

  21. [29]

    V oice- preserving zero-shot multiple accent conversion

    Mumin Jin, Prashant Serai, Jilong Wu, Andros Tjandra, Vimal Manohar, and Qing He. V oice- preserving zero-shot multiple accent conversion. In ICASSP, pages 1–5. IEEE, 2023

    2023

  22. [30]

    Schuller, and Haizhou Li

    Kun Zhou, Berrak Sisman, Rajib Rana, Bj ¨orn W. Schuller, and Haizhou Li. Emotion intensity and its control for emotional voice conversion. IEEE Trans. Affect. Comput. , 14(1):31–48, 2023

    2023

  23. [31]

    PA VITS: exploring prosody-aware VITS for end-to-end emotional voice conversion

    Tianhua Qi, Wenming Zheng, Cheng Lu, Yuan Zong, and Hailun Lian. PA VITS: exploring prosody-aware VITS for end-to-end emotional voice conversion. In ICASSP, pages 12697– 12701. IEEE, 2024

    2024

  24. [32]

    Transfer the linguistic representa- tions from TTS to accent conversion with non-parallel data

    Xi Chen, Jiakun Pei, Liumeng Xue, and Mingyang Zhang. Transfer the linguistic representa- tions from TTS to accent conversion with non-parallel data. In ICASSP, pages 12501–12505. IEEE, 2024

    2024

  25. [33]

    U-style: Cascading u-nets with multi-level speaker and style modeling for zero-shot voice cloning

    Tao Li, Zhichao Wang, Xinfa Zhu, Jian Cong, Qiao Tian, Yuping Wang, and Lei Xie. U-style: Cascading u-nets with multi-level speaker and style modeling for zero-shot voice cloning. IEEE ACM Trans. Audio Speech Lang. Process., 2024

    2024

  26. [34]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998–6008, 2017

    2017

  27. [35]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- oth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aur ´elien Ro- driguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation...

    2023 arXiv

  28. [36]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In ICLR. OpenReview.net, 2023

    2023

  29. [37]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pages 4172–4182. IEEE, 2023

    2023

  30. [38]

    Audiobox: Unified audio generation with natural language prompts

    Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, Chris Summers, ...

    2023 arXiv

  31. [39]

    One-shot voice conversion by vector quantization

    Da-Yi Wu and Hung-Yi Lee. One-shot voice conversion by vector quantization. In ICASSP, pages 7734–7738. IEEE, 2020

    2020

  32. [40]

    Unsupervised learning of disentangled speech content and style representation

    Andros Tjandra, Ruoming Pang, Yu Zhang, and Shigeki Karita. Unsupervised learning of disentangled speech content and style representation. In INTERSPEECH, pages 4089–4093. ISCA, 2020

    2020

  33. [41]

    Text- less speech-to-speech translation on real data

    Ann Lee, Hongyu Gong, Paul-Ambroise Duquenne, Holger Schwenk, Peng-Jen Chen, Chang- han Wang, Sravya Popuri, Yossi Adi, Juan Miguel Pino, Jiatao Gu, and Wei-Ning Hsu. Text- less speech-to-speech translation on real data. In NAACL-HLT, pages 860–872. Association for Computation...

    2022

  34. [42]

    Lifa Sun, Kun Li, Hao Wang, Shiyin Kang, and Helen M. Meng. Phonetic posteriorgrams for many-to-one voice conversion without parallel data training. In ICME, pages 1–6. IEEE Computer Society, 2016

    2016

  35. [43]

    A comparative study of self-supervised speech representation based voice conversion

    Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi, and Tomoki Toda. A comparative study of self-supervised speech representation based voice conversion. IEEE J. Sel. Top. Signal Process., 16(6):1308–1318, 2022

    2022

  36. [44]

    Leveraging diverse semantic-based audio pretrained models for singing voice conversion

    Xueyao Zhang, Zihao Fang, Yicheng Gu, Haopeng Chen, Lexiao Zou, Junan Zhang, Liu- meng Xue, and Zhizheng Wu. Leveraging diverse semantic-based audio pretrained models for singing voice conversion. In SLT. IEEE, 2024. 13 Published as a conference paper at ICLR 2025

    2024

  37. [45]

    Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial networks

    Takuhiro Kaneko and Hirokazu Kameoka. Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial networks. In EUSIPCO, pages 2100–2104. IEEE, 2018

    2018

  38. [46]

    Stargan-vc: non- parallel many-to-many voice conversion using star generative adversarial networks

    Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, and Nobukatsu Hojo. Stargan-vc: non- parallel many-to-many voice conversion using star generative adversarial networks. In SLT. IEEE, 2018

    2018

  39. [47]

    Diffusion-based voice conversion with fast maximum likelihood sam- pling scheme

    Vadim Popov, Ivan V ovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Sergeevich Kudi- nov, and Jiansheng Wei. Diffusion-based voice conversion with fast maximum likelihood sam- pling scheme. In ICLR. OpenReview.net, 2022

    2022

  40. [48]

    Diff-hiervc: Diffusion-based hierar- chical voice conversion with robust pitch generation and masked prior for zero-shot speaker adaptation

    Ha-Yeong Choi, Sang-Hoon Lee, and Seong-Whan Lee. Diff-hiervc: Diffusion-based hierar- chical voice conversion with robust pitch generation and masked prior for zero-shot speaker adaptation. In INTERSPEECH, pages 2283–2287. ISCA, 2023

    2023

  41. [49]

    Tts-guided training for accent conversion without parallel data

    Yi Zhou, Zhizheng Wu, Mingyang Zhang, Xiaohai Tian, and Haizhou Li. Tts-guided training for accent conversion without parallel data. IEEE Signal Process. Lett., 30:533–537, 2023

    2023

  42. [50]

    End-to-end accent conversion without using native utterances

    Songxiang Liu, Disong Wang, Yuewen Cao, Lifa Sun, Xixin Wu, Shiyin Kang, Zhiyong Wu, Xunying Liu, Dan Su, Dong Yu, and Helen Meng. End-to-end accent conversion without using native utterances. In ICASSP, pages 6289–6293. IEEE, 2020

    2020

  43. [51]

    Non-parallel sequence-to-sequence voice conversion with disentangled linguistic and speaker representations

    Jing-Xuan Zhang, Zhen-Hua Ling, and Li-Rong Dai. Non-parallel sequence-to-sequence voice conversion with disentangled linguistic and speaker representations. IEEE ACM Trans. Audio Speech Lang. Process., 28:540–552, 2020

    2020

  44. [52]

    LM-VC: zero-shot voice conversion via speech generation based on language models.IEEE Signal Process

    Zhichao Wang, Yuanzhe Chen, Lei Xie, Qiao Tian, and Yuping Wang. LM-VC: zero-shot voice conversion via speech generation based on language models.IEEE Signal Process. Lett., 30:1157–1161, 2023

    2023

  45. [53]

    Hierspeech++: Bridging the gap between semantic and acoustic representation of speech by hierarchical vari- ational inference for zero-shot speech synthesis

    Sang-Hoon Lee, Ha-Yeong Choi, Seung-Bin Kim, and Seong-Whan Lee. Hierspeech++: Bridging the gap between semantic and acoustic representation of speech by hierarchical vari- ational inference for zero-shot speech synthesis. arXiv preprint, abs/2311.12454, 2023

    2023 arXiv

  46. [54]

    A comparison of discrete and soft speech units for improved voice conversion

    Benjamin van Niekerk, Marc-Andr ´e Carbonneau, Julian Za ¨ıdi, Matthew Baas, Hugo Seut ´e, and Herman Kamper. A comparison of discrete and soft speech units for improved voice conversion. In ICASSP, pages 6562–6566. IEEE, 2022

    2022

  47. [55]

    SEF-VC: speaker embedding free zero-shot voice conversion with cross attention

    Junjie Li, Yiwei Guo, Xie Chen, and Kai Yu. SEF-VC: speaker embedding free zero-shot voice conversion with cross attention. In ICASSP, pages 12296–12300. IEEE, 2024

    2024

  48. [56]

    Neu- ral analysis and synthesis: Reconstructing speech from self-supervised representations

    Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Lee, Hoon Heo, and Kyogu Lee. Neu- ral analysis and synthesis: Reconstructing speech from self-supervised representations. In NeurIPS, pages 16251–16265, 2021

    2021

  49. [57]

    NANSY++: unified voice synthesis with neural analysis and synthesis

    Hyeong-Seok Choi, Jinhyeok Yang, Juheon Lee, and Hyeongju Kim. NANSY++: unified voice synthesis with neural analysis and synthesis. In ICLR. OpenReview.net, 2023

    2023

  50. [58]

    Speechsplit2.0: Unsupervised speech disentanglement for voice conversion without tuning autoencoder bottle- necks

    Chak Ho Chan, Kaizhi Qian, Yang Zhang, and Mark Hasegawa-Johnson. Speechsplit2.0: Unsupervised speech disentanglement for voice conversion without tuning autoencoder bottle- necks. In ICASSP, pages 6332–6336. IEEE, 2022

    2022

  51. [59]

    Cox, Mark Hasegawa-Johnson, and Shiyu Chang

    Kaizhi Qian, Yang Zhang, Heting Gao, Junrui Ni, Cheng-I Lai, David D. Cox, Mark Hasegawa-Johnson, and Shiyu Chang. Contentvec: An improved self-supervised speech repre- sentation by disentangling speakers. In ICML, volume 162 of Proceedings of Machine Learn- ing Research, page...

    2022

  52. [60]

    Multi-speaker expressive speech synthesis via multiple factors decoupling

    Xinfa Zhu, Yi Lei, Kun Song, Yongmao Zhang, Tao Li, and Lei Xie. Multi-speaker expressive speech synthesis via multiple factors decoupling. In ICASSP, pages 1–5. IEEE, 2023

    2023

  53. [61]

    CLUB: A contrastive log-ratio upper bound of mutual information

    Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. CLUB: A contrastive log-ratio upper bound of mutual information. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 1779–1788. PMLR, 2020. 14 Published as a conference paper a...

    2020

  54. [62]

    Repcodec: A speech representation codec for speech tokenization

    Zhichao Huang, Chutong Meng, and Tom Ko. Repcodec: A speech representation codec for speech tokenization. In ACL (1), pages 5777–5790. Association for Computational Linguistics, 2024

    2024

  55. [63]

    Soundstream: An end-to-end neural audio codec

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. Soundstream: An end-to-end neural audio codec. IEEE ACM Trans. Audio Speech Lang. Process., 30:495–507, 2022

    2022

  56. [64]

    Wavlm: Large-scale self- supervised pre-training for full stack speech processing

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. Wavlm: Large-scale self- supervised pre-train...

    2022

  57. [65]

    ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification

    Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification. In IN- TERSPEECH, pages 3830–3834. ISCA, 2020

    2020

  58. [66]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), pages 4171–

  59. [67]

    Bigvgan: A universal neural vocoder with large-scale training

    Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon. Bigvgan: A universal neural vocoder with large-scale training. In ICLR. OpenReview.net, 2023

    2023

  60. [68]

    Tyers, and Gregor Weber

    Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. Common voice: A massively-multilingual speech corpus. In LREC, pages 4218–4222. European Language Resources Association, 2020

    2020

  61. [69]

    The singing voice conversion challenge 2023

    Wen-Chin Huang, Lester Phillip Violeta, Songxiang Liu, Jiatong Shi, and Tomoki Toda. The singing voice conversion challenge 2023. In ASRU, pages 1–8. IEEE, 2023

    2023

  62. [70]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In ICML, volume 202, pages 28492–28518, 2023

    2023

  63. [71]

    Commonaccent: Exploring large acoustic pretrained models for accent classification based on common voice

    Juan Zuluaga-Gomez, Sara Ahmed, Danielius Visockas, and Cem Subakan. Commonaccent: Exploring large acoustic pretrained models for accent classification based on common voice. In INTERSPEECH, pages 5291–5295. ISCA, 2023

    2023

  64. [72]

    emotion2vec: Self-supervised pre-training for speech emotion representation

    Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, and Xie Chen. emotion2vec: Self-supervised pre-training for speech emotion representation. In ACL (Find- ings), pages 15747–15760. Association for Computational Linguistics, 2024

    2024

  65. [73]

    Stuart P. Lloyd. Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28(2):129–136, 1982

    1982

  66. [74]

    Libri-light: A benchmark for ASR with limited or no supervision

    Jacob Kahn, Morgane Rivi `ere, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Pierre- Emmanuel Mazar´e, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, Tatiana Likhomanenko, Gabriel Synnaeve, Armand Joulin, Abdelrahman Mohamed, and Em- manuel Dupoux. Lib...

    2020

  67. [75]

    Librispeech: An ASR corpus based on public domain audio books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: An ASR corpus based on public domain audio books. In ICASSP, pages 5206–5210. IEEE, 2015

    2015

  68. [76]

    Amphion: An open-source audio, music and speech generation toolkit

    Xueyao Zhang, Liumeng Xue, Yicheng Gu, Yuancheng Wang, Jiaqi Li, Haorui He, Chaoren Wang, Ting Song, Xi Chen, Zihao Fang, Haopeng Chen, Junan Zhang, Tze Ying Tang, Lexiao Zou, Mingxuan Wang, Jun Han, Kai Chen, Haizhou Li, and Zhizheng Wu. Amphion: An open-source audio, music a...

    2024

  69. [77]

    V oicecraft: Zero-shot speech editing and text-to-speech in the wild

    Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, and David Harwath. V oicecraft: Zero-shot speech editing and text-to-speech in the wild. InACL (1), pages 12442– 12462. Association for Computational Linguistics, 2024

    2024

  70. [78]

    Overview of the amphion toolkit (v0.2)

    Jiaqi Li, Xueyao Zhang, Yuancheng Wang, Haorui He, Chaoren Wang, Li Wang, Huan Liao, Junyi Ao, Zeyu Xie, Yiqiao Huang, Junan Zhang, and Zhizheng Wu. Overview of the amphion toolkit (v0.2). arXiv preprint, abs/2501.15442, 2025

    2025 arXiv

  71. [79]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation

    Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. I...

    2024

  72. [80]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR (Poster). OpenReview.net, 2019

    2019

  73. [81]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015

    2015

  74. [82]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint , abs/2207.12598, 2022

    2022 arXiv

  75. [83]

    Scaling speech technology to 1, 000+ languages

    Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. Scaling speech technology to 1, 000+ langu...

    2024

  76. [84]

    Conditional variational autoencoder with adver- sarial learning for end-to-end text-to-speech

    Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adver- sarial learning for end-to-end text-to-speech. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 5530–5540. PMLR, 2021

    2021

  77. [85]

    Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu

    Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. LibriTTS: A corpus derived from librispeech for text-to-speech. InINTERSPEECH, pages 1526–1530, 2019

    2019

  78. [86]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In NeurIPS, 2020

    2020

  79. [87]

    BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL, pages 7871–7880. Associ...

    2020

  80. [88]

    CSTR VCTK Corpus: En- glish multi-speaker corpus for cstr voice cloning toolkit (version 0.92)

    Junichi Yamagishi, Christophe Veaux, Kirsten MacDonald, et al. CSTR VCTK Corpus: En- glish multi-speaker corpus for cstr voice cloning toolkit (version 0.92). University of Edin- burgh. The Centre for Speech Technology Research (CSTR), 2019

    2019

  81. [89]

    High fidelity neural audio compression

    Alexandre D ´efossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprint, abs/2210.13438, 2022

    2022 arXiv

  82. [90]

    MLS: A large-scale multilingual dataset for speech research

    Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. MLS: A large-scale multilingual dataset for speech research. In INTERSPEECH, pages 2757–2761. ISCA, 2020

    2020

  83. [91]

    Gigaspeech: An evolving, multi-domain ASR corpus with 10, 000 hours of transcribed audio

    Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watan- abe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Zhao You, and Zhiyong Yan. Gigaspee...

    2021

  84. [92]

    bit” vs. “bet

    Zal ´an Borsos, Rapha ¨el Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matthew Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. Audiolm: A language modeling approach to audio generation. IEEE ACM Trans. Audio Sp...

    2023

  85. [4186]

    Association for Computational Linguistics, 2019

    2019

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.