Pith. sign in

REVIEW 3 major objections 5 minor 48 references

Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Conan converts speech chunk by chunk, keeping the source words while matching an unseen reference voice, with latency as low as 37 ms.

desk verdict A genuinely novel causal pixel-shuffle vocoder and a sensible streaming VC architecture, but the only online baseline comparison rests on borrowed metrics that don't hold up. read the letter →

arxiv 2507.14534 v4 pith:4SLHKW7C submitted 2025-07-19 eess.AS cs.CLcs.SD

Yu Zhang , Baotong Tian , Zhiyao Duan This is my paper · ORCID
classification eess.AScs.CLcs.SD
keywords zero-shotvoiceconversiononlinestreamingspeechprocessingEmformeradaptivestyleencodercausalshufflevocoderpixellow-latencysynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Conan is a voice conversion system designed to work live: it takes ordinary speech in chunks, keeps the words and meaning of the original utterance, and makes the voice sound like an unseen reference speaker. The paper argues that the right combination of streaming content extraction, adaptive style encoding, and a fully causal vocoder solves the three hard problems of online zero-shot conversion: content fidelity under latency, transfer of timbre and speaking style, and naturalness at chunk boundaries. If the claims hold, real-time communications and interactive entertainment could use voice conversion with only tens of milliseconds of delay, without giving up the source content or the target voice identity. The paper reports that Conan beats the compared baselines on word error rate, speaker similarity, and naturalness, with latency as low as 37 ms in its fast setting.

What carries the argument

The load-bearing mechanism is the chunkwise causal pipeline itself, but the component that carries the quality claim is the Causal Shuffle Vocoder: instead of transposed convolutions, it projects the mel features to more channels and rearranges them in time with pixel shuffle, so upsampling never looks at future frames and never creates checkerboard artifacts. The Stream Content Extractor supplies the same role HuBERT plays offline by using Emformer's memory-bank attention to keep cross-chunk context. The Adaptive Style Encoder supplies the unseen speaker's detailed style through a clustered vector-quantization bottleneck plus an align-attention step that fuses style with the content and timbre embeddings. Together these let each 20 ms or 80 ms chunk be converted independently while a sliding context window smooths the boundaries.

What would settle it

A matched head-to-head test would settle the claim: run Conan and StreamVC on identical source-reference pairs with the same ASR for WER and CER, the same speaker-similarity model, and the same listeners for MOS and CMOS. If Conan does not beat StreamVC on these metrics under identical protocols, the paper's central claim of outperforming baselines fails.

Watch

Extended reading notes

Core claim

The central claim is that a fully chunkwise, causal network can preserve the source utterance's content while adopting the timbre and style of a reference speaker, and that this design outperforms prior zero-shot voice conversion systems. Content is handled by a Stream Content Extractor: an Emformer trained to predict the same 20 ms HuBERT content labels an offline encoder would produce, so no future context is needed. Style is handled by an Adaptive Style Encoder that quantizes chunk-level reference representations with clustering-based vector quantization and then aligns them to the content and timbre via attention. Synthesis is handled by a Causal Shuffle Vocoder that rebuilds HiFi-GAN with causal convolutions and pixel-shuffle upsampling, avoiding the artifacts of zero-padded causal vocoders. On LibriTTS test-clean with VCTK reference speakers, the full Conan model reports the lowest WER and highest MOS-S among the compared systems, and the strictly causal fast model runs at 37 ms total latency.

Load-bearing premise

The load-bearing premise is that the numbers reported for the main competing streaming system, StreamVC, were measured under the same conditions as Conan's own evaluations; if the protocols and listening tests differ, the claimed superiority over StreamVC is not established.

Editorial extensions

If this is right

  • Real-time services could add a natural-sounding voice conversion stage with only tens of milliseconds of latency while keeping the original words.
  • The same distillation recipe, training a streaming encoder to match offline HuBERT labels, could be reused for other streaming speech tasks that currently depend on non-causal self-supervised models.
  • The Causal Shuffle Vocoder's pixel-shuffle upsampling gives streaming synthesizers an alternative to zero-padded causal vocoders, removing a known source of spectral artifacts.
  • The configurable right-context chunks give deployers a direct latency-versus-quality trade-off: 80 ms chunks with two lookahead chunks for quality, 20 ms strictly causal chunks for speed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same chunkwise design could be ported to singing voice conversion or streaming text-to-speech, where the content labels and F0 path would carry melody instead of speech prosody.
  • Because the Adaptive Style Encoder uses a discrete codebook, a natural next experiment is to test whether individual codes correspond to interpretable emotions or prosody and whether manipulating them gives real-time style control.
  • Latency is bounded by chunk size plus per-module delay, so smaller chunks would push below 37 ms, presumably at some cost to content accuracy; the paper's full and fast settings bracket that trade-off.
  • The reported comparison with StreamVC relies on published numbers and demo pages, so a matched head-to-head test would be the cleanest way to confirm the superiority claim beyond the paper's own protocol.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Conan, a chunkwise online zero-shot voice conversion system with three main components: a Stream Content Extractor based on Emformer that distills HuBERT content labels for low-latency streaming content encoding, an Adaptive Style Encoder that uses clustering vector quantization and alignment attention to transfer reference-speaker style, and a Causal Shuffle Vocoder that makes HiFi-GAN fully causal via pixel-shuffle upsampling. The system is evaluated on LibriTTS and VCTK with objective metrics (WER, CER, SIM) and subjective MOS/CMOS tests, and the authors report a fast configuration with 37 ms latency. The central claim is that Conan outperforms baseline models in content accuracy, speaker similarity, and quality while operating online.

Significance. If the central claim is established, Conan would be a useful contribution to real-time zero-shot voice conversion, particularly because it addresses three recognized difficulties: streaming content extraction, fine-grained style transfer, and causal high-quality vocoding. The architectural ideas are clearly motivated, and the Causal Shuffle Vocoder is an elegant way to avoid transposed-convolution artifacts in a causal setting. The latency measurements are valuable and the ablation study is a strength, as it isolates the contribution of each component. However, the evaluation protocol for the only online baseline, StreamVC, is not a matched comparison, and the content metric is partly aligned with the training target; these issues currently weaken the evidence for the headline superiority claim.

major comments (3)
  1. [IV-B, Table I] The claim that Conan outperforms StreamVC, the only online baseline, rests on an unmatched comparison. The paper states that StreamVC is not open-sourced and that the authors 'rely on the objective metrics reported in their paper and perform subjective assessments based on their demo pages.' Different ASR versions, resampling, utterance filtering, and listener pools can shift WER, CER, and MOS by amounts comparable to the reported differences (e.g., WER 6.02 vs. 6.22, MOS-S 4.02 vs. 3.86). The implausible internal consistency check in Table I—StreamVC's CER (2.17%) is lower than the Source CER (2.43%) on the same test set—strongly suggests that the evaluation sets or ASR pipelines are not actually identical. This undermines the paper's central claim of superiority over the only streaming baseline. The authors should either re-run StreamVC under the same protocol, or explicitly restrict the superiority claim to the offline baselines and reframe the streaming comparison as suggestive.
  2. [III-B and IV-A] The content metric is partly circular. The Stream Content Extractor is trained by distilling HuBERT content labels (Section III-B), and content accuracy is measured with a HuBERT-Large ASR model (Section IV-A). Since both the training target and the evaluation metric are derived from HuBERT, low WER/CER may reflect how well the extractor mimics HuBERT rather than how faithfully linguistic content is preserved for human listeners. This is a correctness-risk concern for the content-accuracy claims, not merely a philosophical objection. The authors should add a human intelligibility test or use an ASR model from a different family (e.g., Whisper) to verify that the content-accuracy advantage is not an artifact of the HuBERT alignment.
  3. [IV-A and IV-B] Objective results are reported without any measure of variance or statistical significance. WER, CER, and SIM in Table I are point estimates, and the subjective MOS values carry 95% confidence intervals, but the objective differences among the strongest systems are small (e.g., WER 6.02% vs. 7.14% for QuickVC, SIM 85.71% vs. 81.47% for Diff-VCTK). Without confidence intervals, significance tests, or multiple utterance-level resampling, the reader cannot tell whether these differences are stable. The authors should report objective metrics with confidence intervals or bootstrap significance, or at least specify the number of utterances and any test-set construction that would support the comparison.
minor comments (5)
  1. [Abstract and Section IV-B] The abstract states a latency 'as low as 37 ms,' but Table II reports 139.71 ms for the full setting and 36.87 ms for the fast setting. The abstract should clarify that the 37 ms figure corresponds to the fast, strictly causal configuration.
  2. [III-B, Eq. (6)] In Eq. (6), the notation 'arg max_{1≤j≤J} [Softmax(U C_i^{(N)})]^{(j,t)}' is ambiguous because the softmax is taken over the class dimension but the (j,t) indexing is not fully defined. Please clarify the exact tensor dimensions and the role of t.
  3. [Throughout] The component name is inconsistent: 'Stream Content Extractor' appears in the abstract and Section III, while 'Stream Content Encoder' appears in the ablation study (Table III). Please unify the terminology.
  4. [IV-B] The explanation that StreamVC's CER advantage comes from reusing source pitch and energy is presented without supporting evidence. If this claim is retained, it should be backed by an experiment or a citation; otherwise it should be softened.
  5. [II-C] The related-work section lists many style-modeling methods, but the positioning of Conan relative to recent streaming style-transfer work (e.g., Yang et al. [5]) could be sharper: specifically, what does the Adaptive Style Encoder capture that the GST-based approach in [5] does not?

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: Conan's components are evaluated with external metrics and ablations; the StreamVC comparison issue is a validity concern, not a circular derivation.

full rationale

Conan's derivation chain is self-contained. The Stream Content Extractor is trained by cross-entropy to match offline HuBERT labels (Sec. III-E), but content accuracy is measured with WER/CER from a separate HuBERT-Large ASR on the final converted waveform; WER is an external intelligibility metric, not an identity with the training target. Speaker similarity is measured with Resemblyzer, an external speaker embedding model, and quality/naturalness with human MOS/CMOS judgments. The ablations (Table III) compare against removing each proposed component, and the conclusions track the measured metrics. The only notable weakness is the StreamVC comparison (Sec. IV-B), where metrics are borrowed from StreamVC's paper and demo pages rather than measured under a matched protocol; that is a validity/external-comparability risk, not a circularity, because Conan's own numbers are still generated by the stated evaluation pipeline. Self-citations (e.g., [4], [45]) appear only as background or metric references and are not load-bearing. No prediction reduces by construction to a fitted input, and no load-bearing claim rests on a self-citation. Score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions about content representation, speaker decomposition, and evaluation validity. No free parameters are fitted in a mathematical derivation; the system's performance depends on hand-chosen hyperparameters and unstated loss weights.

free parameters (4)
  • chunk size = 80 ms (full) / 20 ms (fast)
    Chosen by hand to trade latency against quality; central to the online design.
  • right context chunks = 2 (full) / 0 (fast)
    Controls lookahead; chosen by hand; ablation shows it affects WER and CMOS.
  • CVQ codebook size = 128
    Default size, affects style discretization capacity.
  • loss weighting hyperparameters = not reported
    Weights for L_CVQ, L_pitch, L_mae, L_ssim, L_GAN are not specified, yet they determine the balance between content preservation and speaker similarity.
assumptions (4)
  • domain assumption HuBERT content labels are a sufficient representation of linguistic content for voice conversion.
    Section III-B: the content extractor is trained to reproduce HuBERT labels at 20 ms intervals; if HuBERT loses prosody or speaker-independent content, Conan inherits that loss.
  • domain assumption Speaker identity can be decomposed into a global timbre embedding and a chunk-level style embedding derived from reference mel-spectrogram.
    Section III-A and III-C: this decomposition is assumed by the architecture; no proof is offered that the two encoders do not entangle.
  • ad hoc to paper Chunkwise causal processing with a sliding context window produces acceptable quality at chunk boundaries.
    Section III-E: the inference procedure assumes overlapping context ensures smooth transitions; no specific boundary artifact analysis is provided.
  • domain assumption The adopted evaluation metrics (WER/CER via HuBERT-Large ASR, SIM via Resemblyzer, MOS) are valid proxies for voice conversion quality.
    Section IV-A: metrics are defined but not validated against human perception beyond MOS; the WER metric aligns with the HuBERT training target.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion." pith.science (2026). https://pith.science/paper/4SLHKW7C

@misc{pith2026250714534,
  author       = {Pith},
  title        = {Pith review of: Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4SLHKW7C}},
  note         = {Machine review of arXiv:2507.14534}
}
read the original abstract

Zero-shot online voice conversion (VC) holds significant promise for real-time communications and entertainment. However, current VC models struggle to preserve semantic fidelity under real-time constraints, deliver natural-sounding conversions, and adapt effectively to unseen speaker characteristics. To address these challenges, we introduce Conan, a chunkwise online zero-shot voice conversion model that preserves the content of the source while matching the voice timbre and styles of reference speech. Conan comprises three core components: 1) a Stream Content Extractor that leverages Emformer for low-latency streaming content encoding; 2) an Adaptive Style Encoder that extracts fine-grained stylistic features from reference speech for enhanced style adaptation; 3) a Causal Shuffle Vocoder that implements a fully causal HiFiGAN using a pixel-shuffle mechanism. Experimental evaluations demonstrate that Conan outperforms baseline models in subjective and objective metrics. Audio samples can be found at https://aaronz345.github.io/ConanDemo.

Figures

Figures reproduced from arXiv: 2507.14534 by the authors.

Figure 1
Figure 1. The overall architecture of Conan (a). Online source speech is fed into the system in chunks, and the synthesized output is likewise produced on a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The architecture of Adaptive Style Encoder. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The chunkwise online inference procedure of Conan. Here, we [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 36 canonical work pages

  1. [1]

    StreamVoice: Streamable context-aware language modeling for real-time zero-shot voice conversion,

    Z. Wang, Y . Chen, X. Wang, L. Xie, and Y . Wang, “StreamVoice: Streamable context-aware language modeling for real-time zero-shot voice conversion,” in Proc. Annual Meeting of the Association for Com- putational Linguistics (ACL) , L.-W. Ku, A. Martins, and V . Srikumar, Eds., Bangkok, Thailand, 2024, pp. 7328–7338

    work page 2024

  2. [2]

    Vqmivc: Vector quantization and mutual information-based unsuper- vised speech representation disentanglement for one-shot voice conver- sion,

    D. Wang, L. Deng, Y . T. Yeung, X. Chen, X. Liu, and H. Meng, “Vqmivc: Vector quantization and mutual information-based unsuper- vised speech representation disentanglement for one-shot voice conver- sion,” in Proc. Interspeech, 2021, pp. 1344–1348

    work page 2021

  3. [3]

    Autovc: Zero-shot voice style transfer with only autoencoder loss,

    K. Qian, Y . Zhang, S. Chang, X. Yang, and M. Hasegawa-Johnson, “Autovc: Zero-shot voice style transfer with only autoencoder loss,” in Proc. International Conference on Machine Learning (ICML) . PMLR, 2019

    work page 2019

  4. [4]

    Controlvc: Zero-shot voice conversion with time-varying controls on pitch and speed,

    M. Chen and Z. Duan, “Controlvc: Zero-shot voice conversion with time-varying controls on pitch and speed,” in Proc. Interspeech, 2023, pp. 2098–2102

    work page 2023

  5. [5]

    Streamable Speech Representation Disentanglement and Multi-Level Prosody Modeling for Live One-Shot V oice Conversion,

    Haoquan Yang and Liqun Deng and Yu Ting Yeung and Nianzu Zheng and Yong Xu, “Streamable Speech Representation Disentanglement and Multi-Level Prosody Modeling for Live One-Shot V oice Conversion,” in Proc. Interspeech, 2022, pp. 2578–2582

    work page 2022

  6. [6]

    Streamvc: Real-time low-latency voice conversion,

    Y . Yang, Y . Kartynnik, Y . Li, J. Tang, X. Li, G. Sung, and M. Grund- mann, “Streamvc: Real-time low-latency voice conversion,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2024

    work page 2024

  7. [7]

    Tcsinger 2: Customizable multilingual zero-shot singing voice synthesis,

    Y . Zhang, W. Guo, C. Pan, D. Yao, Z. Zhu, Z. Jiang, Y . Wang, T. Jin, and Z. Zhao, “Tcsinger 2: Customizable multilingual zero-shot singing voice synthesis,” arXiv preprint arXiv:2505.14910 , 2025

    arXiv 2025

  8. [8]

    Isdrama: Immersive spatial drama generation through multimodal prompting,

    Y . Zhang, W. Guo, C. Pan, Z. Zhu, T. Jin, and Z. Zhao, “Isdrama: Immersive spatial drama generation through multimodal prompting,” arXiv preprint arXiv:2504.20630 , 2025

    arXiv 2025

Show all 48 references
  1. [9]

    A comparison of discrete and soft speech units for improved voice conversion,

    B. Van Niekerk, M.-A. Carbonneau, J. Za ¨ıdi, M. Baas, H. Seut ´e, and H. Kamper, “A comparison of discrete and soft speech units for improved voice conversion,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022

    2022

  2. [10]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  3. [11]

    Phonetic pos- teriorgrams for many-to-one voice conversion without parallel data training,

    L. Sun, K. Li, H. Wang, S. Kang, and H. Meng, “Phonetic pos- teriorgrams for many-to-one voice conversion without parallel data training,” in 2016 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2016, pp. 1–6

    2016

  4. [12]

    Starganv2-vc: A diverse, unsuper- vised, non-parallel framework for natural-sounding voice conversion,

    Y . A. Li, A. Zare, and N. Mesgarani, “Starganv2-vc: A diverse, unsuper- vised, non-parallel framework for natural-sounding voice conversion,” in Proc. Interspeech, 2021, pp. 1349–1353

    2021

  5. [13]

    End-to-end streaming model for low-latency speech anonymization,

    W. Quamer and R. Gutierrez-Osuna, “End-to-end streaming model for low-latency speech anonymization,” in 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 727–734

    2024

  6. [14]

    Contrastive predictive coding supported factorized variational autoen- coder for unsupervised learning of disentangled speech representations,

    J. Ebbers, M. Kuhlmann, T. Cord-Landwehr, and R. Haeb-Umbach, “Contrastive predictive coding supported factorized variational autoen- coder for unsupervised learning of disentangled speech representations,” in Proc. IEEE International Conference on Acoustics, Speech and Signal...

    2021

  7. [15]

    Neural analysis and synthesis: Reconstructing speech from self-supervised representations,

    H.-S. Choi, J. Lee, W. Kim, J. Lee, H. Heo, and K. Lee, “Neural analysis and synthesis: Reconstructing speech from self-supervised representations,” Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  8. [16]

    Lm-vc: Zero-shot voice conversion via speech generation based on language models,

    Z. Wang, Y . Chen, L. Xie, Q. Tian, and Y . Wang, “Lm-vc: Zero-shot voice conversion via speech generation based on language models,” IEEE Signal Processing Letters , 2023

    2023

  9. [17]

    An investigation of streaming non-autoregressive sequence-to-sequence voice conversion,

    T. Hayashi, K. Kobayashi, and T. Toda, “An investigation of streaming non-autoregressive sequence-to-sequence voice conversion,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2022

    2022

  10. [18]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in International Conference on Learning Representations (ICLR) , 2021

    2021

  11. [19]

    Non- autoregressive sequence-to-sequence voice conversion,

    T. Hayashi, W.-C. Huang, K. Kobayashi, and T. Toda, “Non- autoregressive sequence-to-sequence voice conversion,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021

    2021

  12. [20]

    Fasts2s-vc: Streaming non- autoregressive sequence-to-sequence voice conversion,

    H. Kameoka, K. Tanaka, and T. Kaneko, “Fasts2s-vc: Streaming non- autoregressive sequence-to-sequence voice conversion,” arXiv preprint arXiv:2104.06900, 2021

    2021 arXiv

  13. [21]

    Streaming voice conversion via intermediate bottleneck features and non-streaming teacher guidance,

    Y . Chen, M. Tu, T. Li, X. Li, Q. Kong, J. Li, Z. Wang, Q. Tian, Y . Wang, and Y . Wang, “Streaming voice conversion via intermediate bottleneck features and non-streaming teacher guidance,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICA...

    2023

  14. [22]

    Dualvc 2: Dynamic masked convolution for unified streaming and non-streaming voice conversion,

    Z. Ning, Y . Jiang, P. Zhu, S. Wang, J. Yao, L. Xie, and M. Bi, “Dualvc 2: Dynamic masked convolution for unified streaming and non-streaming voice conversion,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024

    2024

  15. [23]

    Alo-vc: Any-to-any low-latency one-shot voice conversion,

    B. Wang, D. Ronssin, and M. Cernak, “Alo-vc: Any-to-any low-latency one-shot voice conversion,” in Proc. Interspeech, 2023, pp. 2073–2077

    2023

  16. [24]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in Neural Information Processing Systems (NeurIPS) , 2020

    2020

  17. [25]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  18. [26]

    Attentron: Few-shot text-to- speech utilizing attention-based variable-length embedding,

    S. Choi, S. Han, D. Kim, and S. Ha, “Attentron: Few-shot text-to- speech utilizing attention-based variable-length embedding,” in Proc. Interspeech, 2020, pp. 2007–2011

    2020

  19. [27]

    Normalization driven zero- shot multi-speaker speech synthesis

    N. Kumar, S. Goel, A. Narang, and B. Lall, “Normalization driven zero- shot multi-speaker speech synthesis.” in Proc. Interspeech , 2021, pp. 1354–1358

    2021

  20. [28]

    Daft-exprt: Robust prosody transfer across speakers for expressive speech synthesis,

    J. Zaıdi, H. Seut ´e, B. Niekerk, and M. Carbonneau, “Daft-exprt: Robust prosody transfer across speakers for expressive speech synthesis,” arXiv preprint arXiv:2108.02271, 2021

    2021 arXiv

  21. [29]

    Generspeech: Towards style transfer for generalizable out-of-domain text-to-speech,

    R. Huang, Y . Ren, J. Liu, C. Cui, and Z. Zhao, “Generspeech: Towards style transfer for generalizable out-of-domain text-to-speech,” Advances in Neural Information Processing Systems (NeurIPS) , 2022

    2022

  22. [30]

    Styler: Style factor modeling with rapidity and robustness via speech decomposition for expressive and controllable neural text to speech,

    K. Lee, K. Park, and D. Kim, “Styler: Style factor modeling with rapidity and robustness via speech decomposition for expressive and controllable neural text to speech,” in Proc. Interspeech, 2021, pp. 4643–4647

    2021

  23. [31]

    Mega-tts 2: Boosting prompting mechanisms for zero-shot speech synthesis,

    Z. Jiang, J. Liu, Y . Ren, J. He, Z. Ye, S. Ji, Q. Yang, C. Zhang, P. Wei, C. Wang et al. , “Mega-tts 2: Boosting prompting mechanisms for zero-shot speech synthesis,” in The Twelfth International Conference on Learning Representations , 2024

    2024

  24. [32]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, E. Liu, Y . Leng, K. Song, S. Tang, Z. Wu, T. Qin, X. Li, W. Ye, S. Zhang, J. Bian, L. He, J. Li, and sheng zhao, “Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,” in Proc. International...

    2024

  25. [33]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Ma et al. , “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  26. [34]

    Fastpitch: Parallel text-to-speech with pitch prediction,

    A. Ła ´ncucki, “Fastpitch: Parallel text-to-speech with pitch prediction,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021

    2021

  27. [35]

    Tcsinger: Zero-shot singing voice synthesis with style transfer and multi-level style control,

    Y . Zhang, Z. Jiang, R. Li, C. Pan, J. He, R. Huang, C. Wang, and Z. Zhao, “Tcsinger: Zero-shot singing voice synthesis with style transfer and multi-level style control,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 1960– 1975

    2024

  28. [36]

    Online clustered codebook,

    C. Zheng and A. Vedaldi, “Online clustered codebook,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 22 798–22 807

    2023

  29. [37]

    Versatile framework for song generation with prompt-based control,

    Y . Zhang, W. Guo, C. Pan, Z. Zhu, R. Li, J. Lu, R. Huang, R. Zhang, Z. Hong, Z. Jiang et al., “Versatile framework for song generation with prompt-based control,” arXiv preprint arXiv:2504.19062 , 2025

    2025

  30. [38]

    Neural discrete representa- tion learning,

    A. Van Den Oord, O. Vinyals et al. , “Neural discrete representa- tion learning,” Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  31. [39]

    Stylesinger: Style transfer for out-of-domain singing voice synthesis,

    Y . Zhang, R. Huang, R. Li, J. He, Y . Xia, F. Chen, X. Duan, B. Huai, and Z. Zhao, “Stylesinger: Style transfer for out-of-domain singing voice synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 19 597–19 605

    2024

  32. [40]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS) , 2017

    2017

  33. [41]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in neural information processing systems , vol. 33, pp. 17 022–17 033, 2020

    2020

  34. [42]

    Deconvolution and checkerboard artifacts,

    A. Odena, V . Dumoulin, and C. Olah, “Deconvolution and checkerboard artifacts,” Distill, 2016. [Online]. Available: http://distill.pub/2016/ deconv-checkerboard

    2016

  35. [43]

    Least squares generative adversarial networks,

    X. Mao, Q. Li, H. Xie, R. Y . Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 2794–2802

    2017

  36. [44]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” in Proc. Interspeech, 2019, pp. 1526–1530

    2019

  37. [45]

    Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,

    Y . Zhang, C. Pan, W. Guo, R. Li, Z. Zhu, J. Wang, W. Xu, J. Lu, Z. Hong, C. Wang et al. , “Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,” Advances in Neural Information Processing Systems (NeurIPS) , 2024

    2024

  38. [46]

    Any-to-any generation via composable diffusion,

    Z. Tang, Z. Yang, C. Zhu, M. Zeng, and M. Bansal, “Any-to-any generation via composable diffusion,” Advances in Neural Information Processing Systems (NeurIPS) , 2023

    2023

  39. [47]

    Any-to-many voice conversion with location-relative sequence-to-sequence modeling,

    S. Liu, Y . Cao, D. Wang, X. Wu, X. Liu, and H. Meng, “Any-to-many voice conversion with location-relative sequence-to-sequence modeling,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 1717–1728, 2021

    2021

  40. [48]

    Quickvc: Any-to-many voice conversion using inverse short-time fourier transform for faster conversion,

    H. Guo, C. Liu, C. T. Ishi, and H. Ishiguro, “Quickvc: Any-to-many voice conversion using inverse short-time fourier transform for faster conversion,” arXiv preprint arXiv:2302.08296 , 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.