Pith. sign in

REVIEW 4 major objections 3 minor 56 references

Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that explicit conditioning on $F_0$, phonemes, intensity, and speaker identity is enough to make voice conversion freely controllable without sacrificing intelligibility.

desk verdict Fast-VGAN's architecture is a real step forward for controllable, lightweight VC, but the headline empirical claims rest on a test set of only two utterances per speaker and need stronger validation before they can be taken at face value. read the letter →

arxiv 2507.04817 v1 pith:ODJPMZ2X submitted 2025-07-07 cs.SD cs.AIeess.AS

Mathilde Abrassart , Nicolas Obin , Axel Roebel This is my paper
classification cs.SDcs.AIeess.AS
keywords voiceconversionprosodycontrolF0manipulationspeechrateexplicitconditioningGANsynthesismelspectrogramgenerationexpressive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fast-VGAN is a lightweight, non-autoregressive, fully convolutional generator that turns four interpretable inputs — fundamental frequency ($F_0$), intensity, an aligned phoneme sequence, and a speaker embedding — into mel spectrograms, which a universal neural vocoder then turns into audio. Because the model conditions on these factors explicitly rather than inferring them from the audio, each can be edited at inference time: pitch contours can be shifted by an octave, vowel durations stretched or compressed by a factor of three, the phoneme sequence changed, and the speaker embedding swapped. The authors test this on voice conversion against ControlVC and HiFi-VC, on static pitch and time manipulations, and on expressive resynthesis from the Expresso corpus without ever training on expressive data. Their central claim is that this controllability does not cost intelligibility or identity: they report a word error rate of 0.00% and a speaker similarity of 0.648 on their seen-speaker conversion test, with MOS naturalness of 3.63 and similarity of 3.47. If the claim holds, prosody control becomes an interpretable dial rather than a learned side effect, useful for dubbing, expressive synthesis, and studying which acoustic cues carry speaker identity.

What carries the argument

The engine is the Fast-VGAN generator: five transposed-convolution blocks mapping a time-frequency tensor formed from log-domain $F_0$ (with speaker log-mean subtracted), per-frame intensity, phoneme embeddings, and a unit-norm speaker lookup vector into a mel spectrogram. Phonemes are encoded as one-hot vectors augmented with frame length, projected through a Conv1D layer, replicated to frame count, and concatenated with two pairs of positional cross-fade channels that localize each frame within the phrase and within the phoneme. Two discriminators, one 2D on local time-frequency patches and one 1D on global spectral structure, are trained adversarially and conditioned on the same $F_0$, speaker, and phoneme inputs; the generator uses a reconstruction loss plus a 0.5-weighted GAN loss, with Swish activations and no normalization or dropout layers. The mel spectrogram is turned into audio by the MBExWN universal vocoder. This machinery carries the controllability claim because every conditioning channel is an explicit, editable input rather than an inferred latent.

What would settle it

Take Fast-VGAN's trained model and run it on a held-out set of many VCTK sentences and unseen speakers, computing WER and speaker-embedding similarity under the same pitch and duration manipulations; if WER rises well above 0.00% or similarity falls below the comparison systems' scores on ordinary sentences, the claim that intelligibility and identity are preserved under explicit prosodic control is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that a small fully convolutional GAN conditioned on high-level speech features can serve as a controllable voice-conversion front end. On the VCTK corpus with eight speakers, Fast-VGAN matches or beats the comparison systems on objective intelligibility, achieving 0.00% WER and cosine speaker similarity 0.648, while remaining robust when the target speaker's pitch ambitus and speech rate are imposed; in subjective tests it reaches MOS 3.63 for naturalness and 3.47 for speaker similarity, the highest among the compared systems. The same model withstands pitch shifts of up to ±1 octave, ambitus scaling of up to ±1 octave, and vowel-duration scaling between 1/3× and 3×, with WER staying low and similarity degrading gradually. Trained on neutral VCTK plus only neutral Expresso utterances, it resynthesizes confused, happy, and sad speech with WER essentially equal to real recordings and similarity around 0.87, suggesting that no expressive data is needed to transfer expressive prosody. The result is a voice converter whose input features are exactly the parameters a user would want to manipulate.

Load-bearing premise

The main voice-conversion result is measured on only two utterances per speaker that every test speaker happened to have recorded, so the near-zero WER and high similarity assume those two sentences represent all the content and speaking styles the system would encounter.

Editorial extensions

If this is right

  • Speakers and prosody can be manipulated independently in a converted utterance: mean pitch, pitch range, and vowel timing are set by direct input features.
  • Because inference is non-autoregressive and the decoder runs faster than real time on CPU, the approach is usable in real-time and resource-constrained settings.
  • Expressive transformations can be applied to neutral speech without expressive training data, since the model transfers $F_0$ and duration contours from an emotion corpus at inference.
  • If the reported zero WER is representative, the model can undergo large prosodic edits while keeping the words intact, which suits dubbing and voice reenactment.
  • Prosodic parameter adaptation to a target speaker does not improve perceived similarity and can hurt it when ambitus and rate are combined, indicating that these identity cues need calibration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two-utterance evaluation base makes the 0.00% WER fragile; a reasonable next step is a held-out set of dozens of sentences per speaker to see whether intelligibility survives.
  • Because the phoneme sequence is an explicit conditioning input, the same architecture could support content editing — replacing a word or reordering phrases — by editing the phoneme alignment before synthesis, which the paper does not demonstrate.
  • Replacing the learned speaker lookup with an encoder trained on unseen speakers would extend Fast-VGAN from any-to-many to any-to-any conversion; the explicit conditioning makes such an extension look feasible.
  • The finding that combined ambitus and rate adaptation degrades subjective similarity suggests a search over the conditioning parameters per target speaker could recover the lost quality, a testable extension of the current fixed rule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. Fast-VGAN is a non-autoregressive, fully convolutional GAN-based voice conversion model that generates mel spectrograms from explicit conditioning features—F0, intensity, aligned phoneme sequences, and speaker embeddings—followed by the MBExWN vocoder. The paper claims that at inference, F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted, enabling voice identity conversion and expressive transformations such as pitch shift, ambitus scaling, and duration stretching without requiring expressive training data. The evaluation compares Fast-VGAN with ControlVC and HiFi-VC on VCTK using ASR WER and speaker-embedding cosine similarity, plus MOS tests, and reports prosodic-scaling curves and an Expresso-based expressive synthesis experiment. The main conclusion is that Fast-VGAN offers substantial flexibility while maintaining high intelligibility and speaker similarity.

Significance. If the results hold, the contribution is practically valuable: the model is lightweight (about 3.2M parameters), runs faster than real-time on CPU, uses interpretable conditioning rather than black-box disentanglement, and provides explicit controls that are directly relevant to expressive voice conversion and dubbing. The paper also includes both objective and subjective evaluations with confidence intervals and a demo page. However, the evidence base for the headline claims is very small—two shared VCTK utterances per speaker—and no significance tests are reported, so the significance is currently conditional rather than established.

major comments (4)
  1. [Section 4.3, Table 2] The core objective VC benchmark is computed on only 2 utterances per speaker, selected because all eight evaluation speakers read them, yielding 16 converted utterances for Fast-VGAN and each baseline. A WER of 0.000±0.000 on this tiny set is not compelling evidence of high intelligibility: short, clean sentences can be transcribed perfectly even when conversion quality varies, and bootstrapping over the same two sentences cannot capture sentence-level variability. The claim in Section 5.1.1 that Fast-VGAN 'outperforms the baselines in WER' is therefore under-supported; the authors should evaluate on a larger held-out set, report per-sentence results, and include significance tests.
  2. [Section 5.1.2, Figure 2] The prosodic scaling experiments reuse the same two shared utterances for all conditions, so each curve in Figure 2 is based on 16 synthesized samples per scaling factor. This is insufficient to support the conclusion in Section 5.1.2 that the model 'preserves intelligibility and speaker consistency across a wide range of expressive transformations.' The same limitation applies to the claims about extreme manipulations such as 3× duration stretching and ±1 octave shifts; the authors should test on a larger and more diverse set of utterances and report statistical comparisons across transformation factors.
  3. [Section 5.2.1, Table 4] The text states that Fast-VGAN 'achieves the highest performance across both dimensions,' but the confidence intervals overlap with ControlVC on naturalness (3.63±0.25 versus 3.60±0.14), and no significance tests are reported for any MOS comparison. The similarity difference (3.47±0.32 versus 2.82±0.18) is suggestive, but the paper should provide paired or mixed-model statistics and listener-level details before claiming superiority. At minimum, the claim should be tempered to reflect the absence of significant differences.
  4. [Abstract and Section 3.1] The paper claims that 'F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted' at inference, but no experiment in Section 5 manipulates the phoneme sequence independently. The voice conversion and scaling experiments keep the source utterance's phonemes fixed and vary F0, ambitus, duration, or speaker embedding; the expressive synthesis experiment uses different sentences from the Expresso speakers but does not isolate phoneme-content control. Either add an evaluation with phoneme substitution or revise the abstract and introduction to limit the claim to F0, duration, and speaker identity.
minor comments (3)
  1. [Section 5.1.1] The sentence 'We note that WER for Fast-VGAN only doing resynthesis and the standalone MBExWN vocoder are 0.00 ± 0.00% and 0.935 ± 0.013 and 0.992 ± 0.001 respectively for similarity' is ungrammatical and unclear; report these resynthesis controls in a table with clearly labeled metrics.
  2. [Section 3.3] The sentence 'It uses Leaky ReLU activations' has an ambiguous antecedent; specify that the discriminator branches use Leaky ReLU.
  3. [Section 4.3] Minor grammar issues: 'we adopted an many-to-many conversion setup' and 'we selected 2 utterances that were spoken by all of them that were used for testing' should be rephrased for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Fast-VGAN's prosody controls are architectural inputs; the central claims are supported by external evaluation metrics.

full rationale

The paper does not derive its target results from its own inputs by construction. Fast-VGAN explicitly conditions a mel-spectrogram generator on F0, intensity, phoneme labels, and a trainable speaker embedding; the ability to adjust F0 contours, phoneme sequences, and speaker embeddings at inference is a direct architectural affordance, not a fitted parameter renamed as a prediction. The evaluation of intelligibility and speaker similarity uses external, pre-trained systems (NeMo ASR for WER and Resemblyzer speaker embeddings for cosine similarity) on held-out utterances, so the headline claims are not statistically forced by the training objective. Self-citations appear only as component choices: F0-FCN for pitch extraction [31], MBExWN as the neural vocoder [40], and a temporal alignment method [42]; none of these is invoked as a uniqueness theorem or as the justification for the paper's central claim. The main weakness is empirical: the objective VC benchmark uses only two utterances per speaker, and the zero WER may reflect the small, easy test set rather than robust generalization. That is a validity and generalization concern, not a circularity in the derivation chain. No step in the paper equates a prediction with an input by definition, and no fitted value is presented as an independent result. The derivation chain is therefore self-contained with respect to the external benchmarks.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claim relies on standard machine-learning assumptions about feature sufficiency and metric validity, plus a small set of hand-chosen hyperparameters. There are no fitted constants presented as predictions and no invented physical or theoretical entities. The main risk is the implicit assumption that the tiny evaluation set captures general behavior.

free parameters (4)
  • Training loss weights = L_rec = 1, L_GAN = 0.5
    Chosen by the authors without an ablation study; affects the balance between reconstruction fidelity and adversarial naturalness.
  • Speaker embedding normalization = Unit-norm constraint applied
    Selected based on initial tests; constrains the speaker embedding space and influences timbre transfer.
  • Decoder channel schedule = [160, 144, 128, 112, 96]
    Hand-designed capacity reduction across transposed conv blocks; no systematic search reported.
  • F0 normalization = Per-speaker global log-mean subtraction
    A preprocessing choice intended to strip speaker-dependent pitch level; its effect on voice conversion is not separately ablated.
assumptions (4)
  • domain assumption Aligned phoneme sequence, F0, and intensity are sufficient to reconstruct intelligible speech for any training speaker.
    The decoder maps these features plus speaker embedding to a mel-spectrogram (Section 3.1); if this information is insufficient, resynthesis quality would degrade.
  • domain assumption Speaker embedding captures vocal timbre independently of F0 and phoneme content.
    No explicit disentanglement loss is used; the model relies on this separation for voice conversion (Sections 3.2 and 3.3).
  • domain assumption WER from a pretrained NeMo ASR model and cosine similarity from Resemblyzer are valid proxies for intelligibility and speaker similarity.
    All objective conclusions are drawn from these two metrics (Section 5.1), including for extreme prosodic modifications.
  • domain assumption The MBExWN vocoder introduces no significant artifacts when synthesizing from Fast-VGAN mel-spectrograms.
    The vocoder is treated as a fixed external component (Section 3.1); its interaction with generated features is not separately evaluated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters." pith.science (2026). https://pith.science/paper/ODJPMZ2X

@misc{pith2026250704817,
  author       = {Pith},
  title        = {Pith review of: Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODJPMZ2X}},
  note         = {Machine review of arXiv:2507.04817}
}
read the original abstract

Precise control over speech characteristics, such as pitch, duration, and speech rate, remains a significant challenge in the field of voice conversion. The ability to manipulate parameters like pitch and syllable rate is an important element for effective identity conversion, but can also be used independently for voice transformation, achieving goals that were historically addressed by vocoder-based methods. In this work, we explore a convolutional neural network-based approach that aims to provide means for modifying fundamental frequency (F0), phoneme sequences, intensity, and speaker identity. Rather than relying on disentanglement techniques, our model is explicitly conditioned on these factors to generate mel spectrograms, which are then converted into waveforms using a universal neural vocoder. Accordingly, during inference, F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted, allowing for intuitively controlled voice transformations. We evaluate our approach on speaker conversion and expressive speech tasks using both perceptual and objective metrics. The results suggest that the proposed method offers substantial flexibility, while maintaining high intelligibility and speaker similarity.

Figures

Figures reproduced from arXiv: 2507.04817 by the authors.

Figure 1
Figure 1. Fast-VGAN architecture. During training: the Lrec loss is the RMSE loss and the LGAN loss is the MSE loss. The weighting is 1 for Lrec and 0.5 for LGAN . mostly encodes speaking style and expressivity of the speaker, while no timber-related information. Intensity, computed as the average energy along the frequency axis of the Mel spectrogram, reflects the perceived loudness and expressivity of the utterance. Aligned… view at source ↗
Figure 2
Figure 2. WER and speakersimilarity under prosodic parameter scaling. Vertical dashed line indicates the neutral setting. WER is shown as a percentage (0–100), similarity is normalized between 0 and 1. Note that for ground truth, WER and similarity are 0.00 ± 0.00% and 1.00 ± 0.00%. Among the baselines, ControlVC shows low WER and good similarity. Our proposed method achieves a WER of 0.00, indicating excellent intelligibilit… view at source ↗
Figure 3
Figure 3. Subjective parameter scaling evaluation. speech while effectively preserving the identity of the target speaker. ControlVC shows strong performance in terms of naturalness (3.60 ± 0.14), but is outperformed by Fast-VGAN in speaker similarity. In contrast, HiFi-VC underperforms in both metrics, particularly in similarity (2.00 ± 0.17). Applying only the target speaker’s ambitus or only vowel dilation results in sligh… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Naturalness over resynthesized and real Expr. audio and similarity to targeted emotion. Green: target, blue: target (fairly), grey: uncertain, orange: neutral (fairly), red: neutral. scale [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 53 canonical work pages

  1. [1]

    By definition, this task requires altering vocal identity, which raises fundamental questions about what features define a speaker’s voice and how they should be manipulated

    Introduction V oice conversion (VC) aims to transform a source speaker’s voice to sound like that of a target speaker, while preserving the linguistic content [1, 2]. By definition, this task requires altering vocal identity, which raises fundamental questions about what features define a speaker’s voice and how they should be manipulated. Many systems tr...

    work page

  2. [2]

    Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters

    Related Works We review key areas relevant to our work, including recent developments in voice conversion models and prosodic control strategies, the role of feature representations in enabling fine- grained control, and the application of GAN-based approaches for high-quality and efficient speech synthesis. arXiv:2507.04817v1 [cs.SD] 7 Jul 2025 2.1. V oi...

    work page Pith review arXiv 2025

  3. [3]

    Our goal is to design a model capable of transferring a specific utterance from one speaker to another speaker’s timbre, while relying on high-level control features

    Fast-VGAN V oice Conversion In this section, we detail the rationale behind our architectural choices. Our goal is to design a model capable of transferring a specific utterance from one speaker to another speaker’s timbre, while relying on high-level control features. To facilitate training on limited data and ensure a lightweight implementation, we impo...

    work page

  4. [4]

    Experiments In the remainder of this section, we present the databases and evaluation tasks performed. We first compare our method to two representative voice conversion models from the literature, ControlVC [9] and HiFi-VC [8], to benchmark the timbre transfer quality in a standard many-to-many conversion setting. We then analyze the role and variability...

    work page

  5. [5]

    Evaluations 5.1. Objective Evaluations For all conducted experiments, including the proposed method and the baseline systems, we evaluate the performance using both intelligibility and speaker similarity metrics. • Intelligibility: we compute the Word Error Rate (WER) by transcribing all generated utterances using the pretrained stt en fastconformer hybri...

    work page

  6. [6]

    Conclusion Fast-VGAN demonstrates strong potential for expressive voice conversion, enabling controllable transformations of prosodic features such as pitch range and speech rate while maintaining naturalness and speaker similarity. Our results suggest that explicit prosodic conditioning can effectively drive perceived expressiveness and provide users wit...

    work page

  7. [7]

    Acknowledgments This work was partly performed using HPC resources from GENCI-IDRIS (Grant 2025-AD011011177R5), and partly funded by the ANR project BRUEL (ANR-22-CE39-0009)

    work page 2025

  8. [8]

    Reimagining speech: a scoping review of deep learning-based methods for non-parallel voice conversion,

    A. R. Bargum, S. Serafin, and C. Erkut, “Reimagining speech: a scoping review of deep learning-based methods for non-parallel voice conversion,”Frontiers in Signal Processing, vol. 4, 2024

    work page 2024

Show all 56 references
  1. [9]

    An Overview of V oice Conversion and Its Challenges: From Statistical Modeling to Deep Learning,

    B. Sisman, J. Yamagishi, S. King, and H. Li, “An Overview of V oice Conversion and Its Challenges: From Statistical Modeling to Deep Learning,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 132–157, 2020

    2020

  2. [10]

    AutoVC: Zero-Shot V oice Style Transfer with Only Autoencoder Loss,

    K. Qian, Y . Zhang, S. C. Chang et al., “AutoVC: Zero-Shot V oice Style Transfer with Only Autoencoder Loss,” in International Conference on Machine Learning (ICML), 2019, pp. 5210–5219

    2019

  3. [11]

    Diffusion-Based V oice Conversion with Fast Maximum Likelihood Sampling Scheme,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, M. S. Kudinov, and J. Wei, “Diffusion-Based V oice Conversion with Fast Maximum Likelihood Sampling Scheme,” in International Conference on Learning Representation (ICLR), 2022

    2022

  4. [12]

    QUICKVC: A Lightweight VITS- Based Any-to-Many V oice Conversion Model using ISTFT for Faster Conversion,

    H. Guo, C. Liu et al. , “QUICKVC: A Lightweight VITS- Based Any-to-Many V oice Conversion Model using ISTFT for Faster Conversion,” in IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023, pp. 1–7

    2023

  5. [13]

    V oice Reenactment with F0 and timing constraints and adversarial learning of conversions,

    F. Bous, L. Benaroya, N. Obin, and A. Roebel, “V oice Reenactment with F0 and timing constraints and adversarial learning of conversions,” in European Signal Processing Conference (EUSIPCO), 2022, pp. 389–393

    2022

  6. [14]

    Highly Controllable Diffusion-based Any-to-Any V oice Conversion Model with Frame-level Prosody Feature,

    K. Byun, S. Moon, and E. Visser, “Highly Controllable Diffusion-based Any-to-Any V oice Conversion Model with Frame-level Prosody Feature,” 2023. [Online]. Available: https: //arxiv.org/abs/2309.03364

    2023 arXiv

  7. [15]

    HiFi-VC: High Quality ASR-based V oice Conversion,

    A. Kashkin, I. Karpukhin, and S. Shishkin, “HiFi-VC: High Quality ASR-based V oice Conversion,” in Speech Synthesis Workshop (SSW), 2023, pp. 100–105

    2023

  8. [16]

    ControlVC: Zero-Shot V oice Conversion with Time-Varying Controls on Pitch and Speed,

    M. Chen and Z. Duan, “ControlVC: Zero-Shot V oice Conversion with Time-Varying Controls on Pitch and Speed,” in Interspeech, 2023, pp. 2098–2102

    2023

  9. [17]

    V oice quality: the 4th prosodic dimension,

    N. Campbell and P. Mokhtari, “V oice quality: the 4th prosodic dimension,” in International Congress of Phonetic Sciences (ICPhS), 2003, pp. 2417–2420

    2003

  10. [18]

    Manipulating V oice Attributes by Adversarial Learning of Structured Disentangled Representations,

    L. Benaroya, N. Obin, and A. Roebel, “Manipulating V oice Attributes by Adversarial Learning of Structured Disentangled Representations,” Entropy, vol. 25, no. 2, 2023

    2023

  11. [19]

    End-to-End Adversarial Text-to-Speech,

    J. Donahue, S. Dieleman, M. Binkowski, E. Elsen, and K. Simonyan, “End-to-End Adversarial Text-to-Speech,” in International Conference on Learning Representation (ICLR) , 2021

    2021

  12. [20]

    Disentangling prosody and timbre embeddings via voice conversion,

    N. Gengembre, O. Le Blouch, and C. Gendrot, “Disentangling prosody and timbre embeddings via voice conversion,” in Interspeech, 2024, pp. 2765–2769

    2024

  13. [21]

    Learning Expressive Disentangled Speech Representations with Soft Speech Units and Adversarial Style Augmentation,

    Y . Deng, J. Wang, X. Zhang, N. Cheng, and J. Xiao, “Learning Expressive Disentangled Speech Representations with Soft Speech Units and Adversarial Style Augmentation,” in International Joint Conference on Neural Networks (IJCNN) . IEEE, 2024, pp. 1–7

    2024

  14. [22]

    Disentangling prosody representations with unsupervised speech reconstruction,

    L. Qu, T. Li, C. Weber, T. Pekarek-Rosin, F. Ren, and S. Wermter, “Disentangling prosody representations with unsupervised speech reconstruction,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 39–54, 2023

    2023

  15. [23]

    Emotional prosody control for speech generation,

    S. Sivaprasad, S. Kosgi, and V . Gandhi, “Emotional prosody control for speech generation,” arXiv preprint arXiv:2111.04730, 2021

    2021 arXiv

  16. [24]

    Advancements in real- time voice conversion technologies: A comprehensive analysis of techniques,

    P. S. Rana, S. K. Modi, A. L. Yadav et al., “Advancements in real- time voice conversion technologies: A comprehensive analysis of techniques,” Chahat and Modi, Soham Kr and Yadav, Anup Lal, Advancements in Real-Time Voice Conversion Technologies: A Comprehensive Analysis of T...

    2024

  17. [25]

    Robust and fine-grained prosody control of end-to-end speech synthesis,

    Y . Lee and T. Kim, “Robust and fine-grained prosody control of end-to-end speech synthesis,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5911–5915

    2019

  18. [26]

    Unsupervised learning of disentangled speech content and style representation,

    A. Tjandra, R. Pang, Y . Zhang, and S. Karita, “Unsupervised learning of disentangled speech content and style representation,” arXiv preprint arXiv:2010.12973, 2020

    2010 arXiv

  19. [27]

    Learning explicit prosody models and deep speaker embeddings for atypical voice conversion,

    D. Wang, S. Liu, L. Sun, X. Wu, X. Liu, and H. Meng, “Learning explicit prosody models and deep speaker embeddings for atypical voice conversion,”arXiv preprint arXiv:2011.01678, 2020

    2011 arXiv

  20. [28]

    Conformer: Convolution-augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Interspeech, 2020, pp. 5036–5040

    2020

  21. [29]

    WORLD: A V ocoder-Based High-Quality Speech Synthesis System for Real- Time Applications,

    M. Morise, F. Yokomori, and K. Ozawa, “WORLD: A V ocoder-Based High-Quality Speech Synthesis System for Real- Time Applications,” in IEICE Transactions on Information and Systems, vol. E99.D, no. 7, 2016, pp. 1877–1884

    2016

  22. [30]

    How far are we from robust voice conversion: A survey,

    T.-h. Huang, J.-h. Lin, and H.-y. Lee, “How far are we from robust voice conversion: A survey,” in 2021 IEEE spoken language technology workshop (SLT). IEEE, 2021, pp. 514–521

    2021

  23. [31]

    HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,

    J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,” in Proc. NeurIPS, 2020, pp. 17 022–17 033

    2020

  24. [32]

    Disentanglement of emotional style and speaker identity for expressive voice conversion,

    Z. Du, B. Sisman, K. Zhou, and H. Li, “Disentanglement of emotional style and speaker identity for expressive voice conversion,”arXiv preprint arXiv:2110.10326, 2021

    2021 arXiv

  25. [33]

    Diphone synthesis using an overlap-add technique for speech waveforms concatenation,

    F. Charpentier and M. Stella, “Diphone synthesis using an overlap-add technique for speech waveforms concatenation,” in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 1986, pp. 2015–2018

    1986

  26. [34]

    HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,” in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, 2021, pp. 3451–3460

    2021

  27. [35]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in NeurIPS, 2020

    2020

  28. [36]

    Fast and reliable f0 estimation method based on the period extraction of vocal fold vibration of singing voice and speech,

    M. Morise, H. Kawahara, and H. Katayose, “Fast and reliable f0 estimation method based on the period extraction of vocal fold vibration of singing voice and speech,” in Audio Engineering Society Conference: 35th International Conference: Audio for Games. Audio Engineering Soci...

    2009

  29. [37]

    Crepe: A Convolutional Representation for Pitch Estimation,

    J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “Crepe: A Convolutional Representation for Pitch Estimation,” in EEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 161–165

    2018

  30. [38]

    Fully-Convolutional Network for Pitch Estimation of Speech Signals,

    L. Ardaillon and A. Roebel, “Fully-Convolutional Network for Pitch Estimation of Speech Signals,” in Insterspeech, 2019

    2019

  31. [39]

    Parallel-Data-Free V oice Conversion Using Cycle-Consistent Adversarial Networks,

    T. Kaneko and H. Kameoka, “Parallel-Data-Free V oice Conversion Using Cycle-Consistent Adversarial Networks,” in IEEE transactions on knowledge and data engineering, 2017

    2017

  32. [40]

    CycleGAN V oice Conversion of Spectral Envelopes using Adversarial Weights,

    R. Ferro, N. Obin, and A. Roebel, “CycleGAN V oice Conversion of Spectral Envelopes using Adversarial Weights,” in European Signal Processing Conference (EUSIPCO), 2020

    2020

  33. [41]

    StarGAN-VC: Non-parallel many- to-many voice conversion using star generative adversarial networks,

    T. Kaneko and H. Kameoka, “StarGAN-VC: Non-parallel many- to-many voice conversion using star generative adversarial networks,” in Spoken Language Technology Workshop (SLT) , 2019, pp. 266–273

    2019

  34. [42]

    StarGAN-VC2: Rethinking conditional methods for stargan-based voice conversion,

    ——, “StarGAN-VC2: Rethinking conditional methods for stargan-based voice conversion,” in Interspeech, 2019, pp. 679– 683

    2019

  35. [43]

    Nvc-net: End-to-end adversarial voice conversion,

    B. Nguyen and F. Cardinaux, “Nvc-net: End-to-end adversarial voice conversion,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 7012–7016

    2022

  36. [44]

    Avqvc: One-shot voice conversion by vector quantization with applying contrastive learning,

    H. Tang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, “Avqvc: One-shot voice conversion by vector quantization with applying contrastive learning,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 4613–4617

    2022

  37. [45]

    Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech,

    V . Popov, A. V ovk, I. Yakovenkoet al., “Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech,” in International Conference on Machine Learning (ICML), 2021

    2021

  38. [46]

    DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust V oice Conversion,

    H.-Y . Choi, S.-H. Lee, and S.-W. Lee, “DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust V oice Conversion,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, pp. 17 862–17 870

    2024

  39. [47]

    Neural V ocoding for Singing and Speaking V oices with the Multi-Band Excited WaveNet,

    A. Roebel and F. Bous, “Neural V ocoding for Singing and Speaking V oices with the Multi-Band Excited WaveNet,” Information, vol. 13, no. 3, p. 103, 2022

    2022

  40. [48]

    Revisiting Over- Smoothness in Text to Speech,

    Y . Ren, X. Tan, T. Qin, Z. Zhao, and T.-Y . Liu, “Revisiting Over- Smoothness in Text to Speech,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , 2022, pp. 8197–8213

    2022

  41. [49]

    On temporal constraints for deep neural voice alignment,

    Y . Teytaut, “On temporal constraints for deep neural voice alignment,” Ph.D. dissertation, Sorbonne Universit´e, 2023

    2023

  42. [50]

    Festival speech synthesis system: system documentation (1.1. 1),

    A. Black, “Festival speech synthesis system: system documentation (1.1. 1),” Human Communication Research Centre Technical Report, 1997

    1997

  43. [51]

    LIA—PHON: Un syst `eme complet de phon ´etisation de textes,

    F. B ´echet, “LIA—PHON: Un syst `eme complet de phon ´etisation de textes,” TAL. Traitement automatique des langues , vol. 42, no. 1, pp. 47–67, 2001

    2001

  44. [52]

    CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),

    J. Yamagishi, C. Veaux, K. MacDonald et al. , “CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),” University of Edinburgh. The Centre for Speech Technology Research (CSTR), pp. 271–350, 2019

    2019

  45. [53]

    EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis,

    T. A. Nguyen, W.-N. Hsu, A. D’Avirro, B. Shi, I. Gat, M. Fazel- Zarani, T. Remez, J. Copet, G. Synnaeve, M. Hassid, F. Kreuk, Y . Adi, and E. Dupoux, “EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis,” inInterspeech, 2023, pp. 4823–4827

    2023

  46. [54]

    Speech Synthesis with Mixed Emotions,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Speech Synthesis with Mixed Emotions,” IEEE Transactions on Affective Computing, vol. 14, no. 4, pp. 3120–3134, 2022

    2022

  47. [55]

    Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 920–924

    2021

  48. [56]

    Sequence-to-Sequence Moelling OF F0 For Speech Emotion Conversion,

    C. Robinson, N. Obin, and A. Roebel, “Sequence-to-Sequence Moelling OF F0 For Speech Emotion Conversion,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Brighton, United Kingdom, May 2019

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.