Pith. sign in

REVIEW 5 major objections 6 minor 40 references

ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new system converts a speaker's emotion from a natural-language prompt or a reference clip, with adjustable intensity, and reports gains over three baselines.

desk verdict Solid EVC systems paper with a clean architecture; the blocking issue is the unverified baseline comparison. read the letter →

arxiv 2505.13805 v1 pith:QVW4UP4H submitted 2025-05-20 cs.SD cs.AIeess.AS

Yu Pan , Yanni Hu , Yuguang Yang , Jixun Yao , Jianhao Ye , Hongbin Zhou , Lei Ma , Jianjun Zhao This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords emotionalvoiceconversionnaturallanguagepromptcontrastivelanguage-audiopretrainingflowmatchingemotionintensityspeechsynthesisPhoneticPosteriorGrams
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces ClapFM-EVC, an emotional voice conversion system that changes the emotion of spoken audio while preserving the words and the speaker's voice. Its central claim is that a user can choose the target emotion by typing a natural-language description such as "whispering in panic" or by playing a reference recording, and can also adjust how strongly the emotion is expressed. To do this, the system aligns emotional meaning between text and audio with a contrastive model, fuses the resulting emotion vector with speech-content features through an intensity-gated encoder, and regenerates the acoustic spectrum with a flow-matching decoder. In the paper's evaluation on a 20-hour Mandarin corpus, the system scores higher than three existing EVC systems on all reported quality and emotion-similarity metrics.

What carries the argument

The load-bearing piece is the EVC-CLAP training objective: a symmetric KL divergence between predicted audio-text similarity matrices and soft ground-truth matrices built from categorical emotion labels and natural-language prompt labels. This pulls speech and text with the same emotional meaning into nearby points in a shared 512-dimensional space, giving the system fine-grained emotional control from text. The second load-bearing piece is the FuEncoder's adaptive intensity gate, a learnable multiplier that scales the emotion embedding before adaptive layer-normalization fusion blocks combine it with content features. The third is the optimal-transport conditional flow matching decoder, which generates the Mel-spectrogram by learning a vector field that transports Gaussian noise to the data distribution along straight paths and samples it with Euler steps.

What would settle it

Run StarGAN-EVC, Seq2seq-EVC, and MixEmo on the same training data, speaker, and reference-speech protocol as ClapFM-EVC, then compare the Table 1 metrics. If the baselines close the gap on MCD, UTMOS, and eMOS, the stated advantage is an artifact of comparison conditions; if the gap persists under identical training and evaluation conditions, the claim is supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that high-fidelity emotional voice conversion can be made flexible by splitting emotional control between a text prompt and a reference speech signal. Its EVC-CLAP module produces a 512-dimensional emotion embedding aligned across text and speech, using a symmetric KL contrastive loss guided by soft labels that blend categorical emotion labels with natural-language prompts. The FuEncoder merges this embedding with Phonetic PosteriorGrams from a pretrained ASR model, while an adaptive intensity gate scales the emotional component. A conditional flow-matching decoder, trained with optimal-transport paths, reconstructs the target Mel-spectrogram, and a pretrained vocoder turns it into speech. The paper reports that on its 20-hour single-speaker Mandarin corpus, ClapFM-EVC improves emotion similarity by at least 26.2% in EECS and 53.1% in eMOS over the best baseline, while also achieving lower MCD, RMSE, and CER and higher UTMOS and nMOS. Prompt-driven conversion was close to reference-driven conversion in a 47-listener ABX test, suggesting that text descriptions can stand in for reference audio in many cases.

Load-bearing premise

The load-bearing premise is that the comparison against existing systems is fair: the paper does not state whether StarGAN-EVC, Seq2seq-EVC, and MixEmo were retrained or fine-tuned on the same 20-hour single-speaker Mandarin corpus used to train ClapFM-EVC, so the reported metric gaps could be inflated by corpus or speaker mismatch.

Editorial extensions

If this is right

  • Emotional voice conversion can be driven by free-form language prompts rather than a fixed set of categorical labels.
  • Users can set emotional intensity continuously through the adaptive intensity gate, not just pick an emotion category.
  • In the reported ABX test, using a prompt to specify emotion gives listeners essentially the same emotional similarity as using a reference clip, with 57.4% of participants reporting no preference.
  • On the reported metrics, converted speech has lower distortion and higher intelligibility than the three comparison systems, so the system could fit into downstream TTS, dubbing, or audiobook pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this pattern transfers to other languages and speakers, the contrastive text-audio alignment used for emotion could also be applied to other speech attributes such as emphasis, speaking style, or perceived age.
  • A natural extension would be to calibrate the adaptive intensity gate against human perception, producing a mapping from gate value to perceived emotional strength that lets users dial emotion precisely.
  • Because the evaluation corpus is private and single-speaker, an immediate testable extension is to retrain the three baselines on the same corpus and speaker; the size of the reported gaps under identical training conditions would reveal how much of the advantage comes from the architecture itself.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. ClapFM-EVC is a two-stage emotional voice conversion system. EVC-CLAP is trained with a symmetric KL contrastive loss on soft labels built from both categorical emotion classes and natural-language prompts, producing cross-modal emotion embeddings from HuBERT speech features and XLM-RoBERTa text features. AdaFM-VC then fuses these embeddings with ASR-derived PPGs in a FuEncoder whose adaptive intensity gate scales the emotion conditioning, and a conditional flow matching decoder generates the target mel-spectrogram, rendered by a pretrained BigVGAN vocoder. The paper reports reference-speech comparisons against StarGAN-EVC, Seq2seq-EVC, and MixEmo, a prompt-versus-reference ABX preference test, and ablations of the emotion-label supervision, symKL loss, and intensity gate.

Significance. If the comparison is controlled, the main contribution is a practically attractive EVC architecture with dual prompt/reference conditioning and a plausible path to better naturalness through flow matching; the ablations provide evidence that the labeled contrastive supervision, the symmetric KL loss, and the adaptive intensity gate each contribute to the reported scores, and the use of an external emotion encoder (emotion2vec) for EECS makes the emotion-similarity metric less circular. The paper also ships demo audio and detailed implementation choices, which aids reproducibility. However, the headline state-of-the-art conclusion depends on evaluation choices that are currently underspecified, so the empirical contribution cannot yet be taken at face value.

major comments (5)
  1. [Section 3.1.1 / 3.1.3 / Table 1] The paper never defines a held-out test split. Section 3.1.1 describes 12,000 selected utterances from a 20-hour corpus, and Section 3.1.2 describes training the two models for fixed epochs/iterations, but there is no statement of how the data were partitioned into training, validation, and test sets or whether the reported nMOS/eMOS, EECS, and objective metrics were computed on utterances seen during training. Without a confirmed held-out evaluation, the absolute scores in Tables 1 and 2 cannot be interpreted as evidence of generalization. Please specify the split, the number of test utterances per condition, and the speakers involved, and confirm that all reported numbers are on unseen data.
  2. [Section 3.2.1, Table 1] The comparison with StarGAN-EVC, Seq2seq-EVC, and MixEmo is missing a training-condition statement. The manuscript only says these baselines employ the reference waveform and gives their repository links; it does not say whether they were retrained or fine-tuned on the same single-speaker Mandarin corpus, which pretrained checkpoints were used, or how hyperparameters were chosen. Since the official versions of these systems were developed under different corpus/language/speaker conditions, the large relative gains in Table 1 (e.g., 49.2% UTMOS and 53.1% eMOS over the best baseline) could be inflated by out-of-domain baseline evaluation. Please report the exact adaptation protocol for each baseline and ensure all systems are evaluated on identical held-out test utterances.
  3. [Section 2.3.1 and Section 3.2/3.3] The mechanism for the claimed adjustable emotion intensity is not specified. Section 2.3.1 describes AIG as multiplying the EVC-CLAP emotional features by a learnable hyperparameter, and the ablation w/o AIG only shows that removing this scalar degrades performance; there is no description of how a user provides a desired intensity value at inference time, no intensity-conditional training objective, and no experiment in which intensity is varied while content and speaker are held fixed. Given that adjustable emotion intensity appears in the title and abstract, please clarify the inference-time control mechanism and provide a quantitative demonstration (e.g., emotion-similarity scores or listening tests at multiple intensity settings).
  4. [Section 3.1.3, Table 1] No statistical significance testing accompanies the subjective MOS comparisons. The abstract and Section 3.2.1 use the word significant for the gains over baselines, and the 95% confidence intervals in Table 1 are useful, but they do not by themselves establish significance for every pairwise comparison, especially in the ABX test in Section 3.2.2 where 57.4% of listeners report no preference. Please add an appropriate paired test on per-listener scores (e.g., Wilcoxon signed-rank or bootstrap) and report the number of rated items per system.
  5. [Section 3.1.3] The objective quality metrics MCD, RMSE, and CER are not fully specified. The paper does not state which utterance is used as the reference for MCD/RMSE (source speech, the reference emotional utterance, or a ground-truth same-content target), nor whether the signals are time-aligned or how CER is computed on converted versus source content. These choices materially affect the numbers, so please spell out the reference signals, alignment procedure, and ASR decoding settings used for Table 1.
minor comments (6)
  1. [Section 2.2] The model is called EVC-CLAP in the title and abstract but Emo-CLAP in Section 2.2; please use one name consistently.
  2. [Equations 1-2] The quantities M_y^GT and M_p^GT in Equation (2) are used before being explicitly defined; please define them and state the normalization procedure before presenting the equation.
  3. [Equation (1)] Calling epsilon_a and epsilon_t learnable hyper-parameters is confusing; these are learned scalar temperatures and should be named accordingly.
  4. [Section 3.2.2 / Figure 2] Please report the axes of Figure 2, the number of judgments per condition, and confidence intervals for the ABX preference percentages, and consider interpreting the 57.4% no-preference result more cautiously in the text.
  5. [Section 2.1 / 3.2] The paper says the system is any-to-one, but all experiments use a single-speaker Mandarin corpus; please clarify whether the system was evaluated on multiple source speakers and, if not, temper the any-to-one claim or define it in the paper's sense.
  6. [Section 2] Some references to the authors' own prior work (GEMO-CLAP, StableVC, Takin-VC, CTEFM-VC) are used as building blocks; consider adding a few sentences stating which components are reused and which are new, to help readers assess novelty.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: EVC-CLAP and AdaFM-VC are trained end-to-end and evaluated against external baselines and external pretrained metrics; no load-bearing claim reduces to its inputs by construction.

full rationale

ClapFM-EVC's central claim is an empirical state-of-the-art claim anchored to Table 1, where the proposed system is compared with StarGAN-EVC, Seq2seq-EVC, and MixEmo. The objective metrics are computed by external pretrained models (emotion2vec for EECS, a CTC ASR for CER, SpeechMOS for UTMOS), and the subjective metrics come from human raters; none of these metrics are defined in terms of ClapFM-EVC's own training targets. The internal derivation chain is a standard conditional generation pipeline: EVC-CLAP is trained with the symKL loss of Eqs. (2)-(3) to produce emotion embeddings, while AdaFM-VC is trained with the flow-matching loss of Eq. (8) to reconstruct Mel-spectrograms conditioned on those embeddings and on PPG content features. The converted speech is not a renaming or algebraic rearrangement of the soft-label matrix M_s_GT; the labels are training supervision, not the output. Self-citations to GEMO-CLAP, StableVC, Takin-VC, and HybridFormer supply architectural components or inspiration, but the paper does not invoke any self-cited uniqueness theorem to force its design, and the SOTA claim is tested against external baselines. The absence of explicit baseline retraining details in Section 3.2.1 is a possible evaluation-fairness risk, but it is not circularity: nothing in the paper's equations makes the Table 1 gaps true by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard pretrained-model compositions and several hand-set hyperparameters. The most consequential assumption is the sufficiency and prompt-annotation quality of a proprietary 20-hour single-speaker corpus, which is not released.

free parameters (5)
  • alpha_e = 0.2
    Balances categorical label similarity and prompt similarity in the soft-label matrix (Eq. 2); set empirically.
  • epsilon_a and epsilon_t = initialized to 2.3
    Learnable temperature parameters in the similarity matrices (Eq. 1); values are trained.
  • alpha (label smoothing) = 1e-8
    Smoothing hyperparameter in Eq. 4 to avoid zero probabilities in KL divergence.
  • sigma_min = 0.0001
    Minimum standard deviation of noise in the flow matching path (Eq. 7).
  • AIG learnable intensity hyperparameter = not reported
    Multiplies emotion features in the adaptive intensity gate; the value is learned and its range or initialization is not given (Section 2.3.1).
assumptions (5)
  • domain assumption Pretrained HuBERT, XLM-RoBERTa, HybridFormer, and BigVGAN contribute reliable representations for emotion, language, phonetic content, and waveform synthesis
    The framework composes pretrained models without fine-tuning most of them; their suitability for the emotional prompt task is assumed (Section 2.1).
  • domain assumption PPGs from HybridFormer encode content while discarding emotion and timbre
    The FuEncoder design relies on PPGs being content-only so emotional embeddings can be added without corrupting content (Section 2.3.1).
  • standard math Optimal transport flow matching (Eq. 6-8) converges to the target mel-spectrogram distribution given the fused features
    Standard CFM theory from [21,22,32] is invoked; no new proof is given.
  • domain assumption SymKL contrastive loss with binary soft labels (same category or same prompt) aligns emotion across modalities
    The soft-label matrix uses binary identical/non-identical conditions (Eq. 2), which assumes that natural language prompt identity and category identity capture all relevant emotional similarity between pairs.
  • ad hoc to paper A single-speaker 20-hour corpus is sufficient to train an any-to-one EVC system
    The paper claims any-to-one conversion but only reports results from one internal single-speaker Mandarin corpus (Section 3.1.1), without cross-speaker validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech." pith.science (2026). https://pith.science/paper/QVW4UP4H

@misc{pith2026250513805,
  author       = {Pith},
  title        = {Pith review of: ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVW4UP4H}},
  note         = {Machine review of arXiv:2505.13805}
}
read the original abstract

Despite great advances, achieving high-fidelity emotional voice conversion (EVC) with flexible and interpretable control remains challenging. This paper introduces ClapFM-EVC, a novel EVC framework capable of generating high-quality converted speech driven by natural language prompts or reference speech with adjustable emotion intensity. We first propose EVC-CLAP, an emotional contrastive language-audio pre-training model, guided by natural language prompts and categorical labels, to extract and align fine-grained emotional elements across speech and text modalities. Then, a FuEncoder with an adaptive intensity gate is presented to seamless fuse emotional features with Phonetic PosteriorGrams from a pre-trained ASR model. To further improve emotion expressiveness and speech naturalness, we propose a flow matching model conditioned on these captured features to reconstruct Mel-spectrogram of source speech. Subjective and objective evaluations validate the effectiveness of ClapFM-EVC.

Figures

Figures reproduced from arXiv: 2505.13805 by the authors.

Figure 1
Figure 1. Overall training architecture of the proposed ClapFM-EVC framework. guage prompts and their corresponding categorical emotion la￾bels. In this way, EVC-CLAP can effectively extract and align emotional representations across audio and text modalities, while enabling ClapFM-EVC to capture fine-grained emotional information conveyed by natural language prompts. Then, we train the AdaFM-VC using the obtained emotional e… view at source ↗
Figure 2
Figure 2. The ABX preference test results compare the Refer￾ence with Prompt. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 17 canonical work pages

  1. [1]

    Introduction Emotional voice conversion (EVC) aims to convert the emo- tional state of source speech to a target category while preserv- ing original content and speaker identity [1]. Recently, EVC has garnered great attention within the speech processing realms and holds great potential for many practical applications such as voice assistant, audiobook p...

    work page

  2. [2]

    System Overview As illustrated in Fig

    METHODOLOGY 2.1. System Overview As illustrated in Fig. 1, ClapFM-EVC can be characterized as a conditional latent model, where the proposed EVC-CLAP, FuEncoder, CFM-based decoder, as well as pretrained ASR

    work page

  3. [3]

    Experimental Setups 3.1.1

    EXPERIMENTS 3.1. Experimental Setups 3.1.1. Datasets Since no open-source EVC corpus with comprehensive emo- tional natural language prompts is currently available, we lever- age an internally developed expressive single-speaker Mandarin corpus for training the proposed ClapFM-EVC system. This corpus encompasses 20 hours of speech data sampled at 24 kHz. ...

    work page

  4. [4]

    Specifically, the proposed ClapFM-EVC initially employs EVC-CLAP to extract and align emotional elements across audio-text modalities

    CONCLUSIONS In this study, we propose ClapFM-EVC, an innovative and ef- fective high-fidelity any-to-one EVC framework that features flexible and interpretable emotion control along with adjustable emotion intensity. Specifically, the proposed ClapFM-EVC initially employs EVC-CLAP to extract and align emotional elements across audio-text modalities. To en...

    work page

  5. [5]

    MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition

    Y . Pan, Y . Yang, Y . Huang, J. Yao, J. Yin, Y . Hu, H. Lu, L. Ma, and J. Zhao, “Msac: Multiple speech attribute control method for reliable speech emotion recognition,”arXiv preprint arXiv:2308.04025, 2023

    work page Pith review arXiv 2023

  6. [6]

    Mixed emotion mod- elling for emotional voice conversion,

    K. Zhou, B. Sisman, C. Busso, and H. Li, “Mixed emotion mod- elling for emotional voice conversion,”computer, vol. 6, p. 7, 2022

    work page 2022

  7. [7]

    A pre-training based personalized dialogue generation model with persona- sparse data,

    Y . Zheng, R. Zhang, M. Huang, and X. Mao, “A pre-training based personalized dialogue generation model with persona- sparse data,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 9693–9700

    work page 2020

  8. [8]

    Takin: A cohort of superior quality zero-shot speech generation models,

    S. Chen, Y . Feng, L. He, T. He, W. He, Y . Hu, B. Lin, Y . Lin, Y . Pan, P. Tanet al., “Takin: A cohort of superior quality zero-shot speech generation models,”arXiv preprint arXiv:2409.12139, 2024

    arXiv 2024

Show all 40 references
  1. [9]

    From speaker to dubber: movie dubbing with prosody and duration consistency learning,

    Z. Zhang, L. Li, G. Cong, H. Yin, Y . Gao, C. Yan, A. v. d. Hengel, and Y . Qi, “From speaker to dubber: movie dubbing with prosody and duration consistency learning,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 7523– 7532

    2024

  2. [10]

    Promptvc: Flexible stylistic voice conversion in latent space driven by natural language prompts,

    J. Yao, Y . Yang, Y . Lei, Z. Ning, Y . Hu, Y . Pan, J. Yin, H. Zhou, H. Lu, and L. Xie, “Promptvc: Flexible stylistic voice conversion in latent space driven by natural language prompts,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Process...

    2024

  3. [11]

    Gmp-atl: Gender- augmented multi-scale pseudo-label enhanced adaptive trans- fer learning for speech emotion recognition via hubert,

    Y . Pan, Y . Yang, H. Lu, L. Ma, and J. Zhao, “Gmp-atl: Gender- augmented multi-scale pseudo-label enhanced adaptive trans- fer learning for speech emotion recognition via hubert,”arXiv preprint arXiv:2405.02151, 2024

    2024 arXiv

  4. [12]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution- augmented transformer for speech recognition,”arXiv preprint arXiv:2005.08100, 2020

    2005 arXiv

  5. [13]

    Squeezeformer: An effi- cient transformer for automatic speech recognition,

    S. Kim, A. Gholami, A. Shaw, N. Lee, K. Mangalam, J. Ma- lik, M. W. Mahoney, and K. Keutzer, “Squeezeformer: An effi- cient transformer for automatic speech recognition,”Advances in Neural Information Processing Systems, vol. 35, pp. 9361–9373, 2022

    2022

  6. [14]

    Cam++: A fast and efficient network for speaker verification using context- aware masking,

    H. Wang, S. Zheng, Y . Chen, L. Cheng, and Q. Chen, “Cam++: A fast and efficient network for speaker verification using context- aware masking,”arXiv preprint arXiv:2303.00332, 2023

    2023 arXiv

  7. [15]

    One-shot emotional voice conversion based on feature separa- tion,

    W. Lu, X. Zhao, N. Guo, Y . Li, J. Wei, J. Tao, and J. Dang, “One-shot emotional voice conversion based on feature separa- tion,”Speech Communication, vol. 143, pp. 1–9, 2022

    2022

  8. [16]

    Stargan for emo- tional speech conversion: Validated by data augmentation of end- to-end emotion recognition,

    G. Rizos, A. Baird, M. Elliott, and B. Schuller, “Stargan for emo- tional speech conversion: Validated by data augmentation of end- to-end emotion recognition,” inICASSP 2020-2020 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, ...

    2020

  9. [17]

    Transforming spectrum and prosody for emotional voice conversion with non-parallel train- ing data,

    K. Zhou, B. Sisman, and H. Li, “Transforming spectrum and prosody for emotional voice conversion with non-parallel train- ing data,”arXiv preprint arXiv:2002.00198, 2020

    2002 arXiv

  10. [18]

    Non-parallel emo- tion conversion using a deep-generative hybrid network and an adversarial pair discriminator,

    R. Shankar, J. Sager, and A. Venkataraman, “Non-parallel emo- tion conversion using a deep-generative hybrid network and an adversarial pair discriminator,”arXiv preprint arXiv:2007.12932, 2020

    2007 arXiv

  11. [19]

    Emotional voice conversion using multitask learning with text-to-speech,

    T.-H. Kim, S. Cho, S. Choi, S. Park, and S.-Y . Lee, “Emotional voice conversion using multitask learning with text-to-speech,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7774– 7778

    2020

  12. [20]

    and vocoder [23] models serve as its core components. Similarly to [24, 25], we first train EVC-CLAP using a symmetric Kullback-Leibler divergence based contrastive loss (symKL-loss) along with soft labels derived from natural lan- arXiv:2505.13805v1 [cs.SD] 20 May 2025 Figure...

    2025 arXiv

  13. [21]

    Attention- based interactive disentangling network for instance-level emo- tional voice conversion,

    Y . Chen, L. Yang, Q. Chen, J.-H. Lai, and X. Xie, “Attention- based interactive disentangling network for instance-level emo- tional voice conversion,”arXiv preprint arXiv:2312.17508, 2023

    2023 arXiv

  14. [22]

    Towards realistic emotional voice conversion using controllable emotional intensity,

    T. Qi, S. Wang, C. Lu, Y . Zhao, Y . Zong, and W. Zheng, “Towards realistic emotional voice conversion using controllable emotional intensity,”arXiv preprint arXiv:2407.14800, 2024

    2024 arXiv

  15. [23]

    Emotion intensity and its control for emotional voice conversion,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Emotion intensity and its control for emotional voice conversion,”IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 31–48, 2022

    2022

  16. [24]

    To- ward any-to-any emotion voice conversion using disentangled dif- fusion framework,

    H.-H. Chou, Y .-S. Lin, C.-C. Sung, Y . Tsao, and C.-C. Lee, “To- ward any-to-any emotion voice conversion using disentangled dif- fusion framework,”arXiv preprint arXiv:2409.03636, 2024

    2024

  17. [25]

    Hybridformer: Improving squeezeformer with hybrid attention and nsr mecha- nism,

    Y . Yang, Y . Pan, J. Yin, J. Han, L. Ma, and H. Lu, “Hybridformer: Improving squeezeformer with hybrid attention and nsr mecha- nism,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  18. [26]

    Sta- blevc: Style controllable zero-shot voice conversion with condi- tional flow matching,

    J. Yao, Y . Yan, Y . Pan, Z. Ning, J. Ye, H. Zhou, and L. Xie, “Sta- blevc: Style controllable zero-shot voice conversion with condi- tional flow matching,”arXiv preprint arXiv:2412.04724, 2024

    2024 arXiv

  19. [27]

    Ctefm-vc: Zero-shot voice conversion based on content-aware timbre ensemble modeling and flow matching,

    Y . Pan, Y . Yang, J. Yao, J. Ye, H. Zhou, L. Ma, and J. Zhao, “Ctefm-vc: Zero-shot voice conversion based on content-aware timbre ensemble modeling and flow matching,”arXiv preprint arXiv:2411.02026, 2024

    2024 arXiv

  20. [28]

    Bigvgan: A universal neural vocoder with large-scale training,

    S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigvgan: A universal neural vocoder with large-scale training,” arXiv preprint arXiv:2206.04658, 2022

    2022 arXiv

  21. [29]

    Clap learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “Clap learning audio concepts from natural language supervision,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  22. [30]

    Gemo-clap: Gender-attribute-enhanced contrastive language- audio pretraining for accurate speech emotion recognition,

    Y . Pan, Y . Hu, Y . Yang, W. Fei, J. Yao, H. Lu, L. Ma, and J. Zhao, “Gemo-clap: Gender-attribute-enhanced contrastive language- audio pretraining for accurate speech emotion recognition,” in ICASSP 2024-2024 IEEE International Conference on Acous- tics, Speech and Signal Pro...

    2024

  23. [31]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

    2021

  24. [32]

    Unsupervised cross-lingual representation learning at scale,

    A. Conneau, “Unsupervised cross-lingual representation learning at scale,”arXiv preprint arXiv:1911.02116, 2019

    1911 arXiv

  25. [33]

    Meta-stylespeech: Multi-speaker adaptive text-to-speech generation,

    D. Min, D. B. Lee, E. Yang, and S. J. Hwang, “Meta-stylespeech: Multi-speaker adaptive text-to-speech generation,” inInterna- tional Conference on Machine Learning. PMLR, 2021, pp. 7748–7759

    2021

  26. [34]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  27. [35]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural In- formation Processing Systems, 2017

    2017

  28. [36]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  29. [37]

    Takin-vc: Zero-shot voice conversion via jointly hy- brid content and memory-augmented context-aware timbre mod- eling,

    Y . Yang, Y . Pan, J. Yao, X. Zhang, J. Ye, H. Zhou, L. Xie, L. Ma, and J. Zhao, “Takin-vc: Zero-shot voice conversion via jointly hy- brid content and memory-augmented context-aware timbre mod- eling,”arXiv preprint arXiv:2410.01350, 2024

    arXiv

  30. [38]

    emotion2vec: Self-supervised pre-training for speech emotion representation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emotion2vec: Self-supervised pre-training for speech emotion representation,”arXiv preprint arXiv:2312.15185, 2023

    2023 arXiv

  31. [39]

    Limited data emotional voice conversion leveraging text-to-speech: Two-stage sequence-to- sequence training,

    K. Zhou, B. Sisman, and H. Li, “Limited data emotional voice conversion leveraging text-to-speech: Two-stage sequence-to- sequence training,”arXiv preprint arXiv:2103.16809, 2021

    2021 arXiv

  32. [40]

    Speech synthesis with mixed emotions,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Speech synthesis with mixed emotions,”IEEE Transactions on Affective Computing, vol. 14, no. 4, pp. 3120–3134, 2022

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.