Pith. sign in

REVIEW 4 major objections 5 minor 43 references

PROEMO: Prompt-Driven Text-to-Speech Synthesis Based on Emotion and Intensity Control

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PROEMO claims that a GPT-4 prompt-control module on top of FastSpeech 2 with emotion and intensity encoders lets TTS generate multi-speaker expressive speech with controllable emotion and intensity, improving emotion classification…

desk verdict PROEMO is a decent incremental expressive-TTS paper with a real advance in multi-speaker emotion plus intensity control, but its headline intensity claim rests on a circular evaluation that the authors describe openly in the text. read the letter →

arxiv 2501.06276 v1 pith:I2KV5TLL submitted 2025-01-10 cs.SD cs.CLeess.AS

Shaozuo Zhang , Ambuj Mehrish , Yingting Li , Soujanya Poria This is my paper · ORCID
classification cs.SDcs.CLeess.AS
keywords text-to-speechemotionalspeechsynthesisemotionintensitycontrolprompt-basedprosodylargelanguagemodelsFast2HuBERTexpressive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PROEMO is an attempt to give text-to-speech systems fine-grained emotional control without retraining for each emotion. The paper extends FastSpeech 2 with two HuBERT-based encoders, one that classifies emotion and one that regresses emotional intensity, and then uses GPT-4 during inference to propose global and per-word scaling factors for pitch, energy, and duration. The claimed effect is that synthesized speech becomes expressive in a controllable way: word-level prompt control raises emotion classification accuracy from 74.80 percent to 79.72 percent on the best model, and listeners sort generated samples into low, medium, and high intensity with roughly 72 percent accuracy. A sympathetic reader would take the central claim to be that prompt-driven prosody scaling, on top of learned emotion and intensity embeddings, is sufficient to steer multi-speaker expressive speech.

What carries the argument

The object that carries the argument is the modified variance adapter of FastSpeech 2, now conditioned by two HuBERT-based encoders and then rescaled at inference by LLM-chosen factors. The emotion encoder adds a classification head to HuBERT; the intensity encoder adds a regression head trained on continuous intensity scores produced by a learned per-speaker, per-emotion relative ranking function $r(x_A)=W x_A$ over openSMILE features. The prompt-control step asks GPT-4 to output scaling values for pitch, energy, and duration, maps them through a quadratic function onto preset ranges (duration 0.74-1.34, energy 0.5-2, pitch within the predicted range), and applies them via $d'_i = d_i G_d \sigma_i$, $e'_i = e_i G_e \epsilon_i$, $p'_i = p_i + G_p + \pi_i$. This is what lets the same trained model shift emotional tone and intensity at inference without additional training.

What would settle it

Rerun the evaluation with GPT-4's scaling factors replaced by random draws from the same mapped ranges (energy 0.5-2, duration 0.74-1.34, pitch scaled within the predicted range); if emotion classification accuracy does not drop below the reported 79.72 percent level, the prompt-control module is not the mechanism driving the gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that a TTS pipeline can generate multi-speaker expressive speech with controllable emotion category and emotional intensity by combining learned emotion and intensity representations with inference-time prosody prompting. On top of a FastSpeech 2 backbone, an emotion encoder and an intensity encoder inject emotional content through the variance adapter, while GPT-4 proposes scaling factors that modify predicted pitch, energy, and duration at both the utterance and word level. The paper reports that local (word-level) prompt control consistently yields higher emotion classification accuracy than global-only or no prompt control, and that the full model with both encoders plus global and local control receives the highest mean opinion score while word error rate and character error rate stay close to the no-prompt baseline.

Load-bearing premise

The load-bearing premise is that GPT-4, called at inference time without fine-tuning and with no reported quantitative check of its outputs, will reliably produce scaling factors for pitch, energy, and duration that match the intended emotion and stay stable; the paper itself notes that the original prompt from [16] produced unstable output and had to be redesigned.

Editorial extensions

If this is right

  • Word-level prompt control, not global scaling, is the operation that most reliably improves emotion classification accuracy across FS2w/Emo and FS2w/Emo&Int.
  • The full FS2w/Emo&Int system with global and local prompt control yields the highest MOS, so combining learned emotion and intensity embeddings with LLM-scaled prosody improves perceived expressiveness without increasing word error rate.
  • A listener-based test places generated low, medium, and high intensity samples into the correct category about 72 percent of the time, which the paper takes as evidence that the intensity encoder produces perceptually meaningful intensity ordering.
  • Because MCD, WER, and CER remain close to baseline when prompt control is applied, the expressiveness gain is not purchased with a large loss in intelligibility or acoustic fidelity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same variance-adapter scaling trick could in principle transfer to other variance-adapter-based TTS backbones with only the allowed ranges re-tuned, since the prompt-control module never touches the learned weights.
  • A cheaper deployed system could distill GPT-4's scaling-factor behavior into a small prosody-prediction network; comparing distilled versus live-LLM outputs would quantitatively separate the prompt-following contribution from the learned encoders' contribution.
  • Because intensity rankings are learned per speaker and per emotion, the current intensity scale is relative rather than globally calibrated; testing whether listeners agree across speakers would show whether a shared intensity axis is needed for cross-speaker intensity transfer.
  • The 72 percent PIR accuracy could be broken down by emotion to see whether intensity is easier to perceive in some emotions (for example, anger) than others; the paper does not report such a breakdown.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces PROEMO, an expressive text-to-speech framework built on FastSpeech 2. It adds two HuBERT-based auxiliary encoders for emotion classification and intensity regression, and a GPT-4 prompt-control module that rescales pitch, duration, and energy at global and word levels during inference. Experiments on the ESD dataset report objective metrics (emotion classification accuracy, MCD, WER, CER) and subjective tests (MOS and perceptual intensity ranking), with the central claims that prompt control, especially local control, improves emotional expressiveness and that the framework generates multi-speaker speech with varying emotional intensity.

Significance. If the claims are substantiated, the paper offers a practical and modular extension of FastSpeech 2: emotion and intensity conditioning without style-prompt annotated data, using only LibriTTS for pretraining and ESD for fine-tuning. The design is clear and reproducible, and the prompt template with step-by-step reasoning is a potentially useful engineering artifact. The most credible evidence is the consistent ECA improvement with local prompt control in the FS2w/E and FS2w/E&I rows of Table 1 (e.g., 79.72% vs. 74.80% for FS2w/E&I). However, the intensity-control claim is not independently validated because the PIR reference labels and the intensity encoder's training target come from the same learned ranking function; the GPT-4 module's behavior is not quantitatively characterized; and the subjective tests lack inferential statistics. These gaps materially weaken the paper's central contribution as currently stated.

major comments (4)
  1. [Section 4.2 / Section 3.3] The PIR validation is circular. Section 4.2 states that intensity annotations are derived from the learned rank function r(x) described in Section 3.3, and Section 3.3 supervises the intensity encoder by regression to that same function. The Low/Medium/High test samples are generated using these labels, and the PIR reference is produced by the same function. Therefore the reported ~72% participant accuracy shows only that listeners can perceive acoustic differences that align with the model's internal ordering; it does not demonstrate control of emotional intensity on an independent scale. The authors should validate perceived intensity against human ratings of natural reference utterances, externally annotated intensity labels, or a forced-choice design built on human-ordered natural speech.
  2. [Section 3.4] The GPT-4 prompt-control module is the core mechanism claimed to improve expressiveness, but the paper provides no quantitative analysis of GPT-4's outputs. The authors note that the prompt from [16] produced unstable output and was redesigned, yet no data are reported on the distribution of predicted scaling factors, failure rate, stability across repeated API calls, or sensitivity to prompt wording. At minimum, an ablation with random scaling factors sampled from the same ranges (Eqs. 1-3) would show whether the ECA improvements come from the prompt semantics or merely from the perturbation of prosodic features.
  3. [Section 5.2] The human subjective results lack error bars and significance testing. MOS values in Table 1 (e.g., 3.408 vs. 3.728 for FS2w/E&I with L vs. G&L control) are reported for 20 participants without confidence intervals or pairwise tests, and the PIR result is reported only as 'approximately 72%' with no per-condition breakdown or chance-level comparison. The authors should report per-item standard errors, confidence intervals, and appropriate statistical tests (e.g., Wilcoxon signed-rank for MOS, binomial or permutation test for PIR), ideally with per-emotion breakdowns.
  4. [Section 5.1, Table 1] The claim that local-level prompt control 'consistently' improves ECA over no prompt control is too broad. In Table 1, Daft-Exprt with local control has ECA 0.481 versus 0.663 without prompt control, and FS2 with local control (0.296) is barely different from FS2 without control (0.247). The consistent trend actually holds only for the FS2w/E and FS2w/E&I models. The statement should be explicitly restricted to those configurations.
minor comments (5)
  1. [Section 4.1] Typo: 'it failes to converge' should be 'it fails to converge'.
  2. [Section 5.1] Typo: 'classification accuracy improves to79.72%' is missing a space before the number.
  3. [Section 5.2] The sentence 'These participants from diverse geographical regions are expertise in speech and NLP' should read 'are experts in speech and NLP.'
  4. [Figure 2 / Section 3.4] The complete prompt template is referenced in Figure 2 but not fully shown in the paper; including the full prompt in an appendix would improve reproducibility.
  5. [Equations (1)-(3)] Pitch is scaled additively while duration and energy are scaled multiplicatively; a brief justification for this asymmetry would help readers interpret the scaling factors produced by GPT-4.

Circularity Check

1 steps flagged · score 6.0 of 10

The PIR intensity evaluation is circular: ground-truth Low/Medium/High labels come from the same learned rank function r(x) that supervises the intensity encoder and labels the test samples, so the reported 72% accuracy only shows listeners agree with the model's own intensity ordering.

  1. self definitional [Section 4.2 (PIR setup), Section 3.3 (Intensity Encoder), Section 5.2 (PIR results)]
    "Since ground truth intensity annotations are lacking, we derive them using the learned ranked function, detailed in Section 3.3, for the evaluation dataset. These annotations serve as a reference during the PIR test. Participants rank generated samples based on perceived intensity, categorizing them into predefined levels, which are then compared with intensity annotations derived from the learned rank function."

    The PIR reference labels are not human ground truth: they are outputs of r(x), the same learned rank function that defines the intensity encoder's regression target and that is used to create the Low/Medium/High reconstructions at test time. Therefore the 72% accuracy measures whether listeners recover the model's own internal intensity ordering, not whether intensity is perceived independently of the model. Because the rank function, the training labels, the conditioning levels, and the evaluation reference all come from one learned function, the intensity-control claim reduces by construction to a self-consistency check.

full rationale

The central circularity is confined to the intensity-evaluation chain. Section 3.3 defines the learned rank function r(x) from acoustic features, and the intensity encoder is supervised by regression to this function. Section 4.2 then uses r(x) to derive the 'ground truth' intensity annotations for the PIR test, and Section 5.2 reports approximately 72% listener accuracy against those same annotations. This is a self-definitional benchmark: the reference standard is produced by the same learned function that generates and conditions the test samples. The claim that the framework 'can generate multi-speaker expressive speech with varying emotional intensity' therefore rests on a self-consistency check rather than an independent calibration of perceived intensity. I do not count the ECA results as circular: the emotion classifier used for evaluation is independently trained, and the comparison across prompt-control conditions is internally consistent. The MOS test is an independent human rating. The paper's reliance on unvalidated GPT-4 outputs is a robustness and correctness concern, not a circularity. Score 6 reflects that the intensity-control verification reduces by construction while other contributions retain independent content.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim relies on a set of standard ML assumptions and one evaluation-specific circularity. No new physical or abstract entities are introduced. The free parameters are the hand-tuned duration range and the learned per-speaker/per-emotion intensity ranking weights. The GPT-4 output reliability is the least-supported assumption.

free parameters (2)
  • Duration scaling range bounds = G_d, sigma_i in [0.74, 1.34]
    Empirically narrowed from [0.5,2] based on experimental observations that duration scaling is more sensitive; this hand-tuned range affects the prompt-control mapping. Section 3.4.
  • Per-speaker, per-emotion rank function weights W = Learned on ESD acoustic features via relative-attribute SVM
    Defines intensity scores used both to supervise the intensity encoder and to label PIR test references. Sections 3.3 and 4.2.
assumptions (6)
  • domain assumption HuBERT self-supervised features contain sufficient emotional and intensity information for downstream classification/regression on a small dataset
    Used to build the emotion and intensity encoders (Sections 3.2, 3.3) with only the transformer layers fine-tuned.
  • domain assumption Relative attributes framework is applicable to emotion intensity in speech
    Basis for the intensity rank function r(x)=Wx and for labeling intensity in the PIR test (Sections 3.3, 4.2).
  • domain assumption GPT-4 produces scaling factors that are emotionally appropriate and stable enough for prosody control
    The entire prompt-control module relies on this; the authors note instability with the original prompt and do not report a quantitative analysis of GPT-4 output reliability (Section 3.4).
  • domain assumption Scaling pitch, energy and duration from the FS2 variance adaptor in the specified ranges translates to perceptible prosody changes without breaking naturalness
    Central to the prompt-control mechanism (Equations 1-3).
  • domain assumption ESD emotion labels and neutral/emotional utterance pairs are reliable for training the encoders and the rank function
    Used as supervision throughout (Sections 4.1, 4.2).
  • domain assumption openSMILE 384-dimensional acoustic features capture intensity-relevant signal
    Input to the relative ranking function (Section 3.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PROEMO: Prompt-Driven Text-to-Speech Synthesis Based on Emotion and Intensity Control." pith.science (2026). https://pith.science/paper/I2KV5TLL

@misc{pith2026250106276,
  author       = {Pith},
  title        = {Pith review of: PROEMO: Prompt-Driven Text-to-Speech Synthesis Based on Emotion and Intensity Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I2KV5TLL}},
  note         = {Machine review of arXiv:2501.06276}
}
read the original abstract

Speech synthesis has significantly advanced from statistical methods to deep neural network architectures, leading to various text-to-speech (TTS) models that closely mimic human speech patterns. However, capturing nuances such as emotion and style in speech synthesis is challenging. To address this challenge, we introduce an approach centered on prompt-based emotion control. The proposed architecture incorporates emotion and intensity control across multi-speakers. Furthermore, we leverage large language models (LLMs) to manipulate speech prosody while preserving linguistic content. Using embedding emotional cues, regulating intensity levels, and guiding prosodic variations with prompts, our approach infuses synthesized speech with human-like expressiveness and variability. Lastly, we demonstrate the effectiveness of our approach through a systematic exploration of the control mechanisms mentioned above.

Figures

Figures reproduced from arXiv: 2501.06276 by the authors.

Figure 1
Figure 1. Overview of proposed expressive speech generation framework composed of 4 modules: TTS backbone based on FS2(yellow￾dotted), HuBERT for Emotion Encoder(red-dotted), HuBERT for Intensity Encoder(purple-dotted), GPT-4 Prompting for prosody control(green-dotted). FE: Feature Extractor, CLS: Classification Head, REG: Regression Head. LRF: Learned Rank Function employs prompt-based control by generating a latent speaking… view at source ↗
Figure 2
Figure 2. Introduction to Prompt Control: The scaling factors suggested by the LLM (shown in red and blue tables) directly affect the Variance Adaptor The prompt design is a crucial aspect of our methodology. Initially, we tried a prompt from [16] but faced instability in GPT-4 output. This instability caused consistent changes in pitch, energy, and duration, leading to poor expressiveness. To tackle this, we created a compre… view at source ↗
Figure 3
Figure 3. Perceptual Intensity Ranking. H,M,L: High, Medium and Low intensities. 5. Results 5.1. Objective Evaluation To assess synthesized speech expressiveness, we analyze emo￾tion recognition across different models. We train an emotion encoder independently using the ESD training set. This encoder achieves 95% accuracy on the ESD evaluation set, showcasing its ability to capture subtle emotional nuances. Further insights … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: t-SNE of embeddings for the ESD validation set. (a) Intensity embeddings computed using the relative ranking func￾tion, r(.). (b) Joint emotion and intensity embeddings. Model PC ECA ↑ MCD ↓ WER ↓ CER ↓ MOS ↑ Daft-Exprt None 0.663 6.278 0.353 0.123 3.450 Daft-Exprt G&L…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 30 canonical work pages

  1. [16]

    An overview of affective speech synthesis and conversion in the deep learning era,

    A. Triantafyllopoulos, B. W. Schuller, G. ˙Iymen, M. Sezgin, X. He, Z. Yang, P. Tzirakis, S. Liu, S. Mertes, E. Andr ´e et al., “An overview of affective speech synthesis and conversion in the deep learning era,” Proceedings of the IEEE, 2023

    work page 2023

  2. [1]

    Introduction Speech synthesis has witnessed remarkable advances in recent years, driven primarily by the integration of deep learning tech- niques [1]. While modern speech synthesis systems can pro- duce increasingly natural-sounding speech [2, 3], the challenge of imbuing synthesized speech with expressive qualities akin to human speech remains a focal p...

    work page

  3. [2]

    PromptTTS [15] introduces a style encoder trained on ’style prompts’, natural language descriptions of desired speaking styles

    Related Works Recent TTS advancements explore specifying target prosody, emotion, or speaking style through natural language prompts. PromptTTS [15] introduces a style encoder trained on ’style prompts’, natural language descriptions of desired speaking styles. This encoder predicts values for categorical parameters (e.g., gender and emotion) to guide spe...

    arXiv 2025

  4. [3]

    Initially, we pre-train a multispeaker English TTS backbone model uti- lizing a large publicly accessible dataset

    Methodology The proposed pipeline is structured into distinct stages. Initially, we pre-train a multispeaker English TTS backbone model uti- lizing a large publicly accessible dataset. Then, we add emo- tion and intensity encoders and fine-tune the model with an emotional dataset. During inference, we use a LLM to pre- dict prosody aligned with the target...

    work page

  5. [4]

    Baselines and Datasets We utilize the LibriTTS [28] for pretraining, which includes 33, 236 training samples (equating to 53.78 hours) collected from 247 speakers

    Experimental Setup 4.1. Baselines and Datasets We utilize the LibriTTS [28] for pretraining, which includes 33, 236 training samples (equating to 53.78 hours) collected from 247 speakers. To capture various emotions, we turn to the 1where a, b, and c are estimated using defined boundary condition of [0.5, 2]. Emotional Speech Database (ESD) for fine-tunin...

    work page

  6. [5]

    Objective Evaluation To assess synthesized speech expressiveness, we analyze emo- tion recognition across different models

    Results 5.1. Objective Evaluation To assess synthesized speech expressiveness, we analyze emo- tion recognition across different models. We train an emotion encoder independently using the ESD training set. This encoder achieves 95% accuracy on the ESD evaluation set, showcasing its ability to capture subtle emotional nuances. Further insights into emotio...

    work page

  7. [6]

    Conclusion In conclusion, our proposed approach, focused on prompt- based emotion control in speech synthesis, represents a signif- icant advancement in infusing synthesized speech with human- like expressiveness and variability. By integrating emotion and intensity control across multi-speakers and utilizing large LLMs to manipulate speech prosody while ...

    work page

  8. [7]

    Multi- speaker expressive speech synthesis via multiple factors decou- pling,

    X. Zhu, Y . Lei, K. Song, Y . Zhang, T. Li, and L. Xie, “Multi- speaker expressive speech synthesis via multiple factors decou- pling,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    work page 2023

Show all 43 references
  1. [8]

    A review of deep learning techniques for speech processing,

    A. Mehrish, N. Majumder, R. Bharadwaj, R. Mihalcea, and S. Po- ria, “A review of deep learning techniques for speech processing,” Information Fusion, p. 101869, 2023

    2023

  2. [9]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  3. [10]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Inter- national Conference on Machine Learning . PMLR, 2021, pp. 5530–5540

    2021

  4. [11]

    Expressive speech synthesis: a review,

    D. Govind and S. M. Prasanna, “Expressive speech synthesis: a review,”International Journal of Speech Technology, vol. 16, pp. 237–260, 2013

    2013

  5. [12]

    V oicebox: Text-guided multilingual universal speech generation at scale,

    M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V . Manohar, Y . Adi, J. Mahadeokar et al. , “V oicebox: Text-guided multilingual universal speech generation at scale,” Advances in neural information processing systems , vol. 36, 2024

    2024

  6. [13]

    Msstyletts: Multi-scale style modeling with hierarchical context information for expressive speech synthesis,

    S. Lei, Y . Zhou, L. Chen, Z. Wu, X. Wu, S. Kang, and H. Meng, “Msstyletts: Multi-scale style modeling with hierarchical context information for expressive speech synthesis,” IEEE/ACM Trans- actions on Audio, Speech, and Language Processing, 2023

    2023

  7. [14]

    Style tokens: Un- supervised style modeling, control and transfer in end-to-end speech synthesis,

    Y . Wang, D. Stanton, Y . Zhang, R.-S. Ryan, E. Battenberg, J. Shor, Y . Xiao, Y . Jia, F. Ren, and R. A. Saurous, “Style tokens: Un- supervised style modeling, control and transfer in end-to-end speech synthesis,” in International conference on machine learn- ing. PMLR, 2018,...

    2018

  8. [15]

    Ensemble prosody prediction for expressive speech synthesis,

    T. H. Teh, V . Hu, D. S. R. Mohan, Z. Hodari, C. G. Wallis, T. G. Ibarrondo, A. Torresquintero, J. Leoni, M. Gales, and S. King, “Ensemble prosody prediction for expressive speech synthesis,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Pro...

    2023

  9. [17]

    Prosody-tts: An end-to-end speech synthesis system with prosody control,

    G. Pamisetty and K. Sri Rama Murty, “Prosody-tts: An end-to-end speech synthesis system with prosody control,”Circuits, Systems, and Signal Processing, vol. 42, no. 1, pp. 361–384, 2023

    2023

  10. [18]

    Speech Synthesis with Self-Supervisedly Learnt Prosodic Representations,

    Z.-C. Liu, Z.-H. Ling, Y .-J. Hu, J. Pan, J.-W. Wang, and Y .-D. Wu, “Speech Synthesis with Self-Supervisedly Learnt Prosodic Representations,” in Proc. INTERSPEECH 2023, 2023, pp. 7–11

    2023

  11. [19]

    Exploiting emotion information in speaker em- beddings for expressive text-to-speech,

    Z. Shaheen, T. Sadekova, Y . Matveeva, A. Shirshova, and M. Kudinov, “Exploiting emotion information in speaker em- beddings for expressive text-to-speech,” in INTERSPEECH, vol. 2023, 2023, pp. 2038–2042

    2023

  12. [20]

    EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Syn- thesis,

    H. Tang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, “EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Syn- thesis,” in Proc. INTERSPEECH 2023, 2023, pp. 12–16

    2023

  13. [21]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. K. Jungil Kong and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” in Ad- vances in Neural Information Processing Systems, vol. 33, 2020, pp. 17 022–17 033

    2020

  14. [22]

    Prompttts: Control- lable text-to-speech with text descriptions,

    Z. Guo, Y . Leng, Y . Wu, S. Zhao, and X. Tan, “Prompttts: Control- lable text-to-speech with text descriptions,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  15. [23]

    We intend to improve upon the algorithm proposed in [23] for predicting emotion intensity levels using a multispeaker emotive speech dataset

    addresses this by using relative attribute labeling, similar to its success in computer vision [24]. We intend to improve upon the algorithm proposed in [23] for predicting emotion intensity levels using a multispeaker emotive speech dataset. We quantify intensity by comparing...

  16. [24]

    Controllable speaking styles using a large language model,

    A. T. Sigurgeirsson and S. King, “Controllable speaking styles using a large language model,” 2023

    2023

  17. [25]

    Emodiff: Intensity con- trollable emotional text-to-speech with soft-label guidance,

    Y . Guo, C. Du, X. Chen, and K. Yu, “Emodiff: Intensity con- trollable emotional text-to-speech with soft-label guidance,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  18. [26]

    Fine-grained quantitative emotion editing for speech generation,

    S. Inoue, K. Zhou, S. Wang, and H. Li, “Fine-grained quantitative emotion editing for speech generation,” 2024

    2024

  19. [27]

    Train- ing language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Train- ing language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022

    2022

  20. [28]

    Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,

    D. Yang, S. Liu, R. Huang, G. Lei, C. Weng, H. Meng, and D. Yu, “Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,”arXiv preprint arXiv:2301.13662, 2023

    2023 arXiv

  21. [29]

    Hubert: Self-supervised speech rep- resentation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “Hubert: Self-supervised speech rep- resentation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 29, pp. 3451–3460, 2021

    2021

  22. [30]

    Emotion intensity and its control for emotional voice conversion,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Emotion intensity and its control for emotional voice conversion,” IEEE Transactions on Affective Computing , vol. 14, no. 1, pp. 31–48, 2022

    2022

  23. [31]

    Learning visual attributes,

    V . Ferrari and A. Zisserman, “Learning visual attributes,” Ad- vances in neural information processing systems, vol. 20, 2007

    2007

  24. [32]

    opensmile: The mu- nich versatile and fast open-source audio feature extractor,

    F. Eyben, M. W ¨ollmer, and B. Schuller, “opensmile: The mu- nich versatile and fast open-source audio feature extractor,” in Proceedings of the 18th ACM International Conference on Multi- media. ACM, 2010, pp. 1459–1462

    2010

  25. [33]

    Relative attributes,

    D. Parikh and K. Grauman, “Relative attributes,” in 2011 Interna- tional Conference on Computer Vision . IEEE, 2011, pp. 503– 510

    2011

  26. [34]

    Gpt-4 system card,

    OpenAI, “Gpt-4 system card,” https://openai.com/research/gpt-4, 2023

    2023

  27. [35]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,” arXiv preprint arXiv:1904.02882, 2019

    1904 arXiv

  28. [36]

    Daft-exprt: Cross-speaker prosody transfer on any text for expressive speech synthesis,

    J. Zaıdi, H. Seut ´e, B. van Niekerk12, and M.-A. Carbonneau, “Daft-exprt: Cross-speaker prosody transfer on any text for expressive speech synthesis,” arXiv preprint arXiv:2108.02271 , 2022

    2022 arXiv

  29. [37]

    ubisoft-laforge-daft-exprt,

    J. Za ¨ıdi, H. Seut ´e, B. van Niekerk, and M.-A. Car- bonneau, “ubisoft-laforge-daft-exprt,” https://github.com/ubisoft/ ubisoft-laforge-daft-exprt.git, 2021

    2021

  30. [38]

    Generalized end-to-end loss for speaker verification,

    L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in 2018 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4879–4883

    2018

  31. [39]

    Hubert-base,

    Facebook, “Hubert-base,” https://huggingface.co/facebook/ hubert-base-ls960, 2021

    2021

  32. [40]

    Synthesizer voice qual- ity of new languages calibrated with mean mel cepstral distor- tion

    J. Kominek, T. Schultz, and A. W. Black, “Synthesizer voice qual- ity of new languages calibrated with mean mel cepstral distor- tion.” in SLTU, 2008, pp. 63–68

    2008

  33. [41]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” in ICML. PMLR, 2023, pp. 28 492–28 518

    2023

  34. [42]

    Mean opinion score (mos) revisited: Methods and applications, limitations and alter- natives,

    R. C. Streijl, S. Winkler, and D. S. Hands, “Mean opinion score (mos) revisited: Methods and applications, limitations and alter- natives,” Multimedia Systems, vol. 22, no. 2, pp. 213–227, 2016

    2016

  35. [43]

    Fine-grained emotional control of text-to-speech: Learning to rank inter-and intra- class emotion intensities,

    S. Wang, J. Guðnason, and D. Borth, “Fine-grained emotional control of text-to-speech: Learning to rank inter-and intra- class emotion intensities,” in ICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.