Pith. sign in

REVIEW 3 major objections 4 minor 30 references

RA-CLAP: Relation-Augmented Emotional Speaking Style Contrastive Language-Audio Pretraining For Speech Retrieval

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding self-distillation to contrastive language-audio pretraining lets a speech retrieval model capture graded, partial matches between a voice and its description, with gains on emotion-rich speech retrieval.

desk verdict The new task is real and the encoder study is useful, but the headline RA-CLAP gain is unproven because the only comparison couples self-distillation with fine-tuning. read the letter →

arxiv 2505.19437 v1 pith:MET3RT2M submitted 2025-05-26 cs.SD eess.AS

Haoqin Sun , Jingguang Tian , Jiaming Zhou , Hui Wang , Jiabei He , Shiwan Zhao , Xiangyu Kong , Desheng Hu
show 3 more authors
Xinkang Xu Xinhui Hu Yong Qin
This is my paper
classification cs.SDeess.AS
keywords emotionalspeakingstyleretrievalcontrastivelanguage-audiopretrainingself-distillationspeech-textcross-modalrepresentationlearningspeechemotiondescriptionpartialmatchingnaturallanguage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces emotional speaking style retrieval (ESSR), the task of ranking speech recordings by how closely their speaking style and emotion match a natural-language description, and builds ESS-CLAP, a dual-encoder contrastive model that embeds speech and text in one space. It argues that standard contrastive training is too rigid because it assumes each speech segment either fully matches a caption or does not, whereas real style descriptions hold only partially. To relax that assumption, the paper proposes RA-CLAP, which freezes the trained ESS-CLAP as a teacher and trains a student to imitate the teacher's soft similarity distribution over the batch using KL divergence. In experiments on three open datasets, RA-CLAP improves retrieval over the contrastive baseline on the emotion-annotated TextrolSpeech benchmark, though the PromptSpeech scores dip slightly. The paper positions this as a reusable recipe for cross-modal speech–language pretraining when caption–audio relations are graded rather than binary.

What carries the argument

The mechanism is self-distillation from a frozen teacher. The teacher is the InfoNCE-trained ESS-CLAP, whose softmax output over a batch already ranks every speech–text pair by similarity; the student is trained to reproduce that ranking via a symmetric KL divergence on the batch similarity matrices. Because the target distribution is soft, a caption can match several speech segments to different degrees, and a speech segment can carry multiple partial matches, which ordinary hard positive/negative labels cannot express. The model also uses a weighted sum over speech-encoder layers and learnable temperature parameters $\epsilon_s$ and $\epsilon_t$ to shape the similarity matrices that enter the distillation loss.

What would settle it

Train RA-CLAP with an uninformative teacher, for example a teacher whose batch similarity matrix is replaced by random noise or whose speech and text encoders are randomly permuted, keeping the same fine-tuning protocol. If TextrolSpeech gains persist, the teacher's soft distribution is not the cause; if they disappear, the teacher's learned matching distribution is doing the claimed work.

Watch

Extended reading notes

Core claim

The central claim is that self-distillation can teach a contrastive speech–text model the graded, part-matching relation between a speech segment and its description. The pretrained ESS-CLAP model is used as a frozen teacher: for each batch it produces a probability matrix $M$ by softmax over the similarity scores between all speech and text items in the batch. The student is initialized from the teacher and trained with the symmetric KL loss $L_d = \frac{1}{2}\left(\mathrm{KL}(\mathrm{logsoftmax}(C_s), \mathrm{softmax}(M)) + \mathrm{KL}(\mathrm{logsoftmax}(C_t), \mathrm{softmax}(M))\right)$, where $C_s$ and $C_t$ are the student's own scaled similarity matrices. The authors report that this distillation, combined with fine-tuning on PromptSpeech and TextrolSpeech, lifts TextrolSpeech retrieval over the balanced-sampling ESS-CLAP baseline (Audio-to-Text R@1 from 23.5 to 26.0, Text-to-Audio R@1 from 25.5 to 27.5) while PromptSpeech scores fall slightly. They interpret the shift as the model trading off between description types, learning the emotion-focused TextrolSpeech distribution at some cost to PromptSpeech's style-and-content descriptions.

Load-bearing premise

The argument stands on the premise that the frozen teacher's soft probability scores over a batch are a faithful signal of which speech–caption pairs partially match, rather than just a reflection of the teacher's own biases; if that premise fails, self-distillation cannot add information and the reported gains could be an artifact of the extra fine-tuning stage.

Editorial extensions

If this is right

  • RA-CLAP provides a two-stage recipe—contrastive pretraining followed by KL distillation from the frozen pretrained model—that can be applied to any dual-encoder speech–text retrieval system without new annotations.
  • The results indicate that emotion-bearing descriptions are where the soft distribution helps most, so datasets whose captions are purely stylistic may not benefit from this form of self-distillation.
  • WavLM with RoBERTa is the best of the tested encoder pairs, and speaker-representation encoders perform comparably to SSL speech encoders, suggesting speaker identity is a usable channel for emotional style retrieval.
  • Joint training on PromptSpeech and TextrolSpeech improves both test sets, while adding SpeechCraft without fine-tuning hurts; the authors argue the large, diverse SpeechCraft data still helps after fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cleaner ablation would fine-tune ESS-CLAP on PromptSpeech+TextrolSpeech exactly as RA-CLAP is fine-tuned; without that control, the reported comparison mixes the effect of self-distillation with the effect of fine-tuning.
  • If the teacher-distribution mechanism is the active ingredient, the same recipe should transfer to other fine-grained retrieval tasks where captions match only partially, such as sound-event retrieval or speaker-description search.
  • Replacing the teacher's softmax targets with labels derived from known partial-match annotations would test whether genuine partial-match knowledge, rather than the teacher's internal biases, is what the student absorbs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces emotional speaking style retrieval (ESSR), a task for retrieving speech by natural-language descriptions of emotion and speaking style. It proposes ESS-CLAP, a contrastive language-audio pretraining model, and RA-CLAP, which adds a self-distillation stage in which a frozen ESS-CLAP teacher provides softmax probability distributions over batch negatives as KL targets for a student model. Experiments on PromptSpeech, TextrolSpeech, and SpeechCraft report retrieval metrics (mAP@10 and R@1/R@5/R@10) for both audio-to-text and text-to-audio directions. The authors claim the results validate RA-CLAP's effectiveness and show that self-distillation improves generalization by learning 'local matching relationships.' The paper's central evidence, however, is weakened by an uncontrolled comparison: the only RA-CLAP row in Table 2 is fine-tuned on PS+TS while the ESS-CLAP baseline is not, so the reported gains cannot be attributed to the distillation mechanism alone.

Significance. If the self-distillation mechanism were cleanly validated, RA-CLAP would be a useful recipe for speech-language pretraining in expressive speech domains, and the ESSR task definition plus the encoder comparison would be a serviceable benchmark contribution. The paper also makes a sensible design choice in balanced sampling across datasets to mitigate severe size imbalance. However, the core claim is not yet supported by the experiments as presented. The comparison that is supposed to isolate the contribution of the teacher-soft-label objective also changes the training protocol, and the acknowledged PromptSpeech regression contradicts the unqualified abstract statement that the results validate RA-CLAP. The manuscript is therefore a promising step but requires a matched experimental design and additional analyses before the central claim can be accepted.

major comments (3)
  1. [§3.3, Table 2] The only RA-CLAP comparison is confounded. The row 'PS+TS+SC∗∗' is fine-tuned on PS+TS, whereas the corresponding ESS-CLAP row 'PS+TS+SC∗' is not fine-tuned at all. The reported TextrolSpeech gains (R@1 audio-to-text 26.5 vs 23.5; text-to-audio 27.5 vs 25.5) and the PromptSpeech regression (audio-to-text R@1 5.5 vs 6.2) could therefore be caused entirely by continued fine-tuning on in-domain data, by the balanced data mixture, or by the distillation hyperparameters, rather than by the soft-label objective. To support the abstract's claim that the experimental results validate RA-CLAP, the authors must add a controlled comparison: an ESS-CLAP model fine-tuned on PS+TS without the KL term, and a RA-CLAP model evaluated without fine-tuning. Without such a control, the central claim is not established.
  2. [§2.2, Eqs. (4)--(7)] The premise that the frozen teacher's softmax distribution over in-batch negatives faithfully models 'local matching relationships' is not supported. Because the student is initialized from the teacher and trained on the same data, self-distillation could merely reinforce the teacher's own biases rather than add new relational information. The paper provides no analysis showing that the soft labels contain information beyond the hard InfoNCE labels, nor an ablation varying the sharpness of the teacher distribution (e.g., through the temperatures ε_s and ε_t). The values of ε_s and ε_t, as well as the InfoNCE temperature τ, are never reported, yet the KL objective in Eq. (6) depends critically on them. Without a sensitivity analysis or a control that replaces the teacher soft labels with an alternative soft-target source, the mechanism claimed for RA-CLAP remains unvalidated.
  3. [§3.2, Table 2] No error bars, multiple seeds, or significance tests are reported for any retrieval metric. Many of the differences used to support the paper's conclusions are small in absolute terms (e.g., R@1 changes of one to three points), and in the absence of variance estimates it is unclear whether the observed differences are reliable. The authors should report mean and standard deviation over at least three seeds for the main comparisons, or provide some other statistical assessment.
minor comments (4)
  1. [§2.2, Eq. (6)] The term 'RA-CALP' appears in the text describing Eq. (6); this should be 'RA-CLAP' for consistency with the rest of the paper.
  2. [§2.1] The sentence 'To extract high-quality speech features, the SS-CLAP model utilizes the speech encoder' refers to a model that has not been introduced; it should read ESS-CLAP or RA-CLAP as appropriate.
  3. [§3.3, Table 2] The table caption defines the asterisk symbols but the row 'PS+TS+SC∗∗' is marked as RA-CLAP without a separate column or notation showing which fine-tuning stage is used; please clarify the training and fine-tuning configuration of every row in the table.
  4. [§3.3] The Discussion of the PromptSpeech regression as a 'trade-off' is reasonable, but the abstract should be revised to reflect this nuance rather than stating without qualification that the experimental results validate RA-CLAP.

Circularity Check

1 steps flagged · score 2.0 of 10

RA-CLAP's retrieval gains are empirical and not derived from its inputs, but its central mechanism claim is self-referential: the 'local matching relationships' are defined as the frozen teacher's own softmax distribution, and the reported comparison is confounded by additional fine-tuning.

  1. self definitional [Section 2.2, Eqs. (4)-(7); Abstract]
    "In this stage, we utilize the pretrained model ESS-CLAP as the teacher model, freezing its parameters and initializing the student model with these parameters. To overcome the limitations of traditional binary correspondence, we use the probability distribution predicted by the teacher model as the learning objective."

    The paper's stated contribution is that self-distillation learns 'potential local matching relationships between speech and descriptions' (Abstract), but the only definition of those relationships is the teacher's softmax distribution M used in Eq. (6). The student is initialized from the teacher and trained with Ld = 1/2[KL(ls(Cs), s(M)) + KL(ls(Ct), s(M))], so the learned 'relationships' are by construction the teacher's own similarity distribution. There is no external annotation of partial or local matches; the teacher itself was trained with the standard binary InfoNCE loss (Eq. 3). Thus the mechanism cannot introduce relational information beyond the teacher's biases, and the causal claim 'thereby enhancing generalization ability' is not independently grounded.

full rationale

The central empirical claim—that RA-CLAP improves retrieval over ESS-CLAP—is tested on held-out splits of PromptSpeech, TextrolSpeech, and SpeechCraft, so the reported retrieval numbers are not derived from the training objective. The paper is self-contained against these external benchmarks and its comparisons in Table 2 are empirical. The self-citations (refs [1]–[6]) are background on emotion recognition and are not load-bearing for the method or results. The one definitional concern is the self-distillation objective: Eq. (6) makes the teacher's own distribution the target, so the phrase 'potential local matching relationships' is by construction the teacher's softmax output; this weakens the method's motivation but does not make the measured retrieval gains tautological. In addition, the RA-CLAP row (PS+TS+SC**) differs from the balanced ESS-CLAP row (PS+TS+SC*) by both the distillation objective and additional fine-tuning on PS+TS, so the ablation does not isolate self-distillation; this is an experimental confound rather than a circular derivation. Overall score 2 reflects the minor self-referentiality in the method framing without a forced or definitional central result.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger lists the hand-set temperatures, the data-balancing design choice, and the assumptions that the captions and the teacher soft labels are trustworthy. Ordinary network weights are not counted as free parameters because they are trained rather than chosen ad hoc; no new physical or conceptual entity is introduced.

free parameters (4)
  • InfoNCE temperature tau = not reported
    Temperature in Eq. (3) controls sharpness of the contrastive softmax and is chosen by hand; the value is not specified.
  • Distillation temperature epsilon_s = not reported
    Eq. (4) scales the student speech-text similarity matrix before the KL loss; the value is not reported.
  • Distillation temperature epsilon_t = not reported
    Eq. (5) scales the student text-speech similarity matrix; the value is not reported.
  • Balanced sampling ratio = 1:1:1
    Authors set each dataset to one third of each batch to counter size imbalance; this choice affects all reported retrieval numbers.
assumptions (4)
  • domain assumption Paired speech-caption instances are correct and complete enough for emotional speaking style retrieval.
    All contrastive training and evaluation rely on PromptSpeech, TextrolSpeech, and SpeechCraft captions as ground truth; no human verification is reported.
  • ad hoc to paper Teacher softmax over batch negatives models true local matching relationships.
    This is the core premise of RA-CLAP in Section 2.2, asserted without proof or analysis of the learned distributions.
  • domain assumption Self-distillation with a frozen teacher on the same training data does not collapse or merely amplify teacher errors.
    No theoretical or empirical analysis of the distillation dynamics is provided.
  • domain assumption The three datasets are compatible enough for joint training with 1:1:1 sampling.
    The paper itself notes that SpeechCraft descriptions are more diverse and hurt transfer, so compatibility is an empirical bet rather than an established fact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RA-CLAP: Relation-Augmented Emotional Speaking Style Contrastive Language-Audio Pretraining For Speech Retrieval." pith.science (2026). https://pith.science/paper/MET3RT2M

@misc{pith2026250519437,
  author       = {Pith},
  title        = {Pith review of: RA-CLAP: Relation-Augmented Emotional Speaking Style Contrastive Language-Audio Pretraining For Speech Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MET3RT2M}},
  note         = {Machine review of arXiv:2505.19437}
}
read the original abstract

The Contrastive Language-Audio Pretraining (CLAP) model has demonstrated excellent performance in general audio description-related tasks, such as audio retrieval. However, in the emerging field of emotional speaking style description (ESSD), cross-modal contrastive pretraining remains largely unexplored. In this paper, we propose a novel speech retrieval task called emotional speaking style retrieval (ESSR), and ESS-CLAP, an emotional speaking style CLAP model tailored for learning relationship between speech and natural language descriptions. In addition, we further propose relation-augmented CLAP (RA-CLAP) to address the limitation of traditional methods that assume a strict binary relationship between caption and audio. The model leverages self-distillation to learn the potential local matching relationships between speech and descriptions, thereby enhancing generalization ability. The experimental results validate the effectiveness of RA-CLAP, providing valuable reference in ESSD.

Figures

Figures reproduced from arXiv: 2505.19437 by the authors.

Figure 1
Figure 1. Overview of the proposed RA-CLAP model. usability and effectiveness of the ESS-CLAP model. To the best of our knowledge, this study marks the first sys￾tematic application of contrastive cross-modal pretraining for the task of emotional speaking style retrieval. We hope that this work serves as a foundation for large-scale speech genera￾tive models applicable to tasks such as emotional speaking style captioning or p… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages

  1. [1]

    Introduction The language-based speech retrieval task refers to the process of searching for speech recordings based on natural language de- scriptions. This approach enables users to intuitively retrieve speech based on desired voice characteristics (e.g., emotion, speaker timbre, and speaking style) without relying on pre- defined labels. Different from...

    work page Pith review arXiv 2025

  2. [2]

    1, the proposed RA-CLAP model is designed to learn a joint representation of speech and text through a con- trastive learning and self-distillation framework

    Proposed Methods As shown in Fig. 1, the proposed RA-CLAP model is designed to learn a joint representation of speech and text through a con- trastive learning and self-distillation framework. The training process consists of a pre-training phase and a self-distillation phase. The process is as follows: 2.1. pre-training phase Given a data pair consisting...

    work page

  3. [3]

    Experiments 3.1. Datasets The PromptSpeech (PS) dataset [9] consists of prompts con- taining style and content information, along with corresponding Table 3:Dataset Statistics for Speech Data Sources. Dataset Language #Duration #Samples SPEECHCRAFTEn 1437.57h 1,097,989 PROMPTSPEECHEn 38h 26,588 TEXTROLSPEECHEn 330h 236,220 speech samples. The dataset is a...

    work page

  4. [4]

    Conclusions In this paper, we conduct a preliminary investigation into the key factors affecting the CLAP model for ESSR task. As part of this study, we propose a novel task ESSR and ESS-CLAP, a CLAP model specifically designed for learning speech rep- resentations by effectively integrating speech with natural lan- guage descriptions. Further, we propose...

    work page

  5. [5]

    Multi-level knowledge distillation for speech emotion recogni- tion in noisy conditions,

    Y . Liu, H. Sun, G. Chen, Q. Wang, Z. Zhao, X. Lu, and L. Wang, “Multi-level knowledge distillation for speech emotion recogni- tion in noisy conditions,” inINTERSPEECH 2023, 2023, pp. 1893–1897

    work page 2023

  6. [6]

    Iterative prototype refinement for ambiguous speech emotion recognition,

    H. Sun, S. Zhao, X. Kong, X. Wang, H. Wang, J. Zhou, and Y . Qin, “Iterative prototype refinement for ambiguous speech emotion recognition,” inProc. Interspeech 2024, 2024, pp. 3200–3204

    work page 2024

  7. [7]

    Fine- grained disentangled representation learning for multimodal emo- tion recognition,

    H. Sun, S. Zhao, X. Wang, W. Zeng, Y . Chen, and Y . Qin, “Fine- grained disentangled representation learning for multimodal emo- tion recognition,” inICASSP 2024-2024 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 051–11 055

    work page 2024

  8. [8]

    Enhancing multimodal emotion recognition through multi- granularity cross-modal alignment,

    X. Wang, S. Zhao, H. Sun, H. Wang, J. Zhou, and Y . Qin, “Enhancing multimodal emotion recognition through multi- granularity cross-modal alignment,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

Show all 30 references
  1. [9]

    Enhancing emotion recognition in incomplete data: A novel cross-modal alignment, reconstruction, and refinement framework,

    H. Sun, S. Zhao, S. Li, X. Kong, X. Wang, J. Zhou, A. Kong, Y . Chen, W. Zeng, and Y . Qin, “Enhancing emotion recognition in incomplete data: A novel cross-modal alignment, reconstruction, and refinement framework,” inICASSP 2025-2025 IEEE Inter- national Conference on Acoust...

    2025

  2. [10]

    Emotion- preserving prosody anonymization network for voice privacy pro- tection,

    J. He, S. Zhao, J. Zhou, H. Sun, H. Wang, and Y . Qin, “Emotion- preserving prosody anonymization network for voice privacy pro- tection,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  3. [11]

    Speaker-text retrieval via contrastive learning,

    X. Liu, X. Wang, E. Cooper, X. Miao, and J. Yamagishi, “Speaker-text retrieval via contrastive learning,”arXiv preprint arXiv:2312.06055, 2023

    2023 arXiv

  4. [12]

    Paraclap– towards a general language-audio model for computational par- alinguistic tasks,

    X. Jing, A. Triantafyllopoulos, and B. Schuller, “Paraclap– towards a general language-audio model for computational par- alinguistic tasks,”arXiv preprint arXiv:2406.07203, 2024

    2024 arXiv

  5. [13]

    Prompttts: Control- lable text-to-speech with text descriptions,

    Z. Guo, Y . Leng, Y . Wu, S. Zhao, and X. Tan, “Prompttts: Control- lable text-to-speech with text descriptions,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  6. [14]

    Prompttts++: Controlling speaker identity in prompt-based text-to-speech using natural language descriptions,

    R. Shimizu, R. Yamamoto, M. Kawamura, Y . Shirahata, H. Doi, T. Komatsu, and K. Tachibana, “Prompttts++: Controlling speaker identity in prompt-based text-to-speech using natural language descriptions,” inICASSP 2024-2024 IEEE Interna- tional Conference on Acoustics, Speech an...

    2024

  7. [15]

    Stylecap: Automatic speaking-style captioning from speech based on speech and lan- guage self-supervised learning models,

    K. Yamauchi, Y . Ijima, and Y . Saito, “Stylecap: Automatic speaking-style captioning from speech based on speech and lan- guage self-supervised learning models,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, ...

    2024

  8. [16]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  9. [17]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (I...

    2023

  10. [18]

    Gemo-clap: Gender-attribute-enhanced contrastive language- audio pretraining for accurate speech emotion recognition,

    Y . Pan, Y . Hu, Y . Yang, W. Fei, J. Yao, H. Lu, L. Ma, and J. Zhao, “Gemo-clap: Gender-attribute-enhanced contrastive language- audio pretraining for accurate speech emotion recognition,” in ICASSP 2024-2024 IEEE International Conference on Acous- tics, Speech and Signal Pro...

    2024

  11. [19]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,”arXiv preprint arXiv:1904.02882, 2019

    1904 arXiv

  12. [20]

    Textrolspeech: A text style control speech corpus with codec language text-to-speech models,

    S. Ji, J. Zuo, M. Fang, Z. Jiang, F. Chen, X. Duan, B. Huai, and Z. Zhao, “Textrolspeech: A text style control speech corpus with codec language text-to-speech models,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, ...

    2024

  13. [21]

    Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (ver- sion 0.92),

    J. Yamagishi, C. Veaux, K. MacDonaldet al., “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (ver- sion 0.92),”University of Edinburgh. The Centre for Speech Tech- nology Research (CSTR), pp. 271–350, 2019

    2019

  14. [22]

    Seen and unseen emo- tional style transfer for voice conversion with a new emotional speech dataset,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Seen and unseen emo- tional style transfer for voice conversion with a new emotional speech dataset,” inICASSP 2021-2021 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 920–924

    2021

  15. [23]

    Toronto emotional speech set (tess)-younger talker happy,

    K. Dupuis and M. K. Pichora-Fuller, “Toronto emotional speech set (tess)-younger talker happy,” 2010

    2010

  16. [24]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation,

    K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y . Qiao, and C. C. Loy, “Mead: A large-scale audio-visual dataset for emotional talking-face generation,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 700–717

    2020

  17. [25]

    Speaker-dependent audio- visual emotion recognition

    S. Haq, P. J. Jackson, and J. Edge, “Speaker-dependent audio- visual emotion recognition.” inA VSP, vol. 2009, 2009, pp. 53–58

    2009

  18. [26]

    Categorical and dimensional ratings of emotional speech: Behavioral findings from the morgan emotional speech set,

    S. D. Morgan, “Categorical and dimensional ratings of emotional speech: Behavioral findings from the morgan emotional speech set,”Journal of Speech, Language, and Hearing Research, vol. 62, no. 11, pp. 4015–4029, 2019

    2019

  19. [27]

    Speechcraft: A fine-grained expressive speech dataset with natural language description,

    Z. Jin, J. Jia, Q. Wang, K. Li, S. Zhou, S. Zhou, X. Qin, and Z. Wu, “Speechcraft: A fine-grained expressive speech dataset with natural language description,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 1255– 1264

    2024

  20. [28]

    Aishell-3: A multi- speaker mandarin tts corpus and the baselines,

    Y . Shi, H. Bu, X. Xu, S. Zhang, and M. Li, “Aishell-3: A multi- speaker mandarin tts corpus and the baselines,”arXiv preprint arXiv:2010.11567, 2020

    2010 arXiv

  21. [29]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhanget al., “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    2021 arXiv

  22. [30]

    Libritts-r: A restored multi-speaker text-to-speech corpus,

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, and A. Bapna, “Libritts-r: A restored multi-speaker text-to-speech corpus,”arXiv preprint arXiv:2305.18802, 2023

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.