Pith. sign in

REVIEW 6 major objections 5 minor 68 references

Leveraging Cross-Attention Transformer and Multi-Feature Fusion for Cross-Linguistic Speech Emotion Recognition

T0 review · 6 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fusing HuBERT with MFCC and prosody through a cross-attention transformer, then fine-tuning on 10% to one third of each target dataset, yields a cross-linguistic emotion recognizer averaging 78.75% accuracy across seven datasets in five…

desk verdict A plausible feature-fusion architecture for cross-linguistic SER, but the headline numbers are not backed by a controlled protocol and the paper needs major revision before its claims can be trusted. read the letter →

arxiv 2501.10408 v1 pith:HLBRDQNH submitted 2025-01-06 eess.AS cs.CLcs.SD

Ruoyu Zhao , Xiantao Jiang , F. Richard Yu , Victor C.M. Leung , Tao Wang , Shaohu Zhang This is my paper · ORCID
classification eess.AScs.CLcs.SD
keywords speechemotionrecognitioncross-linguisticcross-attentiontransformerHuBERTMFCCprosodicfeaturestransferlearningmulti-featurefusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a system called HuMP-CAT can recognize emotion in speech across five languages by combining HuBERT self-supervised representations with MFCCs and prosodic features, fused through a cross-attention transformer. The model is trained on the English IEMOCAP corpus, then fine-tuned on a small slice of each target dataset (10% to one third of the target data). Across seven target datasets the paper reports an average accuracy of 78.75%, with 88.69% on German EMODB and 79.48% on Italian EMOVO. If the result holds, it would mean a single pretrained speech representation plus a small labeled sample can adapt emotion recognition to new languages, which matters because large annotated emotion corpora are scarce for most languages.

What carries the argument

The load-bearing mechanism is the Cross-Attention Transformer (CAT), a transformer block in which one input feature sequence supplies the query and a second supplies the keys and values, letting either stream attend to the other. HuMP-CAT applies it twice: first to fuse a prosodic embedding with an MFCC embedding produced by a Bi-LSTM, then to fuse the result with a 768-dimensional HuBERT embedding taken from transformer layers 1 and 9. The outputs are summarized by their mean and variance into a 64-dimensional vector, which is classified with AM-Softmax. This two-stage fusion is what allows heterogeneous feature types to exchange information and is the component the paper credits for the cross-linguistic gain.

What would settle it

Re-run the CLSER evaluation with strictly speaker-disjoint fine-tuning and test partitions for all seven target datasets, using the same emotion classes and the same amount of target data for every method, and check whether the 78.75% average and the per-dataset leads over GAN-SVM, ADRNN, VACNN+BOVW, FLUDA, MDAT, and EmoBox survive; if the gaps close, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that cross-attention fusion of complementary acoustic features is the key to cross-linguistic transfer. Concretely, the authors argue that HuBERT's learned representations, the spectral detail in MFCCs, and prosody carry different emotion-related information, and that a two-stage cross-attention transformer lets these streams exchange information before classification. After training the full HuMP-CAT on IEMOCAP and fine-tuning on a small subset of each target corpus, the paper reports that it reaches 78.75% average accuracy over EMODB, RAVDESS, TESS, SAVEE, EMOVO, MESD, and ESD, and that this outperforms the compared GAN-SVM, ADRNN, VACNN+BOVW, FLUDA, MDAT, and EmoBox baselines on the overlapping target datasets. The paper also reports an ablation in which combining MFCC with prosody beats either feature alone, for example EMODB accuracy rises from 82.54% or 83.36% to 88.69%.

Load-bearing premise

The load-bearing premise is that the small fine-tuning subset used from each target dataset is a fair, representative sample that does not overlap in speakers with the test set, so the reported accuracies reflect true generalization rather than speaker-identity leakage.

Editorial extensions

If this is right

  • A single English-trained source model can be adapted to German, Italian, Spanish, and Chinese with only a small labeled subset of each language, lowering the data barrier for adding new languages.
  • Combining hand-crafted acoustic features with self-supervised representations improves over either feature type alone, as shown by the ablation where EMODB accuracy reaches 88.69% with both features versus 82.54% with MFCC alone and 83.36% with prosody alone.
  • The transfer advantage appears even for the linguistically distant Chinese target (ESD at 60.35%), which the paper reports as still better than the compared baselines.
  • The two-stage cross-attention fusion with AM-Softmax classification provides a reusable architecture for other speech tasks that need to fuse multiple feature streams.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper never states whether the fine-tuning subset and test set are speaker-disjoint, a strict speaker-disjoint evaluation is the decisive follow-up test; if speakers overlap, the 78.75% average could partly reflect speaker-identity leakage rather than emotion transfer.
  • The comparisons against prior work mix different source corpora, different amounts of target data, and different emotion-class sets, so an apples-to-apples benchmark with identical fine-tuning budgets would be needed to credit the architecture rather than the setup.
  • The paper reports results on SAVEE in Table VI but does not describe that dataset in Section IV-A, making the seven-dataset average hard to reproduce from the text alone.
  • If cross-attention fusion is the true driver, swapping the HuBERT front-end for another self-supervised speech model such as WavLM or data2vec should produce similar cross-linguistic gains; testing that swap would separate the fusion mechanism from the choice of representation model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The manuscript proposes HuMP-CAT, a cross-linguistic speech emotion recognition (CLSER) system that fuses HuBERT self-supervised representations with MFCC and prosodic features via two cross-attention transformer stages, followed by AM-Softmax classification. The source model is trained on IEMOCAP and then fine-tuned on small subsets of seven target datasets (EMODB, MESD, EMOVO, RAVDESS, SAVEE, TESS, and ESD). The authors report an average accuracy of 78.75% across the seven datasets and claim state-of-the-art CLSER performance, with notable results on EMODB (88.69%) and EMOVO (79.48%). The paper also includes ablations on feature combinations and a comparison with other self-supervised models on IEMOCAP.

Significance. If the reported accuracies are reliable, the work provides a useful data point that combining fixed self-supervised embeddings (HuBERT) with classical acoustic features through cross-attention can improve cross-corpus SER under low-resource fine-tuning. The paper evaluates on multiple languages and public corpora, and the architecture is described in enough detail for replication. However, the significance is currently limited by several load-bearing reporting problems: the headline average is arithmetically inconsistent with the per-dataset table, the fine-tuning/test split is ambiguous on speaker overlap, the target-dataset list is inconsistent between Table II and Section IV-E, and the state-of-the-art comparison is uncontrolled. These issues must be resolved before the central superiority claim can be assessed.

major comments (6)
  1. [Abstract / Section IV-E, Table VI] The abstract's claimed average accuracy of 78.75% does not match Table VI: the seven reported values (88.69, 83.56, 85.45, 77.91, 79.48, 72.84, 60.35) sum to 548.28, giving a mean of 78.33%. Since the average is a headline result, this arithmetic inconsistency must be corrected and the discrepancy explained.
  2. [Section IV-E] The fine-tuning protocol is ambiguous on the critical question of speaker overlap. The text states '20% of the speakers' speech from EMODB, one third of the speakers' speech from EMOVO, and 10% of the data from the remaining datasets,' but it does not specify whether the fine-tuning and test partitions are speaker-disjoint. In EMODB (10 speakers) and EMOVO (6 speakers), if any speaker's utterances appear in both partitions, the reported accuracies can be inflated via speaker-identity leakage. Please specify whether splits are made at the speaker level or utterance level, and if utterance-level, provide results with strictly speaker-disjoint partitions.
  3. [Section IV-A and Table II] The list of target datasets is internally inconsistent. Section IV-E says the seven evaluation datasets are EMODB, MESD, EMOVO, RAVDESS, SAVEE, TESS, and ESD, but Table II lists IEMOCAP, RAVDESS, TESS, EMODB, EMOVO, MESD, and ESD, omitting SAVEE and including the source corpus IEMOCAP. Moreover, SAVEE is not described in Section IV-A, even though it contributes one of the seven accuracy values in Table VI. The dataset description must be completed and made consistent.
  4. [Section IV-C] The evaluation protocol is described as '10-fold cross validation' but then as an 80% train / 10% validation / 10% test speaker split, which is not 10-fold cross-validation. This ambiguity affects the credibility of the IEMOCAP source model results, which are the foundation for the transfer learning experiments. Please clarify the exact resampling scheme.
  5. [Section IV-E, Fig. 5, Table VI] The comparison with prior CLSER methods is not controlled: prior results use different source datasets, different amounts of target data for fine-tuning, different emotion label sets, and different evaluation protocols. The paper itself acknowledges this in Fig. 5, stating that the different source datasets 'provide useful reference points.' Consequently, the abstract's statement that HuMP-CAT 'outperforms existing methods across multiple target languages' is not supported by the evidence as presented. Either provide matched baselines re-implemented under identical protocols, or soften the claim to a reference-point comparison.
  6. [Section IV-E] All reported accuracies are single point estimates with no error bars, multiple seeds, or repeated runs. Without variance information, even the within-paper differences (e.g., HuMP-CAT 88.69% vs. VACNN+BOVW 86.92% on EMODB) cannot be judged as meaningful. Please report means and standard deviations over at least three independent runs.
minor comments (5)
  1. [Section III-B] The phrase 'masked masked Lossm' contains a duplicated word and should read 'masked Lossm.'
  2. [Equation (8)] The cosine term appears to use 'm' in the denominator of the argument, which is likely a typo for the number of filter banks 'M' (i.e., cos((m+0.5)iπ/M)).
  3. [Section IV-A7] The ESD dataset description says it contains '350 parallel utterances spoken by 10 native English speakers,' but Table II lists ESD as a Chinese dataset with 420 utterances. Please clarify the language, utterance count, and whether the description matches the actual subset used.
  4. [Section IV-E] The sentence introducing the comparison with Ahn et al. reads 'Zehra et al. Ahn et al. [66] introduced Few-shot Learning and Unsupervised Domain Adaptation'; this appears to be a grammatical error and should be corrected.
  5. [General] Several references are cited with incomplete bibliographic information (e.g., [17] uses 'DistilHuBERT' while the text refers to HuBERT; [36] has inconsistent capitalization). Please review the reference list for consistency and completeness.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports direct empirical measurements rather than deriving its claims from fitted inputs or self-citations.

full rationale

HuMP-CAT's central claims are empirical: Table III–VII report measured accuracy values on IEMOCAP and seven target corpora. There is no equation in the paper from which an accuracy value is derived by construction, and no fitted parameter is renamed as a prediction. The feature combination (HuBERT + MFCC + prosody) is selected via IEMOCAP experiments and then applied to target datasets; this is standard model selection, not circular reasoning. The choice of HuBERT layers 1 and 9 is attributed to an external prior study [37], not to the authors' own unverified result. The only identifiable self-citation is reference [15] by one of the authors, but it appears in related work on voice privacy and is not load-bearing for the CLSER claims. The reviewer concern that the fine-tuning/test split may not be speaker-disjoint is a reproducibility and validity risk, not a circularity: it concerns whether the measurements are trustworthy, not whether the results reduce to their inputs. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a standard ML training setup (pretrained HuBERT, supervised fine-tuning, evaluation on held-out target data). The main hand-chosen quantities are the fine-tuning budgets per dataset, the selected HuBERT layers, the convolution kernel, and the AM-Softmax margin. The key unstated assumption is that the data splits are speaker-disjoint and directly comparable across corpora; this is a domain assumption the paper does not justify.

free parameters (5)
  • Fine-tuning data budget per target dataset = 20% (EMODB), 1/3 (EMOVO), 10% (others)
    Hand-chosen, dataset-dependent budgets. The abstract claims '10% or 20%', but the body uses one third for EMOVO. This choice directly affects the reported accuracies and makes cross-method comparison uncalibrated.
  • HuBERT transformer layer selection = layers 1 and 9
    Layers selected based on [37] and presumably IEMOCAP validation, then fixed for all target datasets. This is a hand-selected architectural choice that could be tuned per dataset.
  • Convolution kernel and stride for HuBERT feature projection = kernel 10x18, stride 4x3 (Fig.3 shows 10x15)
    The paper states one kernel size in the text and another in the figure; this is a hand-chosen hyperparameter affecting the fused representation.
  • AM-Softmax margin = not specified
    The loss margin is not reported, yet it controls the discriminative power of the classifier. Without its value, reproduction is impossible.
  • Training epochs and batch size = 50 epochs, batch 32, learning rate 1e-3
    These hyperparameters are fixed without a search; they determine final accuracy and are not justified.
assumptions (3)
  • domain assumption The four emotions happy, sad, angry and neutral are directly comparable across all seven datasets.
    Used to map each dataset's emotion labels to a common set (Section IV-A). Variations in annotation and acted emotion quality across corpora are ignored.
  • domain assumption HuBERT representations trained on English (LibriSpeech) transfer to German, Italian, Spanish, Chinese and English SER without language-specific adaptation.
    The entire cross-linguistic claim relies on the ability of the self-supervised features to remain emotion-discriminative across languages; no language-id or domain adaptation is used.
  • domain assumption The evaluation protocol, such as speaker-disjoint splits, is standard and implicitly assumed.
    The paper never specifies whether test speakers are disjoint from fine-tuning speakers, yet this is a standard requirement for speaker-independent emotion recognition. The omission affects all reported target accuracies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Cross-Attention Transformer and Multi-Feature Fusion for Cross-Linguistic Speech Emotion Recognition." pith.science (2026). https://pith.science/paper/HLBRDQNH

@misc{pith2026250110408,
  author       = {Pith},
  title        = {Pith review of: Leveraging Cross-Attention Transformer and Multi-Feature Fusion for Cross-Linguistic Speech Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HLBRDQNH}},
  note         = {Machine review of arXiv:2501.10408}
}
read the original abstract

Speech Emotion Recognition (SER) plays a crucial role in enhancing human-computer interaction. Cross-Linguistic SER (CLSER) has been a challenging research problem due to significant variability in linguistic and acoustic features of different languages. In this study, we propose a novel approach HuMP-CAT, which combines HuBERT, MFCC, and prosodic characteristics. These features are fused using a cross-attention transformer (CAT) mechanism during feature extraction. Transfer learning is applied to gain from a source emotional speech dataset to the target corpus for emotion recognition. We use IEMOCAP as the source dataset to train the source model and evaluate the proposed method on seven datasets in five languages (e.g., English, German, Spanish, Italian, and Chinese). We show that, by fine-tuning the source model with a small portion of speech from the target datasets, HuMP-CAT achieves an average accuracy of 78.75% across the seven datasets, with notable performance of 88.69% on EMODB (German language) and 79.48% on EMOVO (Italian language). Our extensive evaluation demonstrates that HuMP-CAT outperforms existing methods across multiple target languages.

Figures

Figures reproduced from arXiv: 2501.10408 by the authors.

Figure 1
Figure 1. General process of SER. they struggled with generalization across languages due to significant variability in linguistic and acoustic features. For instance, Zehra et al. [29] trained their model using an Urdu dataset [30] and tested it on datasets in other languages, including English, German, and Italian, achieving accuracies ranging from 50% ∼ 60%. With advances in deep learning, Braunschweiler et al. [31] applie… view at source ↗
Figure 2
Figure 2. Architecture of Cross-Attention Transformer. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Structure of proposed HuMP-CAT. 2) RAVDESS: The RAVDESS [49] contains 1440 utterance. The database includes 24 professional actors (12 female, 12 male), who voice two lexically matched statements in North American accent. Speech includes calm, happy, sad, angry, fearful, surprised, and disgusted expressions. 3) TESS: Toronto emotional speech set (TESS) [50] is an English speech database consisting of 2800 samples fr… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Confusion matrix of HuMP-CAT on IEMOCAP corpus. approach using adaptive pseudo-label task pretraining (P￾TAPT), which achieved a significant improvement of more than 5% in unweighted accuracy (UA) over state-of-the-art performance on IEMOCAP. HuMP-CAT achieved the accu…
Figure 5
Figure 5. Figure 5: Comparison of three methods on the same target dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 53 canonical work pages

  1. [1]

    Survey on speech emotion recognition: Features, classification schemes, and databases,

    M. El Ayadi, M. S. Kamel, and F. Karray, “Survey on speech emotion recognition: Features, classification schemes, and databases,” Pattern recognition, vol. 44, no. 3, pp. 572–587, 2011

    2011

  2. [2]

    Automated screening for distress: A perspective for the future,

    R. Rana, S. Latif, R. Gururajan, A. Gray, G. Mackenzie, G. Humphris, and J. Dunn, “Automated screening for distress: A perspective for the future,” European journal of cancer care , vol. 28, no. 4, p. e13033, 2019

    work page 2019

  3. [3]

    A comprehensive review of speech emotion recognition systems,

    T. M. Wani, T. S. Gunawan, S. A. A. Qadri, M. Kartiwi, and E. Am- bikairajah, “A comprehensive review of speech emotion recognition systems,” IEEE access , vol. 9, pp. 47 795–47 814, 2021

    work page 2021

  4. [4]

    A systematic review on affective computing: Emotion models, databases, and recent advances,

    Y . Wang, W. Song, W. Tao, A. Liotta, D. Yang, X. Li, S. Gao, Y . Sun, W. Ge, and W. Zhang, “A systematic review on affective computing: Emotion models, databases, and recent advances,” Information Fusion , vol. 83, pp. 19–52, 2022. 9

    work page 2022

  5. [5]

    Speech emotion recognition based on hmm and svm,

    Y .-L. Lin and G. Wei, “Speech emotion recognition based on hmm and svm,” in Proc. IEEE International conference on machine learning and cybernetics, vol. 8, Aug. 2005, pp. 4898–4901

    work page 2005

  6. [6]

    Speech emotion recognition using fourier parameters,

    K. Wang, N. An, B. N. Li, Y . Zhang, and L. Li, “Speech emotion recognition using fourier parameters,” IEEE Transactions on affective computing, vol. 6, no. 1, pp. 69–75, 2015

    work page 2015

  7. [7]

    Implementation and comparison of speech emotion recognition system using gaussian mix- ture model (gmm) and k-nearest neighbor K-NN techniques,

    R. B. Lanjewar, S. Mathurkar, and N. Patel, “Implementation and comparison of speech emotion recognition system using gaussian mix- ture model (gmm) and k-nearest neighbor K-NN techniques,” Procedia computer science , vol. 49, pp. 50–57, 2015

    work page 2015

  8. [8]

    Speech emotion recognition using deep learning techniques: A review,

    R. A. Khalil, E. Jones, M. I. Babar, T. Jan, M. H. Zafar, and T. Alhussain, “Speech emotion recognition using deep learning techniques: A review,” IEEE access , vol. 7, pp. 117 327–117 345, 2019

    2019

Show all 68 references
  1. [9]

    Deep learning approaches for speech emotion recognition: State of the art and research challenges,

    R. Jahangir, Y . W. Teh, F. Hanif, and G. Mujtaba, “Deep learning approaches for speech emotion recognition: State of the art and research challenges,” Multimedia Tools and Applications , vol. 80, no. 16, pp. 23 745–23 812, 2021

    2021

  2. [10]

    Speech emotion recognition: A review,

    A. Thakur and S. Dhull, “Speech emotion recognition: A review,” in Proc. International Conference on Advanced Communication and Computational Technology, Dec. 2019, pp. 815–827

    2019

  3. [11]

    Self-supervised speech representation learning: A review,

    A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløe, T. N. Sainath, and W. Shinji, “Self-supervised speech representation learning: A review,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, ...

    2022

  4. [12]

    Cross-corpus speech emotion recognition using semi-supervised transfer non-negative matrix factorization with adapta- tion regularization

    H. Luo and J. Han, “Cross-corpus speech emotion recognition using semi-supervised transfer non-negative matrix factorization with adapta- tion regularization.” in Proc. INTERSPEECH, 2019, pp. 3247–3251

    2019

  5. [13]

    Multisource i-vectors domain adaptation using maximum mean discrepancy based autoencoders,

    W.-w. Lin, M.-W. Mak, and J.-T. Chien, “Multisource i-vectors domain adaptation using maximum mean discrepancy based autoencoders,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 26, no. 12, pp. 2412–2422, 2018

    2018

  6. [14]

    Self-supervised learning for multimedia recommendation,

    Z. Tao, X. Liu, Y . Xia, X. Wang, L. Yang, X. Huang, and T.-S. Chua, “Self-supervised learning for multimedia recommendation,” IEEE Transactions on Multimedia , vol. 25, pp. 5107–5116, 2022

    2022

  7. [15]

    V oicepm: A robust privacy measurement on voice anonymity,

    S. Zhang, Z. Li, and A. Das, “V oicepm: A robust privacy measurement on voice anonymity,” in Proc. 16th ACM Conference on Security and Privacy in Wireless and Mobile Networks , 2023, pp. 215–226

    2023

  8. [16]

    Emobox: Multilingual multi-corpus speech emotion recog- nition toolkit and benchmark,

    Z. Ma, M. Chen, H. Zhang, Z. Zheng, W. Chen, X. Li, J. Ye, X. Chen, and T. Hain, “Emobox: Multilingual multi-corpus speech emotion recog- nition toolkit and benchmark,” arXiv preprint arXiv:2406.07162 , 2024

    2024 arXiv

  9. [17]

    Distilhubert: Speech rep- resentation learning by layer-wise distillation of hidden-unit bert,

    H.-J. Chang, S.-w. Yang, and H.-y. Lee, “Distilhubert: Speech rep- resentation learning by layer-wise distillation of hidden-unit bert,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, May. 2022, pp. 7087–7091

    2022

  10. [19]

    Cross- corpus speech emotion recognition with hubert self-supervised represen- tation,

    M. A. Pastor, D. Ribas, A. Ortega, A. Miguel, and E. Lleida, “Cross- corpus speech emotion recognition with hubert self-supervised represen- tation,” in Proc. ISCA Conference IberSPEECH , Nov. 2022, pp. 76–80

    2022

  11. [21]

    Representation learning through cross-modal conditional teacher-student training for speech emotion recognition,

    S. Srinivasan, Z. Huang, and K. Kirchhoff, “Representation learning through cross-modal conditional teacher-student training for speech emotion recognition,” in Proc. IEEE International Conference on Acous- tics, Speech and Signal Processing , May. 2022, pp. 6442–6446

    2022

  12. [22]

    Multi-lingual multi-task speech emotion recognition using wav2vec 2.0,

    M. Sharma, “Multi-lingual multi-task speech emotion recognition using wav2vec 2.0,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , May. 2022, pp. 6907–6911

    2022

  13. [23]

    A systematic literature review of speech emotion recognition approaches,

    Y . B. Singh and S. Goel, “A systematic literature review of speech emotion recognition approaches,” Neurocomputing, vol. 492, pp. 245– 263, 2022

    2022

  14. [24]

    Speech emotion recognition using sequential capsule net- works,

    X. Wu, Y . Cao, H. Lu, S. Liu, D. Wang, Z. Wu, X. Liu, and H. Meng, “Speech emotion recognition using sequential capsule net- works,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3280–3291, 2021

    2021

  15. [25]

    Transformer based unsupervised pre-training for acoustic representation learning,

    R. Zhang, H. Wu, W. Li, D. Jiang, W. Zou, and X. Li, “Transformer based unsupervised pre-training for acoustic representation learning,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Jun. 2021, pp. 6933–6937

    2021

  16. [26]

    Contrastive unsupervised learning for speech emotion recognition,

    M. Li, B. Yang, J. Levy, A. Stolcke, V . Rozgic, S. Matsoukas, C. Pa- payiannis, D. Bone, and C. Wang, “Contrastive unsupervised learning for speech emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , Jun. 2021, pp. 6329–6333

    2021

  17. [27]

    Cross-corpus classification of realistic emotions–some pilot experiments,

    F. Eyben, A. Batliner, B. Schuller, D. Seppi, and S. Steidl, “Cross-corpus classification of realistic emotions–some pilot experiments,” in Proc. 7th international conference on language resources and evaluation , 2010

    2010

  18. [28]

    Using multiple databases for training in emotion recognition: To unite or to vote?

    B. Schuller, Z. Zhang, F. Weninger, and G. Rigoll, “Using multiple databases for training in emotion recognition: To unite or to vote?” in Proc. Twelfth Annual Conference of the International Speech Commu- nication Association , Aug. 2011, pp. 1553–1556

    2011

  19. [30]

    Cross lingual speech emotion recognition: Urdu vs. western languages,

    S. Latif, A. Qayyum, M. Usman, and J. Qadir, “Cross lingual speech emotion recognition: Urdu vs. western languages,” in Proc. International conference on frontiers of information technology , Dec. 2018, pp. 88–93

    2018

  20. [31]

    A study on cross-corpus speech emotion recognition and data augmenta- tion,

    N. Braunschweiler, R. Doddipatla, S. Keizer, and S. Stoyanchev, “A study on cross-corpus speech emotion recognition and data augmenta- tion,” in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 24–30

    2021

  21. [32]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  22. [33]

    Emotion recognition from speech using wav2vec 2.0 embeddings,

    L. Pepino, P. Riera, and L. Ferrer, “Emotion recognition from speech using wav2vec 2.0 embeddings,” arXiv preprint arXiv:2104.03502 , 2021

    2021 arXiv

  23. [34]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  24. [35]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    2018 arXiv

  25. [36]

    Unveiling em- bedded features in wav2vec2 and hubert msodels for speech emotion recognition,

    A. CHAKHTOUNA, S. SEKKATE, and A. Abdellah, “Unveiling em- bedded features in wav2vec2 and hubert msodels for speech emotion recognition,” Procedia Computer Science , vol. 232, pp. 2560–2569, 2024

    2024

  26. [37]

    Layer-wise analysis of a self- supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self- supervised speech representation model,” in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) , Dec. 2021, pp. 914–921

    2021

  27. [38]

    Multiple acoustic features speech emotion recognition using cross-attention transformer,

    Y . He, N. Minematsu, and D. Saito, “Multiple acoustic features speech emotion recognition using cross-attention transformer,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , 2023, pp. 1–5

    2023

  28. [39]

    Speech emotion recognition using local and global features,

    Y . Gao, B. Li, N. Wang, and T. Zhu, “Speech emotion recognition using local and global features,” in Proc. Brain Informatics: International Conference, Nov. 2017, pp. 3–13

    2017

  29. [40]

    Modeling prosodic features with joint factor analysis for speaker verification,

    N. Dehak, P. Dumouchel, and P. Kenny, “Modeling prosodic features with joint factor analysis for speaker verification,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 15, no. 7, pp. 2095– 2103, 2007

    2007

  30. [41]

    A novel feature selection method for speech emotion recognition,

    T. ¨Ozseven, “A novel feature selection method for speech emotion recognition,” Applied Acoustics , vol. 146, pp. 320–326, 2019

    2019

  31. [42]

    Analysis of linguistic and prosodic features of bilingual arabic–english speakers for speech emotion recognition,

    L. Abdel-Hamid, N. H. Shaker, and I. Emara, “Analysis of linguistic and prosodic features of bilingual arabic–english speakers for speech emotion recognition,” IEEE Access , vol. 8, pp. 72 957–72 970, 2020

    2020

  32. [43]

    Towards an automatic evaluation of the dysarthria level of patients with parkinson’s disease,

    J. C. V ´asquez-Correa, J. Orozco-Arroyave, T. Bocklet, and E. N ¨oth, “Towards an automatic evaluation of the dysarthria level of patients with parkinson’s disease,” Journal of communication disorders , vol. 76, pp. 21–36, 2018

    2018

  33. [44]

    Speech emotion recognition based on multiple acoustic features and deep convolutional neural network,

    K. Bhangale and M. Kothandaraman, “Speech emotion recognition based on multiple acoustic features and deep convolutional neural network,” Electronics, vol. 12, no. 4, p. 839, 2023

    2023

  34. [45]

    Speech emotion recognition using mel frequency log spectrogram and deep convolutional neural network,

    K. Bhangale and K. Mohanaprasad, “Speech emotion recognition using mel frequency log spectrogram and deep convolutional neural network,” in Proc. International Conference on Futuristic Communication and Network Technologies, Nov. 2020, pp. 241–250

    2020

  35. [46]

    Learning deep features to recognise speech emotion using merged deep CNN,

    J. Zhao, X. Mao, and L. Chen, “Learning deep features to recognise speech emotion using merged deep CNN,” IET Signal Processing , vol. 12, no. 6, pp. 713–721, 2018

    2018

  36. [47]

    The speakin speaker verification system for far-field speaker verification challenge 2022,

    Y . Zheng, J. Peng, Y . Chen, Y . Zhang, J. Wang, M. Liu, and M. Xu, “The speakin speaker verification system for far-field speaker verification challenge 2022,” arXiv preprint arXiv:2209.11625 , 2022

    2022 arXiv

  37. [48]

    IEMOCAP: Interactive emotional dyadic motion capture database,

    C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “IEMOCAP: Interactive emotional dyadic motion capture database,” Language resources and evaluation , vol. 42, pp. 335–359, 2008. 10

    2008

  38. [49]

    The ryerson audio-visual database of emotional speech and song RA VDESS: A dynamic, multimodal set of facial and vocal expressions in north american english,

    S. R. Livingstone and F. A. Russo, “The ryerson audio-visual database of emotional speech and song RA VDESS: A dynamic, multimodal set of facial and vocal expressions in north american english,” PloS one , vol. 13, no. 5, p. e0196391, 2018

    2018

  39. [50]

    Real- time end-to-end speech emotion recognition with cross-domain adapta- tion,

    K. Wongpatikaseree, S. Singkul, N. Hnoohom, and S. Yuenyong, “Real- time end-to-end speech emotion recognition with cross-domain adapta- tion,” Big Data and Cognitive Computing , vol. 6, no. 3, p. 79, 2022

    2022

  40. [51]

    A database of german emotional speech

    F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeier, and B. Weiss, “A database of german emotional speech.” in Proc. Interspeech, vol. 5, Sep. 2005, pp. 1517–1520

    2005

  41. [52]

    Emovo corpus: an italian emotional speech database,

    G. Costantini, I. Iaderola, A. Paoloni, and M. Todisco, “Emovo corpus: an italian emotional speech database,” in Proc. the ninth international conference on language resources and evaluation , 2014, pp. 3501–3504

    2014

  42. [53]

    The mexican emotional speech database (mesd): elaboration and assessment based on machine learning,

    M. M. Duville, L. M. Alonso-Valerdi, and D. I. Ibarra-Zarate, “The mexican emotional speech database (mesd): elaboration and assessment based on machine learning,” in 2021 43rd Annual International Confer- ence of the IEEE Engineering in Medicine & Biology Society (EMBC) . IEE...

    2021

  43. [54]

    Emotional voice conversion: Theory, databases and esd,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Emotional voice conversion: Theory, databases and esd,” Speech Communication, vol. 137, pp. 1–18, 2022

    2022

  44. [55]

    Towards discriminative representations and unbiased predictions: Class-specific angular softmax for speech emotion recognition

    Z. Li, L. He, J. Li, L. Wang, and W.-Q. Zhang, “Towards discriminative representations and unbiased predictions: Class-specific angular softmax for speech emotion recognition.” in Proc. INTERSPEECH , 2019, pp. 1696–1700

    2019

  45. [56]

    Improving speech emotion recognition using graph attentive bi-directional gated recurrent unit network,

    B.-H. Su, C.-M. Chang, Y .-S. Lin, and C.-C. Lee, “Improving speech emotion recognition using graph attentive bi-directional gated recurrent unit network,” in Proc. INTERSPEECH, 2020, pp. 506–510

    2020

  46. [57]

    Hgfm: A hierarchical grained and feature model for acoustic emotion recognition,

    Y . Xu, H. Xu, and J. Zou, “Hgfm: A hierarchical grained and feature model for acoustic emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , May. 2020, pp. 6499–6503

    2020

  47. [58]

    Cross- corpus speech emotion recognition with hubert self-supervised represen- tation,

    M. A. Pastor, D. Ribas, A. Ortega, A. Miguel, and E. Lleida, “Cross- corpus speech emotion recognition with hubert self-supervised represen- tation,” in Proc. IberSPEECH. ISCA, 2022, pp. 76–80

    2022

  48. [59]

    Temporal modeling matters: A novel temporal emotional modeling approach for speech emotion recognition,

    J. Ye, X.-C. Wen, Y . Wei, Y . Xu, K. Liu, and H. Shan, “Temporal modeling matters: A novel temporal emotional modeling approach for speech emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , Jun. 2023, pp. 1–5

    2023

  49. [60]

    Learning multi-scale features for speech emotion recognition with connection attention mechanism,

    Z. Chen, J. Li, H. Liu, X. Wang, H. Wang, and Q. Zheng, “Learning multi-scale features for speech emotion recognition with connection attention mechanism,” Expert Systems with Applications , vol. 214, p. 118943, 2023

    2023

  50. [61]

    Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,

    L.-W. Chen and A. Rudnicky, “Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing , Jun. 2023, pp. 1–5

    2023

  51. [62]

    Unsupervised adversarial domain adaptation for cross-lingual speech emotion recognition,

    S. Latif, J. Qadir, and M. Bilal, “Unsupervised adversarial domain adaptation for cross-lingual speech emotion recognition,” in Proc. IEEE International Conference on Affective Computing and Intelligent Inter- action, Sep. 2019, pp. 732–737

    2019

  52. [63]

    Speech emotion recognition from 3D log-mel spectrograms with deep learning network,

    H. Meng, T. Yan, F. Yuan, and H. Wei, “Speech emotion recognition from 3D log-mel spectrograms with deep learning network,” IEEE access, vol. 7, pp. 125 868–125 881, 2019

    2019

  53. [64]

    Fusing visual attention CNN and bag of visual words for cross-corpus speech emotion recognition,

    M. Seo and M. Kim, “Fusing visual attention CNN and bag of visual words for cross-corpus speech emotion recognition,” Sensors, vol. 20, no. 19, p. 5559, 2020

    2020

  54. [65]

    Cross corpus multi-lingual speech emotion recognition using ensemble learning,

    W. Zehra, A. R. Javed, Z. Jalil, H. U. Khan, and T. R. Gadekallu, “Cross corpus multi-lingual speech emotion recognition using ensemble learning,” Complex & Intelligent Systems , vol. 7, no. 4, pp. 1845–1854, 2021

    2021

  55. [66]

    Cross-corpus speech emotion recognition based on few-shot learning and domain adaptation,

    Y . Ahn, S. J. Lee, and J. W. Shin, “Cross-corpus speech emotion recognition based on few-shot learning and domain adaptation,” IEEE Signal Processing Letters , vol. 28, pp. 1190–1194, 2021

    2021

  56. [67]

    Crema-d: Crowd-sourced emotional multimodal actors dataset,

    H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, “Crema-d: Crowd-sourced emotional multimodal actors dataset,” IEEE transactions on affective computing , vol. 5, no. 4, pp. 377–390, 2014

    2014

  57. [68]

    Enhancing cross-language multimodal emotion recognition with dual attention transformers,

    S. A. M. Zaidi, S. Latif, and J. Qadir, “Enhancing cross-language multimodal emotion recognition with dual attention transformers,” IEEE Open Journal of the Computer Society , 2024

    2024

  58. [69]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” 2019. [Online]. Available: https://arxiv.org/abs/1907.11692

    2019 arXiv

  59. [70]

    Xls-r: Self-supervised cross-lingual speech representation learning at scale,

    A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. V on Platen, Y . Saraf, J. Pinoet al. , “Xls-r: Self-supervised cross-lingual speech representation learning at scale,” arXiv preprint arXiv:2111.09296, 2021

    2021 arXiv

  60. [71]

    Robust speech recognition via large-scale weak supervi- sion,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervi- sion,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.