Pith. sign in

REVIEW 2 major objections 6 minor 34 references

Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 12-model cross-modal ensemble with balanced stacking achieves 0.4094 MacroF1 on 8-class speech emotion recognition, beating the challenge baseline of 0.329.

desk verdict Solid challenge system description with a plausible ensemble result and a test score that survives scrutiny, though it's a new recipe rather than a new idea. read the letter →

arxiv 2505.20007 v2 pith:7LDQTKHA submitted 2025-05-26 eess.AS cs.SD

Lucas Ueda , João Lima , Leonardo Marques , Paula Costa This is my paper · ORCID
classification eess.AScs.SD
keywords speechemotionrecognitioncross-modalattentionensemblestackingclassimbalanceself-supervisedlearningMSP-PodcastmacroF1multimodalfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper enters a challenge on speech emotion recognition in naturalistic conditions and claims that a system built from 12 pretrained cross-modal models, each fusing speech and text (and sometimes prosody) representations through cross-modal attention, can outperform the official baseline by a wide margin. On the 8-class test set it reports MacroF1 0.4094 and accuracy 0.4128, compared to 0.329 for the official baseline. The authors attribute the gain to two design choices: training with weighted cross-entropy plus a neutral-expressive soft margin loss and balanced batches to fight class imbalance, and ensembling the models with a Random Forest meta-learner trained on a balanced subset. The result matters for practical human-computer interaction, where emotions such as fear appear in only about 1.5 percent of natural speech data.

What carries the argument

The central object is the cross-modal attention alignment unit inside each member model. Speech and text frame/token features are projected to a common 512-dimension space, normalized, passed through bidirectional GRUs, and then one modality serves as query against another as key/value; the attention output is added back to the GRU output, pooled by attention weighting, concatenated, and classified. Around this unit, the paper builds 12 pretrained-feature combinations (Whisper, Hubert, and WavLM for speech; Roberta and Deberta for text; FACodec prosody/speaker for paralinguistics) and two loss designs, then a balanced 5-fold Random Forest stack that learns to combine the logits. The machinery's purpose is to align modalities temporally while preserving each modality's own information, and to let the meta-model reweight models on a class-balanced training partition.

What would settle it

Run the same 12-model stack with human-transcribed test audio or with transcripts corrupted by a known error rate; if MacroF1 drops substantially or the cross-modal alignment no longer helps, the reported 0.4094 is partly an artifact of transcript quality rather than a property of the architecture. A direct ablation would compare the test-set transcript source by using oracle transcripts instead of Whisper-generated ones.

Watch

Extended reading notes

Core claim

The central claim is that cross-modal fusion of frozen self-supervised speech and text representations, aligned by a cross-modal attention layer and combined through balanced stacking, is a strong recipe for categorical emotion recognition in spontaneous speech. The paper shows a steady progression: individual bimodal models reach dev bootstrap F1 values around 0.42 at best, and the 12-model balanced Random Forest stack reaches 0.430 on the dev bootstrap and 0.4094 MacroF1 on the test set. Adding prosody/speaker features from a factorized codec gives the best accuracy for individual trimodal models but does not beat the best bimodal pairing in F1; still, the meta-model extracts marginal gains from including them. The authors interpret the weakest single-modality-pair results (two speech SSL models without text) as evidence that modality diversity, not just model count, drives the ensemble's success.

Load-bearing premise

The results assume that Whisper Large V3's automatic transcripts of the test set are accurate enough to serve as the text modality in the same way the provided transcriptions served during development, so systematic transcription errors on emotional or noisy speech would degrade every member of the ensemble.

Editorial extensions

If this is right

  • The balanced stack's test MacroF1 of 0.4094 beats both the official baseline (0.329) and the reproduced baseline (0.284), suggesting that balanced stacking of diverse cross-modal models is a practical way to push speech emotion recognition toward usable performance on natural, imbalanced speech.
  • Weighted cross-entropy plus neutral-expressive soft margin loss and balanced batches gives consistent gains over weighted cross-entropy alone for most model pairs, reinforcing that class-imbalance handling is a key lever in naturalistic speech emotion recognition.
  • Combining audio and text is more valuable than combining two different audio self-supervised models, indicating that modality diversity, not just model count, drives the ensemble's success.
  • Trimodal inclusion of prosody/speaker features yields the best accuracy among individual models, suggesting that paralinguistic cues carry complementary signal that helps when accuracy rather than balanced F1 is the target.
  • The gap between the dev bootstrap score (0.430) and the test score (0.409) indicates some overfitting, but the test score still exceeds the baselines, so robust evaluation on balanced subsets remains important for comparing systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported test numbers depend on Whisper-generated transcripts for the test set; if future organizers supply human transcripts, the audio-text alignment could shift, so a clean test of the method would compare against human transcriptions.
  • Because the meta-model reweights on a balanced training partition, the ensemble may act as an implicit emotion-calibration layer; one could test whether stacking with calibrated probabilities improves calibration further.
  • The architecture's per-modality frozen representations mean compute cost is mostly in training heads and the meta-model, so scaling member diversity (more SSL models or more paralinguistic factors beyond prosody/speaker) may be a cheaper path to further gains than fine-tuning each member.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents the authors' entry to the Interspeech 2025 Speech Emotion Recognition in Naturalistic Conditions Challenge. The system encodes speech, text, and prosody with frozen pretrained models (Whisper, HuBERT, WavLM, RoBERTa, DeBERTa, FACodec), projects the representations to a common space, applies bidirectional GRUs and cross-modal attention with attention pooling, and trains a classifier with either weighted cross-entropy or weighted cross-entropy plus a soft-margin loss and balanced batching. Twelve such models are combined by a balanced Random-Forest stacking meta-model. On the challenge test set, the system reports a MacroF1 of 0.4094 and accuracy of 0.4128, compared with the official baseline MacroF1 of 0.329 and a reproduced baseline of 0.284.

Significance. The central contribution is empirical: a concrete, open-sourced system with a test-set score that exceeds the official baseline by 0.080 in MacroF1 on a balanced 3200-sample test set. The paper has real strengths: it provides a clean ablation of 12 model configurations, uses a bootstrap-balanced dev metric (BS-F1) that is consistent with the test result (0.430 dev vs 0.409 test), and makes code available. The use of Whisper-generated transcripts for both training and testing, together with speech-only and prosodic branches in the ensemble, makes the reported score unlikely to be an artifact of transcript noise. The main weaknesses are the underspecified stacking protocol and the lack of uncertainty quantification on the test result; these are fixable presentation and reproducibility issues rather than errors in the reported numbers.

major comments (2)
  1. [Section 3.2 and Section 4.2 (Stacking)] The description of the stacking procedure is too imprecise to reproduce. The paper says the meta-model is a '5-fold Random Forest' trained on a 'balanced training partition' and gives its hyperparameters (200 estimators, max depth 8, gini, min samples leaf/split 10), but it never states how the meta-features (the logits of the 12 base models) are generated for the stacking training set. In particular, it is unclear whether the RF is trained on out-of-fold predictions (the standard stacking protocol) or on the base models' training-set logits, and whether the balanced subset is drawn from the challenge training set, the dev set, or both. Since the stacking contribution is central to the reported test score, please specify the exact data flow and, if in-sample logits were used, justify why this does not overfit or evaluate the stack with out-of-fold meta-features.
  2. [Section 5 and Table 2] The paper compares the proposed system with both the official baseline (MacroF1 0.329) and a 'Reproduced Baseline' (MacroF1 0.284) but does not explain why the reproduced baseline is so much worse than the official one. If the reproduction used different preprocessing, a different checkpoint, or different evaluation settings, that row cannot be interpreted as a faithful baseline. Please state the exact reproduction protocol or remove the reproduced-baseline comparison from the headline claims.
minor comments (6)
  1. [Equation (1)] Equation (1) is malformed as printed: the attention weights appear to be a softmax over r_i^T m / sqrt(D), but the subscripts and superscripts are transposed and the denominator uses L both as an index and as the sequence length; please rewrite it cleanly.
  2. [Section 3.1] The dimension notation (e.g., 'T T ×Dt − → T T ×D2h') is hard to read; consider writing the dimensions in a table or using a consistent arrow notation.
  3. [Abstract and Table 2] The abstract reports test accuracy 0.4128 while Table 2 reports 0.413; unify the rounding.
  4. [Section 4.3 and Table 2] Table 2 reports only the min-max range of the dev bootstrap F1; a 95% confidence interval or standard deviation would be more informative for assessing the stability of the BS-F1 values.
  5. [Section 6 (Conclusion)] The conclusion states that the system 'remained among the top performers on the challenge leaderboard,' but no leaderboard position or external evidence is provided; either cite the leaderboard or soften the claim.
  6. [References] References [15] and [24] are duplicate entries for NaturalSpeech 3; merge them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported test result is an externally evaluated benchmark, independent of any fitted parameter.

full rationale

This paper reports an empirical system rather than a derivation, so the circularity checks reduce to whether any reported quantity is forced by construction. The central claim is the MacroF1 of 0.4094 on the official challenge test set versus the reproduced and official baselines; that score comes from an externally held-out, balanced 3200-sample test set. The weighted cross-entropy class weights (Equation 3) are ordinary inverse-frequency weights computed from training labels and used inside a standard loss function, not a fitted quantity later renamed as a prediction. The balanced Random Forest stacking meta-model is trained on the training portion and evaluated on the held-out test set, which is standard stacking practice even if the in-sample meta-features are non-standard. The Whisper-generated test transcripts are a feature-extraction consistency choice, not a fitted parameter, and the audio-only speech encoders in the ensemble mean the text channel is not the sole carrier of the result. The BS-F1 bootstrap statistic is a dev-set model-selection aid; any selection-on-dev concern is a correctness or overfitting risk, not a circular reduction. There is no load-bearing self-citation or imported uniqueness theorem. Because no step in the paper reduces by its own equations or by author citation to its inputs, no circular step can be exhibited and the score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The reported score rests on the reliability of the challenge's categorical labels, the accuracy of Whisper-generated transcripts, and the transferability of frozen SSL features. The bootstrap-balanced dev F1 is an ad hoc proxy for the balanced test set. No new physical or theoretical entities are introduced.

free parameters (5)
  • Learning rate = 1e-4 or 1e-5
    Per-model choice described as 'chosen according to observed effects' in Section 4.2; different values across the 12 models affect final ensemble performance.
  • Hidden size h = 512
    Fixed projection dimension for all modalities in Section 3.1; set by hand.
  • Number of attention heads = 1
    Set to 1 to avoid overfitting in Section 3.1.
  • Random Forest hyperparameters = 200 estimators, max depth 8, min samples leaf/split 10
    Meta-model settings in Section 4.2, no tuning procedure reported.
  • Ensemble composition = 12 models (10 bimodal, 2 trimodal)
    Feature combinations were selected based on dev results and 'internal experiments' in Section 4.2.
assumptions (4)
  • domain assumption MSP-Podcast categorical emotion labels are reliable and the 8 classes are exclusive enough for supervised training.
    The whole training and evaluation depends on these labels; the paper reports no inter-annotator agreement analysis (Section 4.1).
  • domain assumption Whisper Large V3 transcripts for train and test are accurate enough that automatic transcription errors do not materially change the text modality.
    Used to create consistent transcripts because test transcripts are unavailable (Section 4.2).
  • domain assumption Frozen pretrained SSL features (Whisper, Hubert, WavLM, Roberta, Deberta, FACodec) transfer to the SER task without fine-tuning.
    The paper freezes all modality encoders (Section 3.1, Figure 1), so the learned emotion signal is limited to the fusion and classifier head.
  • ad hoc to paper The bootstrap balanced subsets of the dev set provide a reliable proxy for the balanced test set.
    Used to report BS-F1 and select models; dev is imbalanced while test is balanced (Sections 4.2, 4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model." pith.science (2026). https://pith.science/paper/7LDQTKHA

@misc{pith2026250520007,
  author       = {Pith},
  title        = {Pith review of: Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7LDQTKHA}},
  note         = {Machine review of arXiv:2505.20007}
}
read the original abstract

Emotion plays a fundamental role in human interaction, and therefore systems capable of identifying emotions in speech are crucial in the context of human-computer interaction. Speech emotion recognition (SER) is a challenging problem, particularly in natural speech and when the available data is imbalanced across emotions. This paper presents our proposed system in the context of the 2025 Speech Emotion Recognition in Naturalistic Conditions Challenge. Our proposed architecture leverages cross-modality, utilizing cross-modal attention to fuse representations from different modalities. To address class imbalance, we employed two training designs: (i) weighted crossentropy loss (WCE); and (ii) WCE with an additional neutralexpressive soft margin loss and balancing. We trained a total of 12 multimodal models, which were ensembled using a balanced stacking model. Our proposed system achieves a MacroF1 score of 0.4094 and an accuracy of 0.4128 on 8-class speech emotion recognition.

Figures

Figures reproduced from arXiv: 2505.20007 by the authors.

Figure 1
Figure 1. Proposed cross-modal architecture. Dashed lines in￾dicates the tri-modal branch approach when applicable. Each text, speech and paralinguistic inputs are frozen during pro￾posed model training. Pooled representation r : r = XL i=1 wiri (2) Where w is the learned attention weights, ri ∈ R D is the modality feature frame/token i with dimension D and m ∈ R D is a trainable parameter. The pooled representation r is the … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages

  1. [1]

    Introduction Conveying emotional content plays a pivotal role in human spoken communication, which makes it essential to consider emotional aspects when developing speech-understanding sys- tems for specific applications. In this scenario, speech emotion recognition (SER) is characterized as the task of automatically identifying a person’s emotional state...

    work page 2025

  2. [2]

    Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model

    Related Works Several works have verified that the combination of multiple modalities led to an overall improvement in the general emo- tion recognition task [16]. This is due to the fact that introduc- ing more information helps the model to get a fuller grasp of one’s emotional state [17]. Some examples of modalities used to detect emotions in literatur...

    work page Pith review arXiv 2025

  3. [3]

    System Description Based on the idea that scaling the number of cross-modal fea- tures can improve the emotion recognition task, our system con- sists on a meta-model that is trained on the representations pro- duced by several cross-modal models. The lasts are obtained us- ing a number of combinations of pre-trained features from dif- ferent modalities a...

    work page

  4. [4]

    Dataset The provided challenge data consist of recordings from the MSP-Podcast dataset [23]

    Experimental setup 4.1. Dataset The provided challenge data consist of recordings from the MSP-Podcast dataset [23]. The speaking turns have been per- ceptually annotated by at least five raters with categorical and attribute-based emotional labels. For categorical classifica- tion there are 8 emotional labels: Anger, contempt, Disgust, Fear, Happiness, N...

    work page

  5. [5]

    Half of these models were trained us- ing only WCE loss, while the remaining half were trained with the additional batch balancing and SML loss

    Results A total of 12 models were trained, comprising 10 bimodal mod- els and 2 trimodal models. Half of these models were trained us- ing only WCE loss, while the remaining half were trained with the additional batch balancing and SML loss. Results are com- pared with a provided WavLM-based baseline [26]. The perfor- mance metrics, including accuracy, F ...

    work page

  6. [6]

    Notably, the pro- posed architecture benefits from the combination of multiple modalities, with its worst performance occurring when only a single modality is used

    Conclusion Our experiments demonstrate that utilizing SSL representations from different modalities is an effective approach for speech emotion recognition in naturalistic conditions. Notably, the pro- posed architecture benefits from the combination of multiple modalities, with its worst performance occurring when only a single modality is used. By mixin...

    work page

  7. [7]

    It is also supported by FAPESP (BI0S #2020/09838-0 and Ho- rus #2023/12865-8)

    Acknowledgements This study is partially funded by CAPES – Finance Code 001. It is also supported by FAPESP (BI0S #2020/09838-0 and Ho- rus #2023/12865-8). Paula Costa, Lucas Ueda, and Jo ˜ao Lima are affiliated with the Dept. of Computer Engineering and Automation (DCA), Faculdade de Engenharia El ´etrica e de Computac ¸˜ao, and are part of the AI Lab., ...

    work page arXiv 2020

  8. [8]

    Speech emotion recognition from voice messages recorded in the wild,

    L. G ´omez-Zaragoz´a, ´O. Valls, R. del Amor, M. J. Castro-Bleda, V . Naranjo, M. A. Raya, and J. Mar´ın-Morales, “Speech emotion recognition from voice messages recorded in the wild,” Mar

    work page

Show all 34 references
  1. [9]

    Acoustic Emotion Recognition for Affective Computer Gaming,

    C. Jones and J. Sutherland, “Acoustic Emotion Recognition for Affective Computer Gaming,” in Affect and Emotion in Human- Computer Interaction, C. Peter and R. Beale, Eds. Berlin, Hei- delberg: Springer Berlin Heidelberg, 2008, vol. 4868, pp. 209– 219

    2008

  2. [10]

    Speech emotion recognition using machine learning — A systematic review,

    S. Madanian, T. Chen, O. Adeleye, J. M. Templeton, C. Poellabauer, D. Parry, and S. L. Schneider, “Speech emotion recognition using machine learning — A systematic review,” In- telligent Systems with Applications, vol. 20, p. 200266, Nov. 2023

    2023

  3. [11]

    Speech Emotion Recognition in Neurological Disorders Using Convolutional Neu- ral Network,

    S. N. Zisad, M. S. Hossain, and K. Andersson, “Speech Emotion Recognition in Neurological Disorders Using Convolutional Neu- ral Network,” in Brain Informatics, M. Mahmud, S. Vassanelli, M. S. Kaiser, and N. Zhong, Eds. Cham: Springer International Publishing, 2020, vol. 12241...

    2020

  4. [12]

    Automatic Assessment of Depression From Speech via a Hierarchical Attention Transfer Network and Attention Autoencoders,

    Z. Zhao, Z. Bao, Z. Zhang, J. Deng, N. Cummins, H. Wang, J. Tao, and B. Schuller, “Automatic Assessment of Depression From Speech via a Hierarchical Attention Transfer Network and Attention Autoencoders,” IEEE Journal of Selected Topics in Sig- nal Processing, vol. 14, no. 2, ...

    2020

  5. [13]

    Negative Emotion Recognition using Deep Learning for Thai Language,

    S. Mekruksavanich, A. Jitpattanakul, and N. Hnoohom, “Negative Emotion Recognition using Deep Learning for Thai Language,” in 2020 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electri- cal, Electronics, Computer ...

    2020

  6. [14]

    Negative emotions detection as an indicator of dialogs quality in call centers,

    C. Vaudable and L. Devillers, “Negative emotions detection as an indicator of dialogs quality in call centers,” in 2012 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar. 2012, pp. 5109–5112

    2012

  7. [15]

    Using Paralinguistic Cues in Speech to Recognise Emotions in Older Car Drivers,

    C. Jones and I.-M. Jonsson, “Using Paralinguistic Cues in Speech to Recognise Emotions in Older Car Drivers,” in Affect and Emo- tion in Human-Computer Interaction, C. Peter and R. Beale, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, vol. 4868, pp. 229–240

    2008

  8. [16]

    Affective Human-Robotic Interac- tion,

    C. Jones and A. Deeming, “Affective Human-Robotic Interac- tion,” in Affect and Emotion in Human-Computer Interaction , C. Peter and R. Beale, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, vol. 4868, pp. 175–185

    2008

  9. [17]

    Multimodal emotion recognition using cross-modal attention and 1d convolutional neural networks

    D. Krishna and A. Patil, “Multimodal emotion recognition using cross-modal attention and 1d convolutional neural networks.” in Interspeech, 2020, pp. 4243–4247

    2020

  10. [18]

    Speech emotion recognition combining acoustic features and linguistic information in a hy- brid support vector machine-belief network architecture,

    B. Schuller, G. Rigoll, and M. Lang, “Speech emotion recognition combining acoustic features and linguistic information in a hy- brid support vector machine-belief network architecture,” in2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, ...

    2004

  11. [19]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selecte...

    2022

  12. [20]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Trans. Audio, Speech and Lang. Proc. , vol. 29, p. 3451–3460, Oct. 2021. [Online]. Avai...

    2021

  13. [21]

    Multimodal Emotion Recognition,

    S. Haq and P. Jackson, “Multimodal Emotion Recognition,” in Machine Audition: Principles, Algorithms and Systems, W. Wang, Ed. IGI Global, 2011, pp. 398–423

    2011

  14. [22]

    1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem,

    M. Chen, H. Zhang, Y . Li, J. Luo, W. Wu, Z. Ma, P. Bell, C. Lai, J. D. Reiss, L. Wang, P. C. Woodland, X. Chen, H. Phan, and T. Hain, “1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem,” in The Speaker and Language Recognition...

    2024

  15. [23]

    Naturalspeech 3: zero-shot speech synthesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tang, Z. Wu, T. Qin, X.-Y . Li, W. Ye, S. Zhang, J. Bian, L. He, J. Li, and S. Zhao, “Naturalspeech 3: zero-shot speech synthesis with factorized codec and diffusion models,” in Proceedings of th...

    2024

  16. [24]

    Emotion recogni- tion through multiple modalities: face, body gesture, speech,

    G. Castellano, L. Kessous, and G. Caridakis, “Emotion recogni- tion through multiple modalities: face, body gesture, speech,” Af- fect and Emotion in Human-Computer Interaction: From Theory to Applications, pp. 92–103, 2008

    2008

  17. [25]

    Amphion: An open-source audio, music and speech generation toolkit,

    X. Zhang, L. Xue, Y . Gu, Y . Wang, H. He, C. Wang, X. Chen, Z. Fang, H. Chen, J. Zhang, T. Y . Tang, L. Zou, M. Wang, J. Han, K. Chen, H. Li, and Z. Wu, “Amphion: An open-source audio, music and speech generation toolkit,”arXiv, vol. abs/2312.09911, 2024

    2024 arXiv

  18. [26]

    Using transformers for mul- timodal emotion recognition: Taxonomies and state of the art review,

    S. Hazmoune and F. Bougamouza, “Using transformers for mul- timodal emotion recognition: Taxonomies and state of the art review,” Engineering Applications of Artificial Intelligence , vol. 133, p. 108339, 2024

    2024

  19. [27]

    Deep neural networks for emotion recognition com- bining audio and transcripts,

    J. Cho, R. Pappagari, P. Kulkarni, J. Villalba, Y . Carmiel, and N. Dehak, “Deep neural networks for emotion recognition com- bining audio and transcripts,” inInterspeech 2018, 2018, pp. 247– 251

    2018

  20. [28]

    Multimodal emotion recognition with transformer-based self supervised feature fusion,

    S. Siriwardhana, T. Kaluarachchi, M. Billinghurst, and S. Nanayakkara, “Multimodal emotion recognition with transformer-based self supervised feature fusion,” Ieee Ac- cess, vol. 8, pp. 176 274–176 285, 2020

    2020

  21. [29]

    The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,

    F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. Andr ´e, C. Busso, L. Y . Devillers, J. Epps, P. Laukka, S. S. Narayanan, and K. P. Truong, “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,” IEEE Transactions on Affec...

    2016

  22. [30]

    Stacked generalization,

    D. H. Wolpert, “Stacked generalization,” Neural Networks, vol. 5, no. 2, pp. 241–259, 1992. [Online]. Available: https://www. sciencedirect.com/science/article/pii/S0893608005800231

    1992

  23. [31]

    The interspeech 2025 challenge on speech emotion recognition in naturalistic conditions,

    A. R. Naini, L. Goncalves, A. N. Salman, P. Mote, I. R. ¨Ulgen, T. Thebaud, L. Velazquez, L. P. Garcia, N. Dehak, B. Sisman, and C. Busso, “The interspeech 2025 challenge on speech emotion recognition in naturalistic conditions,” in Interspeech 2025, vol. To appear, Rotterdam,...

    2025

  24. [32]

    Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tang et al., “Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,”arXiv preprint arXiv:2403.03100, 2024

    2024 arXiv

  25. [34]

    Odyssey 2024 - speech emotion recognition challenge: Dataset, baseline framework, and results,

    L. Goncalves, A. N. Salman, A. R. Naini, L. Moro-Vel ´azquez, T. Thebaud, P. Garcia, N. Dehak, B. Sisman, and C. Busso, “Odyssey 2024 - speech emotion recognition challenge: Dataset, baseline framework, and results,” in The Speaker and Language Recognition Workshop (Odyssey 20...

    2024

  26. [2024]

    Available: http://arxiv.org/abs/2403.02167

    [Online]. Available: http://arxiv.org/abs/2403.02167

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.