Pith. sign in

REVIEW 4 major objections 4 minor 42 references

EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims explicit supervision of modality-level variance (via distributional cluster centers and 2-Wasserstein distance) makes multimodal emotion recognition in conversation more accurate than state-of-the-art methods.

desk verdict A plausible but under-validated uncertainty-supervision recipe for MERC; the headline SOTA claim needs a fair rerun before it holds. read the letter →

arxiv 2607.18336 v1 pith:PVVSONOC submitted 2026-07-19 cs.MM cs.CLcs.LG

Zilong Huang , Kong Aik Lee , Junjie Li , Zhe Li , Man-Wai Mak This is my paper · ORCID
classification cs.MMcs.CLcs.LG
keywords emotionrecognitioninconversationmultimodalfusionuncertaintysupervisiondistributionalrepresentation2-WassersteindistanceIEMOCAPMELDuncertainty-aware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EmoEUS tries to show that uncertainty estimates in multimodal emotion recognition should be trained with a direct target, not just learned indirectly from classification loss. It turns each utterance's text, audio, and visual features into Gaussian distributions, weights each modality during fusion by its predicted variance, and adds an explicitly supervised loss that aligns each variance with the actual spread of the utterance's emotion cluster. On IEMOCAP and MELD, this yields consistent gains over state-of-the-art, mainly by reducing confusion between similar emotions. The core claim is that giving the variance a concrete supervision signal produces better uncertainty estimates and more robust fusion.

What carries the argument

The key machinery is the explicitly supervised loss (ESL), which uses distributional cluster centers. For each emotion class and modality, a Gaussian center (mean and variance) is updated once per epoch from the model's own predictions and held fixed during backprop. The ESL loss for an utterance is the square of the difference between the square root of the predicted variance and the square root of the 2-Wasserstein distance between the utterance's distribution and its class center. This creates a self-referential target that pushes the predicted variance to match how far the utterance's mean and variance are from the aggregate cluster. The learned variance in turn drives UAMF, which comput

What would settle it

Compute per-utterance calibration on IEMOCAP/MELD: if the predicted variance is well-calibrated, it should predict the magnitude of the classifier's error; if the calibration is flat or the ESL can be shown to be exactly zero for a non-trivial number of utterances (e.g., any class with one sample), the claim that explicit supervision produces well-calibrated uncertainty is falsified.

Watch

Extended reading notes

Core claim

The central claim is that explicit, distribution-level supervision of uncertainty — through the ESL loss, which matches each modality's predicted variance to the 2-Wasserstein distance between the utterance's distribution and its emotion-specific cluster center — improves both the accuracy and the reliability of multimodal emotion recognition. The method reports on IEMOCAP an accuracy of 74.33 vs 73.53 for the best baseline FEMI, and on MELD 68.32 vs 67.85 for M2FNet/CFN-ESA. The ablation without ESL drops to 73.32/67.78, and without UAMF to 72.63/66.53, indicating that both explicit variance supervision and uncertainty-based fusion contribute. The paper also reports a large gain on the Happ

Load-bearing premise

The method's central premise is that the distance between an utterance's predicted distribution and the running mean/variance of its emotion class is a valid and non-degenerate training target for the utterance's predicted variance.

Editorial extensions

If this is right

  • If the ESL loss works as claimed, uncertainty estimates in MERC can be treated as first-class trainable targets rather than implicit byproducts of classification, which should transfer to other multimodal sequence-labeling tasks.
  • The large improvement on the Happy class and the reduction in confusion between Happy/Excited and Sad/Frustrated suggest that explicit variance supervision helps most for semantically adjacent emotions — the cases where modality cues are most likely to conflict.
  • The paper's ablations show that both the distributional representation (mean plus variance with 2-Wasserstein) and the residual connection in ContextDEM are needed for the best performance, meaning the gain is not purely from the extra network capacity.
  • Because the cluster centers are updated only once per epoch and detached from gradients, the ESL is stable enough to train alongside cross-entropy with a small weight (λ=2×10^-5) and a delayed start (epoch 10), which may guide how other tasks adopt such supervision.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ESL target is self-referential: cluster centers are running averages of the model's own means and variances, and each utterance's own variance contributes to its class center. For classes with a single training utterance, the 2-Wasserstein distance is exactly zero, so the loss would impose no constraint at all; the paper does not report per-class variance or calibration curves to rule this out
  • The reported gains over state of the art are small (roughly 0.5–1.0 points in accuracy and w-F1), yet the improvement on the Happy class is very large (+16.8 points relative to FEMI). The paper does not explain this discontinuity, so it is worth checking whether it is driven by one test session or by the uncertainty supervision doing something specific for that class.
  • A natural extension would be to use the predicted variance as a rejection or down-weighting signal at test time on out-of-distribution inputs, since the framework already quantifies per-modality reliability; the paper only uses variance for fusion during training, not for filtering at inference.
  • The features are frozen from RoBERTa, wav2vec2.0, and CLIP, so the uncertainty is about the representations, not the raw signals. An end-to-end variant might make the uncertainty estimates more grounded, but would confound the effect of ESL with the effect of fine-tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes EmoEUS, a framework for multimodal emotion recognition in conversation (MERC) that models each modality with a Gaussian distribution (ContextDEM), fuses modalities by uncertainty-based weighting (UAMF), and supervises the predicted variance with an explicitly supervised loss (ESL) that aligns it with a 2-Wasserstein distance to emotion-specific cluster centers. Experiments on IEMOCAP and MELD report consistent improvements over prior baselines, with the full model outperforming the strongest listed baselines (74.33 vs 73.53 on IEMOCAP; 68.32 vs 67.85 on MELD). The main novelty is the explicit supervision of uncertainty estimates through a distributional cluster-center loss.

Significance. If the claims hold, the paper contributes a concrete mechanism for uncertainty-aware fusion in MERC, a relatively underexplored direction compared with implicit uncertainty modeling. The ablation design is reasonable: comparing fusion variants and removing ESL/UAMF is a useful way to isolate the proposed components. The paper also reports per-class results and an ambiguous-pair analysis, which is informative. However, the central loss equation is formally problematic as written, the self-referential nature of the cluster centers could undermine the ESL target, and the SOTA comparison is not controlled. These issues are load-bearing for the main claim that explicit uncertainty supervision and uncertainty-weighted fusion produce the reported gains. The empirical gains are small, and no error bars or significance tests are reported, so the evidence is currently insufficient to establish the claimed advantage.

major comments (4)
  1. [§2.6, Eqs. (13)–(14)] The loss is not a 2-Wasserstein distance as written. For diagonal Gaussians with variance σ, W2^2 = ||μ_i−μ_y||^2 + ||√σ_i−√σ_y||^2, but Eq. (13) uses ||σ_i−σ_y||^2. Moreover, D2W(i,δ) in Eq. (13) is a scalar (sum of two squared L2 norms), while α√σ_i^δ in Eq. (14) is a D_g-dimensional vector; subtracting the scalar √D2W is not a valid vector operation unless broadcast, which would force every variance component toward the same scalar and is not the advertised per-dimension distributional alignment. Please correct the loss to a component-wise W2 form or clearly state the implemented operation and justify it.
  2. [§2.6, Eq. (12)] The cluster centers are epoch averages of the model's own predicted means and variances and include the current utterance's own prediction. For a singleton class, D2W(i,δ)=0 identically, so ESL pulls σ_i toward 0; for small classes the target is dominated by the utterance itself. This self-referential target may act as an uncalibrated regularizer rather than a stable supervision signal. The authors should exclude the utterance from its class center (or use an external/EMA target) and report calibration metrics (e.g., ECE) to support the claim that ESL produces well-calibrated uncertainty. Currently no calibration evaluation is provided.
  3. [§4.1, Table 1] The SOTA comparison is not controlled. Baseline numbers appear to be taken from prior papers without establishing that the same feature extractors (RoBERTa, wav2vec 2.0, CLIP), preprocessing, and evaluation protocol are used. Different feature backbones can easily account for several accuracy points, so the 'consistently outperforms' claim is under-supported. Please re-run baselines under the same pipeline or clearly state the source and configuration of each baseline, and provide multiple-seed results with standard deviations or significance tests.
  4. [§4.1, Table 2 and §4.2, Table 3] The ablation gains are small (IEMOCAP 74.33 vs 73.32 without ESL; MELD 68.32 vs 67.78 without ESL) and are reported without variance or statistical tests. Table 3 claims 'significant reductions' in misclassification rates without any significance testing. Given the size of the differences, the positive effect of ESL/UAMF may not be distinguishable from training noise. Please include multiple seeds and significance tests for the ablations and ambiguous-pair results.
minor comments (4)
  1. [§2.4, footnote 1] Using σ to denote variance (instead of standard deviation or σ²) is confusing, especially in Eq. (13) where ||σ_i−σ_y||² appears. Please adopt a clearer notation, e.g., s_i for variance.
  2. [§4.1] The text states 'achieving w-F1 scores above 70% for every emotion class', but Table 1 shows per-class accuracy columns, not per-class w-F1. Please clarify which metric is being reported.
  3. [Figure 1] The caption says solid and dashed rectangles denote trainable and frozen models, but the text does not describe any frozen component. Please clarify or correct the caption.
  4. [Table 1] Several baseline rows contain '-' entries (e.g., AdaIGN, M2FNet, DER-GCN). Please explain why some per-class or overall scores are unavailable, or complete the table.

Circularity Check

1 steps flagged · score 5.0 of 10

ESL uncertainty-supervision target is self-referential by construction; held-out accuracy claim remains independent.

  1. self definitional [Section 2.6, Eqs. (12)-(14)]
    "global cluster centers are updated once per training epoch using the current feature statistics... we adopt a distributional representation of cluster centers... D2W(i, δ) = ∥µδi − µδyi∥22 + ∥σδi − σδyi∥22; LδESL = 1/k Σ_i ∥α√σδi − √D2W(i, δ)∥22"

    Eq. (12) computes the center σδy from the model's own predicted σδi; Eq. (13) defines the supervision distance for utterance i using σδyi, which includes σδi itself. Eq. (14) then regresses α√σδi toward √D2W(i,δ), so the regressand appears in the target. For a singleton class |I_y|=1, D2W(i,δ)=0 identically and the loss becomes ||α√σδi||^2, forcing variance to zero. The 'explicitly supervised' uncertainty target is thus a self-referential bootstrap, not an independent label, so the claimed well-calibrated uncertainty is not established by construction. Held-out accuracy remains an external empirical comparison.

full rationale

The paper's primary performance claim (SOTA accuracy/F1 on IEMOCAP and MELD) is evaluated on held-out partitions (LOSO and predefined splits) and therefore is not circular; no fitted test-set quantity is reported as a prediction. The self-citations (refs. 2, 5, 8, 10-12, 17) appear only as related work and none is used to justify the architecture or to forbid alternatives; there is no imported uniqueness theorem and no ansatz smuggled via citation. The one genuine circular step is in the ESL construction: the 'explicit supervision' target D2W(i,δ) is made from the model's own predicted means and variances, and even includes the utterance's own variance in its class center. The loss therefore aligns predicted variance with a function of predicted variance rather than with an independent, well-calibrated target; for singleton classes the target collapses to zero. I score this as partial circularity (5/10) because the central accuracy claim is independent, but the uncertainty-supervision 'prediction' is self-referential by construction. The vector-vs-scalar mismatch in Eq. (14) and the absence of calibration metrics are correctness risks rather than circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework introduces architectural components (ContextDEM, UAMF, ESL) but no new physical entities. The main additional assumptions are the Gaussian uncertainty representation, the validity of the self-derived 2-Wasserstein target, the adequacy of frozen encoders, and the comparability of baselines. The free parameters that most affect the claims are the learnable alpha and the hand-tuned lambda/start-epoch.

free parameters (4)
  • alpha (learnable uncertainty scaling factor) = Learned, unconstrained
    Appears in Eq. 14 multiplying sqrt(sigma); can absorb any global scale mismatch between predicted variance and the 2-Wasserstein distance, so variance is not calibrated to an absolute scale.
  • lambda and e_pstart = lambda=2e-5, e_pstart=10
    Hand-set in Section 3.2/eq. 16; controls how much ESL contributes and when it starts. Central to the claimed effect of ESL.
  • epsilon in Eq. 5 = Not specified
    Numerical stability constant in the confidence computation; value is not reported.
  • Architecture dimensions D_g, D_C, Transformer depth/heads = Not reported
    Hidden dimensions and layer counts for ContextDEM and TransEncoder are not stated, yet they determine model capacity and are needed for reproduction.
assumptions (4)
  • domain assumption Diagonal Gaussian (mean, variance) output is an adequate representation of utterance-level multimodal uncertainty.
    Section 2.4 assumes each modality-specific feature can be modeled as a Gaussian with independent variance; no evidence is given that this captures the true uncertainty structure.
  • domain assumption Frozen pre-trained encoders (RoBERTa, wav2vec2.0, CLIP) retain sufficient emotional information for text, audio, and video.
    Section 2.3/3.2 uses these encoders without fine-tuning and without validating that their features are emotion-discriminative across IEMOCAP/MELD.
  • ad hoc to paper The 2-Wasserstein distance to an emotion-specific cluster center is a valid supervision target for uncertainty.
    Section 2.6 introduces the ESL objective; the target is constructed from the model's own statistics and there is no independent ground-truth uncertainty to validate it.
  • domain assumption The evaluation setting (LOSO on IEMOCAP, standard split on MELD) is comparable across all baselines in Table 1.
    Section 3.1 describes the splits, but the paper does not state whether all baselines were re-run under the same feature extractors and preprocessing; without that, the comparison rests on an assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation." pith.science (2026). https://pith.science/paper/PVVSONOC

@misc{pith2026260718336,
  author       = {Pith},
  title        = {Pith review of: EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PVVSONOC}},
  note         = {Machine review of arXiv:2607.18336}
}
read the original abstract

Multimodal emotion recognition in conversation (MERC) can leverage multimodal and contextual cues to boost recognition performance. However, existing fusion approaches in MERC often ignore modality-specific uncertainty across utterances caused by conflicting cues, varying noise, and missing modality-specific signals. We propose EmoEUS, an explicit uncertainty supervision framework for MERC. EmoEUS performs uncertainty-aware multimodal fusion by dynamically weighting modalities using learned variance estimates. We also introduce an explicitly supervised loss that aligns each utterance's predicted variance with the distance between the utterance's distributional representation and its emotion- and modality-specific cluster center. Experiments on IEMOCAP and MELD show that EmoEUS consistently outperforms state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2607.18336 by the authors.

Figure 1
Figure 1. The overall architecture of the EmoEUS framework. EmoEUS enhances MERC performance through explicit uncertainty supervision and uncertainty-aware multimodal fusion. Solid and dashed rectangles denote trainable and frozen models, respectively [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Framework of ContextDEM, where LN and MLP de￾note LayerNorm and multi-layer perceptron, respectively. tributions (µ, σ 2 ) to model modality-specific uncertainty. Sec￾ond, an uncertainty-aware multimodal fusion (UAMF) mod￾ule integrates these distributions by weighting modalities based on their uncertainties. Finally, the explicitly supervised loss LESL aligns the predicted variance with the 2-Wasserstein dis￾tance … view at source ↗
Figure 3
Figure 3. Framework of explicitly supervised loss L δ ESL. ESL aims to align the predicted variance σ δ i of each utterance with the 2-Wasserstein distance D2W (i, δ) between the utterance￾wise distribution and the corresponding global emotion center, providing explicit supervision for uncertainty estimation. We hypothesize that implicit uncertainty modeling, based solely on the cross-entropy loss LCE, is insufficient to accu… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 7 linked inside Pith

  1. [1]

    MERC analyzes dialogue turns from multiple modalities (e.g., text, au- dio, and visual) and assigns one emotion label per utterance [1, 2]

    Introduction Multimodalemotionrecognition inconversation (MERC) is important for applications such as human-computer interaction, intelligent medical care, and affective computing [1]. MERC analyzes dialogue turns from multiple modalities (e.g., text, au- dio, and visual) and assigns one emotion label per utterance [1, 2]. Prior work focuses on integratin...

  2. [2]

    Methodology 2.1. Task Definition Consider a dialogue consisting ofktemporally ordered utter- ances, denoted asU={u i}k i=1, where individual utterances have corresponding emotion labels{y i}k i=1 and speaker labels {si}k i=1. The emotion labely i belongs to a predefined set of emotionsY. Each utteranceu i is accompanied by multimodal data, including a vid...

    arXiv 2026

  3. [3]

    Leave-One-Session-Out

    Experimental Settings 3.1. Datasets and Evaluation Metric IEMOCAP[23] is a widely used dataset for emotion recog- nition in conversation. For partitioning the data, we utilized the commonly used “Leave-One-Session-Out” (LOSO) strat- egy.MELD[24] is a multi-modal, multi-speaker conversa- tional dataset derived from the TV series “Friends”. To en- sure a fa...

  4. [4]

    Concat” directly concatenates features (R δ) from dif- ferent modalities and feeds them to the classifier. “Attention

    Experimental Results and Analysis 4.1. Effect of Different Fusion Methods and Overall Results Table 1 shows the detailed results on both datasets. The pro- posed EmoEUS performs the best among all models, demon- strating its effectiveness. By explicitly modeling uncertainty throughL ESL , EmoEUS effectively identifies and emphasizes reliable modalities wh...

    arXiv 2019

  5. [5]

    Conclusions and Future Work We propose the EmoEUS, a novel framework that incorporates explicit uncertainty supervision for MERC. Our approach in- troduces a context-level distribution estimator module (Con- textDEM) to model utterance-level distribution, dynamically weights modality reliability through an uncertainty-aware mul- timodal fusion (UAMF) mech...

  6. [6]

    Acknowledgment The work presented in this article is supported by the Research Platform for Advanced Audio and Speech Signal Processing (P0049192) funded by Innovation Technology Co. Ltd

  7. [7]

    All scientific content, experiments and analyses were produced and verified by the authors

    Use of Generative AI Disclosure Generative AI tools were used only for language polishing and formatting assistance. All scientific content, experiments and analyses were produced and verified by the authors

  8. [8]

    Emotion recognition in conversation: Research challenges, datasets, and recent advances,

    S. Poria, N. Majumder, R. Mihalcea, and E. Hovy, “Emotion recognition in conversation: Research challenges, datasets, and recent advances,”IEEE Access, vol. 7, pp. 100 943–100 953, 2019

    2019

Show all 42 references
  1. [9]

    MM-NodeFormer: Node transformer multimodal fusion for emotion recognition in conver- sation,

    Z. Huang, M.-W. Mak, and K. A. Lee, “MM-NodeFormer: Node transformer multimodal fusion for emotion recognition in conver- sation,” inProc. Interspeech, 2024, Conference Proceedings, pp. 4069–4073

    2024

  2. [10]

    Multiple classifier systems for the classification of audio-visual emotional states,

    M. Glodek, S. Tschechne, G. Layher, M. Schels, T. Brosch, S. Scherer, M. K¨achele, M. Schmidt, H. Neumann, G. Palmet al., “Multiple classifier systems for the classification of audio-visual emotional states,” inAffective Computing and Intelligent Interac- tion: Fourth Internat...

    2011

  3. [11]

    MultiEMO: An attention-based correlation-aware multimodal fusion framework for emotion recognition in conversations,

    T. Shi and S.-L. Huang, “MultiEMO: An attention-based correlation-aware multimodal fusion framework for emotion recognition in conversations,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-...

    2023

  4. [12]

    BA V-MossFormer2: En- hanced MossFormer2 for binaural audio-visual speech enhance- ment,

    W. Ren, K. Li, R. Chao, J. Li, Z. Huang, S. Ahmed, Y .-J. Li, K.-H. Hung, S.-S. Wang, H.-M. Wanget al., “BA V-MossFormer2: En- hanced MossFormer2 for binaural audio-visual speech enhance- ment,” inProc. AVSEC 2025, 2025, pp. 79–80

    2025

  5. [13]

    M2FNET: Multi-modal fusion network for emotion recognition in conversation,

    V . Chudasama, P. Kar, A. Gudmalwar, N. Shah, P. Wasnik, and N. Onoe, “M2FNET: Multi-modal fusion network for emotion recognition in conversation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4652–4661

    2022

  6. [14]

    CFN-ESA: A cross-modal fusion network with emotion-shift awareness for dialogue emo- tion recognition,

    J. Li, Y . Liu, X. Wang, and Z. Zeng, “CFN-ESA: A cross-modal fusion network with emotion-shift awareness for dialogue emo- tion recognition,”arXiv preprint arXiv:2307.15432, 2023

    2023 arXiv

  7. [15]

    IDIR: Identifying and Distilling Informative Relations for Speaker Verification,

    C.-X. Gan, Z. Li, Z. Jin, Z. Huang, M.-W. Mak, and K. A. Lee, “IDIR: Identifying and Distilling Informative Relations for Speaker Verification,” inInterspeech 2025, 2025, pp. 5758–5762

    2025

  8. [16]

    Unimodal-driven distillation in multimodal emotion recognition with dynamic fusion,

    J. Li, R. Yu, H. Huang, and H. Yan, “Unimodal-driven distillation in multimodal emotion recognition with dynamic fusion,” in2025 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2025, pp. 1–6

    2025

  9. [17]

    Distilling attention knowledge for speaker verification,

    Z. Jin, S. Liu, Z. Li, C.-X. Gan, Z. Huang, M.-W. Mak, and K. A. Lee, “Distilling attention knowledge for speaker verification,” in ICASSP 2026 - 2026 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), 2026, pp. 16 447– 16 451

    2026

  10. [18]

    UNet-based fusion and exponential moving average adaptation for noise-robust speaker recognition,

    C.-X. Gan, P. Bell, M.-W. Mak, Z. Li, Z. Jin, Z. Huang, and K. A. Lee, “UNet-based fusion and exponential moving average adaptation for noise-robust speaker recognition,”arXiv preprint arXiv:2604.25624, 2026

    2026 arXiv

  11. [19]

    De- noising student features with diffusion models for knowledge dis- tillation in speaker verification,

    Z. Jin, Y . Tu, Z. Li, Z. Huang, C.-X. Gan, and M.-W. Mak, “De- noising student features with diffusion models for knowledge dis- tillation in speaker verification,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2025, pp. 1–5

    2025

  12. [20]

    Survey on audiovisual emotion recognition: Databases, features, and data fusion strate- gies,

    C.-H. Wu, J.-C. Lin, and W.-L. Wei, “Survey on audiovisual emotion recognition: Databases, features, and data fusion strate- gies,”APSIPA Transactions on Signal and Information Process- ing, vol. 3, p. e12, 2014

    2014

  13. [21]

    A transformer-based model with self-distillation for multimodal emotion recognition in conversations,

    H. Ma, J. Wang, H. Lin, B. Zhang, Y . Zhang, and B. Xu, “A transformer-based model with self-distillation for multimodal emotion recognition in conversations,”IEEE Transactions on Multimedia, vol. 26, pp. 776–788, 2023

    2023

  14. [22]

    Enhancing multimodal emotion recognition through multi- granularity cross-modal alignment,

    X. Wang, S. Zhao, H. Sun, H. Wang, J. Zhou, and Y . Qin, “Enhancing multimodal emotion recognition through multi- granularity cross-modal alignment,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2025, pp. 1–5

    2025

  15. [23]

    Enhancing emotion recognition in incomplete data: A novel cross-modal alignment, reconstruction, and refinement framework,

    H. Sun, S. Zhao, S. Li, X. Kong, X. Wang, J. Zhou, A. Kong, Y . Chen, W. Zeng, and Y . Qin, “Enhancing emotion recognition in incomplete data: A novel cross-modal alignment, reconstruction, and refinement framework,” inICASSP 2025 - 2025 IEEE Inter- national Conference on Acou...

    2025

  16. [24]

    Disentangling speech representations learning with latent dif- fusion for speaker verification,

    Z. Li, M.-W. Mak, J.-T. Chien, M. Pilanci, Z. Jin, and H. Meng, “Disentangling speech representations learning with latent dif- fusion for speaker verification,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 3896–3907, 2025

    2025

  17. [25]

    Probabilistic face embeddings,

    Y . Shi and A. K. Jain, “Probabilistic face embeddings,” inPro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 6902–6911

    2019

  18. [26]

    Data uncertainty learning in face recognition,

    J. Chang, Z. Lan, C. Cheng, and Y . Wei, “Data uncertainty learning in face recognition,” inProceedings of the IEEE/CVF cConference on Computer Vision and Pattern Recognition, 2020, pp. 5710–5719

    2020

  19. [27]

    Map: Multimodal uncertainty-aware vision-language pre-training model,

    Y . Ji, J. Wang, Y . Gong, L. Zhang, Y . Zhu, H. Wang, J. Zhang, T. Sakai, and Y . Yang, “Map: Multimodal uncertainty-aware vision-language pre-training model,” in2023 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 23 262–23 271

    2023

  20. [28]

    Masked graph learning with recurrent alignment for multimodal emotion recognition in conversation,

    T. Meng, F. Zhang, Y . Shou, H. Shao, W. Ai, and K. Li, “Masked graph learning with recurrent alignment for multimodal emotion recognition in conversation,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  21. [29]

    COLD Fusion: Calibrated and or- dinal latent distribution fusion for uncertainty-aware multimodal emotion recognition,

    M. K. Tellamekala, S. Amiriparian, B. W. Schuller, E. Andr ´e, T. Giesbrecht, and M. Valstar, “COLD Fusion: Calibrated and or- dinal latent distribution fusion for uncertainty-aware multimodal emotion recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence,...

    2024

  22. [30]

    IEMOCAP: Interactive emotional dyadic motion capture database,

    C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “IEMOCAP: Interactive emotional dyadic motion capture database,”Language Resources and Evaluation, vol. 42, pp. 335–359, 2008

    2008

  23. [31]

    MELD: A multimodal multi-party dataset for emotion recognition in conversations,

    S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cam- bria, and R. Mihalcea, “MELD: A multimodal multi-party dataset for emotion recognition in conversations,”arXiv preprint arXiv:1810.02508, 2018

    2018 arXiv

  24. [32]

    MM-DFN: Mul- timodal dynamic fusion network for emotion recognition in con- versations,

    D. Hu, X. Hou, L. Wei, L. Jiang, and Y . Mo, “MM-DFN: Mul- timodal dynamic fusion network for emotion recognition in con- versations,” inProc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2022, pp. 7037–7041

    2022

  25. [33]

    RoBERTa: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “RoBERTa: A robustly optimized bert pretraining approach,”arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  26. [34]

    wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,”Advances in Neural Information Processing Systems, vol. 33, pp. 12 449–12 460, 2020

    2020

  27. [35]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” in Proc. International Conference on Machine Learning, 2021, pp. 8748–8763

    2021

  28. [36]

    DialogueRNN: An attentive rnn for emotion de- tection in conversations,

    N. Majumder, S. Poria, D. Hazarika, R. Mihalcea, A. Gelbukh, and E. Cambria, “DialogueRNN: An attentive rnn for emotion de- tection in conversations,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 6818–6825

    2019

  29. [37]

    DialogueGCN: A graph convolutional neural net- work for emotion recognition in conversation,

    D. Ghosal, N. Majumder, S. Poria, N. Chhaya, and A. Gel- bukh, “DialogueGCN: A graph convolutional neural net- work for emotion recognition in conversation,”arXiv preprint arXiv:1908.11540, 2019

    1908 arXiv

  30. [38]

    MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation,

    J. Hu, Y . Liu, J. Zhao, and Q. Jin, “MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation,”arXiv preprint arXiv:2107.06779, 2021

    2021 arXiv

  31. [39]

    Adaptive graph learning for multimodal conversational emotion detection,

    G. Tu, T. Xie, B. Liang, H. Wang, and R. Xu, “Adaptive graph learning for multimodal conversational emotion detection,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, 2024. [Online]. Available: https://ojs.aaai.org/ index.php/AAAI/article/view/29876

    2024

  32. [40]

    Curriculum learning meets directed acyclic graph for multimodal emotion recogni- tion,

    C.-B. Nguyen, D.-T. Le, Q. T. Haet al., “Curriculum learning meets directed acyclic graph for multimodal emotion recogni- tion,” inProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evalua- tion (LREC-COLING 2024), 2024,...

    2024

  33. [41]

    DER-GCN: Dialog and event relation-aware graph convolutional neural network for mul- timodal dialog emotion recognition,

    W. Ai, Y . Shou, T. Meng, and K. Li, “DER-GCN: Dialog and event relation-aware graph convolutional neural network for mul- timodal dialog emotion recognition,”IEEE Transactions on Neu- ral Networks and Learning Systems, vol. 36, no. 3, pp. 4908– 4921, 2025

    2025

  34. [42]

    Feature-enhanced multi- modal interaction model for emotion recognition in conversation,

    Y . Fu, X. Yan, W. Chen, and J. Zhang, “Feature-enhanced multi- modal interaction model for emotion recognition in conversation,” Knowledge-Based Systems, vol. 309, p. 112876, 2025

    2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.