Pith. sign in

REVIEW 5 major objections 8 minor 53 references

Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion

T0 review · 5 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sync-TVA is a graph-attention fusion framework that reports the best accuracy and weighted F1 on the MELD and IEMOCAP emotion recognition benchmarks.

desk verdict Plausible architecture, internally contradictory headline numbers—deserves review only after the authors reconcile text and tables. read the letter →

arxiv 2507.21395 v1 pith:RW3KQQSM submitted 2025-07-29 cs.MM cs.AIcs.SDeess.AS

Zeyu Deng , Yanhui Lu , Jiashu Liao , Shuang Wu , Chongfeng Wei This is my paper · ORCID
classification cs.MMcs.AIcs.SDeess.AS
keywords multimodalemotionrecognitioncross-modalfusiongraphneuralnetworkattentionmechanismclassimbalanceconversationalMELDbenchmarkIEMOCAP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sync-TVA is an end-to-end system for multimodal emotion recognition in conversations. It takes text, audio, and visual features, enhances each modality with a dynamic gating and self-attention module, then builds three learned cross-modal graphs (visual-audio, text-visual, audio-text) whose edge weights are set by the enhancement module. Graph convolution lets each modality borrow context from its cross-modal neighbors, and a gated cross-attention fusion module combines the results. The paper reports consistent gains over prior state-of-the-art models on MELD and IEMOCAP, with the largest improvements on minority emotion classes such as fear and disgust. If these results hold, the framework offers a concrete answer to the recurring problems of modality imbalance and rigid fusion in emotion recognition.

What carries the argument

The central object is the trio of cross-modal graphs constructed by the MSDE (Modality-Specific Dynamic Enhancement) module, which combines dynamic gating, multi-head self-attention, and a residual feedforward network. For each graph, MSDE computes edge weights such as $e^{VA}_{ij}=f_{\mathrm{MSDE}}(F^v_i, F^a_j)$, passes them through a sigmoid to form the adjacency matrix, and feeds the graph into a GCN update $H^{(l+1)}=\sigma(\tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}H^{(l)}W^{(l)})$. The updated features are refined by 1D convolution and layer norm, then fused through cross-attention with a GRU-style gate $F_{\mathrm{CAF}}=\sigma(W_f\tilde{U}+b_f)\odot\tanh(W_g\tilde{U}+b_g)$, iterating over graph pairs before classification. The graph edges are what carry cross-modal information, and the gating controls how much of it enters the final representation.

What would settle it

Replace the learned adjacency function $f_{\mathrm{MSDE}}$ with a fixed, non-trainable kernel such as cosine similarity or random edges, keeping every other component identical; if accuracy and weighted F1 remain within the reported margin over the concatenation baseline, then the learned edge weights are not the cause of the improvement. A second check would evaluate the same architecture on a third conversational emotion dataset or with a different set of pretrained encoders, where a sharp performance drop would show the gains depend on the encoder-specific feature spaces.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that making cross-modal relationships data-dependent, through learned edge weights in heterogeneous graphs rather than static concatenation, improves emotion classification on both standard benchmarks. On IEMOCAP, Sync-TVA reports 73.10% accuracy and 73.35% weighted F1, surpassing the previously best GraphSmile by 0.33 and 0.54 percentage points; on MELD, the text reports 68.25% accuracy and 67.40% weighted F1, with category-level gains on all seven emotions. The ablations attribute most of this to the MSDE module and the graph structure: removing all cross-modal edges lowers weighted F1 by roughly 5 to 6 points, and removing MSDE lowers it by approximately 4 points. The authors conclude that the three components, dynamic enhancement, enforced graph construction, and gated cross-attention fusion, work together to produce more separable emotion representations.

Load-bearing premise

The load-bearing premise is that features from three unrelated pretrained encoders, RoBERTa, ResNet-50, and OpenSMILE, occupy comparable numeric ranges after L2 normalization, so the learned graph-edge function can assign meaningful cross-modal weights; if the feature spaces remain incompatible, the graph convolution may add noise, and the reported gains over the simple concatenation baseline would not transfer to other datasets.

Editorial extensions

If this is right

  • On IEMOCAP, Sync-TVA reports state-of-the-art numbers of 73.10% accuracy and 73.35% weighted F1, ahead of the previous best by about a third and a half of a percentage point.
  • Ablations imply the three design choices are additive: removing MSDE costs about 4 WF1 points, removing all cross-modal graph edges costs about 5 to 6 points, and removing the gated cross-attention fusion costs about 1 to 1.6 points.
  • The per-class results show the largest relative gains on minority emotions, which is exactly where class imbalance usually hurts; on MELD, Fear and Disgust both improve by over 2 points.
  • Because the framework is end-to-end and uses off-the-shelf features, it can be dropped into existing dialogue or human-robot interaction pipelines without manual cross-modal alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-parity of the two-graph variant (V-A plus T-V) with the full three-graph model suggests the audio-text graph is partly redundant on these benchmarks; a leaner two-graph version could be tested for deployment.
  • The paper's own conclusion lists contrastive learning and class-balanced training as future work; coupling the learned graph edges to an explicit imbalance-aware loss might push the minority-class gains further.
  • An ablation that replaces $f_{\mathrm{MSDE}}$ with a fixed similarity function would clarify whether the module learns task-specific cross-modal relations or simply re-discovers cosine similarity.
  • The same three-graph recipe could transfer to other heterogeneous multimodal tasks, but the L2-normalization premise would need to be revisited for encoders with very different output distributions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes Sync-TVA, an end-to-end graph-attention framework for multimodal emotion recognition. The architecture consists of a modality-specific dynamic enhancement (MSDE) module, construction of three heterogeneous cross-modal graphs (V-A, T-V, A-T), graph convolutional updates, and a cross-modal attention fusion (CAF) module. The authors report experiments on MELD and IEMOCAP, claiming consistent improvements over state-of-the-art baselines in accuracy and weighted F1, particularly for class-imbalanced minority emotions. The paper also includes ablation studies of the MSDE module, graph structure, and fusion strategies.

Significance. If the reported results are correct, the proposed framework would be a competitive new model on two widely used MER benchmarks, with a sensible design combining modality-specific enhancement and structured cross-modal graph propagation. The ablation study is a useful step toward understanding the contribution of each component. However, the paper's central empirical claim is currently undermined by internal inconsistencies between the text and tables, missing statistical evidence, and incomplete figure references. The architecture is a modular combination of known components rather than a fundamentally new paradigm, but the application to MER with heterogeneous cross-modal graphs is a reasonable contribution. The strengths are the clear modular design and the attempt to assess component contributions; the weakness is that the quantitative support is not reliable in its present form.

major comments (5)
  1. [IV.E.1 and IV.E.2 vs Tables II and III] The text and the tables report irreconcilable results for the full model. On IEMOCAP, Section IV.E.1 states Accuracy=73.10% and WF1=73.35% with per-class values (Happy 64.12, Sadness 84.34, Anger 72.21), while Table II reports Accuracy=73.42, WF1=73.68 and per-class values (Happy 64.45, Sadness 84.62, Anger 72.54). On MELD, Section IV.E.2 states 68.25/67.40 while Table III reports 68.60/67.75. The claimed improvements over GraphSmile differ accordingly (e.g., 0.33 vs 0.65 on IEMOCAP accuracy). Because the headline claim is 'consistent improvements' over state-of-the-art models, the reader cannot tell which set of numbers corresponds to the actual experiments, and the paper's own evidence is internally contradictory.
  2. [IV.C] Section IV.C states that a paired t-test with alpha=0.05 was used to verify statistical significance, but no p-values, t-statistics, confidence intervals, or variance estimates appear anywhere in the paper. Moreover, the ablation section reports that each configuration was repeated three times but gives only averages, not standard deviations or individual runs. Consequently, the claims of significance and robustness are not supported by the reported evidence.
  3. [IV.B and IV.E] The definition of the accuracy metric is ambiguous and potentially inconsistent with the baseline comparisons. Section IV.B says 'we supplement the average accuracy scores by calculating the arithmetic mean of the accuracy rates for each category,' which describes macro (unweighted) average accuracy. However, Tables II and III are headed 'Accuracy' and the values match the overall accuracy commonly reported for the cited baselines (e.g., DialogueGCN 65.54 on IEMOCAP). If Sync-TVA's Accuracy is macro-averaged while the baselines' Accuracies are overall accuracy, the comparison is not valid. The paper should state explicitly which accuracy is used and verify that all compared numbers use the same definition.
  4. [III.A-III.C and IV.C] The model and training details are not fully specified. The hyperparameters P (position encoding dimension), B (number of attention heads), and the loss weights λs and λo are given in Section IV.C, but none of them appear in the model equations (Eqs. 1-20) or in the text describing the architecture; Eq. (20) is a standard cross-entropy loss with no weighting terms. In addition, Eq. (18) defines the final representation z from two CAF outputs, although three cross-modal graphs are constructed and the fusion is described as iterative; the aggregation of the third branch is not defined. These omissions make the method non-reproducible from the paper alone.
  5. [IV.G] The confusion-matrix analysis in Section IV.G refers to 'Figure X', and Figure 2 is an empty placeholder with only a caption and no actual confusion matrix plots. The quantitative statements about specific confusions (e.g., Fear confused with Sadness 21.5% and Disgust 15.1%) cannot be verified because the corresponding figure is missing. The figure should be included or the analysis should be removed.
minor comments (8)
  1. [IV.A, Table I] The IEMOCAP row in Table I contains only three numbers ('120 31 5810 1623') under a header with four columns for train/valid/test splits, leaving the table layout ambiguous; presumably a column value is missing.
  2. [IV.E.1] There is a typo: 'One the IEMOCAP dataset' should read 'On the IEMOCAP dataset'.
  3. [IV.F.2] The text refers to a 'motion-acoustic channel' when discussing the A-T graph; this appears to be a misnomer for the audio-text interactive channel.
  4. [III.B] The module is introduced as 'Modality-Specific Dependency Encoder (MSDE)' in Section III.B, but the abstract, introduction, and Table IV use 'Modality-Specific Dynamic Enhancement (MSDE)'; please use one name consistently.
  5. [IV.H] Figure 3, the feature-space visualization, has no axis labels and does not state the dimensionality-reduction method (e.g., PCA, t-SNE, UMAP) or how the projection was computed, which limits its interpretability.
  6. [I] The sentence 'Modern computer systems can already capable of exhibiting a degree of empathy' is missing a verb; it should be 'are already capable'.
  7. [V] The conclusion introduces the abbreviation 'GFM' (Gate Fusion Module), which is not used in the methodology; the fusion gating is described as 'GRU-style gating' within the CAF module, so the terminology should be aligned.
  8. [IV.E.3] The baseline name is written as 'SAC-LSTM' in the text but as 'SACL-LSTM' in Table III and reference [53]; please standardize the name.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Sync-TVA's claims rest on external benchmark evaluations, not on self-derived definitions or self-citations.

full rationale

The paper's central claim is empirical: Sync-TVA outperforms existing models on MELD and IEMOCAP. The derivation chain is a standard supervised multimodal architecture: pretrained feature extractors (Eqs. 1-3), a learned adjacency function f_MSDE (Eq. 7), GCN updates (Eq. 10), cross-attention fusion (Eqs. 13-17), and a softmax classifier (Eq. 19). The adjacency weights are learned from input features and used inside the model; they are not fitted to the target labels and then renamed as predictions. Hyperparameters are tuned per dataset, which is conventional fitting rather than circular reasoning, and the evaluation is on held-out test splits of external benchmarks. The reference list contains no self-citations by the authors, and no load-bearing claim is justified by a uniqueness theorem or ansatz imported from the authors' own prior work. The main concern raised by the manuscript is internal inconsistency between the numbers in Section IV.E and Tables II/III, and a missing Figure X in the confusion-matrix section; these are empirical reproducibility and reporting issues, not circularity. No step in the derivation reduces, by definition or by self-citation, to its own inputs. Therefore the circularity score is 0.

Assumptions & free parameters 7 free parameters · 4 assumptions · 2 invented entities

The central claim rests on two named architectural components (MSDE, CAF) and a set of dataset-specific hyperparameters. These are validated only on the two target benchmarks via the paper's own ablations, with no external handles or released artifacts.

free parameters (7)
  • Number of layers L = 7 (IEMOCAP), 5 (MELD)
    Hand-set per dataset without sensitivity analysis.
  • Position encoding dimension P = 17 (IEMOCAP), 3 (MELD)
    Chosen per dataset; no justification given.
  • Number of attention heads B = 19 (IEMOCAP), 3 (MELD)
    Chosen per dataset; no sensitivity analysis.
  • Loss weight lambda_s = 1.0 (IEMOCAP), 0.5 (MELD)
    Tuned per dataset.
  • Loss weight lambda_o = 0.7 (IEMOCAP), 0.2 (MELD)
    Tuned per dataset.
  • Learning rate = 1e-4 (IEMOCAP), 7e-5 (MELD)
    Per-dataset choice.
  • Dropout rate = 0.2
    Fixed for both datasets.
assumptions (4)
  • standard math The GCN update rule in Eq. (10) is a standard message-passing operator that improves node representations.
    Standard symmetric normalized graph convolution; no proof needed.
  • domain assumption Pretrained ResNet-50, RoBERTa, and OpenSMILE features are sufficiently aligned after L2 normalization to support cross-modal edge learning.
    The MSDE scoring function assumes comparable feature distributions; not verified empirically across modalities.
  • ad hoc to paper Sigmoid-normalized pairwise scores in Eq. (8) define a valid and useful adjacency matrix for the GCN.
    This specific adjacency construction is introduced without independent theoretical or empirical justification.
  • ad hoc to paper Three pairwise cross-modal graphs (V-A, T-V, A-T) capture the emotion-relevant interactions better than a single joint graph or simple concatenation.
    Only tested via ablation on the same two benchmarks; no external evidence.
invented entities (2)
  • Modality-Specific Dynamic Enhancement (MSDE) module
    purpose: Per-modality gating, self-attention, and residual feedforward before graph construction.
    Its contribution is measured only through ablations inside this paper; no external falsifiable handle.
  • Cross-modal Attention Fusion (CAF) module
    purpose: Fuses cross-graph representations using cross-attention and GRU-style gating.
    Evaluated only through internal ablations; no independent evidence of efficacy outside this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion." pith.science (2026). https://pith.science/paper/RW3KQQSM

@misc{pith2026250721395,
  author       = {Pith},
  title        = {Pith review of: Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RW3KQQSM}},
  note         = {Machine review of arXiv:2507.21395}
}
read the original abstract

Multimodal emotion recognition (MER) is crucial for enabling emotionally intelligent systems that perceive and respond to human emotions. However, existing methods suffer from limited cross-modal interaction and imbalanced contributions across modalities. To address these issues, we propose Sync-TVA, an end-to-end graph-attention framework featuring modality-specific dynamic enhancement and structured cross-modal fusion. Our design incorporates a dynamic enhancement module for each modality and constructs heterogeneous cross-modal graphs to model semantic relations across text, audio, and visual features. A cross-attention fusion mechanism further aligns multimodal cues for robust emotion inference. Experiments on MELD and IEMOCAP demonstrate consistent improvements over state-of-the-art models in both accuracy and weighted F1 score, especially under class-imbalanced conditions.

Figures

Figures reproduced from arXiv: 2507.21395 by the authors.

Figure 1
Figure 1. Overview of the proposed tri-modal emotion recognition framework. Visual, audio, and textual inputs are processed using [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Confusion matrices of the proposed model on the MELD and IEMOCAP datasets... [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Visualization of feature space before and after processing by Sync-TVA on MELD and IEMOCAP datasets. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 47 canonical work pages

  1. [1]

    Exploring emotional connections: A systematic literature review of attachment in human-robot interaction,

    J. J. Mitchell and M. Jeon, “Exploring emotional connections: A systematic literature review of attachment in human-robot interaction,” International Journal of Human–Computer Interaction , pp. 1–22, 2025

    work page 2025

  2. [2]

    Ai, robotics, medicine and health sciences,

    N. Gasteiger and E. Broadbent, “Ai, robotics, medicine and health sciences,” in The Routledge Social Science Handbook of AI. Routledge, 2021, pp. 313–338

    work page 2021

  3. [3]

    A scoping review of the literature on prosodic elements related to emotional speech in human-robot interaction,

    N. Gasteiger, J. Lim, M. Hellou, B. A. MacDonald, and H. S. Ahn, “A scoping review of the literature on prosodic elements related to emotional speech in human-robot interaction,” International Journal of Social Robotics, vol. 16, no. 4, pp. 659–670, 2024

    work page 2024

  4. [4]

    A novel emotion recognition system for human– robot interaction (hri) using deep ensemble classification,

    K. Zaman, G. Zengkang, S. Zhaoyun, S. M. Shah, W. Riaz, J. Ji, T. Hus- sain, and R. W. Attar, “A novel emotion recognition system for human– robot interaction (hri) using deep ensemble classification,” International Journal of Intelligent Systems , vol. 2025, no. 1, p. 6611276, 2025

    work page 2025

  5. [5]

    Towards the development of affective facial expression recognition for human-robot interaction,

    D. R. Faria, M. Vieira, and F. C. Faria, “Towards the development of affective facial expression recognition for human-robot interaction,” in Proceedings of the 10th international conference on pervasive technolo- gies related to assistive environments , 2017, pp. 300–304

    work page 2017

  6. [6]

    Multimodal human–robot interaction for human-centric smart manufacturing: a survey,

    T. Wang, P. Zheng, S. Li, and L. Wang, “Multimodal human–robot interaction for human-centric smart manufacturing: a survey,” Advanced Intelligent Systems, vol. 6, no. 3, p. 2300359, 2024

    work page 2024

  7. [7]

    Beyond sentiment analysis: A review of recent trends in text based sentiment analysis and emotion detection,

    L. P. Hung and S. Alias, “Beyond sentiment analysis: A review of recent trends in text based sentiment analysis and emotion detection,” Journal of Advanced Computational Intelligence and Intelligent Informatics , vol. 27, no. 1, pp. 84–95, 2023

    work page 2023

  8. [8]

    Deep learning–based text classification: a comprehensive review,

    S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep learning–based text classification: a comprehensive review,”ACM computing surveys (CSUR), vol. 54, no. 3, pp. 1–40, 2021

    work page 2021

Show all 53 references
  1. [9]

    Building emotional dictionary for sentiment analysis of online news,

    Y . Rao, J. Lei, L. Wenyin, Q. Li, and M. Chen, “Building emotional dictionary for sentiment analysis of online news,” World Wide Web , vol. 17, pp. 723–742, 2014

    2014

  2. [10]

    Deep convolutional neural networks for sentiment analysis of short texts,

    C. Dos Santos and M. Gatti, “Deep convolutional neural networks for sentiment analysis of short texts,” in Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers, 2014, pp. 69–78

    2014

  3. [11]

    A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification,

    Y . Zhang and B. Wallace, “A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification,” arXiv preprint arXiv:1510.03820 , 2015

    2015 arXiv

  4. [12]

    Recurrent neural networks for emotion recognition in video,

    S. Ebrahimi Kahou, V . Michalski, K. Konda, R. Memisevic, and C. Pal, “Recurrent neural networks for emotion recognition in video,” in Pro- ceedings of the 2015 ACM on international conference on multimodal interaction, 2015, pp. 467–474

    2015

  5. [13]

    Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network,

    A. Sherstinsky, “Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network,” Physica D: Nonlinear Phe- nomena, vol. 404, p. 132306, 2020

    2020

  6. [14]

    Text sentiment analysis based on long short-term memory,

    D. Li and J. Qian, “Text sentiment analysis based on long short-term memory,” in 2016 First IEEE International Conference on Computer Communication and the Internet (ICCCI) . IEEE, 2016, pp. 471–475

    2016

  7. [15]

    Emotion recognition from facial expressions using hybrid feature descriptors,

    T. Kalsum, S. M. Anwar, M. Majid, B. Khan, and S. M. Ali, “Emotion recognition from facial expressions using hybrid feature descriptors,” IET Image Processing , vol. 12, no. 6, pp. 1004–1012, 2018

    2018

  8. [16]

    Using multiple steerable filters and bayesian regularization for facial expression recognition,

    H. Mahersia and K. Hamrouni, “Using multiple steerable filters and bayesian regularization for facial expression recognition,” Engineering Applications of Artificial Intelligence , vol. 38, pp. 190–202, 2015

    2015

  9. [17]

    Automatic speech emotion recogni- tion using support vector machine,

    P. Shen, Z. Changjun, and X. Chen, “Automatic speech emotion recogni- tion using support vector machine,” in Proceedings of 2011 international conference on electronic & mechanical engineering and information technology, vol. 2. IEEE, 2011, pp. 621–625

    2011

  10. [18]

    Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network,

    G. Trigeorgis, F. Ringeval, R. Brueckner, E. Marchi, M. A. Nicolaou, B. Schuller, and S. Zafeiriou, “Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network,” in 2016 IEEE international conference on acoustics, speech and signal proce...

    2016

  11. [19]

    Micro-expression recognition based on 3d flow convolutional neural network,

    J. Li, Y . Wang, J. See, and W. Liu, “Micro-expression recognition based on 3d flow convolutional neural network,” Pattern Analysis and Applications, vol. 22, pp. 1331–1339, 2019

    2019

  12. [20]

    Applying segment-level attention on bi- modal transformer encoder for audio-visual emotion recognition,

    J.-H. Hsu and C.-H. Wu, “Applying segment-level attention on bi- modal transformer encoder for audio-visual emotion recognition,” IEEE Transactions on Affective Computing , vol. 14, no. 4, pp. 3231–3243, 2023

    2023

  13. [21]

    Multimodal and temporal perception of audio-visual cues for emotion recognition,

    E. Ghaleb, M. Popa, and S. Asteriadis, “Multimodal and temporal perception of audio-visual cues for emotion recognition,” in 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2019, pp. 552–558

    2019

  14. [22]

    Multimodal emotion recognition using deep generalized canonical correlation analysis with an attention mechanism,

    Y .-T. Lan, W. Liu, and B.-L. Lu, “Multimodal emotion recognition using deep generalized canonical correlation analysis with an attention mechanism,” in 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020, pp. 1–6

    2020

  15. [23]

    Outlier processing in multimodal emotion recognition,

    G. Zhang, T. Luo, W. Pedrycz, M. A. El-Meligy, M. A. F. Sharaf, and Z. Li, “Outlier processing in multimodal emotion recognition,” IEEE Access, vol. 8, pp. 55 688–55 701, 2020

    2020

  16. [24]

    Graph-based hierarchical semantic consistency network for remote sensing image-text retrieval,

    M. Wang, J. Guo, B. Song, and K. Su, “Graph-based hierarchical semantic consistency network for remote sensing image-text retrieval,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025

    2025

  17. [25]

    Fine-grained alignment and interaction for video grounding with cross- modal semantic hierarchical graph,

    R. Ran, J. Wei, S. He, Y . Zhou, P. Wang, Y . Yang, and H. T. Shen, “Fine-grained alignment and interaction for video grounding with cross- modal semantic hierarchical graph,” IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  18. [26]

    Hybrid multi-attention network for audio–visual emotion recognition through multimodal feature fusion,

    S. Moorthy and Y .-K. Moon, “Hybrid multi-attention network for audio–visual emotion recognition through multimodal feature fusion,” Mathematics, vol. 13, no. 7, p. 1100, 2025

    2025

  19. [27]

    Aia-net: Adaptive interactive attention network for text–audio emotion recognition,

    T. Zhang, S. Li, B. Chen, H. Yuan, and C. P. Chen, “Aia-net: Adaptive interactive attention network for text–audio emotion recognition,” IEEE Transactions on Cybernetics , vol. 53, no. 12, pp. 7659–7671, 2022

    2022

  20. [28]

    Dynamic and static fusion mechanisms of infrared and visible images,

    A. Fang and Y . Li, “Dynamic and static fusion mechanisms of infrared and visible images,” Pattern Recognition, vol. 155, p. 110689, 2024

    2024

  21. [29]

    Adaptive fusion techniques for multi- modal data,

    G. Sahu and O. Vechtomova, “Adaptive fusion techniques for multi- modal data,” arXiv preprint arXiv:1911.03821 , 2019

    1911 arXiv

  22. [30]

    Cortx: Contrastive framework for real-time explanation,

    Y .-N. Chuang, G. Wang, F. Yang, Q. Zhou, P. Tripathi, X. Cai, and X. Hu, “Cortx: Contrastive framework for real-time explanation,” arXiv preprint arXiv:2303.02794, 2023

    2023 arXiv

  23. [31]

    Higru: Hierarchical gated recurrent units for utterance-level emotion recognition,

    W. Jiao, H. Yang, I. King, and M. R. Lyu, “Higru: Hierarchical gated recurrent units for utterance-level emotion recognition,” arXiv preprint arXiv:1904.04446, 2019

    1904 arXiv

  24. [32]

    Semantic-emotion neural network for emotion recognition from text,

    E. Batbaatar, M. Li, and K. H. Ryu, “Semantic-emotion neural network for emotion recognition from text,” IEEE access , vol. 7, pp. 111 866– 111 878, 2019

    2019

  25. [33]

    Aspect-based sen- timent analysis via affective knowledge enhanced graph convolutional networks,

    B. Liang, H. Su, L. Gui, E. Cambria, and R. Xu, “Aspect-based sen- timent analysis via affective knowledge enhanced graph convolutional networks,” Knowledge-Based Systems, vol. 235, p. 107643, 2022

    2022

  26. [34]

    Graph con- volutional network with multiple weight mechanisms for aspect-based sentiment analysis,

    Z. Zhao, M. Tang, W. Tang, C. Wang, and X. Chen, “Graph con- volutional network with multiple weight mechanisms for aspect-based sentiment analysis,” Neurocomputing, vol. 500, pp. 124–134, 2022

    2022

  27. [35]

    Multimodal emotion recognition in polish (student consortium),

    K. Rupauliha, A. Goyal, A. Saini, A. Shukla, and S. Swaminathan, “Multimodal emotion recognition in polish (student consortium),” in 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM). IEEE, 2020, pp. 307–311

    2020

  28. [36]

    Cfn-esa: A cross-modal fusion network with emotion-shift awareness for dialogue emotion recognition,

    J. Li, X. Wang, Y . Liu, and Z. Zeng, “Cfn-esa: A cross-modal fusion network with emotion-shift awareness for dialogue emotion recognition,” IEEE Transactions on Affective Computing , 2024

    2024

  29. [37]

    Multi-modal emotion recognition in conversation based on prompt learning with text-audio fusion features,

    Y . Wu, S. Zhang, and P. Li, “Multi-modal emotion recognition in conversation based on prompt learning with text-audio fusion features,” Scientific Reports, vol. 15, no. 1, p. 8855, 2025

    2025

  30. [38]

    Dia- loguegcn: A graph convolutional neural network for emotion recognition in conversation,

    D. Ghosal, N. Majumder, S. Poria, N. Chhaya, and A. Gelbukh, “Dia- loguegcn: A graph convolutional neural network for emotion recognition in conversation,” arXiv preprint arXiv:1908.11540 , 2019

    1908 arXiv

  31. [39]

    Rgat: A deeper look into syntactic dependency information for coreference resolution,

    Y . Meng, X. Pan, J. Chang, and Y . Wang, “Rgat: A deeper look into syntactic dependency information for coreference resolution,” in 2023 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2023, pp. 1–8

    2023

  32. [40]

    Sdr-gnn: Spectral domain reconstruction graph neural network for incomplete multimodal learning in conversational emotion recognition,

    F. Fu, W. Ai, F. Yang, Y . Shou, T. Meng, and K. Li, “Sdr-gnn: Spectral domain reconstruction graph neural network for incomplete multimodal learning in conversational emotion recognition,”Knowledge- Based Systems, vol. 309, p. 112825, 2025

    2025

  33. [41]

    Multivariate, multi-frequency and multimodal: Rethinking graph neural networks for emotion recog- nition in conversation,

    F. Chen, J. Shao, S. Zhu, and H. T. Shen, “Multivariate, multi-frequency and multimodal: Rethinking graph neural networks for emotion recog- nition in conversation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 10 761–10 770

    2023

  34. [42]

    Multimodal decoupled dis- tillation graph neural network for emotion recognition in conversation,

    Y . Dai, Y . Li, D. Chen, J. Li, and G. Lu, “Multimodal decoupled dis- tillation graph neural network for emotion recognition in conversation,” IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  35. [43]

    Conversational memory network for emotion recognition in dyadic dialogue videos,

    D. Hazarika, S. Poria, A. Zadeh, E. Cambria, L.-P. Morency, and R. Zim- mermann, “Conversational memory network for emotion recognition in dyadic dialogue videos,” in Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting , vo...

    2018

  36. [44]

    Icon: Interactive conversational memory network for multimodal emo- tion detection,

    D. Hazarika, S. Poria, R. Mihalcea, E. Cambria, and R. Zimmermann, “Icon: Interactive conversational memory network for multimodal emo- tion detection,” in Proceedings of the 2018 conference on empirical methods in natural language processing , 2018, pp. 2594–2604

    2018

  37. [45]

    Dialoguernn: An attentive rnn for emotion detection in conversations,

    N. Majumder, S. Poria, D. Hazarika, R. Mihalcea, A. Gelbukh, and E. Cambria, “Dialoguernn: An attentive rnn for emotion detection in conversations,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 6818–6825

    2019

  38. [46]

    Dialoguecrn: Contextual reasoning net- works for emotion recognition in conversations,

    D. Hu, L. Wei, and X. Huai, “Dialoguecrn: Contextual reasoning net- works for emotion recognition in conversations,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proce...

    2021

  39. [47]

    Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversation,

    J. Hu, Y . Liu, J. Zhao, and Q. Jin, “Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversation,” arXiv preprint arXiv:2107.06779 , 2021

    2021 arXiv

  40. [48]

    Cogmen: Con- textualized gnn based multimodal emotion recognition,

    A. Joshi, A. Bhat, A. Jain, A. V . Singh, and A. Modi, “Cogmen: Con- textualized gnn based multimodal emotion recognition,” arXiv preprint arXiv:2205.02455, 2022

    2022 arXiv

  41. [49]

    Joyful: Joint modality fusion and graph contrastive learning for multimoda emotion recogni- tion,

    D. Li, Y . Wang, K. Funakoshi, and M. Okumura, “Joyful: Joint modality fusion and graph contrastive learning for multimoda emotion recogni- tion,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguist...

    2023

  42. [50]

    Mm-dfn: Multimodal dynamic fusion network for emotion recognition in conversations,

    D. Hu, X. Hou, L. Wei, L. Jiang, and Y . Mo, “Mm-dfn: Multimodal dynamic fusion network for emotion recognition in conversations,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 7037–7041

    2022

  43. [51]

    Tracing intricate cues in dialogue: Joint graph structure and sentiment dynamics for multimodal emotion recognition,

    J. Li, X. Wang, and Z. Zeng, “Tracing intricate cues in dialogue: Joint graph structure and sentiment dynamics for multimodal emotion recognition,” arXiv preprint arXiv:2407.21536 , 2024

    2024 arXiv

  44. [52]

    Der-gcn: Dialog and event relation-aware graph convolutional neural network for multimodal dialog emotion recognition,

    W. Ai, Y . Shou, T. Meng, and K. Li, “Der-gcn: Dialog and event relation-aware graph convolutional neural network for multimodal dialog emotion recognition,” IEEE Transactions on Neural Networks and Learning Systems, 2024

    2024

  45. [53]

    Supervised adversarial contrastive learning for emotion recognition in conversations,

    D. Hu, Y . Bao, L. Wei, W. Zhou, and S. Hu, “Supervised adversarial contrastive learning for emotion recognition in conversations,” arXiv preprint arXiv:2306.01505, 2023. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. XX, NO. X, MONTH 2025 13 VI. B IOGRAPHY SE...

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.