Pith. sign in

REVIEW 4 cited by

AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.07653 v1 pith:CCNAEKEI submitted 2024-07-10 cs.HC

Zheng Lian , Haiyang Sun , Licai Sun , Jiangyan Yi , Bin Liu , Jianhua Tao This is my paper · ORCID
classification cs.HC
keywords datasetaffectgptannotationemotionmultimodalrecognitioncostemer
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Explainable Multimodal Emotion Recognition (EMER) is an emerging task that aims to achieve reliable and accurate emotion recognition. However, due to the high annotation cost, the existing dataset (denoted as EMER-Fine) is small, making it difficult to perform supervised training. To reduce the annotation cost and expand the dataset size, this paper reviews the previous dataset construction process. Then, we simplify the annotation pipeline, avoid manual checks, and replace the closed-source models with open-source models. Finally, we build \textbf{EMER-Coarse}, a coarsely-labeled dataset containing large-scale samples. Besides the dataset, we propose a two-stage training framework \textbf{AffectGPT}. The first stage exploits EMER-Coarse to learn a coarse mapping between multimodal inputs and emotion-related descriptions; the second stage uses EMER-Fine to better align with manually-checked results. Experimental results demonstrate the effectiveness of our proposed method on the challenging EMER task. To facilitate further research, we will make the code and dataset available at: https://github.com/zeroQiaoba/AffectGPT.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. InCarEmo: A Multimodal Dataset for In-Cabin Emotion Recognition and Driver State Monitoring

    cs.AI 2026-07 conditional novelty 7.0 of 10

    A new in-cabin dataset pairs RGB/IR video, audio, and Chinese dialogue text with emotion, fatigue, and distraction labels, and baselines show fusion beats single modalities on the Chinese partition.

  2. DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    Schema-constrained MLLM reports plus SICS/DMC modules and the T4-Deception dataset yield SOTA in-domain, cross-domain, and cross-cultural multimodal deception detection.

  3. HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Requiring omni-modal models to summarize context before reasoning, with LLM-judged context and logical rewards, improves human-intent reasoning benchmarks.

  4. Advancing the Foundation Model for Music Understanding

    cs.SD 2025-08 unverdicted novelty 5.0 of 10

    MuFun is proposed as a unified music foundation model that jointly handles instrumental and lyrical content, and it is claimed to outperform existing audio language models on the authors' new MuCUE benchmark.

Pith tools