REVIEW 4 cited by
AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Explainable Multimodal Emotion Recognition (EMER) is an emerging task that aims to achieve reliable and accurate emotion recognition. However, due to the high annotation cost, the existing dataset (denoted as EMER-Fine) is small, making it difficult to perform supervised training. To reduce the annotation cost and expand the dataset size, this paper reviews the previous dataset construction process. Then, we simplify the annotation pipeline, avoid manual checks, and replace the closed-source models with open-source models. Finally, we build \textbf{EMER-Coarse}, a coarsely-labeled dataset containing large-scale samples. Besides the dataset, we propose a two-stage training framework \textbf{AffectGPT}. The first stage exploits EMER-Coarse to learn a coarse mapping between multimodal inputs and emotion-related descriptions; the second stage uses EMER-Fine to better align with manually-checked results. Experimental results demonstrate the effectiveness of our proposed method on the challenging EMER task. To facilitate further research, we will make the code and dataset available at: https://github.com/zeroQiaoba/AffectGPT.
Forward citations
Cited by 4 Pith papers
-
InCarEmo: A Multimodal Dataset for In-Cabin Emotion Recognition and Driver State Monitoring
A new in-cabin dataset pairs RGB/IR video, audio, and Chinese dialogue text with emotion, fatigue, and distraction labels, and baselines show fusion beats single modalities on the Chinese partition.
-
DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning
Schema-constrained MLLM reports plus SICS/DMC modules and the T4-Deception dataset yield SOTA in-domain, cross-domain, and cross-cultural multimodal deception detection.
-
HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context
Requiring omni-modal models to summarize context before reasoning, with LLM-judged context and logical rewards, improves human-intent reasoning benchmarks.
-
Advancing the Foundation Model for Music Understanding
MuFun is proposed as a unified music foundation model that jointly handles instrumental and lyrical content, and it is claimed to outperform existing audio language models on the authors' new MuCUE benchmark.
Discussion (0). Sign in to comment.