REVIEW 14 cited by
R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we present the first application of Reinforcement Learning with Verifiable Reward (RLVR) to an Omni-multimodal large language model in the context of emotion recognition, a task where both visual and audio modalities play crucial roles. We leverage RLVR to optimize the Omni model, significantly enhancing its performance in three key aspects: reasoning capability, emotion recognition accuracy, and generalization ability. The introduction of RLVR not only improves the model's overall performance on in-distribution data but also demonstrates superior robustness when evaluated on out-of-distribution datasets. More importantly, the improved reasoning capability enables clear analysis of the contributions of different modalities, particularly visual and audio information, in the emotion recognition process. This provides valuable insights into the optimization of multimodal large language models.
Forward citations
Cited by 14 Pith papers
-
InCarEmo: A Multimodal Dataset for In-Cabin Emotion Recognition and Driver State Monitoring
A new in-cabin dataset pairs RGB/IR video, audio, and Chinese dialogue text with emotion, fatigue, and distraction labels, and baselines show fusion beats single modalities on the Chinese partition.
-
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
A training-free, query-anchored, modality-decoupled token compression method outperforms unidirectional audio/video compression baselines on Qwen2.5-Omni while cutting prefill cost.
-
EmoAgent-R1: Towards Multimodal Emotion Understanding with Reinforcement Learning-based Dynamic Agent Specialization
EmoAgent-R1 combines dynamic agent routing with a token-reweighted GRPO variant (P-GRPO) to reach 77.85% mean on MER-UniBench, exceeding AffectGPT-R1 by 1.90 points.
-
TimeThink: Reasoning with Time for Video LLMs
TimeThink adds step-wise temporal process rewards (max IoU of referenced intervals) to GRPO for Video-LLMs, improving grounding and reasoning over outcome-only RL baselines.
-
Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captions
A new large-scale facial emotion caption dataset and a global-local contrastive training framework with positive mining improve zero-shot facial expression recognition.
-
LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs
A training-free token compression method using semantic connected components in space and time keeps video understanding accuracy high even when retaining only 5-10% of visual tokens.
-
HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context
Requiring omni-modal models to summarize context before reasoning, with LLM-judged context and logical rewards, improves human-intent reasoning benchmarks.
-
GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning
GRPO-CARE improves answer accuracy and reasoning coherence over standard GRPO on a new video reasoning benchmark, with a 6.7 point gain on the hardest level and a 24.5 point higher consistency rate.
-
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Rationale-privileged on-policy self-distillation reaches 84.19 mean on MER-UniBench by scoring student rollouts with a local teacher that alone sees frontier-generated multimodal evidence.
-
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
EMO-R3, which combines a three-step emotional reasoning prompt with a reward for the model agreeing with its own image–emotion judgments, raises visual emotion-recognition accuracy by about one point over plain GRPO.
-
Group Relative Policy Optimization for Speech Recognition
Applying GRPO with rule-based rewards to LLM-based ASR improves WER by up to 18.4% relative and reduces hallucination errors on unseen acoustic conditions.
-
Advancing the Foundation Model for Music Understanding
MuFun is proposed as a unified music foundation model that jointly handles instrumental and lyrical content, and it is claimed to outperform existing audio language models on the authors' new MuCUE benchmark.
-
From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
Self-reflection in LLMs can be steered up or down by a single activation-space vector, improving accuracy when amplified and cutting output length when suppressed.
-
Privacy-Preserving Approximate Nearest Neighbor Search on High-Dimensional Data
The submission's title and abstract describe a single-server privacy-preserving ANNS system with distance comparison encryption, but the manuscript body is an unrelated activity-recognition paper, leaving the advertis...
Discussion (0). Sign in to comment.