SOTAVerified

Multimodal Emotion Recognition

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 180 papers

TitleStatusHype
A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition—0
GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition—0
Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution ShiftsCode1
Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition—0
TACFN: Transformer-based Adaptive Cross-modal Fusion Network for Multimodal Emotion RecognitionCode0
PsyCounAssist: A Full-Cycle AI-Powered Psychological Counseling Assistant System—0
Leveraging Label Potential for Enhanced Multimodal Emotion Recognition—0
BeMERC: Behavior-Aware MLLM-based Framework for Multimodal Emotion Recognition in Conversation—0
Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion—0
GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations—0
Feature-Based Dual Visual Feature Extraction Model for Compound Multimodal Emotion RecognitionCode0
United we stand, Divided we fall: Handling Weak Complementary Relationships for Audio-Visual Emotion Recognition in Valence-Arousal Space—0
Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts—0
R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement LearningCode5
Latent Distribution Decoupling: A Probabilistic Framework for Uncertainty-Aware Multimodal Emotion RecognitionCode1
A Novel Approach to for Multimodal Emotion Recognition : Multimodal semantic information fusion—0
Milmer: a Framework for Multiple Instance Learning based Multimodal Emotion RecognitionCode1
Cross-modal Context Fusion and Adaptive Graph Convolutional Network for Multimodal Conversational Emotion Recognition—0
EmoTech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information with Hybrid Recurrent Network—0
LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations—0
MVP: Multimodal Emotion Recognition based on Video and Physiological Signals—0
EMOE: Modality-Specific Enhanced Dynamic Emotion Experts—0
Enhancing Multimodal Emotion Recognition through Multi-Granularity Cross-Modal Alignment—0
A Multimodal Emotion Recognition System: Integrating Facial Expressions, Body Movement, Speech, and Spoken Language—0
EmoVerse: Exploring Multimodal Large Language Models for Sentiment and Emotion UnderstandingCode1
Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation—0
SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion RecognitionCode1
CMATH: Cross-Modality Augmented Transformer with Hierarchical Variational Distillation for Multimodal Emotion Recognition in Conversation—0
Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviors—0
PHemoNet: A Multimodal Network for Physiological SignalsCode2
Hierarchical Hypercomplex Network for Multimodal Emotion RecognitionCode2
Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better—0
Recent Trends of Multimodal Affective Computing: A Survey from NLP PerspectiveCode2
Multimodal Emotion Recognition with Vision-language Prompting and Modality Dropout—0
Leveraging Contrastive Learning and Self-Training for Multimodal Emotion Recognition with Limited Labeled SamplesCode0
Video Emotion Open-vocabulary Recognition Based on Multimodal Large Language Model—0
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion RecognitionCode4
Enhancing Modal Fusion by Alignment and Label Matching for Multimodal Emotion RecognitionCode1
Multi Teacher Privileged Knowledge Distillation for Multimodal Expression RecognitionCode0
Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion RecognitionCode1
Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross AttentionCode1
MicroEmo: Time-Sensitive Multimodal Emotion Recognition with Micro-Expression Dynamics in Video Dialogues—0
Masked Graph Learning with Recurrent Alignment for Multimodal Emotion Recognition in Conversation—0
Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the WildCode0
Enhancing Emotion Recognition in Incomplete Data: A Novel Cross-Modal Alignment, Reconstruction, and Refinement Framework—0
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction TuningCode4
Multimodal Emotion Recognition based on Facial Expressions, Speech, and EEG—0
Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition—0
Empathy Through Multimodality in Conversational Interfaces—0
MultiMAE-DER: Multimodal Masked Autoencoder for Dynamic Emotion RecognitionCode1
Show:102550
← PrevPage 1 of 4Next →
All datasetsIEMOCAP-4MELDIEMOCAPCMU-MOSEI-SentimentCMU-MOSEI-Sentiment-3Expressive hands and faces dataset (EHF).MELD-Sentiment

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F186.52—Unverified
2JoyfulWeighted F185.7—Unverified
3COGMENWeighted F184.5—Unverified
4DANNAccuracy82.7—Unverified
5MMERAccuracy81.7—Unverified
6PATHOSnet v2Accuracy80.4—Unverified
7Self-attention weight correction (A+T)Accuracy76.8—Unverified
8CHFusionAccuracy76.5—Unverified
9bc-LSTMWeighted F174.1—Unverified
10Audio + Text (Stage III)F170.5—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F166.71—Unverified
2Audio + Text (Stage III)Weighted F165.8—Unverified
3JoyfulWeighted F161.77—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F172.81—Unverified
2JoyfulWeighted F170.5—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F144.93—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F166.73—Unverified
#ModelMetricClaimedVerifiedStatus
1SMPLify-Xv2v error52.9—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F174.31—Unverified