SOTAVerified

Multimodal Emotion Recognition

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 101–150 of 180 papers

TitleStatusHype
Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data—0
Cross-modal Context Fusion and Adaptive Graph Convolutional Network for Multimodal Conversational Emotion Recognition—0
cross-modal fusion techniques for utterance-level emotion recognition from text and speech—0
CSAT‑FTCN: A Fuzzy‑Oriented Model with Contextual Self‑attention Network for Multimodal Emotion Recognition—0
Deep CNN with late fusion for realtime multimodal emotion recognition—0
Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations—0
DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognition—0
Do Multimodal Emotion Recognition Models Tackle Ambiguity?—0
Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation—0
Dynamic Modality and View Selection for Multimodal Emotion Recognition with Missing Modalities—0
Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better—0
EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge—0
EMOE: Modality-Specific Enhanced Dynamic Emotion Experts—0
EmoTech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information with Hybrid Recurrent Network—0
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle—0
Emotion recognition based on multi-modal electrophysiology multi-head attention Contrastive Learning—0
Emotion recognition by fusing time synchronous and time asynchronous representations—0
Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals—0
Empathy Through Multimodality in Conversational Interfaces—0
Enhancing Emotion Recognition in Incomplete Data: A Novel Cross-Modal Alignment, Reconstruction, and Refinement Framework—0
Enhancing Multimodal Emotion Recognition through Multi-Granularity Cross-Modal Alignment—0
Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus—0
FAF: A novel multimodal emotion recognition approach integrating face, body and text—0
Framewise approach in multimodal emotion recognition in OMG challenge—0
GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations—0
GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition—0
United we stand, Divided we fall: Handling Weak Complementary Relationships for Audio-Visual Emotion Recognition in Valence-Arousal Space—0
HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognition—0
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023—0
ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection—0
Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition—0
Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors—0
Interpretable Multimodal Emotion Recognition using Facial Features and Physiological Signals—0
Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning—0
Multimodal Emotion Recognition Using Multimodal Deep Learning—0
Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models—0
Multimodal Emotion Recognition with Vision-language Prompting and Modality Dropout—0
Multimodal End-to-End Group Emotion Recognition using Cross-Modal Attention—0
Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition—0
MVP: Multimodal Emotion Recognition based on Video and Physiological Signals—0
Noise-Resistant Multimodal Transformer for Emotion Recognition—0
Progressive Modality Reinforcement for Human Multimodal Emotion Recognition From Unaligned Multimodal Sequences—0
PsyCounAssist: A Full-Cycle AI-Powered Psychological Counseling Assistant System—0
Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition—0
Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum—0
Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring—0
Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviors—0
Speech Emotion Recognition Based on Self-Attention Weight Correction for Acoustic and Text Features—0
TACOformer:Token-channel compounded Cross Attention for Multimodal Emotion Recognition—0
Towards Multimodal Emotion Recognition in German Speech Events in Cars using Transfer Learning—0
Show:102550
← PrevPage 3 of 4Next →
All datasetsIEMOCAP-4MELDIEMOCAPCMU-MOSEI-SentimentCMU-MOSEI-Sentiment-3Expressive hands and faces dataset (EHF).MELD-Sentiment

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F186.52—Unverified
2JoyfulWeighted F185.7—Unverified
3COGMENWeighted F184.5—Unverified
4DANNAccuracy82.7—Unverified
5MMERAccuracy81.7—Unverified
6PATHOSnet v2Accuracy80.4—Unverified
7Self-attention weight correction (A+T)Accuracy76.8—Unverified
8CHFusionAccuracy76.5—Unverified
9bc-LSTMWeighted F174.1—Unverified
10Audio + Text (Stage III)F170.5—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F166.71—Unverified
2Audio + Text (Stage III)Weighted F165.8—Unverified
3JoyfulWeighted F161.77—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F172.81—Unverified
2JoyfulWeighted F170.5—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F144.93—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F166.73—Unverified
#ModelMetricClaimedVerifiedStatus
1SMPLify-Xv2v error52.9—Unverified
#ModelMetricClaimedVerifiedStatus
1GraphSmileWeighted F174.31—Unverified