Multimodal Emotion Recognition

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 71–80 of 180 papers

Title	Date	Tasks	Status	Hype
Towards Emotion Analysis in Short-form Videos: A Large-Scale Dataset and Baseline	Nov 29, 2023	audio-visual learningForm	CodeCode Available	1
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition	Nov 18, 2023	Contrastive LearningEmotion Recognition	CodeCode Available	1
Accommodating Missing Modalities in Time-Continuous Multimodal Emotion Recognition	Nov 16, 2023	Emotion RecognitionMultimodal Emotion Recognition	—Unverified	0
Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction	Nov 8, 2023	Emotion RecognitionGraph Neural Network	CodeCode Available	1
A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations	Oct 31, 2023	Emotion RecognitionEmotion Recognition in Conversation	CodeCode Available	1
A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learning	Oct 24, 2023	Emotion ClassificationEmotion Recognition	—Unverified	0
Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals	Oct 11, 2023	EEGElectroencephalogram (EEG)	CodeCode Available	1
Learning Noise-Robust Joint Representation for Multimodal Emotion Recognition under Incomplete Data Scenarios	Sep 21, 2023	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	0
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023	Sep 11, 2023	Emotion ClassificationEmotion Recognition	—Unverified	0
Leveraging Label Information for Multimodal Emotion Recognition	Sep 5, 2023	Emotion ClassificationEmotion Recognition	—Unverified	0

Show:10 25 50

← PrevPage 8 of 18Next →

All datasets IEMOCAP-4 MELD IEMOCAP CMU-MOSEI-Sentiment CMU-MOSEI-Sentiment-3 Expressive hands and faces dataset (EHF).MELD-Sentiment

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	86.52	—	Unverified
2	Joyful	Weighted F1	85.7	—	Unverified
3	COGMEN	Weighted F1	84.5	—	Unverified
4	DANN	Accuracy	82.7	—	Unverified
5	MMER	Accuracy	81.7	—	Unverified
6	PATHOSnet v2	Accuracy	80.4	—	Unverified
7	Self-attention weight correction (A+T)	Accuracy	76.8	—	Unverified
8	CHFusion	Accuracy	76.5	—	Unverified
9	bc-LSTM	Weighted F1	74.1	—	Unverified
10	Audio + Text (Stage III)	F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.71	—	Unverified
2	Audio + Text (Stage III)	Weighted F1	65.8	—	Unverified
3	Joyful	Weighted F1	61.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	72.81	—	Unverified
2	Joyful	Weighted F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	44.93	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SMPLify-X	v2v error	52.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	74.31	—	Unverified