Multimodal Emotion Recognition

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1–10 of 180 papers

Title	Date	Tasks	Status	Hype
R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning	Mar 7, 2025	Emotion RecognitionLanguage Modeling	CodeCode Available	5
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning	Jun 17, 2024	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	4
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition	Aug 20, 2024	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	4
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition	Apr 26, 2024	Emotion RecognitionMulti-Label Learning	CodeCode Available	3
Recent Trends of Multimodal Affective Computing: A Survey from NLP Perspective	Sep 11, 2024	Aspect-Based Sentiment AnalysisEmotion Recognition	CodeCode Available	2
MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild	Apr 13, 2024	cross-modal alignmentDynamic Facial Expression Recognition	CodeCode Available	2
Hierarchical Hypercomplex Network for Multimodal Emotion Recognition	Sep 13, 2024	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	2
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning	Apr 18, 2023	Emotion RecognitionMulti-Label Learning	CodeCode Available	2
PHemoNet: A Multimodal Network for Physiological Signals	Sep 13, 2024	Brain Computer InterfaceEEG	CodeCode Available	2
COGMEN: COntextualized GNN based Multimodal Emotion recognitioN	May 5, 2022	Emotion RecognitionEmotion Recognition in Conversation	CodeCode Available	1

Show:10 25 50

← PrevPage 1 of 18Next →

All datasets IEMOCAP-4 MELD IEMOCAP CMU-MOSEI-Sentiment CMU-MOSEI-Sentiment-3 Expressive hands and faces dataset (EHF).MELD-Sentiment

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	86.52	—	Unverified
2	Joyful	Weighted F1	85.7	—	Unverified
3	COGMEN	Weighted F1	84.5	—	Unverified
4	DANN	Accuracy	82.7	—	Unverified
5	MMER	Accuracy	81.7	—	Unverified
6	PATHOSnet v2	Accuracy	80.4	—	Unverified
7	Self-attention weight correction (A+T)	Accuracy	76.8	—	Unverified
8	CHFusion	Accuracy	76.5	—	Unverified
9	bc-LSTM	Weighted F1	74.1	—	Unverified
10	Audio + Text (Stage III)	F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.71	—	Unverified
2	Audio + Text (Stage III)	Weighted F1	65.8	—	Unverified
3	Joyful	Weighted F1	61.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	72.81	—	Unverified
2	Joyful	Weighted F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	44.93	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SMPLify-X	v2v error	52.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	74.31	—	Unverified