Multimodal Emotion Recognition

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 111–120 of 180 papers

Title	Date	Tasks	Status	Hype
Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Data	Aug 25, 2022	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	0
VISTANet: VIsual Spoken Textual Additive Net for Interpretable Multimodal Emotion Recognition	Aug 24, 2022	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	0
GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection	Jul 25, 2022	Emotion RecognitionEmotion Recognition in Conversation	CodeCode Available	1
Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss	Jul 23, 2022	Cultural Vocal Bursts Intensity PredictionEmotion Recognition	CodeCode Available	1
A Multibias-mitigated and Sentiment Knowledge Enriched Transformer for Debiasing in Multimodal Conversational Emotion Recognition	Jul 17, 2022	Emotion RecognitionMultimodal Emotion Recognition	—Unverified	0
Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognition	Jul 11, 2022	Emotion RecognitionMultimodal Emotion Recognition	CodeCode Available	0
GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition	Jul 6, 2022	Emotion ClassificationEmotion Recognition	CodeCode Available	1
0/1 Deep Neural Networks via Block Coordinate Descent	Jun 19, 2022	10-shot image generation	—Unverified	0
COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition	Jun 12, 2022	Emotion RecognitionMultimodal Emotion Recognition	—Unverified	0
Do Multimodal Emotion Recognition Models Tackle Ambiguity?	Jun 1, 2022	Emotion RecognitionMultimodal Emotion Recognition	—Unverified	0

Show:10 25 50

← PrevPage 12 of 18Next →

All datasets IEMOCAP-4 MELD IEMOCAP CMU-MOSEI-Sentiment CMU-MOSEI-Sentiment-3 Expressive hands and faces dataset (EHF).MELD-Sentiment

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	86.52	—	Unverified
2	Joyful	Weighted F1	85.7	—	Unverified
3	COGMEN	Weighted F1	84.5	—	Unverified
4	DANN	Accuracy	82.7	—	Unverified
5	MMER	Accuracy	81.7	—	Unverified
6	PATHOSnet v2	Accuracy	80.4	—	Unverified
7	Self-attention weight correction (A+T)	Accuracy	76.8	—	Unverified
8	CHFusion	Accuracy	76.5	—	Unverified
9	bc-LSTM	Weighted F1	74.1	—	Unverified
10	Audio + Text (Stage III)	F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.71	—	Unverified
2	Audio + Text (Stage III)	Weighted F1	65.8	—	Unverified
3	Joyful	Weighted F1	61.77	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	72.81	—	Unverified
2	Joyful	Weighted F1	70.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	44.93	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	66.73	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SMPLify-X	v2v error	52.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraphSmile	Weighted F1	74.31	—	Unverified