Visual Dialog

Visual Dialog requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a follow-up question about the image, the task is to answer the question.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 11–20 of 118 papers

Title	Date	Tasks	Status	Hype
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts	May 24, 2023	Dialogue State TrackingImage Retrieval	CodeCode Available	0
Unified Multimodal Model with Unlikelihood Training for Visual Dialog	Nov 23, 2022	Answer GenerationChatbot	CodeCode Available	1
A survey on knowledge-enhanced multimodal learning	Nov 19, 2022	Conditional Image GenerationFactual Visual Question Answering	—Unverified	0
Knowledge Transfer with Visual Prompt in multi-modal Dialogue Understanding and Generation	Oct 1, 2022	Dialogue UnderstandingKnowledge Distillation	—Unverified	0
LAVIS: A Library for Language-Vision Intelligence	Sep 15, 2022	BenchmarkingImage Captioning	CodeCode Available	0
Video Dialog as Conversation about Objects Living in Space-Time	Jul 8, 2022	ObjectRelational Reasoning	CodeCode Available	1
Adversarial Robustness of Visual Dialog	Jul 6, 2022	Adversarial RobustnessVisual Dialog	—Unverified	0
ENRICH4ALL: A First Luxembourgish BERT Model for a Multilingual Chatbot	Jun 1, 2022	ChatbotLanguage Modeling	—Unverified	0
VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolution	May 29, 2022	AI Agentcoreference-resolution	CodeCode Available	1
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training	May 25, 2022	Conditional Text GenerationOut-of-Distribution Detection	CodeCode Available	1

Show:10 25 50

← PrevPage 2 of 12Next →

All datasets Visual Dialog v1.0 test-std VisDial v0.9 val VisDial v1.0 test-std BlendedSkillTalk ConvAI2 EmpatheticDialogues Image-Chat Wizard of Wikipedia

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Single	NDCG (x 100)	78.7	—	Unverified
2	P1P2+Distill+Ensemble	NDCG (x 100)	77.92	—	Unverified
3	Ensemble + Fine-tuning	NDCG (x 100)	76.43	—	Unverified
4	ensemble, finetune	NDCG (x 100)	76.17	—	Unverified
5	VD-PCR	NDCG (x 100)	76.14	—	Unverified
6	Ensemble	NDCG (x 100)	75.35	—	Unverified
7	Ensemble + Finetune	NDCG (x 100)	74.88	—	Unverified
8	bert-double-stream-finetuning	NDCG (x 100)	74.62	—	Unverified
9	CE-finetuned, single model	NDCG (x 100)	74.47	—	Unverified
10	2	NDCG (x 100)	73.36	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	9xFGA (VGG)	MRR	68.92	—	Unverified
2	DAN	MRR	66.38	—	Unverified
3	CorefNMN (ResNet-152)	MRR	64.1	—	Unverified
4	CoAtt	MRR	63.98	—	Unverified
5	CorefNMN	MRR	63.6	—	Unverified
6	DualVD	MRR	62.94	—	Unverified
7	SF-QIH-se-2	MRR	62.42	—	Unverified
8	HCIAE-NP-ATT	MRR	62.22	—	Unverified
9	HieCoAtt-QI	MRR	57.88	—	Unverified
10	AMEM	R@1	48.53	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	5xFGA + LS	NDCG	64.04	—	Unverified
2	5xFGA + LS*+	MRR	0.71	—	Unverified
3	Two-Step	MRR	0.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	40	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	2.2	—	Unverified