Visual Dialog

Visual Dialog requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a follow-up question about the image, the task is to answer the question.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 51–75 of 118 papers

Title	Date	Tasks	Status	Score
TAB-VCR: Tags and Attributes based Visual Commonsense Reasoning Baselines	Oct 31, 2019	AttributeQuestion Answering	CodeCode Available	5
TAB-VCR: Tags and Attributes based VCR Baselines	Dec 1, 2019	AttributeQuestion Answering	CodeCode Available	5
Ask No More: Deciding when to guess in referential visual dialogue	May 17, 2018	Decision MakingVisual Dialog	CodeCode Available	5
DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog	Dec 18, 2019	AI AgentDecoder	CodeCode Available	5
Dual Attention Networks for Visual Reference Resolution in Visual Dialog	Feb 25, 2019	AI AgentQuestion Answering	CodeCode Available	5
DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue	Nov 17, 2019	feature selectionQuestion Answering	CodeCode Available	5
Visual Reference Resolution using Attention Memory for Visual Dialog	Sep 23, 2017	Parameter PredictionQuestion Answering	—Unverified	0
Visual-Textual Alignment for Graph Inference in Visual Dialog	Dec 1, 2020	Visual Dialog	—Unverified	0
VU-BERT: A Unified framework for Visual Dialog	Feb 22, 2022	Language ModelingLanguage Modelling	—Unverified	0
What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.	Jul 1, 2019	Reinforcement LearningVisual Dialog	—Unverified	0
What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialog	Jul 28, 2019	Reinforcement LearningVisual Dialog	—Unverified	0
What's to know? Uncertainty as a Guide to Asking Goal-oriented Questions	Dec 16, 2018	Visual Dialog	—Unverified	0
Video Dialog via Progressive Inference and Cross-Transformer	Nov 1, 2019	Answer GenerationQuestion Answering	—Unverified	0
Adversarial Robustness of Visual Dialog	Jul 6, 2022	Adversarial RobustnessVisual Dialog	—Unverified	0
Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations	Aug 30, 2023	Explanation GenerationQuestion Answering	—Unverified	0
A Generative Adversarial Density Estimator	Jun 1, 2019	Density EstimationVisual Dialog	—Unverified	0
Gold Seeker: Information Gain from Policy Distributions for Goal-oriented Vision-and-Langauge Reasoning	Dec 16, 2018	Reinforcement LearningVisual Dialog	—Unverified	0
Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning	Nov 21, 2017	Question AnsweringReinforcement Learning	—Unverified	0
A survey on knowledge-enhanced multimodal learning	Nov 19, 2022	Conditional Image GenerationFactual Visual Question Answering	—Unverified	0
Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation	Sep 6, 2019	DecoderReinforcement Learning	—Unverified	0
Evaluating and Improving Interactions with Hazy Oracles	Oct 19, 2021	Object TrackingReferring Expression	—Unverified	0
Connecting Language and Vision to Actions	Jul 1, 2018	Image CaptioningLanguage Modeling	—Unverified	0
Discourse Analysis for Evaluating Coherence in Video Paragraph Captions	Jan 17, 2022	Video CaptioningVisual Dialog	—Unverified	0
Effective questions in referential visual dialogue	Jul 1, 2020	Visual Dialog	—Unverified	0
Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversations	Aug 13, 2024	dialog state trackingDialogue State Tracking	—Unverified	0

Show:10 25 50

← PrevPage 3 of 5Next →

All datasets Visual Dialog v1.0 test-std VisDial v0.9 val VisDial v1.0 test-std BlendedSkillTalk ConvAI2 EmpatheticDialogues Image-Chat Wizard of Wikipedia

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Single	NDCG (x 100)	78.7	—	Unverified
2	P1P2+Distill+Ensemble	NDCG (x 100)	77.92	—	Unverified
3	Ensemble + Fine-tuning	NDCG (x 100)	76.43	—	Unverified
4	ensemble, finetune	NDCG (x 100)	76.17	—	Unverified
5	VD-PCR	NDCG (x 100)	76.14	—	Unverified
6	Ensemble	NDCG (x 100)	75.35	—	Unverified
7	Ensemble + Finetune	NDCG (x 100)	74.88	—	Unverified
8	bert-double-stream-finetuning	NDCG (x 100)	74.62	—	Unverified
9	CE-finetuned, single model	NDCG (x 100)	74.47	—	Unverified
10	2	NDCG (x 100)	73.36	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	9xFGA (VGG)	MRR	68.92	—	Unverified
2	DAN	MRR	66.38	—	Unverified
3	CorefNMN (ResNet-152)	MRR	64.1	—	Unverified
4	CoAtt	MRR	63.98	—	Unverified
5	CorefNMN	MRR	63.6	—	Unverified
6	DualVD	MRR	62.94	—	Unverified
7	SF-QIH-se-2	MRR	62.42	—	Unverified
8	HCIAE-NP-ATT	MRR	62.22	—	Unverified
9	HieCoAtt-QI	MRR	57.88	—	Unverified
10	AMEM	R@1	48.53	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	5xFGA + LS	NDCG	64.04	—	Unverified
2	5xFGA + LS*+	MRR	0.71	—	Unverified
3	Two-Step	MRR	0.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	1.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	40	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Multi-Modal BlenderBot	BLEU-4	2.2	—	Unverified