SOTAVerified

Visual Dialog

Visual Dialog requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a follow-up question about the image, the task is to answer the question.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 118 papers

TitleStatusHype
Mini-Gemini: Mining the Potential of Multi-modality Vision Language ModelsCode7
Hawk: Learning to Understand Open-World Video AnomaliesCode3
Unified Multimodal Model with Unlikelihood Training for Visual DialogCode1
Video Dialog as Conversation about Objects Living in Space-TimeCode1
VD-PCR: Improving Visual Dialog with Pronoun Coreference ResolutionCode1
The Dialog Must Go On: Improving Visual Dialog via Generative Self-TrainingCode1
Ensemble of MRR and NDCG models for Visual DialogCode1
Where Are You? Localization from Embodied DialogCode1
History for Visual Dialog: Do we really need it?Code1
Multi-View Attention Network for Visual DialogCode1
VD-BERT: A Unified Vision and Dialog Transformer with BERTCode1
Reasoning Visual Dialog with Sparse Graph Learning and Knowledge TransferCode1
Iterative Context-Aware Graph Inference for Visual DialogCode1
Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art BaselineCode1
An Annotated Corpus of Reference Resolution for Interpreting Common GroundingCode1
Visual Dialogue State Tracking for Question GenerationCode1
Large-Scale Answerer in Questioner's Mind for Visual Dialog Question GenerationCode1
Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7Code1
Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual DialogCode1
Learning Cooperative Visual Dialog Agents with Deep Reinforcement LearningCode1
Visual DialogCode1
Hierarchical Question-Image Co-Attention for Visual Question AnsweringCode1
V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts—0
V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts—0
Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversations—0
ICCV23 Visual-Dialog Emotion Explanation Challenge: SEU_309 Team Technical Report—0
FlexCap: Describe Anything in Images in Controllable Detail—0
VD-GR: Boosting Visual Dialog with Cascaded Spatial-Temporal Multi-Modal GRaphs—0
Collecting Visually-Grounded Dialogue with A Game Of SortsCode0
Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations—0
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional ExpertsCode0
A survey on knowledge-enhanced multimodal learning—0
Knowledge Transfer with Visual Prompt in multi-modal Dialogue Understanding and Generation—0
LAVIS: A Library for Language-Vision IntelligenceCode0
Adversarial Robustness of Visual Dialog—0
ENRICH4ALL: A First Luxembourgish BERT Model for a Multilingual Chatbot—0
UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog—0
Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning—0
Reasoning with Multi-Structure Commonsense Knowledge in Visual Dialog—0
Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable SceneCode0
Modeling Coreference Relations in Visual Dialog—0
VU-BERT: A Unified framework for Visual Dialog—0
Discourse Analysis for Evaluating Coherence in Video Paragraph Captions—0
How to Fool Systems and Humans in Visually Grounded Interaction: A Case Study on Adversarial Attacks on Visual Dialog—0
UNITER-Based Situated Coreference Resolution with Rich Multimodal InputCode0
Region under Discussion for visual dialog—0
Enriching Language Models with Visually-grounded Word Vectors and the Lancaster Sensorimotor Norms—0
Perceptual Score: What Data Modalities Does Your Model Perceive?Code0
ViDA-MAN: Visual Dialog with Digital Humans—0
Evaluating and Improving Interactions with Hazy Oracles—0
Show:102550
← PrevPage 1 of 3Next →
All datasetsVisual Dialog v1.0 test-stdVisDial v0.9 valVisDial v1.0 test-stdBlendedSkillTalkConvAI2EmpatheticDialoguesImage-ChatWizard of Wikipedia

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1SingleNDCG (x 100)78.7—Unverified
2P1P2+Distill+EnsembleNDCG (x 100)77.92—Unverified
3Ensemble + Fine-tuningNDCG (x 100)76.43—Unverified
4ensemble, finetuneNDCG (x 100)76.17—Unverified
5VD-PCRNDCG (x 100)76.14—Unverified
6EnsembleNDCG (x 100)75.35—Unverified
7Ensemble + FinetuneNDCG (x 100)74.88—Unverified
8bert-double-stream-finetuningNDCG (x 100)74.62—Unverified
9CE-finetuned, single modelNDCG (x 100)74.47—Unverified
102NDCG (x 100)73.36—Unverified
#ModelMetricClaimedVerifiedStatus
19xFGA (VGG)MRR68.92—Unverified
2DANMRR66.38—Unverified
3CorefNMN (ResNet-152)MRR64.1—Unverified
4CoAttMRR63.98—Unverified
5CorefNMNMRR63.6—Unverified
6DualVDMRR62.94—Unverified
7SF-QIH-se-2MRR62.42—Unverified
8HCIAE-NP-ATTMRR62.22—Unverified
9HieCoAtt-QIMRR57.88—Unverified
10AMEMR@148.53—Unverified
#ModelMetricClaimedVerifiedStatus
15xFGA + LSNDCG64.04—Unverified
25xFGA + LS*+MRR0.71—Unverified
3Two-StepMRR0.7—Unverified
#ModelMetricClaimedVerifiedStatus
1Multi-Modal BlenderBotBLEU-41—Unverified
#ModelMetricClaimedVerifiedStatus
1Multi-Modal BlenderBotBLEU-41.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Multi-Modal BlenderBotBLEU-41.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Multi-Modal BlenderBotBLEU-440—Unverified
#ModelMetricClaimedVerifiedStatus
1Multi-Modal BlenderBotBLEU-42.2—Unverified