Multimodal Reasoning

Reasoning over multimodal inputs.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 276–300 of 302 papers

Title	Date	Tasks	Status	Hype
Deep Neural Networks for Visual Reasoning	Sep 24, 2022	Multimodal ReasoningVisual Reasoning	—Unverified	0
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering	Sep 20, 2022	Multimodal Deep LearningMultimodal Reasoning	CodeCode Available	2
Reducing the Vision and Language Bias for Temporal Sentence Grounding	Jul 27, 2022	Information RetrievalMultimodal Reasoning	—Unverified	0
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation	May 25, 2022	Multimodal ReasoningOptical Character Recognition (OCR)	—Unverified	0
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language	Apr 1, 2022	DiversityImage Captioning	CodeCode Available	0
Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?	Mar 31, 2022	Fine-Grained Visual RecognitionMultimodal Reasoning	CodeCode Available	0
Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding	Mar 29, 2022	Multimodal ReasoningVisual Grounding	CodeCode Available	1
Fine-Grained Visual Entailment	Mar 29, 2022	Multimodal ReasoningVisual Entailment	CodeCode Available	1
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning	Mar 21, 2022	Common Sense ReasoningMultimodal Reasoning	CodeCode Available	1
Learning from Inside: Self-driven Siamese Sampling and Reasoning for Video Question Answering	Dec 1, 2021	Multimodal ReasoningQuestion Answering	—Unverified	0
Improving Pre-trained Vision-and-Language Embeddings for Phrase Grounding	Nov 1, 2021	Multimodal ReasoningPhrase Grounding	—Unverified	0
TxT: Crossmodal End-to-End Learning with Transformers	Sep 9, 2021	Multimodal ReasoningQuestion Answering	—Unverified	0
WebQA: Multihop and Multimodal QA	Sep 1, 2021	Image RetrievalMultimodal Reasoning	CodeCode Available	1
Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision	Aug 12, 2021	3D geometryDescriptive	CodeCode Available	1
C^3: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues	Jun 16, 2021	Contrastive Learningcounterfactual	—Unverified	0
MERLOT: Multimodal Neural Script Knowledge Models	Jun 4, 2021	Multimodal ReasoningVisual Commonsense Reasoning	CodeCode Available	1
Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues	May 15, 2021	Multimodal ReasoningNatural Language Inference	—Unverified	0
Visual Goal-Step Inference using wikiHow	Apr 12, 2021	Multimodal ReasoningVGSI	CodeCode Available	0
UniT: Multimodal Multitask Learning with a Unified Transformer	Feb 22, 2021	DecoderMultimodal Reasoning	CodeCode Available	0
DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents	Jan 28, 2021	Document SummarizationMultimodal Reasoning	—Unverified	0
A Multimodal Framework for the Detection of Hateful Memes	Dec 23, 2020	Ensemble LearningMultimodal Reasoning	CodeCode Available	1
Sound2Sight: Generating Visual Dynamics from Sound and Context	Jul 23, 2020	Multimodal ReasoningVideo Forecasting	—Unverified	0
e-SNLI-VE: Corrected Visual-Textual Entailment with Natural Language Explanations	Apr 7, 2020	Multimodal ReasoningNatural Language Inference	CodeCode Available	1
DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog	Dec 18, 2019	AI AgentDecoder	CodeCode Available	0
Multimodal Transformer with Multi-View Visual Representation for Image Captioning	May 20, 2019	DecoderImage Captioning	—Unverified	0

Show:10 25 50

← PrevPage 12 of 13Next →

All datasets REBUS MATH-V AlgoPuzzleVQA

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GPT-4V	Accuracy	24	—	Unverified
2	Gemini Pro	Accuracy	13.2	—	Unverified
3	LLaVa-1.5-13B	Accuracy	1.8	—	Unverified
4	LLaVa-1.5-7B	Accuracy	1.5	—	Unverified
5	BLIP2-FLAN-T5-XXL	Accuracy	0.9	—	Unverified
6	QWEN	Accuracy	0.9	—	Unverified
7	CogVLM	Accuracy	0.9	—	Unverified
8	InstructBLIP	Accuracy	0.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT4V	Accuracy	22.76	—	Unverified
2	Gemini Pro	Accuracy	17.66	—	Unverified
3	Qwen-VL-Max	Accuracy	15.59	—	Unverified
4	InternLM-XComposer2-VL	Accuracy	14.54	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT-4	Acc	30.3	—	Unverified