SOTAVerified|Agents Browse Leaderboard About

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 31–40 of 108 papers

Title	Date	Tasks	Status	Hype	Score
Multi30K: Multilingual English-German Image Descriptions	May 2, 2016	Image DescriptionMachine Translation	CodeCode Available	0	5
A Visual Attention Grounding Neural Model for Multimodal Machine Translation	Aug 24, 2018	Machine TranslationMultimodal Machine Translation	CodeCode Available	0	5
Multimodal Lexical Translation	May 1, 2018	Machine TranslationMultimodal Lexical Translation	CodeCode Available	0	5
HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Language	May 28, 2023	Machine TranslationMultimodal Machine Translation	CodeCode Available	0	5
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs	Oct 26, 2023	AttributeMachine Translation	CodeCode Available	0	5
Findings of the Third Shared Task on Multimodal Machine Translation	Oct 1, 2018	Machine TranslationMultimodal Machine Translation	CodeCode Available	0	5
Cultural and Geographical Influences on Image Translatability of Words across Languages	Jun 1, 2021	Cultural Vocal Bursts Intensity PredictionLow Resource Neural Machine Translation	CodeCode Available	0	5
Latent Variable Model for Multi-modal Translation	Nov 1, 2018	DecoderMachine Translation	CodeCode Available	0	5
Multimodal Machine Translation with Embedding Prediction	Apr 1, 2019	Machine TranslationMultimodal Machine Translation	CodeCode Available	0	5
Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding	Dec 18, 2020	Machine TranslationMultimodal Machine Translation	—Unverified	0	0

Show:10 25 50

← PrevPage 4 of 11Next →

All datasets Multi30K Hindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	del	Meteor (EN-FR)	74.6	—	Unverified
2	ERNIE-UniX2	BLEU (EN-DE)	49.3	—	Unverified
3	IKD-MMT	BLEU (EN-DE)	41.28	—	Unverified
4	DCCN	BLEU (EN-DE)	39.7	—	Unverified
5	Caglayan	BLEU (EN-DE)	39.4	—	Unverified
6	Gumbel-Attention MMT	BLEU (EN-DE)	39.2	—	Unverified
7	Multimodal Transformer	BLEU (EN-DE)	38.7	—	Unverified
8	ImagiT	BLEU (EN-DE)	38.4	—	Unverified
9	del+obj	BLEU (EN-DE)	38	—	Unverified
10	VMMTF	BLEU (EN-DE)	37.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ViTA	BLEU (EN-HI)	51.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ViTA	BLEU (EN-HI)	44.6	—	Unverified