SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 108 papers

TitleStatusHype
ViTA: Visual-Linguistic Translation by Aligning Object TagsCode0
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation—0
Gumbel-Attention for Multi-modal Machine Translation—0
Good for Misconceived Reasons: Revisiting Neural Multimodal Machine Translation—0
Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding—0
MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish—0
TMU Japanese-English Multimodal Machine Translation System for WAT 2020—0
Generative Imagination Elevates Machine Translation—0
A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking—0
Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead Approach—0
Multimodal Machine Translation through Visuals and Speech—0
Adaptive Fusion Techniques for Multimodal Data—0
Understanding the Effect of Textual Adversaries in Multimodal Machine Translation—0
Transformer-based Cascaded Multimodal Speech Translation—0
On Leveraging the Visual Modality for Neural Machine Translation—0
Probing Representations Learned by Multimodal Recurrent and Transformer Models—0
Multilingual Multimodal Machine Translation for Dravidian Languages utilizing Phonetic Transcription—0
Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation—0
Distilling Translations with Visual AwarenessCode0
Grounded Word Sense Translation—0
Debiasing Word Embeddings Improves Multimodal Machine Translation—0
Multimodal Machine Translation with Embedding PredictionCode0
Probing the Need for Visual Context in Multimodal Machine Translation—0
Latent Variable Model for Multi-modal TranslationCode0
UMONS Submission for WMT18 Multimodal Translation TaskCode0
Input Combination Strategies for Multi-Source Transformer Decoder—0
Adversarial Evaluation of Multimodal Machine Translation—0
CUNI System for the WMT18 Multimodal Translation Task—0
Findings of the 2018 Conference on Machine Translation (WMT18)—0
Findings of the Third Shared Task on Multimodal Machine TranslationCode0
Sheffield Submissions for WMT18 Multimodal Translation Shared Task—0
The AFRL-Ohio State WMT18 Multimodal System: Combining Visual with Traditional—0
The MeMAD Submission to the WMT18 Multimodal Translation Task—0
Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report—0
A Visual Attention Grounding Neural Model for Multimodal Machine TranslationCode0
Doubly Attentive Transformer Machine Translation—0
Multimodal Machine Translation with Reinforcement Learning—0
Multimodal Lexical TranslationCode0
A Dataset and Reranking Method for Multimodal MT of User-Generated Image Captions—0
MultiNews: A Web collection of an Aligned Multimodal and Multilingual Corpus—0
Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description—0
OSU Multimodal Machine Translation System Report—0
Findings of the 2017 Conference on Machine Translation (WMT17)—0
Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation—0
Generating Image Descriptions using Multilingual Data—0
The AFRL-OSU WMT17 Multimodal Translation System: An Image Processing Approach—0
NICT-NAIST System for WMT17 Multimodal Translation Task—0
NMTPY: A Flexible Toolkit for Advanced Neural Machine Translation SystemsCode0
Doubly-Attentive Decoder for Multi-modal Neural Machine Translation—0
Incorporating Global Visual Features into Attention-Based Neural Machine Translation—0
Show:102550
← PrevPage 2 of 3Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified