SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 108 papers

TitleStatusHype
MultiNews: A Web collection of an Aligned Multimodal and Multilingual Corpus—0
NICT-NAIST System for WMT17 Multimodal Translation Task—0
On Leveraging the Visual Modality for Neural Machine Translation—0
On Vision Features in Multimodal Machine Translation—0
OSU Multimodal Machine Translation System Report—0
Probing Representations Learned by Multimodal Recurrent and Transformer Models—0
Probing the Need for Visual Context in Multimodal Machine Translation—0
Rakuten’s Participation in WAT 2021: Examining the Effectiveness of Pre-trained Models for Multilingual and Multimodal Machine Translation—0
Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation—0
Sheffield Submissions for WMT18 Multimodal Translation Shared Task—0
SHEF-Multimodal: Grounding Machine Translation on Images—0
Supervised Visual Attention for Simultaneous Multimodal Machine Translation—0
The AFRL-Ohio State WMT18 Multimodal System: Combining Visual with Traditional—0
The AFRL-OSU WMT17 Multimodal Translation System: An Image Processing Approach—0
The Case for Evaluating Multimodal Translation Models on Text Datasets—0
The MeMAD Submission to the WMT18 Multimodal Translation Task—0
TMU Japanese-English Multimodal Machine Translation System for WAT 2020—0
Understanding the Effect of Textual Adversaries in Multimodal Machine Translation—0
Transformer-based Cascaded Multimodal Speech Translation—0
WMT 2016 Multimodal Translation System Description based on Bidirectional Recurrent Neural Networks with Double-Embeddings—0
A Dataset and Reranking Method for Multimodal MT of User-Generated Image Captions—0
Adding Multimodal Capabilities to a Text-only Translation Model—0
Adversarial Evaluation of Multimodal Machine Translation—0
A Shared Task on Multimodal Machine Translation and Crosslingual Image Description—0
A Survey of Vision-Language Pre-training from the Lens of Multimodal Machine Translation—0
A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking—0
CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation—0
CUNI System for the WMT18 Multimodal Translation Task—0
CUNI System for WMT16 Automatic Post-Editing and Multimodal Translation Tasks—0
DCU-UvA Multimodal MT System Report—0
Debiasing Word Embeddings Improves Multimodal Machine Translation—0
Detecting Concrete Visual Tokens for Multimodal Machine Translation—0
Doubly-Attentive Decoder for Multi-modal Neural Machine Translation—0
Doubly Attentive Transformer Machine Translation—0
Adaptive Fusion Techniques for Multimodal Data—0
Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding—0
EMMeTT: Efficient Multimodal Machine Translation Training—0
Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report—0
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation—0
Experiences of Adapting Multimodal Machine Translation Techniques for Hindi—0
Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasets—0
Findings of the 2016 Conference on Machine Translation—0
Findings of the 2017 Conference on Machine Translation (WMT17)—0
Findings of the 2018 Conference on Machine Translation (WMT18)—0
Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description—0
Florenz: Scaling Laws for Systematic Generalization in Vision-Language Models—0
Generalization algorithm of multimodal pre-training model based on graph-text self-supervised training—0
Generating Image Descriptions using Multilingual Data—0
Generative Imagination Elevates Machine Translation—0
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation—0
Show:102550
← PrevPage 2 of 3Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified