SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 26–50 of 108 papers

TitleStatusHype
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation—0
LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation—0
Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective—0
Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine TranslationCode1
VALHALLA: Visual Hallucination for Machine TranslationCode1
Neural Machine Translation with Phrase-Level Universal Visual RepresentationsCode1
On Vision Features in Multimodal Machine TranslationCode1
MSCTD: A Multimodal Sentiment Chat Translation DatasetCode1
Supervised Visual Attention for Simultaneous Multimodal Machine Translation—0
VISA: An Ambiguous Subtitles Dataset for Visual Scene-Aware Machine TranslationCode1
On Vision Features in Multimodal Machine Translation—0
Vision Matters When It Should: Sanity Checking Multimodal Machine Translation ModelsCode0
Multimodal Neural Machine Translation System for English to Bengali—0
Low Resource Multimodal Neural Machine Translation of English-Hindi in News Domain—0
Experiences of Adapting Multimodal Machine Translation Techniques for Hindi—0
Make the Blind Translator See The World: A Novel Transfer Learning Solution for Multimodal Machine Translation—0
Rakuten’s Participation in WAT 2021: Examining the Effectiveness of Pre-trained Models for Multilingual and Multimodal Machine Translation—0
BERTGEN: Multi-task Generation through BERTCode1
Cultural and Geographical Influences on Image Translatability of Words across LanguagesCode0
ViTA: Visual-Linguistic Translation by Aligning Object TagsCode0
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation—0
Gumbel-Attention for Multi-modal Machine Translation—0
Cross-lingual Visual Pre-training for Multimodal Machine TranslationCode1
Good for Misconceived Reasons: Revisiting Neural Multimodal Machine Translation—0
Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding—0
Show:102550
← PrevPage 2 of 5Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified