SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 26–50 of 108 papers

TitleStatusHype
Towards Zero-Shot Multimodal Machine TranslationCode0
Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasets—0
Detecting Concrete Visual Tokens for Multimodal Machine Translation—0
Adding Multimodal Capabilities to a Text-only Translation Model—0
The Case for Evaluating Multimodal Translation Models on Text Datasets—0
Video-Helpful Multimodal Machine TranslationCode0
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering PairsCode0
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine TranslationCode0
A Survey of Vision-Language Pre-training from the Lens of Multimodal Machine Translation—0
HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa LanguageCode0
Iterative Adversarial Attack on Image-guided Story Ending Generation—0
Generalization algorithm of multimodal pre-training model based on graph-text self-supervised training—0
Beyond Triplet: Leveraging the Most Data for Multimodal Machine TranslationCode0
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation—0
LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation—0
Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective—0
Supervised Visual Attention for Simultaneous Multimodal Machine Translation—0
On Vision Features in Multimodal Machine Translation—0
Vision Matters When It Should: Sanity Checking Multimodal Machine Translation ModelsCode0
Multimodal Neural Machine Translation System for English to Bengali—0
Low Resource Multimodal Neural Machine Translation of English-Hindi in News Domain—0
Experiences of Adapting Multimodal Machine Translation Techniques for Hindi—0
Rakuten’s Participation in WAT 2021: Examining the Effectiveness of Pre-trained Models for Multilingual and Multimodal Machine Translation—0
Make the Blind Translator See The World: A Novel Transfer Learning Solution for Multimodal Machine Translation—0
Cultural and Geographical Influences on Image Translatability of Words across LanguagesCode0
Show:102550
← PrevPage 2 of 5Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified