SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 21–30 of 108 papers

TitleStatusHype
Multimodal Machine Translation with Visual Scene Graph Pruning—0
TopicVD: A Topic-Based Dataset of Video-Guided Multimodal Machine Translation for DocumentariesCode0
Florenz: Scaling Laws for Systematic Generalization in Vision-Language Models—0
Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation—0
EMMeTT: Efficient Multimodal Machine Translation Training—0
Towards Zero-Shot Multimodal Machine TranslationCode0
Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasets—0
The Case for Evaluating Multimodal Translation Models on Text Datasets—0
Detecting Concrete Visual Tokens for Multimodal Machine Translation—0
Adding Multimodal Capabilities to a Text-only Translation Model—0
Show:102550
← PrevPage 3 of 11Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified