SOTAVerified

Multimodal Machine Translation

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

( Image credit: Findings of the Third Shared Task on Multimodal Machine Translation )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 11–20 of 108 papers

TitleStatusHype
Detecting Concrete Visual Tokens for Multimodal Machine Translation—0
The Case for Evaluating Multimodal Translation Models on Text Datasets—0
Seamless: Multilingual Expressive and Streaming Speech TranslationCode6
Video-Helpful Multimodal Machine TranslationCode0
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering PairsCode0
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine TranslationCode0
CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine TranslationCode1
A Survey of Vision-Language Pre-training from the Lens of Multimodal Machine Translation—0
HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa LanguageCode0
BigVideo: A Large-scale Video Subtitle Translation Dataset for Multimodal Machine TranslationCode1
Show:102550
← PrevPage 2 of 11Next →
All datasetsMulti30KHindi Visual Genome (Challenge Set)Hindi Visual Genome (Test Set)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1delMeteor (EN-FR)74.6—Unverified
2ERNIE-UniX2BLEU (EN-DE)49.3—Unverified
3IKD-MMTBLEU (EN-DE)41.28—Unverified
4DCCNBLEU (EN-DE)39.7—Unverified
5CaglayanBLEU (EN-DE)39.4—Unverified
6Gumbel-Attention MMTBLEU (EN-DE)39.2—Unverified
7Multimodal TransformerBLEU (EN-DE)38.7—Unverified
8ImagiTBLEU (EN-DE)38.4—Unverified
9del+objBLEU (EN-DE)38—Unverified
10VMMTFBLEU (EN-DE)37.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)51.6—Unverified
#ModelMetricClaimedVerifiedStatus
1ViTABLEU (EN-HI)44.6—Unverified