Unified Embedding for Universal Multilingual Neural Machine Translation

Abstract

In this paper, we propose a novel neural machine translation method that enables translation of multiple languages using a single universal model. Our approach introduces a small shared character space across languages and a language-specific linear transformation after the decoder output. Through mapping each token of language difference to a fixed-length sequence of characters and then building corresponding embeddings, our approach enables a subword system using a small shared pseudo vocabulary. A dictionary-specific inference is represented by a specific component to project the decoder output onto a language-specific token, which eliminates the effect of the off-target problem described in many previous studies, and improves the language discrimination ability of the model. Experiments on the International Workshop on Spoken Language Translation dataset showed that our approach achieves improvement of 3-4 BLEU for zero-shot translation, and without lag on supervised translation under separate byte-pair encoding. Statistical analysis indicated that, with our model, there is no off-target problem for low-resource multilingual translation. Our experiments demonstrate the feasibility of sharing vocabulary about characters rather than at the word/subword or character/byte levels mentioned in previous studies.

Tasks

Reproductions