MTG: A Benchmarking Suite for Multilingual Text Generation

Abstract

We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation. It is the first and largest multilingual multiway text generation benchmark with 400k human-annotated data for four tasks (story generation, question generation, title generation and text summarization) across five languages (English, German, French, Spanish and Chinese). Its multiway characteristic makes it possible to create cross-lingual data between any of two languages and thus boosts the direct cross-lingual knowledge transfer. Based on it, we set various evaluation scenarios and make a deep analysis of several popular multilingual generation models from different aspects. Our benchmark suite will encourage multilingualism for the text generation community with more human-annotated parallel data and more diverse generation scenarios.

Tasks

Reproductions