Automatic Song Translation for Tonal Languages

Abstract

This paper addresses automatic song translation (AST) for tonal languages and the unique challenge of aligning words' tones with the melody of a song in addition to conveying the original meaning. We propose three criteria for effective AST---preserving semantics, singability and intelligibility---and develop objectives for these criteria. We develop a new benchmark for English--Mandarin song translation and develop an unsupervised AST system, the Guided AliGnment for Automatic Song Translation (GagaST), which combines pre-training with three decoding constraints. Both automatic and human evaluations show GagaST successfully balances semantics and singability.

Tasks

Reproductions