Hybrid-Regressive Neural Machine Translation

Abstract

Non-autoregressive translation (NAT) with iterative refinement mechanism has shown comparable performance with the auto-regressive counterpart. However, we have empirically found that decoding acceleration is fragile when using a large batch size and running on the CPU. We demonstrate that one-pass NAT is sufficient when providing a few target contexts in advance through synthetic experiments. Inspired by this, we propose a two-stage translation prototype -- Hybrid-Regressive Translation (HRT) to combine the strengths of autoregressive and non-autoregressive. Specifically, HRT first generates a discontinuous sequence by autoregression (e.g., make a prediction every k tokens, k>1) and then fills all previously skipped tokens at once in a non-autoregressive manner. We also propose a bag of techniques to effectively and efficiently train HRT, with almost no increase in parameters. Experimental results on WMT En-Ro, En-De, and NIST Zh-En show that our model outperforms existing semi-autoregressive models and is competitive with current state-of-the-art non-autoregressive models. Moreover, compared to its autoregressive counterpart, HRT has a stable 1.5x acceleration, regardless of batch size and device.

Tasks

Reproductions