Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
2018-02-19EMNLP 2018Code Available0· sign in to hype
Jason Lee, Elman Mansimov, Kyunghyun Cho
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/nyu-dl/dl4mt-nonautoOfficialIn paperpytorch★ 0
- github.com/zhajiahe/Token_Droppytorch★ 5
Abstract
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine translation (En-De and En-Ro) and image caption generation, and observe that it significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| IWSLT2015 English-German | Denoising autoencoders (non-autoregressive) | BLEU score | 27.01 | — | Unverified |
| IWSLT2015 German-English | Denoising autoencoders (non-autoregressive) | BLEU score | 32.43 | — | Unverified |
| WMT2014 English-German | Denoising autoencoders (non-autoregressive) | BLEU score | 21.54 | — | Unverified |
| WMT2014 German-English | Denoising autoencoders (non-autoregressive) | BLEU score | 25.43 | — | Unverified |
| WMT2016 English-Romanian | Denoising autoencoders (non-autoregressive) | BLEU score | 29.66 | — | Unverified |
| WMT2016 Romanian-English | Denoising autoencoders (non-autoregressive) | BLEU score | 30.3 | — | Unverified |