Nearest Neighbor Machine Translation

Code

• github.com/urvashik/knnlm
  OfficialIn paperpytorch★ 329
• github.com/neulab/knn-transformers
  pytorch★ 285
• github.com/urvashik/knnmt
  pytorch★ 45
• github.com/hubreb/imitkd_ast
  pytorch★ 1

Abstract

We introduce k-nearest-neighbor machine translation (kNN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. kNN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results -- without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, kNN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.

Tasks

Reproductions