Implementing Neural Turing Machines

Code

• github.com/MarkPKCollier/NeuralTuringMachine
  OfficialIn papertf★ 610
• github.com/SourangshuGhosh/NeuralTuringMachine
  tf★ 4
• github.com/mdabagia/NeuralTuringMachine
  pytorch★ 0
• github.com/Baichenjia/NeuralTuringMachine
  tf★ 0
• github.com/theneuralbeing/ntm
  pytorch★ 0
• github.com/ajithcodesit/Neural_Turing_Machine
  tf★ 0

Abstract

Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.

Reproductions