Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Code

• github.com/zaczou/keras_summary
  tf★ 0
• github.com/dtsbourg/ff-attention
  pytorch★ 0
• github.com/firfre/Chinese-Text-sentiment-analysis-with-key-word
  tf★ 0
• github.com/shawnyxiao/textclassification-keras
  tf★ 0
• github.com/WenYanger/Contextual-Attention
  pytorch★ 0

Abstract

We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.

Reproductions