Deep Exploration via Bootstrapped DQN

Code

• github.com/tensorflow/models
  tf★ 77,694
• github.com/johannah/bootstrap_dqn
  pytorch★ 0
• github.com/yaringal/HeteroscedasticDropoutUncertainty
  none★ 0
• github.com/mrahtz/learning-from-human-preferences
  tf★ 0
• github.com/tensorflow/models/tree/master/research/deep_contextual_bandits
  tf★ 0
• github.com/NervanaSystems/coach
  tf★ 0

Abstract

Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy exploration, bootstrapped DQN carries out temporally-extended (or deep) exploration; this can lead to exponentially faster learning. We demonstrate these benefits in complex stochastic MDPs and in the large-scale Arcade Learning Environment. Bootstrapped DQN substantially improves learning times and performance across most Atari games.

Tasks

Benchmark Results

Reproductions