Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning

Code

• github.com/facebookresearch/drqv2
  OfficialIn paperpytorch★ 432
• github.com/denisyarats/drq
  pytorch★ 419
• github.com/mazpie/mastering-urlb
  pytorch★ 41
• github.com/Asap7772/understanding-rlhf
  pytorch★ 32
• github.com/zhaoyi11/tcrl
  pytorch★ 24
• github.com/tajwarfahim/proactive_interventions
  pytorch★ 9
• github.com/architsharma97/medal
  pytorch★ 7
• github.com/zhou-henry/distributed-distributional-drq
  pytorch★ 3

Abstract

We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control. DrQ-v2 builds on DrQ, an off-policy actor-critic approach that uses data augmentation to learn directly from pixels. We introduce several improvements that yield state-of-the-art results on the DeepMind Control Suite. Notably, DrQ-v2 is able to solve complex humanoid locomotion tasks directly from pixel observations, previously unattained by model-free RL. DrQ-v2 is conceptually simple, easy to implement, and provides significantly better computational footprint compared to prior work, with the majority of tasks taking just 8 hours to train on a single GPU. Finally, we publicly release DrQ-v2's implementation to provide RL practitioners with a strong and computationally efficient baseline.

Tasks

Reproductions