Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
Kurtland Chua, Roberto Calandra, Rowan Mcallister, Sergey Levine
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/kchua/handful-of-trialsOfficialIn papertf★ 0
- github.com/facebookresearch/mbrl-libpytorch★ 1,058
- github.com/jingwu6/handful-of-trials-in-pytorchpytorch★ 1
- github.com/github-jnauta/pytorch-pnepytorch★ 0
- github.com/quanvuong/handful-of-trials-pytorchpytorch★ 0
- github.com/sradicwebster/mbrl-libpytorch★ 0
- github.com/ByMic/PETSpytorch★ 0
- github.com/natolambert/dynamicslearnpytorch★ 0
- github.com/Shunichi09/PythonLinearNonlinearControlnone★ 0
Abstract
Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge this gap, by employing uncertainty-aware dynamics models. We propose a new algorithm called probabilistic ensembles with trajectory sampling (PETS) that combines uncertainty-aware deep network dynamics models with sampling-based uncertainty propagation. Our comparison to state-of-the-art model-based and model-free deep RL algorithms shows that our approach matches the asymptotic performance of model-free algorithms on several challenging benchmark tasks, while requiring significantly fewer samples (e.g., 8 and 125 times fewer samples than Soft Actor Critic and Proximal Policy Optimization respectively on the half-cheetah task).