The Arcade Learning Environment: An Evaluation Platform for General Agents
Marc G. Bellemare, Yavar Naddaf, Joel Veness, Michael Bowling
Code Available — Be the first to reproduce this paper.
ReproduceCode
Abstract
In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| Atari 2600 Alien | UCT | Score | 7,785 | — | Unverified |
| Atari 2600 Alien | Best Learner | Score | 939.2 | — | Unverified |
| Atari 2600 Amidar | Best Learner | Score | 103.4 | — | Unverified |
| Atari 2600 Amidar | UCT | Score | 180.3 | — | Unverified |
| Atari 2600 Assault | UCT | Score | 1,512.2 | — | Unverified |
| Atari 2600 Assault | Best Learner | Score | 628 | — | Unverified |
| Atari 2600 Asterix | Best Learner | Score | 987.3 | — | Unverified |
| Atari 2600 Asterix | UCT | Score | 290,700 | — | Unverified |
| Atari 2600 Asteroids | Best Learner | Score | 907.3 | — | Unverified |
| Atari 2600 Asteroids | UCT | Score | 4,660.6 | — | Unverified |
| Atari 2600 Atlantis | Best Learner | Score | 62,687 | — | Unverified |
| Atari 2600 Atlantis | UCT | Score | 193,858 | — | Unverified |
| Atari 2600 Bank Heist | UCT | Score | 497.8 | — | Unverified |
| Atari 2600 Bank Heist | Best Learner | Score | 190.8 | — | Unverified |
| Atari 2600 Battle Zone | UCT | Score | 70,333.3 | — | Unverified |
| Atari 2600 Battle Zone | Best Learner | Score | 15,819.7 | — | Unverified |
| Atari 2600 Beam Rider | Best Learner | Score | 929.4 | — | Unverified |
| Atari 2600 Beam Rider | UCT | Score | 6,624.6 | — | Unverified |
| Atari 2600 Berzerk | Best Baseline | Score | 670 | — | Unverified |
| Atari 2600 Berzerk | Best Learner | Score | 501.3 | — | Unverified |
| Atari 2600 Bowling | Best Learner | Score | 43.9 | — | Unverified |
| Atari 2600 Boxing | UCT | Score | 100 | — | Unverified |
| Atari 2600 Boxing | Best Learner | Score | 44 | — | Unverified |
| Atari 2600 Breakout | Best Learner | Score | 5.2 | — | Unverified |
| Atari 2600 Breakout | UCT | Score | 364.4 | — | Unverified |
| Atari 2600 Carnival | UCT | Score | 5,132 | — | Unverified |
| Atari 2600 Centipede | Best Learner | Score | 8,803.8 | — | Unverified |
| Atari 2600 Centipede | Full Tree | Score | 125,123 | — | Unverified |
| Atari 2600 Chopper Command | UCT | Score | 34,018.8 | — | Unverified |
| Atari 2600 Chopper Command | Best Learner | Score | 1,581.5 | — | Unverified |
| Atari 2600 Crazy Climber | Best Learner | Score | 23,410.6 | — | Unverified |
| Atari 2600 Crazy Climber | UCT | Score | 98,172.2 | — | Unverified |
| Atari 2600 Demon Attack | Best Learner | Score | 520.5 | — | Unverified |
| Atari 2600 Demon Attack | UCT | Score | 28,158.8 | — | Unverified |
| Atari 2600 Double Dunk | Best Learner | Score | -13.1 | — | Unverified |
| Atari 2600 Double Dunk | UCT | Score | 24 | — | Unverified |
| Atari 2600 Elevator Action | UCT | Score | 18,100 | — | Unverified |
| Atari 2600 Enduro | Best Learner | Score | 129.1 | — | Unverified |
| Atari 2600 Enduro | UCT | Score | 286.3 | — | Unverified |
| Atari 2600 Fishing Derby | UCT | Score | 37.8 | — | Unverified |
| Atari 2600 Fishing Derby | Best Learner | Score | -89.5 | — | Unverified |
| Atari 2600 Freeway | Best Learner | Score | 19.1 | — | Unverified |
| Atari 2600 Freeway | Best Baseline | Score | 22.5 | — | Unverified |
| Atari 2600 Freeway | UCT | Score | 0.4 | — | Unverified |
| Atari 2600 Frostbite | Best Learner | Score | 216.9 | — | Unverified |
| Atari 2600 Frostbite | UCT | Score | 270.5 | — | Unverified |
| Atari 2600 Gopher | Best Learner | Score | 1,288.3 | — | Unverified |
| Atari 2600 Gopher | UCT | Score | 20,560 | — | Unverified |
| Atari 2600 Gravitar | Best Learner | Score | 387.7 | — | Unverified |
| Atari 2600 Gravitar | UCT | Score | 2,850 | — | Unverified |