ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

Code

• github.com/askforalfred/alfred
  Officialpytorch★ 0
• github.com/alfworld/alfworld
  pytorch★ 688
• github.com/alexpashevich/E.T.
  pytorch★ 93
• github.com/gistvision/moca
  pytorch★ 40
• github.com/snumprlab/cl-alfred
  pytorch★ 31
• github.com/facebookresearch/EgoTV
  pytorch★ 27
• github.com/snumprlab/capeam
  pytorch★ 16
• github.com/yonseivnl/mcr-agent
  pytorch★ 7
• github.com/yenchehsiao/autonomousllmagentwithadaptingplanning
  pytorch★ 3
• github.com/caisarl76/alfred
  pytorch★ 0

Abstract

We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks. ALFRED includes long, compositional tasks with non-reversible state changes to shrink the gap between research benchmarks and real-world applications. ALFRED consists of expert demonstrations in interactive visual environments for 25k natural language directives. These directives contain both high-level goals like "Rinse off a mug and place it in the coffee maker." and low-level language instructions like "Walk to the coffee maker on the right." ALFRED tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets. We show that a baseline model based on recent embodied vision-and-language tasks performs poorly on ALFRED, suggesting that there is significant room for developing innovative grounded visual language understanding models with this benchmark.

Tasks

Reproductions