On First-Order Meta-Learning Algorithms

Code

• github.com/openai/supervised-reptile
  Officialtf★ 0
• github.com/learnables/learn2learn
  pytorch★ 2,878
• github.com/sanowar-raihan/nerf-meta
  pytorch★ 98
• github.com/Yuzhe-CHEN/NerfSNN
  pytorch★ 1
• github.com/aravindMahadevan/metaLearningAlgos
  pytorch★ 0
• github.com/peisungtsai/Reptile-Pytorch-Implementation
  pytorch★ 0
• github.com/gebob19/cscd94_metalearning
  pytorch★ 0
• github.com/gabrielhuang/reptile-pytorch
  pytorch★ 0
• github.com/hfahrudin/reptile_implement_tf2
  tf★ 0
• github.com/MaximeVandegar/Papers-in-100-Lines-of-Code/tree/main/On_First_Order_Meta_Learning_Algorithms
  pytorch★ 0

Abstract

This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.

Tasks

Benchmark Results

Reproductions