RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

Code

• github.com/princeton-vl/RAFT
  OfficialIn paperpytorch★ 3,986
• github.com/pytorch/vision
  pytorch★ 17,584
• github.com/open-mmlab/mmflow
  pytorch★ 1,050
• github.com/ykasten/layered-neural-atlases
  pytorch★ 613
• github.com/neu-vi/ezflow
  pytorch★ 137
• github.com/neu-vig/ezflow
  pytorch★ 137
• github.com/Visual-Behavior/aloception
  pytorch★ 93
• github.com/plusgood-steven/id-blau
  pytorch★ 58
• github.com/duke-vision/optical-flow-active-learning-release
  pytorch★ 18
• github.com/ToyotaResearchInstitute/att-aware
  pytorch★ 14

Abstract

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance. On KITTI, RAFT achieves an F1-all error of 5.10%, a 16% error reduction from the best published result (6.10%). On Sintel (final pass), RAFT obtains an end-point-error of 2.855 pixels, a 30% error reduction from the best published result (4.098 pixels). In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count. Code is available at https://github.com/princeton-vl/RAFT.

Tasks

Benchmark Results

Reproductions