What Matters in Unsupervised Optical Flow
Rico Jonschkowski, Austin Stone, Jonathan T. Barron, Ariel Gordon, Kurt Konolige, Anelia Angelova
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/google-research/google-research/tree/master/uflowOfficialtf★ 0
- github.com/junbongjang/contour-trackingtf★ 22
- github.com/cfreshgirl/uflowmindspore★ 1
- github.com/2024-MindSpore-1/Code3/tree/main/Flowmindspore★ 0
- github.com/2023-MindSpore-1/ms-code-36mindspore★ 0
Abstract
We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective. Alongside this investigation we construct a number of novel improvements to unsupervised flow models, such as cost volume normalization, stopping the gradient at the occlusion mask, encouraging smoothness before upsampling the flow field, and continual self-supervision with image resizing. By combining the results of our investigation with our improved model components, we are able to present a new unsupervised flow technique that significantly outperforms the previous unsupervised state-of-the-art and performs on par with supervised FlowNet2 on the KITTI 2015 dataset, while also being significantly simpler than related approaches.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| Sintel Clean unsupervised | UFlow | Average End-Point Error | 5.21 | — | Unverified |
| Sintel Final unsupervised | UFlow | Average End-Point Error | 6.5 | — | Unverified |