ReFIn: A Refinement Approach for Video Frame Interpolation

Abstract

Video Frame Interpolation is an important video enhancement problem which aims to generate one or multiple frames between consecutive frames in video. Optical flow-based frame interpolation approaches estimate intermediate optical flow from interpolated frame to input frames and warped frames are fused to generate interpolated frame. However, intermediate flow estimates can itself be erroneous leading to inaccurate interpolation results. In this work, we improve an flow-based intrtpolation algorithm, Super-SloMo by residual refinement. Specifically, we feed intermediate flowmaps, visibility map, warped input frames and intermediate interpolation estimate to a refinement network to predict a frame residual. We have also experimented with different architecture choices to be used in different modules to further improve the results. We found out that GridNet with four pyramid levels achieves the best results whereas UNet++ performs moderately well with significantly less number of parameters.

Tasks

Reproductions