Mobile Video Object Detection with Temporally-Aware Feature Maps

Code

• github.com/tensorflow/models
  tf★ 77,694
• github.com/vikrant7/mobile-vod-bottleneck-lstm
  pytorch★ 0
• github.com/kyscg/tf-models
  tf★ 0

Abstract

This paper introduces an online model for object detection in videos designed to run in real-time on low-powered mobile and embedded devices. Our approach combines fast single-image object detection with convolutional long short term memory (LSTM) layers to create an interweaved recurrent-convolutional architecture. Additionally, we propose an efficient Bottleneck-LSTM layer that significantly reduces computational cost compared to regular LSTMs. Our network achieves temporal awareness by using Bottleneck-LSTMs to refine and propagate feature maps across frames. This approach is substantially faster than existing detection methods in video, outperforming the fastest single-frame models in model size and computational cost while attaining accuracy comparable to much more expensive single-frame models on the Imagenet VID 2015 dataset. Our model reaches a real-time inference speed of up to 15 FPS on a mobile CPU.

Tasks

Reproductions