MediaPipe Hands: On-device Real-time Hand Tracking

Code

• github.com/vidursatija/BlazePalm
  pytorch★ 92
• github.com/luizhss/Gesture_Hand_Controller
  none★ 40
• github.com/lizhu1126/CNN-for-PD-Action
  tf★ 4
• github.com/Matheus-Vyctor/Gesture_Hand_Controller-1
  none★ 2

Abstract

We present a real-time on-device hand tracking pipeline that predicts hand skeleton from single RGB camera for AR/VR applications. The pipeline consists of two models: 1) a palm detector, 2) a hand landmark model. It's implemented via MediaPipe, a framework for building cross-platform ML solutions. The proposed model and pipeline architecture demonstrates real-time inference speed on mobile GPUs and high prediction quality. MediaPipe Hands is open sourced at https://mediapipe.dev.

Reproductions