Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars
Egor Zakharov, Aleksei Ivakhnenko, Aliaksandra Shysheya, Victor Lempitsky
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/saic-violet/bilayer-modelOfficialpytorch★ 246
Abstract
We propose a neural rendering-based system that creates head avatars from a single photograph. Our approach models a person's appearance by decomposing it into two layers. The first layer is a pose-dependent coarse image that is synthesized by a small neural network. The second layer is defined by a pose-independent texture image that contains high-frequency details. The texture image is generated offline, warped and added to the coarse image to ensure a high effective resolution of synthesized head views. We compare our system to analogous state-of-the-art systems in terms of visual quality and speed. The experiments show significant inference speedup over previous neural head avatar models for a given visual quality. We also report on a real-time smartphone-based implementation of our system.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| VoxCeleb2 - 1-shot learning | Fast Bi-layer Avatars (medium size) | CSIM | 0.65 | — | Unverified |
| VoxCeleb2 - 1-shot learning | First Order Motion Model (medium size) | CSIM | 0.64 | — | Unverified |
| VoxCeleb2 - 1-shot learning | Few-shot Vid-to-vid (medium size) | CSIM | 0.6 | — | Unverified |