SOTAVerified

Talking Head Generation

Talking head generation is the task of generating a talking face from a set of images of a person.

( Image credit: Few-Shot Adversarial Learning of Realistic Neural Talking Head Models )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 119 papers

TitleStatusHype
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face AnimationCode6
Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head GenerationCode3
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video GenerationCode3
DreamTalk: When Emotional Talking Head Generation Meets Diffusion Probabilistic ModelsCode3
SyncTalk: The Devil is in the Synchronization for Talking Head SynthesisCode3
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The WildCode3
InsTaG: Learning Personalized 3D Talking Head from Few-Second VideoCode2
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking HeadsCode2
MoDiTalker: Motion-Disentangled Diffusion Model for High-Fidelity Talking Head GenerationCode2
Adaptive Super Resolution For One-Shot Talking-Head GenerationCode2
Efficient Emotional Adaptation for Audio-Driven Talking-Head GenerationCode2
Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video GenerationCode2
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head AvatarsCode2
DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video GenerationCode2
Emotionally Enhanced Talking Face GenerationCode2
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits AnimationCode2
StyleTalk: One-shot Talking Head Generation with Controllable Speaking StylesCode2
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized AdaptationCode2
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head SynthesisCode2
Depth-Aware Generative Adversarial Network for Talking Head Video GenerationCode2
Live Speech Portraits: Real-Time Photorealistic Talking-Head AnimationCode2
MakeItTalk: Speaker-Aware Talking-Head AnimationCode2
Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss FunctionsCode1
Silence is Golden: Leveraging Adversarial Examples to Nullify Audio Control in LDM-based Talking-Head GenerationCode1
Joint Co-Speech Gesture and Expressive Talking Face Generation using Diffusion with AdaptersCode1
GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic ExpressionCode1
Comparative Analysis of Audio Feature Extraction for Real-Time Talking Portrait SynthesisCode1
EmoVOCA: Speech-Driven Emotional 3D Talking HeadsCode1
A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head VideosCode1
DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoderCode1
Learning Landmarks Motion from Speech for Speaker-Agnostic 3D Talking Heads GenerationCode1
Face Animation with an Attribute-Guided Diffusion ModelCode1
DisCoHead: Audio-and-Video-Driven Talking Head Generation by Disentangled Control of Head Pose and Facial ExpressionsCode1
Autoregressive GAN for Semantic Unconditional Head Motion GenerationCode1
Perceptual Conversational Head Generation with Regularized Driver and Enhanced RendererCode1
AI-generated characters for supporting personalized learning and well-beingCode1
AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head ReenactmentCode1
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head MotionCode1
Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via TextCode1
Write-a-speaker: Text-based Emotional and Rhythmic Talking-head GenerationCode1
Fast Bi-layer Neural Synthesis of One-Shot Realistic Head AvatarsCode1
MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face GenerationCode1
Talking-head Generation with Rhythmic Head MotionCode1
What comprises a good talking-head video generation?: A Survey and BenchmarkCode1
Text-based Editing of Talking-head VideoCode1
MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding—0
DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations—0
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution—0
OmniTalker: Real-Time Text-Driven Talking Head Generation with In-Context Audio-Visual Style Replication—0
Audio-Plane: Audio Factorization Plane Gaussian Splatting for Real-Time Talking Head Synthesis—0
Show:102550
← PrevPage 1 of 3Next →
All datasetsVoxCeleb2 - 1-shot learningVoxCeleb1 - 1-shot learningVoxCeleb1 - 32-shot learningVoxCeleb1 - 8-shot learningVoxCeleb2 - 8-shot learning100 sleep nights of 8 caregiversVoxCeleb2 - 32-shot learning

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Few-shot Adversarial ModelFID48.5—Unverified
2CainGANFID35—Unverified
3Fast Bi-layer Avatars (medium size)CSIM0.65—Unverified
4First Order Motion Model (medium size)CSIM0.64—Unverified
5Few-shot Vid-to-vid (medium size)CSIM0.6—Unverified
#ModelMetricClaimedVerifiedStatus
1X2FaceFID45.8—Unverified
2Few-shot Adversarial ModelFID43—Unverified
#ModelMetricClaimedVerifiedStatus
1X2FaceFID56.5—Unverified
2Few-shot Adversarial ModelFID29.5—Unverified
#ModelMetricClaimedVerifiedStatus
1X2FaceFID51.5—Unverified
2Few-shot Adversarial ModelFID38—Unverified
#ModelMetricClaimedVerifiedStatus
1Few-shot Adversarial ModelFID42.2—Unverified
2CainGANFID24.9—Unverified
#ModelMetricClaimedVerifiedStatus
1Ashok10%12—Unverified
#ModelMetricClaimedVerifiedStatus
1Few-shot Adversarial ModelFID30.6—Unverified