SOTAVerified

Lip Reading

Lip Reading is a task to infer the speech content in a video by using only the visual information, especially the lip movements. It has many crucial applications in practice, such as assisting audio-based speech recognition, biometric authentication and aiding hearing-impaired people.

Source: Mutual Information Maximization for Effective Lip Reading

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 153 papers

TitleStatusHype
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model—0
Audio-Visual Efficient Conformer for Robust Speech RecognitionCode1
Audio-visual video face hallucination with frequency supervision and cross modality support by speech based lip reading loss—0
Lip Sync Matters: A Novel Multimodal Forgery DetectorCode0
Streaming Audio-Visual Speech Recognition with Alignment Regularization—0
Audio-Visual Speech Enhancement and Separation by Utilizing Multi-Modal Self-Supervised Embeddings—0
A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids—0
Clean Text and Full-Body Transformer: Microsoft's Submission to the WMT22 Shared Task on Sign Language Translation—0
VCSE: Time-Domain Visual-Contextual Speaker Extraction Network—0
Relaxed Attention for Transformer ModelsCode0
Training Strategies for Improved Lip-readingCode2
Visual Speech Recognition in a Driver Assistance System—0
Towards MOOCs for Lipreading: Using Synthetic Talking Heads to Train Humans in Lipreading at Scale—0
Speaker-adaptive Lip Reading with User-dependent PaddingCode0
Lip-Listening: Mixing Senses to Understand Lips using Cross Modality Knowledge Distillation for Word-Based Models—0
Learning Speaker-specific Lip-to-Speech Generation—0
RUSAVIC Corpus: Russian Audio-Visual Speech in Cars—0
Expression-preserving face frontalization improves visually assisted speech processing—0
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip ReadingCode1
Multi-modality Associative Bridging through Memory: Speech Sound Recollected from Face VideoCode1
A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning—0
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech RecognitionCode1
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster PredictionCode2
Multi-Grained Spatio-Temporal Features Perceived Network for Event-Based Lip-Reading—0
LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip Reading—0
Audio-Visual Synchronisation in the wild—0
Contrastive Learning of Global and Local Video Representations—0
Leveraging Uni-Modal Self-Supervised Learning for Multimodal Audio-visual Speech Recognition—0
Visual Keyword Spotting with AttentionCode1
Advances and Challenges in Deep Lip Reading—0
Sub-word Level Lip Reading With Visual Attention—0
Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks—0
Audio-Visual Speech Recognition is Worth 32328 Voxels—0
LRWR: Large-Scale Benchmark for Lip Reading in Russian language—0
SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory—0
Adaptive Semantic-Spatio-Temporal Graph Convolutional Network for Lip Reading—0
Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading—0
Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations—0
Learning From the Master: Distilling Cross-Modal Advanced Knowledge for Lip Reading—0
LiRA: Learning Visual Speech Representations from Audio through Self-supervision—0
Selective Listening by Synchronizing Speech with LipsCode1
Multi-Perspective LSTM for Joint Visual Representation LearningCode0
End-to-End Video-To-Speech Synthesis using Generative Adversarial Networks—0
Fusing information streams in end-to-end audio-visual speech recognition—0
Lip reading using external viseme decoding—0
Contrastive Learning of Global-Local Video RepresentationsCode1
End-to-end Audio-visual Speech Recognition with ConformersCode1
Contrastive Self-Supervised Learning of Global-Local Audio-Visual Representations—0
Lip-reading with Hierarchical Pyramidal Convolution and Self-Attention—0
AuthNet: A Deep Learning based Authentication Mechanism using Temporal Facial Feature MovementsCode0
Show:102550
← PrevPage 2 of 4Next →
All datasetsGRID corpus (mixed-speech)LRWTCD-TIMIT corpus (mixed-speech)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Lip2WavWER14.08—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER34.2—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER31.26—Unverified