SOTAVerified

Lip Reading

Lip Reading is a task to infer the speech content in a video by using only the visual information, especially the lip movements. It has many crucial applications in practice, such as assisting audio-based speech recognition, biometric authentication and aiding hearing-impaired people.

Source: Mutual Information Maximization for Effective Lip Reading

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 153 papers

TitleStatusHype
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face SynthesisCode4
Where Visual Speech Meets Language: VSP-LLM Framework for Efficient and Context-Aware Visual Speech ProcessingCode3
Training Strategies for Improved Lip-readingCode2
Seeing What You Said: Talking Face Generation Guided by a Lip Reading ExpertCode2
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster PredictionCode2
Learn an Effective Lip Reading Model without PainsCode1
Lipreading using Temporal Convolutional NetworksCode1
Deep Audio-Visual Speech RecognitionCode1
Do VSR Models Generalize Beyond LRS3?Code1
Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech RecognitionCode1
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality AlignmentCode1
Deformation Flow Based Two-Stream Network for Lip ReadingCode1
Contrastive Learning of Global-Local Video RepresentationsCode1
Visual Keyword Spotting with AttentionCode1
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip ReadingCode1
Audio-Visual Efficient Conformer for Robust Speech RecognitionCode1
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech RecognitionCode1
End-to-End Speech-Driven Facial Animation with Temporal GANsCode1
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and RecognitionCode1
End-to-end Audio-visual Speech Recognition with ConformersCode1
Mutual Information Maximization for Effective Lip ReadingCode1
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realismCode1
Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and LanguageCode1
Seeing wake words: Audio-visual Keyword SpottingCode1
Selective Listening by Synchronizing Speech with LipsCode1
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking FacesCode1
Learning Individual Speaking Styles for Accurate Lip to Speech SynthesisCode1
Lip-reading with Densely Connected Temporal Convolutional NetworksCode1
OLKAVS: An Open Large-Scale Korean Audio-Visual Speech DatasetCode1
Multi-modality Associative Bridging through Memory: Speech Sound Recollected from Face VideoCode1
LipVoicer: Generating Speech from Silent Videos Guided by Lip ReadingCode1
Transforming faces into video stories -- VideoFace2.0Code0
Audio-Visual Speech Recognition based on Regulated Transformer and Spatio-Temporal Fusion Strategy for Driver Assistive SystemsCode0
Talking Face Generation by Adversarially Disentangled Audio-Visual RepresentationCode0
Speaker-adaptive Lip Reading with User-dependent PaddingCode0
Combining Residual Networks with LSTMs for LipreadingCode0
Synchronous Bidirectional Learning for Multilingual Lip ReadingCode0
XFlow: Cross-modal Deep Neural Networks for Audiovisual ClassificationCode0
Multi-Perspective LSTM for Joint Visual Representation LearningCode0
MTGA: Multi-View Temporal Granularity Aligned Aggregation for Event-Based Lip-ReadingCode0
DualTalker: A Cross-Modal Dual Learning Approach for Speech-Driven 3D Facial AnimationCode0
LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the WildCode0
Lip Sync Matters: A Novel Multimodal Forgery DetectorCode0
AuthNet: A Deep Learning based Authentication Mechanism using Temporal Facial Feature MovementsCode0
Lip2AudSpec: Speech reconstruction from silent lip movements videoCode0
Lend a Hand: Semi Training-Free Cued Speech Recognition via MLLM-Driven Hand Modeling for Barrier-free CommunicationCode0
A Novel Interpretable and Generalizable Re-synchronization Model for Cued Speech based on a Multi-Cuer CorpusCode0
Estimating speech from lip dynamicsCode0
Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-ReadingCode0
Cross-Attention Fusion of Visual and Geometric Features for Large Vocabulary Arabic Lipreading—0
Show:102550
← PrevPage 1 of 4Next →
All datasetsGRID corpus (mixed-speech)LRWTCD-TIMIT corpus (mixed-speech)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Lip2WavWER14.08—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER34.2—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER31.26—Unverified