SOTAVerified

Lip Reading

Lip Reading is a task to infer the speech content in a video by using only the visual information, especially the lip movements. It has many crucial applications in practice, such as assisting audio-based speech recognition, biometric authentication and aiding hearing-impaired people.

Source: Mutual Information Maximization for Effective Lip Reading

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 153 papers

TitleStatusHype
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face SynthesisCode4
Where Visual Speech Meets Language: VSP-LLM Framework for Efficient and Context-Aware Visual Speech ProcessingCode3
Training Strategies for Improved Lip-readingCode2
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster PredictionCode2
Seeing What You Said: Talking Face Generation Guided by a Lip Reading ExpertCode2
Deformation Flow Based Two-Stream Network for Lip ReadingCode1
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality AlignmentCode1
Deep Audio-Visual Speech RecognitionCode1
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking FacesCode1
Selective Listening by Synchronizing Speech with LipsCode1
End-to-end Audio-visual Speech Recognition with ConformersCode1
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip ReadingCode1
Contrastive Learning of Global-Local Video RepresentationsCode1
Learning Individual Speaking Styles for Accurate Lip to Speech SynthesisCode1
Audio-Visual Efficient Conformer for Robust Speech RecognitionCode1
Mutual Information Maximization for Effective Lip ReadingCode1
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and RecognitionCode1
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realismCode1
Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and LanguageCode1
Do VSR Models Generalize Beyond LRS3?Code1
Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech RecognitionCode1
Visual Keyword Spotting with AttentionCode1
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech RecognitionCode1
Learn an Effective Lip Reading Model without PainsCode1
Seeing wake words: Audio-visual Keyword SpottingCode1
Lipreading using Temporal Convolutional NetworksCode1
OLKAVS: An Open Large-Scale Korean Audio-Visual Speech DatasetCode1
Multi-modality Associative Bridging through Memory: Speech Sound Recollected from Face VideoCode1
LipVoicer: Generating Speech from Silent Videos Guided by Lip ReadingCode1
Lip-reading with Densely Connected Temporal Convolutional NetworksCode1
End-to-End Speech-Driven Facial Animation with Temporal GANsCode1
Cross-Attention Fusion of Visual and Geometric Features for Large Vocabulary Arabic Lipreading—0
Contrastive Self-Supervised Learning of Global-Local Audio-Visual Representations—0
An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition—0
Contrastive Learning of Global and Local Video Representations—0
Audio-Visual Speech Enhancement and Separation by Utilizing Multi-Modal Self-Supervised Embeddings—0
Advances and Challenges in Deep Lip Reading—0
Contextual Audio-Visual Switching For Speech Enhancement in Real-World Environments—0
Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition—0
Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices—0
Clean Text and Full-Body Transformer: Microsoft's Submission to the WMT22 Shared Task on Sign Language Translation—0
End-to-End Video-To-Speech Synthesis using Generative Adversarial Networks—0
A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset—0
Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations—0
Finding phonemes: improving machine lip-reading—0
End-to-End Lip Reading in Romanian with Cross-Lingual Domain Adaptation and Lateral Inhibition—0
Chinese-LiPS: A Chinese audio-visual speech recognition dataset with Lip-reading and Presentation Slides—0
Enhancing Lip Reading with Multi-Scale Video and Multi-Encoder—0
Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert—0
Emotional Speech-Driven Animation with Content-Emotion Disentanglement—0
Show:102550
← PrevPage 1 of 4Next →
All datasetsGRID corpus (mixed-speech)LRWTCD-TIMIT corpus (mixed-speech)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Lip2WavWER14.08—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER34.2—Unverified
#ModelMetricClaimedVerifiedStatus
1Lip2WavWER31.26—Unverified