SOTAVerified

Audio-Visual Speech Recognition

Audio-visual speech recognition is the task of transcribing a paired audio and visual stream into text.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 100 papers

TitleStatusHype
Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices—0
AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations—0
A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset—0
OLKAVS: An Open Large-Scale Korean Audio-Visual Speech DatasetCode1
ReVISE: Self-Supervised Speech Resynthesis With Visual Input for Universal and Generalized Speech Regeneration—0
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement—0
Jointly Learning Visual and Auditory Speech Representations from Raw DataCode1
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning—0
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning—0
Streaming Audio-Visual Speech Recognition with Alignment Regularization—0
Visual Context-driven Audio Feature Enhancement for Robust End-to-End Audio-Visual Speech RecognitionCode1
Kaggle Competition: Cantonese Audio-Visual Speech Recognition for In-car Commands—0
CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command RecognitionCode1
RUSAVIC Corpus: Russian Audio-Visual Speech in Cars—0
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech RecognitionCode1
Learning Contextually Fused Audio-visual Representations for Audio-visual Speech Recognition—0
Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video—0
Recent Progress in the CUHK Dysarthric Speech Recognition System—0
CI-AVSR: A Cantonese Audio-Visual Speech Dataset for In-car Command RecognitionCode1
Robust Self-Supervised Audio-Visual Speech RecognitionCode2
Leveraging Uni-Modal Self-Supervised Learning for Multimodal Audio-visual Speech Recognition—0
Audio-Visual Speech Recognition is Worth 32328 Voxels—0
Large-vocabulary Audio-visual Speech Recognition in Noisy Environments—0
Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading—0
Fusing information streams in end-to-end audio-visual speech recognition—0
End-to-end Audio-visual Speech Recognition with ConformersCode1
Part-based Lipreading for Audio-Visual Speech Recognition—0
AV Taris: Online Audio-Visual Speech RecognitionCode1
Lip Graph Assisted Audio-Visual Speech Recognition Using Bidirectional Synchronous Fusion—0
Should we hard-code the recurrence concept or learn it instead ? Exploring the Transformer architecture for Audio-Visual Speech RecognitionCode1
Discriminative Multi-modality Speech RecognitionCode1
How to Teach DNNs to Pay Attention to the Visual Modality in Speech RecognitionCode1
Audio-visual Recognition of Overlapped speech for the LRS2 dataset—0
Detecting Adversarial Attacks On Audiovisual Speech Recognition—0
Recurrent Neural Network Transducer for Audio-Visual Speech RecognitionCode0
Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition—0
Modality Attention for End-to-End Audio-visual Speech Recognition—0
Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture—0
Deep Audio-Visual Speech RecognitionCode1
LRS3-TED: a large-scale dataset for visual speech recognitionCode0
Towards Lipreading Sentences with Active Appearance Models—0
Multimodal Machine Learning: Integrating Language, Vision and Speech—0
Deep Multimodal Representation Learning from Temporal Data—0
Auxiliary Multimodal LSTM for Audio-visual Speech Recognition and Lipreading—0
Audio Visual Speech Recognition using Deep Recurrent Neural Networks—0
Deep Multimodal Learning for Audio-Visual Speech Recognition—0
Visual Speech Recognition—0
Recognition of Isolated Words using Zernike and MFCC features for Audio Visual Speech Recognition—0
Building a synchronous corpus of acoustic and 3D facial marker data for adaptive audio-visual speech synthesis—0
SUTAV: A Turkish Audio-Visual Database—0
Show:102550
← PrevPage 2 of 2Next →
All datasetsLRS2LRS3-TEDLRWCAS-VSR-S101

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hybrid CTC / AttentionWord Error Rate (WER)39.1—Unverified
2TM-Seq2seqTest WER8.5—Unverified
3TM-CTCTest WER8.2—Unverified
4CTC/AttentionTest WER7—Unverified
5CTC/AttentionTest WER1.5—Unverified
6Whisper-FlamingoTest WER1.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Hyb-ConformerWord Error Rate (WER)2.3—Unverified
2Zero-AVSRWord Error Rate (WER)1.5—Unverified
3AV-HuBERT LargeWord Error Rate (WER)1.4—Unverified
4Whisper-FlamingoWord Error Rate (WER)0.76—Unverified
5MMS-LLaMAWord Error Rate (WER)0.74—Unverified
#ModelMetricClaimedVerifiedStatus
1AVCRFormerTop-1 Accuracy98.81—Unverified
22DCNN + BiLSTM + ResNet + MLFTop-1 Accuracy98.76—Unverified
3PBLTop-1 Accuracy98.3—Unverified
#ModelMetricClaimedVerifiedStatus
1ES³ Base*Word Error Rate (WER)11—Unverified