SOTAVerified

Audio-Visual Speech Recognition

Audio-visual speech recognition is the task of transcribing a paired audio and visual stream into text.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–75 of 100 papers

TitleStatusHype
SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition—0
Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading—0
ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations—0
Fusing information streams in end-to-end audio-visual speech recognition—0
Streaming Audio-Visual Speech Recognition with Alignment Regularization—0
SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer—0
Towards Lipreading Sentences with Active Appearance Models—0
Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video—0
MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition—0
Uncovering the Visual Contribution in Audio-Visual Speech Recognition—0
Modality Attention for End-to-End Audio-visual Speech Recognition—0
MoHAVE: Mixture of Hierarchical Audio-Visual Experts for Robust Speech Recognition—0
MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization—0
Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer—0
Multimodal Machine Learning: Integrating Language, Vision and Speech—0
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning—0
A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset—0
ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition—0
Visual-Aware Speech Recognition for Noisy Scenarios—0
Part-based Lipreading for Audio-Visual Speech Recognition—0
Adapter-Based Multi-Agent AVSR Extension for Pre-Trained ASR Models—0
Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic Perspective—0
Recent Progress in the CUHK Dysarthric Speech Recognition System—0
Recognition of Isolated Words using Zernike and MFCC features for Audio Visual Speech Recognition—0
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement—0
Show:102550
← PrevPage 3 of 4Next →
All datasetsLRS2LRS3-TEDLRWCAS-VSR-S101

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hybrid CTC / AttentionWord Error Rate (WER)39.1—Unverified
2TM-Seq2seqTest WER8.5—Unverified
3TM-CTCTest WER8.2—Unverified
4CTC/AttentionTest WER7—Unverified
5CTC/AttentionTest WER1.5—Unverified
6Whisper-FlamingoTest WER1.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Hyb-ConformerWord Error Rate (WER)2.3—Unverified
2Zero-AVSRWord Error Rate (WER)1.5—Unverified
3AV-HuBERT LargeWord Error Rate (WER)1.4—Unverified
4Whisper-FlamingoWord Error Rate (WER)0.76—Unverified
5MMS-LLaMAWord Error Rate (WER)0.74—Unverified
#ModelMetricClaimedVerifiedStatus
1AVCRFormerTop-1 Accuracy98.81—Unverified
22DCNN + BiLSTM + ResNet + MLFTop-1 Accuracy98.76—Unverified
3PBLTop-1 Accuracy98.3—Unverified
#ModelMetricClaimedVerifiedStatus
1ES³ Base*Word Error Rate (WER)11—Unverified