SOTAVerified

Audio-Visual Speech Recognition

Audio-visual speech recognition is the task of transcribing a paired audio and visual stream into text.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 76–100 of 100 papers

TitleStatusHype
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs—0
Building a synchronous corpus of acoustic and 3D facial marker data for adaptive audio-visual speech synthesis—0
Chinese-LiPS: A Chinese audio-visual speech recognition dataset with Lip-reading and Presentation Slides—0
RUSAVIC Corpus: Russian Audio-Visual Speech in Cars—0
Cocktail-Party Audio-Visual Speech Recognition—0
Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices—0
Scaling and Enhancing LLM-based AVSR: A Sparse Mixture of Projectors Approach—0
DCIM-AVSR : Efficient Audio-Visual Speech Recognition via Dual Conformer Interaction Module—0
Visual Speech Recognition—0
Deep Multimodal Learning for Audio-Visual Speech Recognition—0
Deep Multimodal Representation Learning from Temporal Data—0
Detecting Adversarial Attacks On Audiovisual Speech Recognition—0
SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition—0
Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading—0
ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations—0
Fusing information streams in end-to-end audio-visual speech recognition—0
Streaming Audio-Visual Speech Recognition with Alignment Regularization—0
SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer—0
SUTAV: A Turkish Audio-Visual Database—0
Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition—0
XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception—0
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction—0
Kaggle Competition: Cantonese Audio-Visual Speech Recognition for In-car Commands—0
Audio-visual Recognition of Overlapped speech for the LRS2 dataset—0
Large-vocabulary Audio-visual Speech Recognition in Noisy Environments—0
Show:102550
← PrevPage 4 of 4Next →
All datasetsLRS2LRS3-TEDLRWCAS-VSR-S101

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hybrid CTC / AttentionWord Error Rate (WER)39.1—Unverified
2TM-Seq2seqTest WER8.5—Unverified
3TM-CTCTest WER8.2—Unverified
4CTC/AttentionTest WER7—Unverified
5CTC/AttentionTest WER1.5—Unverified
6Whisper-FlamingoTest WER1.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Hyb-ConformerWord Error Rate (WER)2.3—Unverified
2Zero-AVSRWord Error Rate (WER)1.5—Unverified
3AV-HuBERT LargeWord Error Rate (WER)1.4—Unverified
4Whisper-FlamingoWord Error Rate (WER)0.76—Unverified
5MMS-LLaMAWord Error Rate (WER)0.74—Unverified
#ModelMetricClaimedVerifiedStatus
1AVCRFormerTop-1 Accuracy98.81—Unverified
22DCNN + BiLSTM + ResNet + MLFTop-1 Accuracy98.76—Unverified
3PBLTop-1 Accuracy98.3—Unverified
#ModelMetricClaimedVerifiedStatus
1ES³ Base*Word Error Rate (WER)11—Unverified