SOTAVerified

Audio-Visual Speech Recognition

Audio-visual speech recognition is the task of transcribing a paired audio and visual stream into text.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–10 of 100 papers

TitleStatusHype
ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition—0
Cocktail-Party Audio-Visual Speech Recognition—0
The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition—0
Scaling and Enhancing LLM-based AVSR: A Sparse Mixture of Projectors Approach—0
SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer—0
CoGenAV: Versatile Audio-Visual Representation Learning via Contrastive-Generative SynchronizationCode2
Chinese-LiPS: A Chinese audio-visual speech recognition dataset with Lip-reading and Presentation Slides—0
Visual-Aware Speech Recognition for Noisy Scenarios—0
MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech TokensCode1
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs—0
Show:102550
← PrevPage 1 of 10Next →
All datasetsLRS2LRS3-TEDLRWCAS-VSR-S101

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hyb-ConformerWord Error Rate (WER)2.3—Unverified
2Zero-AVSRWord Error Rate (WER)1.5—Unverified
3AV-HuBERT LargeWord Error Rate (WER)1.4—Unverified
4Whisper-FlamingoWord Error Rate (WER)0.76—Unverified
5MMS-LLaMAWord Error Rate (WER)0.74—Unverified