Robust Speech Recognition via Large-Scale Weak Supervision

Code

• github.com/openai/whisper
  OfficialIn paperpytorch★ 96,448
• github.com/huggingface/transformers
  pytorch★ 158,292
• github.com/ggerganov/whisper.cpp
  none★ 47,845
• github.com/m-bain/whisperx
  pytorch★ 20,861
• github.com/sanchit-gandhi/whisper-jax
  jax★ 4,689
• github.com/whisperspeech/whisperspeech
  pytorch★ 4,576
• github.com/collabora/whisperspeech
  pytorch★ 4,576
• github.com/collabora/whisperlive
  pytorch★ 3,909
• github.com/kadirnar/whisper-plus
  pytorch★ 1,938
• github.com/k2-fsa/icefall
  pytorch★ 1,379

Abstract

We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.

Tasks

Benchmark Results

Reproductions