DiffWave: A Versatile Diffusion Model for Audio Synthesis

Code

• github.com/lmnt-com/diffwave
  pytorch★ 889
• github.com/neillu23/cdiffuse
  pytorch★ 250
• github.com/keonlee9420/DiffSinger
  pytorch★ 247
• github.com/albertfgu/diffwave-sashimi
  pytorch★ 128
• github.com/philsyn/diffwave-vocoder
  pytorch★ 90
• github.com/philsyn/diffwave-unconditional
  pytorch★ 43
• github.com/revsic/tf-diffwave
  tf★ 42
• github.com/revsic/jax-variational-diffwave
  jax★ 40
• github.com/neillu23/DiffuSE
  pytorch★ 36
• github.com/revsic/torch-diffusion-wavegan
  pytorch★ 17

Abstract

In this work, we propose DiffWave, a versatile diffusion probabilistic model for conditional and unconditional waveform generation. The model is non-autoregressive, and converts the white noise signal into structured waveform through a Markov chain with a constant number of steps at synthesis. It is efficiently trained by optimizing a variant of variational bound on the data likelihood. DiffWave produces high-fidelity audios in different waveform generation tasks, including neural vocoding conditioned on mel spectrogram, class-conditional generation, and unconditional generation. We demonstrate that DiffWave matches a strong WaveNet vocoder in terms of speech quality (MOS: 4.44 versus 4.43), while synthesizing orders of magnitude faster. In particular, it significantly outperforms autoregressive and GAN-based waveform models in the challenging unconditional generation task in terms of audio quality and sample diversity from various automatic and human evaluations.

Tasks

Benchmark Results

Reproductions