Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling

Code

• github.com/Daniil-Selikhanovych/ebm-wgan
  pytorch★ 7
• github.com/JHpark1677/CGAN-DDLS
  none★ 2
• github.com/ramonpeter/LaSeR
  pytorch★ 2

Abstract

We show that the sum of the implicit generator log-density p_g of a GAN with the logit score of the discriminator defines an energy function which yields the true data density when the generator is imperfect but the discriminator is optimal, thus making it possible to improve on the typical generator (with implicit density p_g). To make that practical, we show that sampling from this modified density can be achieved by sampling in latent space according to an energy-based model induced by the sum of the latent prior log-density and the discriminator output score. This can be achieved by running a Langevin MCMC in latent space and then applying the generator function, which we call Discriminator Driven Latent Sampling~(DDLS). We show that DDLS is highly efficient compared to previous methods which work in the high-dimensional pixel space and can be applied to improve on previously trained GANs of many types. We evaluate DDLS on both synthetic and real-world datasets qualitatively and quantitatively. On CIFAR-10, DDLS substantially improves the Inception Score of an off-the-shelf pre-trained SN-GAN~sngan from 8.22 to 9.09 which is even comparable to the class-conditional BigGAN~biggan model. This achieves a new state-of-the-art in unconditional image synthesis setting without introducing extra parameters or additional training.

Tasks

Reproductions