PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

Code

• github.com/openai/pixel-cnn
  OfficialIn papertf★ 0
• github.com/ajayjain/lmconv
  pytorch★ 79
• github.com/tuatruog/astrocompress
  pytorch★ 1
• github.com/pclucas14/pixel-cnn-pp
  pytorch★ 0
• github.com/kamenbliznashki/pixel_models
  pytorch★ 0
• github.com/sarus-tech/tf2-published-models/tree/master/pixelcnn_plus
  tf★ 0
• github.com/andrecianflone/vector_quantization
  pytorch★ 0
• github.com/google/flax/tree/master/examples/pixelcnn
  jax★ 0

Abstract

PixelCNNs are a recently proposed class of powerful generative models with tractable likelihood. Here we discuss our implementation of PixelCNNs which we make available at https://github.com/openai/pixel-cnn. Our implementation contains a number of modifications to the original model that both simplify its structure and improve its performance. 1) We use a discretized logistic mixture likelihood on the pixels, rather than a 256-way softmax, which we find to speed up training. 2) We condition on whole pixels, rather than R/G/B sub-pixels, simplifying the model structure. 3) We use downsampling to efficiently capture structure at multiple resolutions. 4) We introduce additional short-cut connections to further speed up optimization. 5) We regularize the model using dropout. Finally, we present state-of-the-art log likelihood results on CIFAR-10 to demonstrate the usefulness of these modifications.

Tasks

Benchmark Results

Reproductions