Understanding disentangling in β-VAE
Christopher P. Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, Alexander Lerchner
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/ema-marconato/glancenetpytorch★ 25
- github.com/mmrl/disent-and-genpytorch★ 21
- github.com/lanzhang128/disentanglementtf★ 7
- github.com/JohanYe/Beta-VAEpytorch★ 1
- github.com/adityabingi/Beta-VAEtf★ 0
- github.com/Knight13/beta-VAE-disentanglementpytorch★ 0
- github.com/Minzhe/VAE_animefacepytorch★ 0
- github.com/1Konny/Beta-VAEpytorch★ 0
- github.com/seymayucer/VAEspytorch★ 0
- github.com/alexbooth/Beta-VAE-Tensorflow-2.0tf★ 0
Abstract
We present new intuitions and theoretical assessments of the emergence of disentangled representation in variational autoencoders. Taking a rate-distortion theory perspective, we show the circumstances under which representations aligned with the underlying generative factors of variation of data emerge when optimising the modified ELBO bound in -VAE, as training progresses. From these insights, we propose a modification to the training regime of -VAE, that progressively increases the information capacity of the latent code during training. This modification facilitates the robust learning of disentangled representations in -VAE, without the previous trade-off in reconstruction accuracy.