On weight and variance uncertainty in neural networks for regression tasks

Code

• github.com/mortamini/vbnet
  OfficialIn papernone★ 1

Abstract

We consider the problem of weight uncertainty proposed by [Blundell et al. (2015). Weight uncertainty in neural network. In International conference on machine learning, 1613-1622, PMLR.] in neural networks (NNs) specialized for regression tasks. We further investigate the effect of variance uncertainty in their model. We show that including the variance uncertainty can improve the prediction performance of the Bayesian NN. Variance uncertainty enhances the generalization of the model by considering the posterior distribution over the variance parameter. We examine the generalization ability of the proposed model using a function approximation example and further illustrate it with the riboflavin genetic data set. We explore fully connected dense networks and dropout NNs with Gaussian and spike-and-slab priors, respectively, for the network weights.

Tasks

Reproductions