Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein space

Code

• github.com/apooladian/mfvi
  OfficialIn papernone★ 5

Abstract

We develop a theory of finite-dimensional polyhedral subsets over the Wasserstein space and optimization of functionals over them via first-order methods. Our main application is to the problem of mean-field variational inference, which seeks to approximate a distribution over R^d by a product measure ^. When is strongly log-concave and log-smooth, we provide (1) approximation rates certifying that ^ is close to the minimizer ^_ of the KL divergence over a polyhedral set P_, and (2) an algorithm for minimizing KL(\|) over P_ based on accelerated gradient descent over ^d. As a byproduct of our analysis, we obtain the first end-to-end analysis for gradient-based algorithms for MFVI.

Tasks

Reproductions