Bayesian Learning via Q-Exponential Process

Code

• github.com/lanzithinking/q-exp
  OfficialIn papernone★ 1

Abstract

Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter uR^d, an _q penalty term, u_q, is usually added to the objective function. What is the probabilistic distribution corresponding to such _q penalty? What is the correct stochastic process corresponding to u_q when we model functions u L^q? This is important for statistically modeling large dimensional objects, e.g. images, with penalty to preserve certainty properties, e.g. edges in the image. In this work, we generalize the q-exponential distribution (with density proportional to) (- 12|u|^q) to a stochastic process named Q-exponential (Q-EP) process that corresponds to the L_q regularization of functions. The key step is to specify consistent multivariate q-exponential distributions by choosing from a large family of elliptic contour distributions. The work is closely related to Besov process which is usually defined by the expanded series. Q-EP can be regarded as a definition of Besov process with explicit probabilistic formulation and direct control on the correlation length. From the Bayesian perspective, Q-EP provides a flexible prior on functions with sharper penalty (q<2) than the commonly used Gaussian process (GP). We compare GP, Besov and Q-EP in modeling functional data, reconstructing images, and solving inverse problems and demonstrate the advantage of our proposed methodology.

Reproductions