Universal Regular Conditional Distributions

Code

• github.com/AnastasisKratsios/Universal_Regular_Conditional_Distributions_Kratsios_2021
  OfficialIn papertf★ 1

Abstract

We introduce a deep learning model that can universally approximate regular conditional distributions (RCDs). The proposed model operates in three phases: first, it linearizes inputs from a given metric space X to R^d via a feature map, then a deep feedforward neural network processes these linearized features, and then the network's outputs are then transformed to the 1-Wasserstein space P_1(R^D) via a probabilistic extension of the attention mechanism of Bahdanau et al.\ (2014). Our model, called the probabilistic transformer (PT), can approximate any continuous function from R^d to P_1(R^D) uniformly on compact sets, quantitatively. We identify two ways in which the PT avoids the curse of dimensionality when approximating P_1(R^D)-valued functions. The first strategy builds functions in C(R^d,P_1(R^D)) which can be efficiently approximated by a PT, uniformly on any given compact subset of R^d. In the second approach, given any function f in C(R^d,P_1(R^D)), we build compact subsets of R^d whereon f can be efficiently approximated by a PT.

Reproductions