A Canonical Transform for Strengthening the Local L^p-Type Universal Approximation Property

Code

• github.com/bzamanlooy/Architopes
  OfficialIn papernone★ 1
• github.com/AnastasisKratsios/Random_Lipschitz_Partition
  none★ 0

Abstract

Most L^p-type universal approximation theorems guarantee that a given machine learning model class F C(R^d,R^D) is dense in L^p_(R^d,R^D) for any suitable finite Borel measure on R^d. Unfortunately, this means that the model's approximation quality can rapidly degenerate outside some compact subset of R^d, as any such measure is largely concentrated on some bounded subset of R^d. This paper proposes a generic solution to this approximation theoretic problem by introducing a canonical transformation which "upgrades F's approximation property" in the following sense. The transformed model class, denoted by F-tope, is shown to be dense in L^p_,strict(R^d,R^D) which is a topological space whose elements are locally p-integrable functions and whose topology is much finer than usual norm topology on L^p_(R^d,R^D); here is any suitable -finite Borel measure on R^d. Next, we show that if F is any family of analytic functions then there is always a strict "gap" between F-tope's expressibility and that of F, since we find that F can never dense in L^p_,strict(R^d,R^D). In the general case, where F may contain non-analytic functions, we provide an abstract form of these results guaranteeing that there always exists some function space in which F-tope is dense but F is not, while, the converse is never possible. Applications to feedforward networks, convolutional neural networks, and polynomial bases are explored.

Reproductions