Approximation in L^p(μ) with deep ReLU neural networks

Abstract

We discuss the expressive power of neural networks which use the non-smooth ReLU activation function (x) = \0,x\ by analyzing the approximation theoretic properties of such networks. The existing results mainly fall into two categories: approximation using ReLU networks with a fixed depth, or using ReLU networks whose depth increases with the approximation accuracy. After reviewing these findings, we show that the results concerning networks with fixed depth--- which up to now only consider approximation in L^p() for the Lebesgue measure --- can be generalized to approximation in L^p(), for any finite Borel measure . In particular, the generalized results apply in the usual setting of statistical learning theory, where one is interested in approximation in L^2(P), with the probability measure P describing the distribution of the data.

Tasks

Reproductions