Embedding Inequalities for Barron-type Spaces

Abstract

An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space B_s() and the spectral Barron space F_s(), where the index s [0,) indicates the smoothness of functions within these spaces and R^d denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any (0,1), s N^+ and f: R, it holds that \[ \|f\|_F_s- ( ) _s \|f\|_B_s( ) _s \|f\|_F_s+1( ). \] Importantly, the constants do not depend on the input dimension d, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.

Tasks

Reproductions