L_2BN: Enhancing Batch Normalization by Equalizing the L_2 Norms of Features

Abstract

In this paper, we analyze batch normalization from the perspective of discriminability and find the disadvantages ignored by previous studies: the difference in l_2 norms of sample features can hinder batch normalization from obtaining more distinguished inter-class features and more compact intra-class features. To address this issue, we propose a simple yet effective method to equalize the l_2 norms of sample features. Concretely, we l_2-normalize each sample feature before feeding them into batch normalization, and therefore the features are of the same magnitude. Since the proposed method combines the l_2 normalization and batch normalization, we name our method L_2BN. The L_2BN can strengthen the compactness of intra-class features and enlarge the discrepancy of inter-class features. The L_2BN is easy to implement and can exert its effect without any additional parameters or hyper-parameters. We evaluate the effectiveness of L_2BN through extensive experiments with various models on image classification and acoustic scene classification tasks. The results demonstrate that the L_2BN can boost the generalization ability of various neural network models and achieve considerable performance improvements.

Tasks

Reproductions