A new Linear Time Bi-level _1, projection ; Application to the sparsification of auto-encoders neural networks

Code

• github.com/MichelBarlaud/SAE-Supervised-Autoencoder-Omics
  Officialpytorch★ 5

Abstract

The _1, norm is an efficient-structured projection, but the complexity of the best algorithm is, unfortunately, O(n m (n m)) for a matrix n m.\\ In this paper, we propose a new bi-level projection method, for which we show that the time complexity for the _1, norm is only O(n m ) for a matrix n m. Moreover, we provide a new _1, identity with mathematical proof and experimental validation. Experiments show that our bi-level _1, projection is 2.5 times faster than the actual fastest algorithm and provides the best sparsity while keeping the same accuracy in classification applications.

Reproductions