A Simple Sparse Denoising Layer for Robust Deep Learning

Abstract

Deep models have achieved great success in many applications. However, vanilla deep models are not well-designed against the input perturbation. In this work, we take an initial step to designing a simple robust layer as a lightweight plug-in for vanilla deep models. To achieve this goal, we first propose a fast sparse coding and dictionary learning algorithm for sparse coding problem with an exact k-sparse constraint or l_0 norm regularization. Our method comes with a closed-form solution for the sparse coding phase by taking advantage of a novel structured dictionary. With this handy solution, we propose a simple sparse denoising layer (SDL) as a lightweight robust plug-in. Extensive experiments on both classification and reinforcement learning tasks manifest the effectiveness of our methods.

Tasks

Reproductions