Penalized versus constrained generalized eigenvalue problems

Abstract

We investigate the difference between using an _1 penalty versus an _1 constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an _1 penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an _1 constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of an _1 constraint in the context of discriminant analysis and principal component analysis.

Tasks

Reproductions