Clustering, multicollinearity, and singular vectors

Abstract

Let A be a matrix with its pseudo-matrix A^ and set S=I-A^A. We prove that, after re-ordering the columns of A, the matrix S has a block-diagonal form where each block corresponds to a set of linearly dependent columns. This allows us to identify redundant columns in A. We explore some applications in supervised and unsupervised learning, specially feature selection, clustering, and sensitivity of solutions of least squares solutions.

Tasks

Reproductions