Element-wise matrix sparsification and reconstruction

Abstract

(3) CUR-Decomposition using Element-wise Sampling: Here we consider another reconstruction method, namely the CUR-Decomposition, by sampling elements from a matrix. Existing CUR algorithms use all the elements of a matrix in order to achieve certain level of reconstruction accuracy. In this work, we discuss some of the reconstruction algorithms that need only a handful of elements of a matrix in order to reconstruct it with some provable guarantee.

The work presented here is focused mainly on sampling elements from a matrix. We study three topics of current research in Theoretical Computer Science, Machine Learning, and Compressed Sensing involving element-wise sampling: (1) Element-wise Matrix Sparsification, (2) Low-rank Matrix Completion, and (3) CUR Decomposition using Element-wise Sampling. Below we give a high-level description of the topics while leaving the details in subsequent chapters.

(1) Fast Low-rank Approximation: Given a matrix we want to sample elements from it based on some probability distribution defined on its elements, such that, we can (approximately) reconstruct the matrix, in some matrix norm, based only on these sampled elements. We want to sample a small number of elements in order to achieve a certain degree of reconstruction accuracy. We propose a generalization of two existing popular sampling methods, and show that our method requires strictly smaller sample size than existing methods. Further, we show that the computation time of the PCA of such sparsified data is significantly faster than that of the full data, while the quality of the PCA of the sparsified data is nearly as good as the true PCA.

(2) Low-rank Matrix Completion: We use the nuclear norm minimization framework to reconstruct a low-rank matrix by observing only few of its elements. We seek to reduce the number of elements to be observed in order to reconstruct the matrix exactly. For this, we investigate a novel form of distribution on the elements of a matrix. We show theoretical analysis and experimental results to highlight some of the properties of this distribution in the context of low-rank matrix completion. Our proposed method outperforms the best-known method.