Complementarity formulation for sparse optimization

Abstract

Complementarity constraints are used to enforce the orthogonality between two decision vectors (or matrices). They naturally appear in mathematical programs for many applications and also in reformulations of certain non-convex optimization problems. Compared to a convex relaxation, a complementarity reformulation of a non-convex model can often yield a better-quality solution. In this thesis, I explore the complementarity reformulation to sparse optimization problems that can simultaneously include $\ell_0$ and rank minimization. Two applications are studied. One is the robust principal component analysis (RPCA), and the other is the low-rank representation (LRR) problem. For RPCA, I introduce complementarity constraints to both $\ell_0$-term and the rank function. For the LRR problem, I give complementarity reformulation to the original model and also a low-rank matrix decomposition based model. For the original model, the complementarity constraints are introduced to the rank function and also the column $\ell_0$-term, and for the low-rank decomposition model, the complementarity constraints are imposed to both low-rank factor matrices. Numerically, I apply the penalty method to each complementarity reformulated problem, with the block coordinate descent method for solving a sequence of penalty subproblems. Global (or whole-sequence) convergence and also local convergence (to global optimality) are shown. The effectiveness and efficiency of my methods are demonstrated by comparing to convex relaxation methods. In terms of recoverability, my method tends to give higher rate of successful recovery. Also, for the LRR problem, solving the low-rank matrix decomposition based model can save the computing time, as compared to solving the original model by the complementarity reformulation.