Low-order methods for nonconvex functional constrained optimization

Abstract

Recently, many real-world problems in engineering and data science not only have very large scales and complicated functional constraints, but also go beyond the scope of convex optimization and inevitably include nonconvex structures. This thesis focuses on developing and analyzing low-order methods for nonconvex functional constrained optimization. In this thesis, I propose several low-order methods, and analyze the complexity of the proposed methods for finding near-KKT points of nonconvex composite problems with either convex or nonconvex functional constraints. All proposed methods generally combine the frameworks of the augmented Lagrangian method, the proximal point method, and my designed subroutines to solve certain unconstrained subproblems. The best-known complexity results are established to all proposed methods on corresponding classes of problems. Numerical experiments demonstrate the efficiency of the proposed methods on a large number of both classical optimization problems and real-world machine learning examples.