High-dimensional data analysis by exploiting low-dimensional models with applications in synchrophasor data analysis in power systems

Abstract

Motivated by missing data recovery in power system monitoring, we then study the problem of recovering missing entries of high-dimensional signals that exhibit low-dimensional nonlinear structures. We propose a novel model, termed as ``union and sums of subspaces'', to characterize practical nonlinear datasets. In this model, each data point belongs to either one of a few low-dimensional subspaces or the sum of a subset of subspaces. We propose convex-optimization-based methods to recover missing entries under this model. We theoretically analyze the recovery guarantee of our proposed methods with both noiseless and noisy measurements. Numerical experiments on synthetic data and simulated power system data are conducted to verify the effectiveness of the proposed methods.

We first consider recovering missing synchrophasor measurements. Leveraging the approximate low-rank property of PMU data, we connect the problem of recovering PMU data erasures with recent advances in low-rank matrix completion methods. Since the existing analysis for matrix completion methods assumes an independent-erasure model that does not capture the correlations in PMU erasures, we propose two models to characterize the temporal and the channel correlations in PMU erasures and provide theoretical guarantees of a matrix completion method in recovering correlated erasures in both models. We also propose an online algorithm that can fill in the missing PMU measurements for real-time applications. Numerical experiments on actual PMU data are conducted to verify the effectiveness of the proposed methods.

We also consider recovering PMU data from quantized and partially corrupted measurements. The data recovery is achieved through solving a constrained maximum likelihood estimation problem that exploits the low-rank property of the actual measurements. The recovery error is proven to be order-optimal and decays in the same order as that of the state-of-the-art method when no corruption exists. The data accuracy is thus maintained while the data privacy is enhanced. A new application of this method for data privacy in power systems is discussed. Experiments on synthetic data and real synchrophasor data in power systems demonstrate the effectiveness of our method.

We next present a new method for identifying a series of cyber data attacks on power system synchrophasor measurements. We focus on detecting "unobservable" cyber data attacks that cannot be detected by any existing method that purely relies on measurements received at one time instant. Leveraging the approximate low-rank property of PMU data, we formulate the identification problem of successive unobservable cyber attacks as a matrix decomposition problem of a low-rank matrix plus a transformed column-sparse matrix. We propose a convex-optimization-based method and provide its theoretical guarantee in the data identification. Numerical experiments on actual PMU data from the Central New York power system and synthetic data are conducted to verify the effectiveness of the proposed method.

With the increasing deployment of phasor measurement units (PMUs) in the power system, utilities and power grid operators are collecting an unprecedented amount of high sampling rate PMU data including frequency, bus voltage phasor, and line current phasor. The data owners are interested in developing efficient algorithms to process and extract as much information as possible from such PMU data for real-time and off-line analysis. Traditional data analysis methods typically analyze each channel of PMU data separately, and then combine the results from the individual analysis to get some conclusions. In this thesis, a spatial-temporal framework for efficient processing of PMU data blocks is proposed. A key property of PMU data blocks is that they have some low-dimensional structures. By exploiting the low-dimensional structures in PMU data, we propose the optimization methods to deal with various data management issues such as missing data recovery, cyber data attack detection, and data privacy enhancement in this thesis.