Predicting change points in multivariate time series data
Loading...
Authors
Khan, Haidar
Issue Date
2019-05
Type
Electronic thesis
Thesis
Thesis
Language
ENG
Keywords
Computer science
Alternative Title
Abstract
Time series data produced by a dynamic system may contain multiple states characterized by distinct patterns. Predicting the phase transitions between them is critical for many important applications including automatic speech recognition, network intrusion detection, machine failure prediction, and epileptic seizure prediction. This thesis presents methods that combine change point detection algorithms and deep neural networks to detect phase transitions in multichannel time series data.
Thus this thesis contains several contributions to the fields of change point detection and machine learning on time series. A method for end-to-end learning on time series is developed and demonstrated on a diverse range of settings. Novel techniques for change point detection are proposed and evaluated. Finally, the proposed methods are combined to improve seizure prediction, paving the way for seizure prediction devices that can improve the lives of people suffering from epilepsy.
Second, we discover hidden states in multichannel time series data to improve the performance of predictive time series models. For example, epileptic seizure prediction models consist of defining two essential quantities: (i) a preseizure period in which prediction of the seizure is as early and accurate as possible and (ii) a set of features based on brain activity that distinguishes between the nonseizure and preseizure periods. We determine these two components jointly using deep learning and change point detection. In our model, convolutional networks trained on the wavelet transformation of the scalp EEG signal are used to learn representations for each relevant period while the preseizure period is learned from the data. Our results on a large dataset of scalp EEG recordings show an approximately ten-minute patient independent preseizure period is most accurate for seizure prediction. Furthermore, our method significantly outperforms a random predictor and other seizure prediction algorithms with a higher sensitivity and a lower false prediction rate. As a result, we demonstrate that a robust feature representation can be learned from scalp EEG that characterizes the preseizure state of focal seizures.
Third, we introduce a method for change point detection based on density ratio estimation. In particular, we train deep neural networks as function approximators to the density ratio function. State of the art density ratio estimation methods solve a convex constrained minimization problem to fit non-parametric kernel functions to the density ratio. The same minimization problem cannot be used to train a deep neural network to approximate the density ratio function. We formulate new objective functions for this problem that can be minimized using gradient descent. Our results show that the network can be trained to effectively approximate the density ratio function, even for complex distributions. On a seizure detection task, our method outperforms other (kernel and neural network based) density ratio estimation methods and change point detection algorithms.
First, we consider the transformation of multichannel time series data into spectral decompositions. These transformations are typically computed during a preprocessing step that involves many hyperparameters, such as window length, filter width, and filter shape. We propose an efficient alternative, called the wavelet deconvolution (WD) layer, to eliminate a significant number of hyperparameters. The WD layer learns the spectral decomposition using the wavelet transform with adjustable scale parameters applied directly to the signal. Learning the scale parameters allows the WD layer to extract the frequency content relevant to the classification task. Applying this method to automatic speech recognition results in a relative improvement of 4% over a strong baseline. Experiments on a time series classification task where engineered features are not available showed the WD layer combined with a convolutional network is the best performing method. The results demonstrate that using the WD layer can improve neural network based time series classifiers in accuracy, sample complexity, and interpretability by learning directly from the input signal.
Thus this thesis contains several contributions to the fields of change point detection and machine learning on time series. A method for end-to-end learning on time series is developed and demonstrated on a diverse range of settings. Novel techniques for change point detection are proposed and evaluated. Finally, the proposed methods are combined to improve seizure prediction, paving the way for seizure prediction devices that can improve the lives of people suffering from epilepsy.
Second, we discover hidden states in multichannel time series data to improve the performance of predictive time series models. For example, epileptic seizure prediction models consist of defining two essential quantities: (i) a preseizure period in which prediction of the seizure is as early and accurate as possible and (ii) a set of features based on brain activity that distinguishes between the nonseizure and preseizure periods. We determine these two components jointly using deep learning and change point detection. In our model, convolutional networks trained on the wavelet transformation of the scalp EEG signal are used to learn representations for each relevant period while the preseizure period is learned from the data. Our results on a large dataset of scalp EEG recordings show an approximately ten-minute patient independent preseizure period is most accurate for seizure prediction. Furthermore, our method significantly outperforms a random predictor and other seizure prediction algorithms with a higher sensitivity and a lower false prediction rate. As a result, we demonstrate that a robust feature representation can be learned from scalp EEG that characterizes the preseizure state of focal seizures.
Third, we introduce a method for change point detection based on density ratio estimation. In particular, we train deep neural networks as function approximators to the density ratio function. State of the art density ratio estimation methods solve a convex constrained minimization problem to fit non-parametric kernel functions to the density ratio. The same minimization problem cannot be used to train a deep neural network to approximate the density ratio function. We formulate new objective functions for this problem that can be minimized using gradient descent. Our results show that the network can be trained to effectively approximate the density ratio function, even for complex distributions. On a seizure detection task, our method outperforms other (kernel and neural network based) density ratio estimation methods and change point detection algorithms.
First, we consider the transformation of multichannel time series data into spectral decompositions. These transformations are typically computed during a preprocessing step that involves many hyperparameters, such as window length, filter width, and filter shape. We propose an efficient alternative, called the wavelet deconvolution (WD) layer, to eliminate a significant number of hyperparameters. The WD layer learns the spectral decomposition using the wavelet transform with adjustable scale parameters applied directly to the signal. Learning the scale parameters allows the WD layer to extract the frequency content relevant to the classification task. Applying this method to automatic speech recognition results in a relative improvement of 4% over a strong baseline. Experiments on a time series classification task where engineered features are not available showed the WD layer combined with a convolutional network is the best performing method. The results demonstrate that using the WD layer can improve neural network based time series classifiers in accuracy, sample complexity, and interpretability by learning directly from the input signal.
Description
May 2019
School of Science
School of Science
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY