Context modeling with inertial mobile sensor

Abstract

The system has also been further evaluated under a more constructive application. "UbiKeyboard" has been developed to detect and predict user's intentional input by analyzing patterns in time series data generated by a wearable smart-glove that is equipped with an accelerometer and gyroscope. With the help of a Web scale natural language model also leveraged, the system is able to recognize user's intentional input with even higher accuracy

Mobile sensors have been around for decades and the number of different kinds are increasing rapidly. With ubiquitous sensors in public facilities, home surveillance equipment and personal mobile devices, there is a great opportunity to leverage those sensors to expand the horizon of humans' ''sensitivity'' to understand the surroundings as well as each individual in a better way. However, mining the time series data produced by those sensors requires domain knowledge and skills in signal processing, data mining and machine learning techniques, which are not everyone's expertise. Motif discovery algorithms have played an important role so far to extract the patterns in time series data. Usually, motifs are acting as crucial medium-high level features in representing a complex event.

However, a few numbers of parameters need to be manually configured that requires expertise in whichever domain the time series data is produced. Meanwhile, lots of the sensors in different mobile devices share same or similar specifications therefore models developed for data produced by one sensor can be applied to those with similar specifications. It is also interesting to notice that inertial sensors, e.g. accelerometers and gyroscopes, have been especially catching a lot of attention as they are widely equipped on mobile devices that humans carry everyday. Therefore, in order to make full use of the sensor data and understand the environment and the human, it is vital to have a system which is efficient, scalable and reusable, and that is capable of analyzing and gaining knowledge from time series data produced by various kinds of sensors.

In this dissertation, the first part of the work focuses on developing the motif detection algorithm to extract time series data patterns efficiently in a scalable approach. We present an unsupervised algorithm which does away with most of the complicated configurations, aggregating sequences of small, atomic motifs to detect compound motifs of arbitrary length. The algorithm finds single dimensional motifs with linear time complexity. With respect to multi-dimensional time series data, the algorithm has also been demonstrated to be able to find the correlated motifs from different dimensions of time series efficiently. The algorithm also consider the presence of noise, lag and jitter as well as other complex relationships between signals in the real application scenario.

The second part of the work is to demonstrate the practicability of the algorithm along with the time series data analysis system in real applications of understanding different perspectives of human activity via inertial sensors on mobile devices. A real time human physical activity recognition web service was developed for understanding sensor data produced by mobile phones. The capability of the system has also been demonstrated via a ''hacker'' system that is able to detect and recover user's virtual keyboard input on a mobile phone by sampling and analyzing data from the background running accelerometer and gyroscope without direct access to user's touch screen.