Head pose estimation on deep CNN models

Abstract

Head pose estimation is an important computer vision task. Its applications can benefit people's daily life. In this thesis, our goal is to solve the head pose estimation problem using deep convolutional neural networks. The first task is to perform head pose estimation as a classification task. We propose a multimodal convolutional neural network(CNN) for head pose classification. The architecture of the model consists of three pathways whose inputs are face image, head image, and facial landmarks, which respectively capture the face appearance, facial context, and facial shape. We first perform the experiments on benchmark datasets. Then we perform head pose classification on low-quality driving videos. In order to deal with the noises in the videos, we propose the Max-Feature Map(MFM) with the help of Network-In-Network(NIN) for the CNN model, which has a better capability of handling the noises and the feature selection.

We also use training techniques such as _ne-tuning and joint training to improve the performance on driving videos. The second task is to predict head pose angles using regression. We propose a deep CNN model which uses larger face image as the input and outputs all three head pose angles. The model has more layers and more parameters. Because head pose estimation needs more face images with various large head poses, face detection method is an essential part of data processing. Thus besides the face detection method we use in head pose classification, we also use a deep-learning face detection method based on region-based convolution neural network(R-CNN). We compare the performance of these two face detection methods on some datasets with continuous head poses, including benchmark datasets, driving videos, and some videos we record with a head tracker device. From head pose classification on benchmark datasets to head pose estimation on arbitrary data, we move on to more challenging tasks step by step. The experimental results on all these tasks and the comparison with other state-of-art methods show that our methods achieve a good robustness as well as a better accuracy, compared to the baseline and existing methods.