Contour-based instance recognition of animals

Abstract

We address the problem of identifying individual animals from contours that contain identifying markings. The objective is to accelerate the photo identification component of an ecological field survey by presenting potential matches for an image of a query animal in order of similarity. For the solution to be scalable to large populations and new species, it must not require human interaction and should be frugal with the number of labeled training examples required.

We demonstrate qualitatively and quantitatively that the contour extraction algorithm is able to extract the identifying contour from an image, with the learned fine-grained probability map outperforming the gradient-based baseline. Finally, we evaluate the matching algorithm on datasets that are used for photo identification of bottlenose dolphins, humpback whales, and elephants. The matching algorithm achieves top-1 accuracy scores of 95%, 85%, and 84% for these three species, respectively.

The algorithm consists of three major stages, namely, contour extraction, representation, and matching. First, the location of the identifying contour in an image is approximated using an algorithm that learns the contour appearance from data. To avoid the need for a large, labeled training set, the contour appearance is first learned at a very coarse level. This coarse approximation is then refined to a fine-grained cost matrix using either a traditional image gradient-based method or a novel learning algorithm that learns the contour appearance at a finer scale. We introduce a method for generating synthetic contour boundary patches to avoid the need for fine-grained training data, which is labor-intensive to collect. The identifying contour is then extracted from this probability map using either a top-down or bottom-up approach to solving one or more shortest-path problems. Second, the contour is encoded using an integral curvature representation that is robust to changes in viewpoint and pose. Finally, we combine this representation with two matching algorithms. The first treats the curvature representation as a sequence and defines the similarity of two contours in terms of their alignment cost using a dynamic time-warping algorithm. The second is a descriptor-based approach using the local naive Bayes nearest neighbors algorithm.