A study of classification and embedding methods for identifying humpback whales

Abstract

The two approaches that we consider are a classification-based approach and an embedding-based one. We were able to achieve a top-1 accuracy of 83.0% for the classifier and of 80.5% when using embeddings. While both approaches showed good results towards identifying novel individuals, there were drawbacks and benefits to using one over the other. Most importantly, we show that a classification-based approach is most appropriate for quickly learning the weights for the used model. It also consistently performs better overall than the embedding-based approach. When using embeddings however, because of the use of an embedding function to acquire the feature vectors that represent our known individuals, there is the possibility to convert new individuals to known individuals without the need for retraining. We achieve a top-1 accuracy of 76.5% with our embedding approach on newly added individuals with no retraining. We also show that a trained classifier can be converted to an embedding model with no or minimal retraining needed.

Many current methods for the identification of individuals of a species do not consider the problem of identifying previously unseen individuals. To be used in a real-world setting, these methods must be able to recognize that all individuals they encounter will not all necessarily be part of the set of individuals the methods were trained to recognize. In this thesis, we explore two different approaches for the identification of new individuals.