Computer-assisted human annotation for animal identification

Abstract

Photographic wildlife censusing (PWC) -- in which animals are surveilled by way of photography, entered into a database, and counted -- has historically required significant labor on the part of human annotators, largely due to small and rare well-annotated training datasets. One framework, photographic mark-recapture (or sight-resight), leverages photographs taken by volunteers, scientists, and camera traps, and necessitates the identification of individuals based on visual similarity. State-of-the-art methods for this kind of PWC leverage a detect-classify-rank-verify-annotate pipeline. We focus on the latter three steps in an effort to help spur broader community interest that the other constituent components (detection and classification) have enjoyed for decades. To that end, we formalize the Computer-Assisted Human Annotation (CAHA) problem and explore several metrics and evaluation protocols that indicate algorithmic correctness and expected human labor, including the trade-off between them.