Automatic recognition and immersive representation of environmental soundscapes

Abstract

The amount of audio data required for bioacoustics monitoring and other applications is often too large to be manually sorted and analyzed. Consequently, automatic sound recognition techniques have to be applied to automatically (1) identify and extract relevant acoustic stimuli and (2) classify these stimuli after a training period. Using audio data collected in the field over a period of several months in Troy, NY, the ability of three different automatic sound recognition techniques at classifying environmental sound stimuli is compared. These three algorithms include a hidden Markov model, a multilayer perceptron, and a convolutional neural network pretrained on Google's Inception-v3. The most accurate algorithm, the convolutional neural network, is then used to automatically classify an unvalidated dataset to allow seasonal soundscape changes to be observed. The analysis can then be used to create an automated non-linear time-lapse to summarize the events that occurred and meaningfully re-represent them in our immersive virtual environment, the CRAIVE-Lab -- together with collected visual material.