Integrating spatial audio and 360° time-lapse imagery for spatial-temporal data exploration

Abstract

Scientific data is typically presented visually, with little thought to conveying additional information aurally. However, it is clear that there is an interaction between hearing and vision in human perception, and in some cases, acoustic cues are even stronger than visual cues. This research aims to integrate both aural and visual weather information in an intuitive way to facilitate understanding and learning. Time-lapse landscape footage will provide context for the data and will be presented in addition to a traditional visual display of weather information. The audio component is wind and rain noise synthesized in Max/MSP from temperature, wind speed, wind direction, and precipitation data, as well as ambient sound recorded throughout a 24hour period. The goal will be to create an immersive audio environment using ambisonics to enhance and clarify the presentation of the complex visual data.