An investigation of usability factors in designing a companion system for an immersive environment
Loading...
Authors
Chang, Albert
Issue Date
2024-12
Type
Electronic thesis
Thesis
Thesis
Language
en_US
Keywords
Architectural sciences
Alternative Title
Abstract
Dialogue systems have become a popular research medium as recent advances in task-orientedand open-domain systems combined with deep learning technologies have increased the po-
tential for practical applications across many disciplines. One such domain of applications
involves multimodal dialogue systems deployed in interactive spaces that seek to provide
an immersive experience for participants. To take advantage of the opportunities allowed
by advances in the hardware and software aspects of immersive environments, this thesis
proposes a design concept for companion system deployed in the EMPAC Research Lab at
Rensselaer Polytechnic Institute and conducts two experiment aimed to demonstrate the
practicality of the audiovisual capabilities of the system. The companion system, named
Scalable Cognitive Immersive Learning & Analysis Room (SCILAR), will utilize a two-part
combination system that provides users with their own local device that is connected to a
shared global space. This shared global space is capable of spatial awareness with a multi-
modal immersive dialogue system and an array of audio and visual sensors that can track
multiple participants within this interactive space across both physical and virtual mediums.
The system will also provide a transcription of conversations that can be used for enabling
further functionality and is provided as-is to track conversation history. Testing on the us-
ability and practicality of this system was conducted via two user studies. The first study
involved a collaborative data analytics scenario that has participants analyze New York City
taxi trips using a made-to-task Google Maps web app and the Google Maps website. The
second study investigated the effect of an acoustic pointer to help with locating alphanumeric
pairs located in a virtual environment displayed on a panoramic screen. The findings of both
of these experiments show statistical significance in the reduction of task completion time
when compared to baseline conditions. This thesis aims to provide insight into interactive
spaces for education and analyst applications and demonstrate the potential capabilities of
combining spatial awareness with a multimodal dialogue system.
Description
December 2024
School of Architecture
School of Architecture
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY