Augmenting collaborative immersive systems through cognitive technologies for educational scenarios
Loading...
Authors
Chabot, Samuel Roger Vincent
Issue Date
2022-08
Type
Electronic thesis
Thesis
Thesis
Language
en_US
Keywords
Architectural sciences
Alternative Title
Abstract
Virtual reality continues to proliferate in the classroom. Human-scale immersive virtual environments provide natural benefits and assuage many of the issues raised with head-mounted displays. While a headset removes a user from his or her physical environment, these spaces allow users to experience virtual realities in groups, maintaining authentic communication between peers. This work contributes the creation and utilization of necessary cognitive technologies and services for educational scenarios as well as conducts the first controlled quantitative study on learning within these environments this author is aware of. A believed benefit of learning within these environments is the ability to enter virtual spaces without abandoning physical and social connections. In a successful educational environment, students and instructors must be able simultaneously to see content on the screen and each other. A designed intelligent, steerable lighting system illuminates the workspace for the task at hand while reducing illumination at the screen by over 97 %. Students and instructors can affect the system from their personal devices, while a network layer maintains input from spatial tracking and user-developed systems. This combination of experienced simplicity and retained flexibility for a lighting system within a human-scale immersive environment is an important contribution to educational scenarios. The irregular geometry of the CRAIVE-Lab screen introduces distortion to panoramic imagery, which is counteractable via a geometric transformation. Whereas for traditional cube-shaped CAVE systems, this transformation is linear and can be described with simple calculations, that required for the CRAIVE-Lab is more complex due to the nonlinearity introduced by the rounded corners. This work determines the mathematically-derived solution for describing this necessary transformation. This generated image warping solution is adaptable to other environments of irregular projections. It is executed via a Python script and performed for users without needing any additional software or files or knowledge of the underlying mathematics. The script does so 100 times faster than the previous method, in a fraction of a second, and in a live classroom setting. To generate spatialized audio for headsets, developers are presented with APIs. However the application of this model to a human-scale immersive environment is a necessary development. The produced Spatial Audio Worker changes the way developers and users consider the environments’ audio arrays. It presents developers with standardized inputs to the loudspeaker arrays, requiring no knowledge of underlying hardware or software. Developers of educational applications (and beyond) can spatialize audio and create congruent content with a few lines of familiar code. This method has been adopted by multiple applications, which now require no tailoring to be used across multiple arrays and environments. Many internet ``cloud’’ services enable new functions and accelerate content creation. A produced navigation interface utilizes the Street View Static API to retrieve and properly format for the screen the necessary imagery for panoramic scene generation. This navigator, the only such known for human-scale immersive environments, instantly transports classes to locations around the globe. This is combined with additional cloud services such as natural language processing and object detection for scenarios such as the Language Learning Environment.Additionally, as these environments proliferate, their limited available time is most valuable for final use cases and courses. Therefore, a virtual testing environment is produced which can be experienced at scale in a head-mounted display. This testing environment enables parallel and remote testing while keeping the physical environment free for educational purposes, providing instructors and students alike with the first known headset-based testing application of a human-scale environment. In a holistic presentation of these cognitive technologies and services, the Language Learning Environment at Rensselaer transports students to virtual- and real-world scenes in which to practice their foreign language skills. This experience is made possible directly through the discussed cognitive technologies and services. It is assessed in both a qualitative and quantitative study. The quantitative study is believed to be the first of its kind. Results indicate that learning in the environment is effective, knowledge is retained over time, and students rate the experience as enjoyable and engaging. This work has profoundly altered interaction with the environment. The ability for parallel contributions of content by students and instructors to a visual and aural human-scale immersive environment via personal devices for the purposes of educational scenarios is a contribution of this work. Users engage directly with the panoramic display via the Pin-Up application, and tasks that previously consumed large portions of class are now completed quickly, asynchronously, and remotely. Multiple novel courses are held in the environments, including Aural Architecture taught by Jonas Braasch in the Acoustics Department and Nuclear Phenomena for Engineering Applications taught by Emily Liu in the Department of Mechanical, Aerospace, & Nuclear Engineering. The wide variety of use cases which have already adopted these cognitive technologies and services indicate much more future research is available for pursuit.
Description
August 2022
School of Architecture
School of Architecture
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY