In collaboration with Dr. Jennifer Shields (Department of Architecture) we aim to investigates the combined application of spatial computing (including mixed reality technologies) alongside Emotion-AI (artificial intelligence focused on recognizing human emotional states in real-time) at cultural heritage sites. Spatial computing has the potential to enhance multi-sensory experiences and deepen contextual understanding for all visitors, particularly aiding those with cognitive or sensory impairments. This is achieved by integrating video and audio sensory enhancements into the physical world. Emotion AI facilitates a detailed assessment of visitor engagement through the analysis of neurological and physiological responses. It provides insights into the emotional triggers of visitors, making it possible to tailor a more personalized and accessible experience, catering to the unique needs and preferences of each visitor.

We are developing a case study focusing on the 15th-century Wildmen statues at Hearst Castle‘s entrance. With the support of Cara O’Brien, the Director of the castle, we will create a permanent exhibit in the Visitor Center Museum that will utilize video and audio projections to underscore the statues’ historical importance and context, providing accessible information for visitors with sensory impairments. We will then do a study utilizing mixed-reality headsets equipped with eye-tracking capabilities and consumer-grade wireless EEG devices, in which we will explore more effective methods of merging digital content with the physical environment. Furthermore, this project intends to enhance the educational experience of architecture and computer science students, offering them the opportunity to learn-by-doing exhibit design, artificial intelligence in enhancing human experiences, and technology to support connecting people more deeply with cultural and historical contexts.

Funding

Funded by the Research, Scholarly, and Creative Activities Grant Program (RSCA) from the Division of Research and the Summer Undergraduate Research Program (SURP) of the College of Engineering (CENG) at California State University, San Luis Obispo.