Real-Time Binaural Spatial Audio Rendering Coupled with Dynamic Head Tracking to Improve Accessibility in Augmented and Virtual Reality Environments

Abstract

AR and VR technologies have developed quickly and provide an extremely compelling visual experience. Nevertheless, it is a matter of concern as efficacy in spatial audio to bring the same immersion and availability is yet not completed, particularly with the users having sensory limitations, including sight or sound inabilities. In this paper, a spatial audio rendering system based on binaural real-time spatial audio incorporating dynamic head tracking, which is intended to increase accessibility in AR/VR, is proposed. It applies personalized Head-Related Transfer Functions (HRTF) and high frequency / low latency inertial measurement data to head track user and provide continuous audio spatialization in real time based on head orientation and position. This dynamic reproduction does not distort the spatial information so that recognition of a source of sound is better, distance can be judged more properly and immersion of listening is enhanced. The framework applies adjustable HRTF profiles and spatial cues enhancement algorithms to support a wide variety of accessibility requirements to assist auditory discrimination and navigation. Large-scale experimental tests with objective tests of localization accuracy and subjective user questionnaires with both neurotypical and visually impaired users prove a high degree of improvement in the perception of spatial audio, decreased cognitive load, and better task performance in an AR/VR environment. The proposed solution is critical to make immersive technologies inclusive and expand the reaches of users with other means. The results reveal the relevance of designing advanced audio rendering coupled with dynamic user response mechanisms to promote accessibility and maximize user experience of an AR/VR system in the future.