Real-Time Speech Enhancement on Edge Devices Using Optimized Deep Learning Models

Abstract

With edge computing increasing becoming a popular implementation for real-time processing and storing data, voice driven applications need effective and lightweight Speech Enhancement Systems to meet real-time processing requirement on constrained hardware platforms. In this work, we propose a deep learning – based speech enhancement framework optimized for the real – time improvement of speech quality on the edge. The system proposed in this thesis contains a convolutional recurrent neural network (CRNN) architecture with pruning and quantization techniques that reduce complexity of the model without reducing performance. The experiments were conducted extensively across the TIMIT and DNS Challenge datasets with different types of noises. We finally show that the model achieves PESQ score of 3.4, 9.1 dB SNR gain over baseline models while maintaining an inference latency under 25ms on ARM Cortex-A53 processors. The results show that the proposed model suitably compromises accuracy, latency, and resource efficiency, hence making it a suitable component for real time applications, for instance, hearing aids, smart assistants, and mobile devices.