Edge-Accelerated Neural Vocoder for High-Fidelity Low-Bitrate Speech Synthesis

Abstract

Neural vocoders have transformed speech synthesis into one of the most natural sounding waves of text-to-speech (TTS) and neural speech coding systems, but they require a lot of computation power and huge model size, making them inaccessible to the resource-constrained edge devices. The current paper describes EdgeVocoder, an edge-computed neural vocoder that was chiefly created with a focus on high-fidelity speech reconstruction at low bitrates (≤6 kbps) with extreme latency and memory requirements. The presented structure unites multi-band waveform generation based on polyphase quadrature mirror filtering to minimise time resolution and computational encoding, a lightweight GAN based generator structure with depth wise separable convolutions and grouped residual blocks as well as channel pruning that structures to remove redundant parameters. Moreover, quantization-aware training is used to allow efficient inference in 8-bit integer format at a very small perceptual loss, so that it can be compatible with the ARM NEON and other edge inference engines. An architecture optimization plan that is sensitive to hardware is also integrated into training as a latency-constrained objective functional that directly correlates to the model design with real-time deployment demands. Experimental tests on 24 kHz multi-speaker data sets show that in comparison to baseline models of HiFi-GAN EdgeVocoder is 4 times fewer parameters and 3.2 times faster in inference and retains the same competitiveness in terms of perceptual quality measured by Mean Opinion Score (MOS) and Perceptual Evaluation of Speech Quality (PESQ). On ARM Cortex-class processors with a memory footprint less than 20 MB, the system has been shown reached real-time factors below 0.02 showing its suitability to embedded and mobile systems. Such findings make EdgeVocoder a viable option to use in deploying high-quality neural speech synthesis in low-bandwidth, low-resource constraint edge realms, without dramatic loss to acoustic quality.