Robust Speech-to-Text and Text-to-Speech Systems for Noisy and Real-World Acoustic Environments

Abstract

Strong text-to-speech (TTS), speech-to-text (STT), systems form the basis of strong technologies that support natural human-computer interaction in varying surfaces such as voice assistant, automated transcription, and communication assistance. But they often perform poorly in the real-world acoustic signal-handling scenarios because of background noise, reverberation and channel variability. The following article offers a retrospective overview of the recent developments in creating sound-robust STT and TTS systems paying special attention to latest achievements in state-of-the-art acoustic representation, deep architectures, and data centric approaches like augmentation and domain adaptation. In the case of STT, we look at the new end-to-end models, self-supervised pretraining, and reliable feature extraction which could lead to the increased recognition accuracy in harsh conditions. Regarding TTS, we discuss progress in the research into implementing neural vocoding, model prosody, and adaptive synthesis algorithms with the goal of maintaining the naturalness and intelligibility of speech during playback in noisy conditions. Moreover, we provide associated benchmark datasets and evaluation metrics which help do a rigorous evaluation of the robustness of a system. Just before closing we address key challenges, such as latency, resource limitations, and ethics and present directions of future research towards scaling, low latency, and privacy-conscious speech interfaces that will be deployed in a variety of real-life contexts.