Adversarially Robust Speech Recognition Against Audio Perturbation Attacks

Abstract

Deep neural network powered Automatic Speech Recognition (ASR) systems have nearly human accuracy in transcription when operating in clean acoustic environments. Nevertheless, those systems are also highly susceptible to adversarial audio perturbation attacks, in which well design, imperceptible waveform distortions may cause serious transcription errors. These vulnerabilities are very dangerous in security sensitive and real-time voice driven applications. In the current research, I will propose a hybrid adversarial robustness structure that aims to improve the resilience of ASR to both the white-box and black-box perturbation attacks. The key components of the proposed method include multi-step adversarial training, regularisation of spectral consistency of features in the Mel-frequency domain and inference time stochastic smoothing to enhance stability of the model when perturbed with a limited amount of noise. The presence of a Conformer-based ASR architecture is compared to the LibriSpeech dataset under the conditions of FGSM, Projected Gradient Descent (PGD), and Carlini-Wanger (CW) attack. The experimental findings are that there are remarkable changes to the adversarial Word Error Rate (WER) improvements of more than 50 percent compared with typical trainings baselines even when using strong multi-step attacks, and the accuracy of clean speech is competitive with minimum deterioration. Ablation analysis establishes that every element of defence is adding incrementally to gain robustness. Its results illustrate the benefit-accuracy trade-off of adversarial defence design and a systematic scheme of assessment in terms of safe deployment of ASR systems in the real world. The paper progresses feasible adversarial resilience methods to contemporary speech recognition with systems.