Real-Time Gesture Recognition Using mmWave Radar Signal Processing and CNN Models

Abstract

The current paper presents a model of real-time gesture recognition, where the usage of millimeter-wave (mmWave) radar signal processing is combined with convolutional neural networks (CNN) in order to effectively and precisely interact with machines as a human being. The system employs a 60 GHz frequency-modulated continuous-wave (FMCW) radar sensor to capture reflected signals of dynamic hand gestures to identify them through spatial and temporal signs of the motion. These raw appreciative intermediate frequency (IF) signals are pre-processed using an uncover of DNA Doppler-range transformation that entails Fast Fourier Transform (FFT)-based profiling of range and velocity, clutter repression and considering spectrograms to deliver range-Doppler maps (RDMs). The RDMs are used as powerful input features to a compact and optimized CNN model that is pretrained to classify ten different gesture patterns including, swipe, push, pull, and rotate. CNN architecture should have low computation complexity, which is possible to implement on the resource-limited edge devices like Raspberry Pi 4 and NVIDIA Jetson Nano. We used 20 subjects across all the gestures at different orientations and distances to collect a huge dataset using this technique to make it robust. Data augmentation and quantization-aware optimization techniques have been used to train the model and strike the right compromise between real time and accuracy performance. According to experimental analysis, the suggested system obtains a gesture classification precision above 95 percent, and the inference latency is averagely less than 100 milliseconds with little power demands, thus richly appliable to embedded systems. In addition, the environment is highly robust where the framework is economical working in changing lighting conditions, occlusion, and environmental noise as compared to conventional vision-based systems in areas of privacy and low-light performance. Extendable and no-contact smart environment The suggested solution would allow a touchless interface to be used in smart environments, such as home automation, industrial control panels, and wearable assistive solutions. To improve upon cross-user generalization in terms of domain adaptation, further research will be based on incorporating attention-based deep learning models. On the whole, this study presents an effective and power-efficient approach to realizing a gesture recognition system based on radar with deep-learning in real-time on embedded systems.