Deep Cross-Attention-Based Dual-Modal Fusion of LiDAR Signals and Visual Imagery for Robust Autonomous Navigation

Abstract

This paper introduces a new deep learning architecture where a dual-modal fusion mechanism is based on cross-attention scheme to achieve resilient autonomous navigation. The architecture combines LiDAR point cloud data and visual image in a manner such that transformer-based cross-attention modules learn about fine-grained spatial-temporal cross-modal dependence. The system described overcomes the limitations of unimodal and early fusion approaches in recognizing inter-modal features through an instance-based balancing weighting scheme where the system balances these limitations under difficult circumstances when the light source is low, occlusion, and white noise. The dataset that the model is trained and tested on are benchmark autonomous driving datasets such as KITTI and nuScenes. Quantitative findings show significant improvements in accuracy levels of object detection (IoU+6.2), trajectory prediction (ADE/FDE improvements) and collision-free path planning in dynamic settings. Ablation experiments prove that cross-attention fusion is better than standard concatenation and late fusion networks. In further addition, the framework enables real-time inference to take place on embedded systems thereby, it can be deployed very easily on resource limited autonomous vehicles and mobile robots. The given proposed method is just not only useful to improve perception robustness but also in decision-making in safety-critical settings to form a scalable and adaptive solution to next-generation autonomous navigation systems.