Zero-Shot Learning for Remote Sensing Image Segmentation Using Cross-Domain Transfer and Self-Training

Abstract

Segmentation of remote sensing (RS) images plays a pivotal role in signal and image processing as it helps in performing pixel-level interpretation of these images to help land cover mapping, environmental monitoring, studying urban infrastructure, and disaster assessment, etc. Although distilled deep learning architectures (e.g., U-Net, DeepLab, SegFormer) have produced good outcomes, their usage of high annotations (particularly with pixel-level) large-scale datasets conditions them to lack generalizability. The work presents a Zero-Shot Learning (ZSL) structure-the Cross-Domain Transfer and Self-Training (CDT-ST) model-to perform RS image segmentation without target-domain labelled information. It combines a domain-invariant feature extraction module based on signal processing, with cross-domain class-mapping based on semantic embedding, and a sequence of refinements to pseudo-labels, namely confidence thresholding, spatial consistency filtering, and Conditional Random Fields (CRFs). The combination of these techniques makes such adaptation strong when dealing with extreme changes in the domain of spatial resolution, illumination, and scene structure. The experiments include evaluating on SpaceNet, DeepGlobe, and LoveDA datasets where the mean Intersection over Union (mIoU) was found to be 87.2% with no target-domain labels, only 233 fewer than fully supervised counterparts. Combining transfer learning, semantic mapping and primitive signal/image processing methods, CDT-ST proposes a scalable, no annotation required, high-accuracy system with application of large-scale, heterogeneous RS segmentation.