Spectral-Spatial Symmetrical Aggregation Cross-Linking Multi-Modal Data Fusion Network

In this paper, a spectral-spatial symmetrical aggregation cross-linking network (SACLNet) is developed for multi-modal data classification, which contains three modules as follows. First, the Spectro-Spatial Feature Learning Module is proposed, using the involution operation sliding over the spectral channels of hyperspectral image (HSI) and fused-sharing weight obtained from HSI and light detection and ranging (HSI-LiDAR) data for spectral and spatial information representation. Second, the pyramid feature fusion module interacts among spectral and spatial information to further share and guide each other. In this step, multistage features, including low-level, middle-level, and high-level, are fused and adjusted in a pyramidal and mutually guided learning process. Third, the fused Spectro-Spatial features are embedded into the Multimodal Data Fusion Module, obtaining the final classification results. Experimental results show that the proposed SACLNet has a satisfactory classification performance than the state-of-the-art methods.

1. INTRODUCTION

Recently, the combination of multi-modal data, especially hyperspectral image (HSI) and light detection and ranging (Li-DAR) image data fusion, has been developed and successfully used in many applications [1],[2].