Shared-Private Decoupling-Based Multilevel Feature Alignment Semisupervised Learning for HSI and LiDAR Classification

The joint classification methods of hyperspectral image (HSI) and light detection and ranging (LiDAR) data based on deep learning have demonstrated exceptional classification performance with sufficient labeled samples. However, it is expensive and time-consuming to acquire labeled data. To address this limitation, we propose a shared-private decoupling-based multilevel feature alignment semisupervised (SASS) learning method for HSI and LiDAR classification, which introduces the idea of domain adaptation (DA) to capture the shared features of labeled and unlabeled data for classification and circularly selects reliable pseudolabels based on these features to retrain the model. Specifically, we treat labeled data as the source domain (SD) and unlabeled data as the target domain (TD) and propose a shared-private feature decoupling (SPFD) module to acquire shared representations of SD and TD by separating domain private features. The multilevel shared feature alignment (MSFA) strategy is designed to synthetically consider both spatial details and semantic information by minimizing the maximum mean discrepancy (MMD) between these shared features. In addition, we design a graph transformer-based class-balanced pseudolabel generation (GBPG) strategy for iterative model training with reliable pseudolabels, which exploits the graph transformer network-based sample acquisition (GTSA) strategy to select valuable samples and generate corresponding pseudolabels using the adaptive class-specific threshold-based sample annotation (ATSA) strategy. Experimental results on three public datasets validate the effectiveness of the proposed method. The code is available at https://github.com/Jiahuiqu/SASS.

I. Introduction

Remote sensing (RS) images have extensive applications across various fields, including environmental protection, urban planning, and resource exploration [1], [2], [3]. In various RS images, hyperspectral images (HSIs) can capture the response of ground objects in different wavelength bands. Therefore, HSI has significant advantages in distinguishing ground objects of various materials and has become one of the most important data sources in RS tasks [4], [5], [6], [7]. However, it performs poorly in distinguishing land cover types with the same spectral characteristics but different heights. Light detection and ranging (LiDAR) is an active mapping technology that provides high-precision ground elevation information for distinguishing ground objects at various heights [8]. However, for objects with the same height but different materials, its discriminative ability may be limited. Therefore, the joint classification of multisource data can leverage their advantages to further improve classification accuracy [9], [10].