A Sam-Empowered Dual-Stream Framework for Scene-Level Local Climate Zone Classification Using Google Earth and Sentinel Images

Recent advancements in remote sensing (RS)-based methods have shown remarkable effectiveness in large-scale local climate zone classification. However, conventional convolutional neural network (CNN)-based methods encounter limitations in effectively incorporating ground object priors. Additionally, commonly used medium-scale data sources, such as Sentinel-2, face challenges in capturing detailed ground object information. In light of these obstacles, we propose a data fusion method that integrates ground object priors extracted from high-resolution Google imagery with Sentinel-2 multi-spectral imagery. The proposed method introduces a novel dual-stream fusion framework (DF4LCZ-Net), which fully leverages instance-based location features from Google imagery and combines them with the scene-level spatial-spectral features extracted from Sentinel-2 images. For effective feature extraction of Google imagery, a SAM-based Graph Convolutional Network (GCN) branch is designed in the framework. We conducted experiments on a newly created multi-source remote sensing image dataset, and the final classification results show the superiority of our method.