Mining Hierarchical Information of CNNs for Scene Classification of VHR Remote Sensing Images

Scene classification of very high resolution (VHR) images is an active research subject in remote sensing community, and it has provided data or decision supports for many practical applications. Although existing CNN-based methods have achieved good classification results, they have not fully exploited rich potential information contained in pre-trained models. In this paper, a novel framework termed hierarchical features fusion of convolutional neural network (HFFCNN) is developed for scene classification of VHR images. On the whole, the HFFCNN covers two parallel modules to severally process convolutional features and fully connected (FC) features. At the first module, an adaptive spatial-wise attention based multi-scale nonlinear bag-of-visual-words (ASA-MNBoVW) model is designed to encoding convolutional feature maps, and the responses of discriminative regions are highlighted without introducing any additional parameters. For the second module, a weighted image pyramid structure is adopted to reveal geometric information and spatial layouts by aggregating local image patch-based FC features. Finally, these hierarchical features are combined for mutually complementing, and a linear classifier is adopted to predict semantic labels. Experimental results organized on two challenging data sets prove that the developed HFFCNN approach obtains more dramatic performance of scene classification than some state-of-the-art methods in terms of OAs.

1 Introduction

With the prosperousness of earth observation technology, multi-source and multi-temporal remote sensing (RS) “Big Data” are accumulated hundreds of Terabytes daily, and the volume of RS data will be increasingly growing in the future [1], [2], [3]. Undoubtedly, RS Big Data era is coming, and it is an inescapable challenge to effectively mine and analysis RS Big Data [4]. As a significant part of RS Big Data, very high resolution (VHR) images provide more abundant spatial patterns and ground information [5], [6], [7], and they have become an important resource to observe Earth's surface, such as land resource management [8], [9], key object detection [10], precision agriculture [11], disaster prevention [12], etc.