Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks

Deep learning methods such as convolutional neural networks (CNNs) can deliver highly accurate classification results when provided with large enough data sets and respective labels. However, using CNNs along with limited labeled data can be problematic, as this leads to extensive overfitting. In this letter, we propose a novel method by considering a pretrained CNN designed for tackling an entirely different classification problem, namely, the ImageNet challenge, and exploit it to extract an initial set of representations. The derived representations are then transferred into a supervised CNN classifier, along with their class labels, effectively training the system. Through this two-stage framework, we successfully deal with the limited-data problem in an end-to-end processing scheme. Comparative results over the UC Merced Land Use benchmark prove that our method significantly outperforms the previously best stated results, improving the overall accuracy from 83.1% up to 92.4%. Apart from statistical improvements, our method introduces a novel feature fusion algorithm that effectively tackles the large data dimensionality by using a simple and computationally efficient approach.

I. Introduction

Supervised classification of very high spatial resolution (VHSR) images is still an open research topic in the remote sensing (RS) field. Monitoring urbanization trends has become a crucial objective, and there is currently a high demand for such automatic RS classification techniques. Toward this direction in the last years, advanced methodologies have significantly contributed to the solution of the VHSR classification problem. Predominantly, methods based on the bag-of-visual-words (BoVW) approach have been proposed for solving this task by learning a dictionary for representing the image content in an unsupervised manner through the use of well-established feature descriptors (HOG, SIFT, etc.) and clustering algorithms. Such approaches include the spatial pyramid matching kernel (SPMK) [1], spatial pyramid cooccurrence kernel (SPCK++) [2], min-tree kd-tree [3], and sparse coding [4] methods. However, the main drawback of all these techniques lies in the assumption that a general feature descriptor can adequately represent the complex image structures by employing expert knowledge through manually designed all-around purpose features.