NWPU-MOC: A Benchmark for Fine-Grained Multicategory Object Counting in Aerial Images

Object counting is a hot topic in computer vision, which aims to estimate the number of objects in a given image. However, most methods only count objects of a single category for an image, which cannot be applied to scenes that need to count objects with multiple categories simultaneously, especially in aerial scenes. To this end, this article introduces a multicategory object-counting (MOC) task to estimate the numbers of different objects (cars, buildings, ships, etc.) in an aerial image. Considering the absence of a dataset for this task, a large-scale dataset (NWPU-MOC) is collected, consisting of 3416 scenes with a resolution of $1024\times1024$ pixels, and well annotated using 14 fine-grained object categories. Besides, each scene contains RGB and near infrared (NIR) images, of which the NIR spectrum can provide richer characterization information compared with only the RGB spectrum. Based on NWPU-MOC, the article presents a multispectrum, MOC framework, which employs a dual-attention module to fuse the features of RGB and NIR and subsequently regress multichannel density maps corresponding to each object category. In addition to modeling the dependence between different channels in the density map with each object category, a spatial contrast loss is designed as a penalty for overlapping predictions at the same spatial position. Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared with some mainstream counting algorithms. The dataset, code, and models are publicly available at https://github.com/lyongo/NWPU-MOC.

I. Introduction

Object counting in aerial scenes aims to estimate the number of objects with a specific category in given images, which can be applied to urban planning [1], [2], [3], environmental monitoring and mapping [4], [5], [6], disaster detection [7], [8], [9], and other practical applications [10], [11], [12]. Over the past years, benefiting from the development of deep learning and neural networks, some object (pedestrians, cars, boats, etc.) counting algorithms designed for remote sensing scenes have been proposed. Specifically, Bahmanyar et al. [13] introduce a crowd-counting dataset captured from the UAV view. They also propose a multiresolution network to estimate the pedestrian count. Gao et al. [14] construct an RSOC dataset for remote sensing object counting, containing four subsets: buildings, small vehicles, large vehicles, and boats. Based on the dataset, Gao et al. [15] propose the PSGCNet, which addresses challenges such as scale variations and complex backgrounds in remote sensing scenes by extracting and fusing multiscale and global feature information.