Dynamic Parallel Pyramid Networks for Scene Recognition

Scene recognition is considered a challenging task of image recognition, mainly due to the presence of multiscale information of global layout and local objects in a given scene. Recent convolutional neural networks (CNNs) that can learn multiscale features have achieved remarkable progress in scene recognition. They have two limitations: 1) the receptive field (RF) size is fixed even though a scene may have large-scale variations and 2) they are computing and memory intensive, partially due to the representation of multiscales. To address these limitations, we propose a lightweight dynamic scene recognition approach based on a novel architectural unit, namely, a dynamic parallel pyramid (DPP) block, that can adaptively select RF size based on multiscale information from the input regarding channel dimensions. We encode multiscale features by applying different convolutional (CONV) kernels on different input tensor channels and then dynamically merge their output using a group attention mechanism followed by channel shuffling to generate the parallel feature pyramid. DPP can be easily incorporated with existing CNNs to develop new deep models, called DPP networks (DPP-Nets). Extensive experiments on large-scale scene image datasets, Places365 standard, Places365 challenge, the Massachusetts Institute of Technology (MIT) Indoor67, and Sun397 confirmed that the proposed method provides significant performance improvement compared with current state-of-the-art (SOTA) approaches. We also verified general applicability from compelling results on lightweight models of MobileNetV2 and ShuffleNetV2 on ImageNet-1k and small object centralized benchmarks on CIFAR-10 and CIFAR-100.

I. Introduction

Scene recognition is a challenging task in computer vision [1], [2] and has attracted significant research attention due to its potential for indoor and outdoor applications, such as autonomous driving and robotic applications. Recognizing the scene from a single image is a challenging problem, particularly for large datasets [3], [4]. The challenges include large intraclass variations, high interclass similarities, scale variations, and viewpoint differences. Fig. 1 shows that images in library-outdoor and general_store-outdoor categories can be easily confused. Similar ambiguities can be found for computer_room and office_cubicles. The fact that scene images not only include various local objects but also have meaningful global spatial properties, makes scene recognition challenging. Fig. 1.

Example images from the Places2 dataset, including outdoor to indoor scene categories. (a) Library-outdoor. (b) General_store-outdoor. (c) Computer_room. (d) Office_cubicles.