Regional-Confidence Weight for Accurate Multi-Label Image Retrieval

Recently, most methods proposed for multi-label image retrieval focus on using hashing functions to improve retrieval efficiency. However, hashing functions may reduce retrieval accuracy and be sub-optimal for some scenarios where accuracy is more important than efficiency. In this paper, we propose a deep feature descriptor without hashing for multi-label image retrieval. We use Faster R-CNN with FPN to perform object detection on images and extract feature maps. Especially, we propose Regional-Confidence Weight, which can evaluate the importance of different regions, to generate more powerful feature maps. Additionally, GeM pooling is used to produce compact feature vectors from feature maps for retrieval. Extensive experiments demonstrate that our approach significantly outperforms several state-of-the-art methods with higher retrieval accuracy for multi-label image retrieval.

I. INTRODUCTION

Multi-label image retrieval focuses on several objects of every query image and returns images with similar objects from the database. An example of multi-label image retrieval can be seen in Fig. 1. Recently, many methods [23], [25], [26], [28] have been proposed for multi-label image retrieval. These methods mainly concentrate on hashing functions to produce extremely compact feature vectors. Thus, they are very effective in large-scale retrieval tasks. However, they lose too much information of original images by hashing, which may affect retrieval accuracy and be sub-optimal for small-scale retrieval tasks or some scenarios where accuracy is more important than efficiency. In this paper, we propose a deep feature descriptor without hashing functions for multi-label image retrieval to get higher retrieval accuracy.