Large-scale Multi-label Image Retrieval Using Residual Network with Hash Layer

In recent years, increasing deep hashing methods have been applied in large-scale multi-label image retrieval. However, in the existing deep network models, the extracted low-level features cannot effectively integrate the multi-level semantic information and the similarity ranking information of pairwise multi-label images into one hash coding learning scheme. Therefore, we cannot obtain an efficient and accurate index method. Motivated by this, in this paper, we proposed a novel approach adopting the cosine distance of pairwise multi-label images semantic vector to quantify existing multi-level similarity in a multi-label image. Meanwhile, we utilized the residual network to learn the final representation of multi-label images features. Finally, we constructed a deep hashing framework to extract features and generate binary codes simultaneously. On the one hand, the improved model uses a deeper network and more complex network structures to enhance the ability of low-level features extraction. On the other hand, the improved model was trained by a fine-tuning strategy, which can accelerate the convergence speed. Extensive experiments on two popular multi-label datasets demonstrate that the improved model outperforms the reference models regarding accuracy. The mean average precision is improved by 1.0432 and 1.1114 times on two datasets, respectively.

I. Introduction

In the era of Internet Web3.0 and with the popularity of social networking sites like the Facebook and Flickr, the amount of unstructured data (such as images, video, audio, text, etc.) have been rapidly increasing. Due to multi-label images containing rich semantic information, how to conveniently, quickly and accurately retrieve images in massive, high-dimensional, multi-label multimedia data has become a research focus of multi-label image retrieval. In the academic field, image retrieval algorithms based on the nearest neighbor are the mainstream methods. However, in the engineering field, the common practice for image retrieval system is usually based on the hashing methods [1]. The hashing-based image retrieval methods can significantly reduce the space complexity and improve the retrieval speed. Fig. 1.

An example of the traditional similarity quantization for multi-label images.