Gat-Assisted Deep Hashing For Multi-Label Image Retrieval

Multi-Label hash methods have achieved excellent performance in multi-label image retrieval, but how to leverage the semantic information of label to improve retrieval quality is still a challenge in this field. This paper proposes GAT-Assisted Deep Hashing (DHGAT). Our model uses Convolutional Neural Network (CNN) to extract image-level features, along with graph attention network (GAT) to extract label-level features. Assisted by GAT, DHGAT is able to pay more attention on the co-occurrence of label. In order to solve the problem of feature fusion, we propose Multi-modal Max-Pooling Bilinear (MMB) mechanism, which MMB fuses image-level feature and label-level feature to generate abundant semantic features, so that the model can output discriminative hash code. Extensive experiments demonstrate that the proposed method can generate hash codes which achieve better retrieval performance on two benchmark datasets, NUS-WIDE and MS-COCO.

1. Introduction

Deep hashing has received widely attention in image retrieval task due to its outstanding performance in code expression and retrieval efficiency. In current image retrieval applications, an image is usually associated with multi-label to describe rich semantic information, and how to utilize multilabel information to improve the retrieval performance has become a challenge.