Deep Adversarial Quantization Network for Cross-Modal Retrieval

In this paper, we propose a seamless multimodal binary learning method for cross-modal retrieval. First, we utilize adversarial learning to learn modality-independent representations of different modalities. Second, we formulate loss function through the Bayesian approach, which aims to jointly maximize correlations of modality-independent representations and learn the common quantizer codebooks for both modalities. Based on the common quantizer codebooks, our method performs efficient and effective cross-modal retrieval with fast distance table lookup. Extensive experiments on three cross-modal datasets demonstrate that our method outperforms state-of-the-art methods. The source code is available at https://github.com/zhouyu1996/DAQN.

1. Introduction

Cross-modal retrieval plays an important role in the abundant appearance of multimedia data on the Internet [1], [2], [3]. The cross-modal retrieval task aims to establish an information retrieval system, which can support querying across content domains, e.g., searching for the related texts through a query image. Due to its low memory consumption and fast computation speed, the binary-based method, which includes hashing and quantization, is one of the most promising solutions for cross-modal retrieval [4], [5].