Joint Versus Independent Multiview Hashing for Cross-View Retrieval

Thanks to the low storage cost and high query speed, cross-view hashing (CVH) has been successfully used for similarity search in multimedia retrieval. However, most existing CVH methods use all views to learn a common Hamming space, thus making it difficult to handle the data with increasing views or a large number of views. To overcome these difficulties, we propose a decoupled CVH network (DCHN) approach which consists of a semantic hashing autoencoder module (SHAM) and multiple multiview hashing networks (MHNs). To be specific, SHAM adopts a hashing encoder and decoder to learn a discriminative Hamming space using either a few labels or the number of classes, that is, the so-called flexible inputs. After that, MHN independently projects all samples into the discriminative Hamming space that is treated as an alternative ground truth. In brief, the Hamming space is learned from the semantic space induced from the flexible inputs, which is further used to guide view-specific hashing in an independent fashion. Thanks to such an independent/decoupled paradigm, our method could enjoy high computational efficiency and the capacity of handling the increasing number of views by only using a few labels or the number of classes. For a newly coming view, we only need to add a view-specific network into our model and avoid retraining the entire model using the new and previous views. Extensive experiments are carried out on five widely used multiview databases compared with 15 state-of-the-art approaches. The results show that the proposed independent hashing paradigm is superior to the common joint ones while enjoying high efficiency and the capacity of handling newly coming views.

I. Introduction

With the rapid growth of multiview data, such as image, text, and video on the Internet, there are increasing demands on developing cross-view methods for a variety of applications [1]–[6]. Among them, cross-view retrieval has arisen great interest from the community, which aims to retrieve the interested content across different views/modalities, for example, retrieving the corresponding text counterpart for a given image query. Due to the low storage cost and high query speed of hash codes [7], [8], cross-view hashing (CVH) has achieved promising performance and is becoming increasingly popular for the large-scale multimedia retrieval. Although CVH has been paid more attention to by both academia and industry [9], [10], there still remain many challenges. Especially, different views may lie in completely disparate spaces with large semantic gaps, thus resulting in inferior retrieval performance.