Neighborhood Preserving Hashing for Scalable Video Retrieval

In this paper, we propose a Neighborhood Preserving Hashing (NPH) method for scalable video retrieval in an unsupervised manner. Unlike most existing deep video hashing methods which indiscriminately compress an entire video into a binary code, we embed the spatial-temporal neighborhood information into the encoding network such that the neighborhood-relevant visual content of a video can be preferentially encoded into a binary code under the guidance of the neighborhood information. Specifically, we propose a neighborhood attention mechanism which focuses on partial useful content of each input frame conditioned on the neighborhood information. We then integrate the neighborhood attention mechanism into an RNN-based reconstruction scheme to encourage the binary codes to capture the spatial-temporal structure in a video which is consistent with that in the neighborhood. As a consequence, the learned hashing functions can map similar videos to similar binary codes. Extensive experiments on three widely-used benchmark datasets validate the effectiveness of our proposed approach.

1. Introduction

Scalable video retrieval seeks similar videos from a large database given a query video. Usually, videos are represented by sampled frames and each frame is characterized by a representative feature. The set of frame features are utilized to identify relevant videos or nearest neighbors. In the face of high dimensional features and large scale datasets, hashing methods have attracted a lot of attention in scalable visual retrieval [1–10]. Video hashing methods encode frame features of each video into a compact binary code while enabling the similarity between videos to be preserved in the Hamming space [11–22]. Among them, learning-based video hashing methods which learn data-dependent and task-specific hashing functions have achieved good search accuracy [23–25].