Deep Supervised Multi-View Learning With Graph Priors

This paper presents a novel method for supervised multi-view representation learning, which projects multiple views into a latent common space while preserving the discrimination and intrinsic structure of each view. Specifically, an apriori discriminant similarity graph is first constructed based on labels and pairwise relationships of multi-view inputs. Then, view-specific networks progressively map inputs to common representations whose affinity approximates the constructed graph. To achieve graph consistency, discrimination, and cross-view invariance, the similarity graph is enforced to meet the following constraints: 1) pairwise relationship should be consistent between the input space and common space for each view; 2) within-class similarity is larger than any between-class similarity for each view; 3) the inter-view samples from the same (or different) classes are mutually similar (or dissimilar). Consequently, the intrinsic structure and discrimination are preserved in the latent common space using an apriori approximation schema. Moreover, we present a sampling strategy to approach a sub-graph sampled from the whole similarity structure instead of approximating the graph of the whole dataset explicitly, thus benefiting lower space complexity and the capability of handling large-scale multi-view datasets. Extensive experiments show the promising performance of our method on five datasets by comparing it with 18 state-of-the-art methods.

I. Introduction

As multi-view data, such as images, textual descriptions, and videos, continues to rapidly grow, there is an increasing demand for developing multi-view learning approaches to cater to a wide range of applications, including multimedia retrieval [1], [2], [3], [4], image annotation [5], heterogeneous face recognition [6], and cross-view retrieval [7], [8]. It is a fundamental challenge in multi-view learning to measure the similarity between samples from different views, commonly known as the ‘heterogeneous gap’ [1], [9], [10].