Correlational Gaussian Processes for Cross-Domain Visual Recognition

We present a probabilistic model that captures higher order co-occurrence statistics for joint visual recognition in a collection of images and across multiple domains. More importantly, we predict the structured output across multiple domains by correlating outputs from the multi-classes Gaussian process classifiers in each individual domain. A set of correlational tensors is adopted to model the relationship within a single domain as well as across multiple domains. This renders it possible to explore a high-order relational model instead of using just a set of pairwise relational models. Such tensor relations are based on both the positive and negative co-occurrences of different categories of visual instances across multi-domains. This is in contrast to most previous models where only pair-wise relationships are explored. We conduct experiments on four challenging image collections. The experimental results clearly demonstrate the efficacy of our proposed model.

1. Introduction

The cross-domain visual recognition problem was firstly explicitly proposed in [33], although many previous works [29], [3], [40], [4], [36]–[38], [7] also implicitly tackled part of such a problem. In such a problem, multiple visual recognition problems in different semantic domains are simultaneously solved through a joint formulation instead of being handled independently. This is based on the intuition that the semantics across different domains are associated with the same visual entity and hence there are intrinsic correlations among them to facilitate the joint inference of all of these visual semantics. For example, we can interpret each photo from people, location and event domain, and then employ the estimated cross-domain correlations to improve the recognition accuracy in each domain, e.g., face recognition in people domain.