Cross-modal retrieval based on deep correlated network

The rapid development of deep learning provides opportunities for exploring cross-modal retrieval from a semantic perspective. How to exploit correlations of multimodal data which are heterogeneous in an unsupervised way is a challenging task. Previous models for this task still have defects, such as the result of retrieval is unstable and etc. To solve these problems, an unsupervised Deep Correlation Network (DCN) is proposed. DCN firstly incorporates multiple correspondence correlations, different neural networks into a whole. And the model can be divided into two parts. The first part models modalities by restricted Boltzmann machines, and using correlation leaning to learn a common information shared by different modalities. The second part uses autoencoder and the correlation learning to do the same work as the first part. Experiments on three real data sets illustrated that DCN has obvious advantages than existing models.