Multiview Semantic Representation for Visual Recognition

Due to interclass and intraclass variations, the images of different classes are often cluttered which makes it hard for efficient classifications. The use of discriminative classification algorithms helps to alleviate this problem. However, it is still an open problem to accurately model the relationships between visual representations and human perception. To alleviate these problems, in this paper, we propose a novel multiview semantic representation (MVSR) algorithm for efficient visual recognition. First, we leverage visually based methods to get initial image representations. We then use both visual and semantic similarities to divide images into groups which are then used for semantic representations. We treat different image representation strategies, partition methods, and numbers as different views. A graph is then used to combine the discriminative power of different views. The similarities between images can be obtained by measuring the similarities of graphs. Finally, we train classifiers to predict the categories of images. We evaluate the discriminative power of the proposed MVSR method for visual recognition on several public image datasets. Experimental results show the effectiveness of the proposed method.

I. Introduction

Correctly classifying images based on their visual contents have been widely explored in recent years. Local feature-based methods are used along with the bag-of-visual-word (BoW) strategies [1]–[4]. Local features [5] and [6] are well designed by experts to cope with various visual changes and noisy information. Although effective, local features are very hard to design which requires rich domain knowledge and experiences. The deep convolutional neural network (CNN)-based methods [7]–[12] try to directly learn the representations from image pixels. This strategy has been proven very effective for various visual applications [13]–[18]. The performances can be further improved with deeper networks and architectures.