Two-Stream Part-Based Deep Representation for Human Attribute Recognition

Recognizing human attributes in unconstrained environments is a challenging computer vision problem. State-of-the-art approaches to human attribute recognition are based on convolutional neural networks (CNNs). The de facto practice when training these CNNs on a large labeled image dataset is to take RGB pixel values of an image as input to the network. In this work, we propose a two-stream part-based deep representation for human attribute classification. Besides the standard RGB stream, we train a deep network by using mapped coded images with explicit texture information, that complements the standard RGB deep model. To integrate human body parts knowledge, we employ the deformable part-based models together with our two-stream deep model. Experiments are performed on the challenging Human Attributes (HAT-27) Dataset consisting of 27 different human attributes. Our results clearly show that (a) the two-stream deep network provides consistent gain in performance over the standard RGB model and (b) that the attribute classification results are further improved with our two-stream part-based deep representations, leading to state-of-the-art results.

I. Introduction

Recognizing human attributes such as gender, age, hair style, and clothing style in unconstrained environments is a challenging problem since humans can appear in different poses, under changing illumination and scale, and at low resolution. Human attribute recognition has many potential applications such as, including people search, person reidentification, and human identification. In case of visual surveillance recognizing fine-grained human attributes, to be used as soft-biometrics, has gained much importance due to many intelligent surveillance applications ranging from the monitoring of railway stations and airports to citizen-oriented applications such as monitoring assistants for the aged people. Initially, most approaches for human attribute recognition relied on face information with images having high resolution aligned frontal faces. However, humans appear in different scales and viewpoints in real-world situations, such as far-view video surveillance scenarios. In such scenarios, recognition solely based on facial cues could provide below-expected results, and cues from clothes and hairstyle are likely to provide valuable additional information. In this paper, we also investigate the problem of human attribute recognition in real-world images.