Object detection and segmentation from joint embedding of parts and pixels

We present a new framework in which image segmentation, figure/ground organization, and object detection all appear as the result of solving a single grouping problem. This framework serves as a perceptual organization stage that integrates information from low-level image cues with that of high-level part detectors. Pixels and parts each appear as nodes in a graph whose edges encode both affinity and ordering relationships. We derive a generalized eigen-problem from this graph and read off an interpretation of the image from the solution eigenvectors. Combining an off-the-shelf top-down part-based person detector with our low-level cues and grouping formulation, we demonstrate improvements to object detection and segmentation.

1. Introduction

Many high-performance object detection algorithms operate top-down and do not exploit grouping or segmentation processes. The best algorithms [9], [26] in the PASCAL VOC challenge [8] fall into this category as do top systems for important applications such as finding people in images [2] and detecting pedestrians specifically [6], [7], [22]. When object segmentation is desired as an output, it is often obtained in a post-processing step, for example, by aligning the predictions of a top-down detector to image contours [2].