Layered object detection for multi-class segmentation

We formulate a layered model for object detection and multi-class segmentation. Our system uses the output of a bank of object detectors in order to define shape priors for support masks and then estimates appearance, depth ordering and labeling of pixels in the image. We train our system on the PASCAL segmentation challenge dataset and show good test results with state of the art performance in several categories including segmenting humans.

1. Introduction

Object detection is a fundamental task in computer vision. Most approaches formulate the problem as that of predicting a bounding box enclosing the object of interest – this is, for example, the evaluation criteria in the popular PASCAL Visual Object Recognition Challenge (VOC) [3]. However, a bounding-box output is clearly limited. For many objects, particularly those with complex, articulated shapes, a bounding box provides a poor description of the support of the object in the image. At the other extreme, one can attempt to produce an object class label for every pixel in the image. This is usually termed multi-class segmentation.