Constrained parametric min-cuts for automatic object segmentation

We present a novel framework for generating and ranking plausible objects hypotheses in an image using bottom-up processes and mid-level cues. The object hypotheses are represented as figure-ground segmentations, and are extracted automatically, without prior knowledge about properties of individual object classes, by solving a sequence of constrained parametric min-cut problems (CPMC) on a regular image grid. We then learn to rank the object hypotheses by training a continuous model to predict how plausible the segments are, given their mid-level region properties. We show that this algorithm significantly outperforms the state of the art for low-level segmentation in the VOC09 segmentation dataset. It achieves the same average best segmentation covering as the best performing technique to date, 0.61 when using just the top 7 ranked segments, instead of the full hierarchy in. Our method achieves 0.78 average best covering using 154 segments. In a companion paper, we also show that the algorithm achieves state-of-the art results when used in a segmentation-based recognition pipeline.

1. Introduction

The challenge of organizing the elements of an image into plausible object regions, or segments, without knowing a-priori which objects are present in that image is one of the remarkable abilities of the human visual system, which we often take for granted. A more vivid conscious experience arises, perhaps, when observing abstract paintings. Clearly, in our perceived visual world not every hypothesis is equally likely, for example objects are usually compact, resulting in their projection in the image being connected; it is also common for strong contrast edges to mark objects boundaries.