Using Multiple Segmentations to Discover Objects and their Extent in Image Collections

Given a large dataset of images, we seek to automatically determine the visually similar object and scene classes together with their image segmentation. To achieve this we combine two ideas: (i) that a set of segmented objects can be partitioned into visual object classes using topic discovery models from statistical text analysis; and (ii) that visual object classes can be used to assess the accuracy of a segmentation. To tie these ideas together we compute multiple segmentations of each image and then: (i) learn the object classes; and (ii) choose the correct segmentations. We demonstrate that such an algorithm succeeds in automatically discovering many familiar objects in a variety of image datasets, including those from Caltech, MSRC and LabelMe.

1. Introduction

In [21] we posed the question, given a (Gargantuan) number of images, “Is it possible to learn visual object classes simply from looking at images?”. That is, if our data set contains many instances of (visually similar) object classes, can we discover these object classes? In this paper we extend this question to “Is it possible to learn visual object classes and their segmentations simply from looking at images?”