Scene analysis using appearance-based models and relational indexing

The analysis of scenes containing multiple 3D objects remains an active research topic in computer vision. Particularly challenging are scenes containing non-polyhedral objects. In general, conventional object models based on junctions and line segments are not suitable for use in this type of recognition. To address this issue, we have developed a representation scheme in which objects are defined by the features that can be reliably extracted from a training set of real images. For a given object the set of such features is called the appearance-based model of the object. We have created a database of appearance-based model of industrial objects containing both flat and curved surfaces, holes, and threads. A matching technique, called relational indexing, has been developed to work with the appearance-based representation of our objects. Each model in the database is described by a relational graph of its appearance-based features and small relational subgraphs of the scene features are wed to index the database and to retrieve appropriate 3D model hypotheses. This paper describes the new models, the matching algorithm, and preliminary results.