Unsupervised segmentation of color-texture regions in images and video

A method for unsupervised segmentation of color-texture regions in images and video is presented. This method, which we refer to as JSEG, consists of two independent steps: color quantization and spatial segmentation. In the first step, colors in the image are quantized to several representative classes that can be used to differentiate regions in the image. The image pixels are then replaced by their corresponding color class labels, thus forming a class-map of the image. The focus of this work is on spatial segmentation, where a criterion for "good" segmentation using the class-map is proposed. Applying the criterion to local windows in the class-map results in the "J-image," in which high and low values correspond to possible boundaries and interiors of color-texture regions. A region growing method is then used to segment the image based on the multiscale J-images. A similar approach is applied to video sequences. An additional region tracking scheme is embedded into the region growing process to achieve consistent segmentation and tracking results, even for scenes with nonrigid object motion. Experiments show the robustness of the JSEG algorithm on real images and video.

1 Introduction

Image and video segmentation is useful in many applications for identifying regions of interest in a scene or annotating the data. The MPEG-4 standard needs segmentation for object-based video coding. However, the problem of unsupervised segmentation is ill-defined because semantic objects do not usually correspond to homogeneous spatiotemporal regions in color, texture, or motion. Some of the recent work in image segmentation include stochastic model-based approaches [1], [6], [13], [17], [24], [25], morphological watershed-based region growing [18], energy diffusion [14], and graph partitioning [20]. The work on video segmentation includes motion-based segmentation [3], [19], [21], [23], spatial segmentation and motion tracking [8], [22], moving objects extraction [12], [15], and region growing using spatiotemporal similarity [4], [16]. Quantitative evaluation methods have also been suggested [2].