Shape from Focus using Color Segmentation and Bilateral Filter

In this paper, we describe a new method for shape from focus using a set of the color segmentation and the bilateral filtering. The shape from focus estimates the geometrical shape of objects from the defocus level of images taken with different focal state. However, the conventional methods tend to fail in the shape estimation around the edge parts of objects and thin texture areas. To address the problems, we use the color segmentation in order to merge such areas by color, and then we interpolate the shape of lost part at each segment. The bilateral smoothing filter is used as the noise suppressor to improve the segmentation accuracy and also this bilateral filter is used to obtain the detail of textures, in other words, high frequency components of the focus measure. In addition, we also acquire the All-in-Focus image by this algorithm and the depth estimation is complemented by this image. By using this method, the lossless and noiseless shape of the object is obtained

1. Introduction

Shape reconstraction from images is one of the challenge in the field of the image processing and the computer vision. The shape from focus (SFF) is a shape estimation method that enables to estimate the object shape from a mono-camera, and is mainly used in the field of microscope where the object-distance¹

camera-to-object distance

is very short and difficult to set the stereo cameras for well-known stereo-matching. The first SFF method was presented by [?], the main idea of the SFF is using a relationship between the focus level and the object distance from the focused plane. The conventional works proposed to estimate the shape from image-sequence that has been taken while changing the focused plane on the object gradually. In [?, ?], Laplacian oparator is used to evaluate the high frequency components of spatial domain, to know the focus level at each pixel, whereas, Gray value variance is used in [?]. To observe the transition of the focus level for z-direction(depth), the SFF enables to detect the focal distance at each pixel which is related by the object distance. Nevertheless conventional methods supposed that an object has rough and uniform texture on the surface, they were not able to estimate the shape in thin textures areas and edge areas correctly (see Fig. 1). While these areas are often included in the photo picture, our research is motivated to improve accuracy of the measurement of the focus level (called the focus measure) onto those areas. To address the above problems, we propose a method to use the color segmentation and the bilateral filtering.