High-Quality Depth Estimation Using an Exemplar 3D Model for Stereo Conversion

High-quality depth painting for each object in a scene is a challenging task in 2D to 3D stereo conversion. One way to accurately estimate the varying depth within the object in an image is to utilize existing 3D models. Automatic pose estimation approaches based on 2D-3D feature correspondences have been proposed to obtain depth from a given 3D model. However, when the 3D model is not identical to the target object, previous methods often produce erroneous depth in the vicinity of the silhouette of the object. This paper introduces a novel 3D model-based depth estimation method that effectively produces high-quality depth information for rigid objects in a stereo conversion workflow. Given an exemplar 3D model and user correspondences, our method generates detailed depth of an object by optimizing the initial depth obtained by the application of structural fitting and silhouette matching in the image domain. The final depth is accurate up to the given 3D model, while consistent with the image. Our method was applied to various image sequences containing objects with different appearances and varying poses. The experiments show that our method can generate plausible depth information that can be utilized for high-quality 2D to 3D stereo conversion.

1 Introduction

Professional stereo conversion requires the estimation of a high quality depth map for objects in a 2D image sequence. Manual depth painting is one of the most popular approaches for approximating the internal depth of the scene object. For successful conversion, the resulting depth should satisfy the following three requirements. 1) The internal depth of an object should be sufficiently volumetric while smoothly varying. Otherwise, an unnatural depth perception, such as the cardboard effect, may result. 2) Discontinuity of depth should be correctly represented across internal edges and the contours of the object. 3) The depth should be temporally coherent over a sequence of images. While meeting the above requirements, depth painting for each object in every frame is a tedious and challenging task.