View Scalable Multiview Video Coding Using 3-D Warping With Depth Map

Multiview video coding demands high compression rates as well as view scalability, which enables the video to be displayed on a multitude of different terminals. In order to achieve view scalability, it is necessary to limit the inter-view prediction structure. In this paper, we propose a new multiview video coding scheme that can improve the compression efficiency under such a limited inter-view prediction structure. All views are divided into two groups in the proposed scheme: base view and enhancement views. The proposed scheme first estimates a view-dependent geometry of the base view. It then uses a video encoder to encode the video of base view. The view-dependent geometry is also encoded by the video encoder. The scheme then generates prediction images of enhancement views from the decoded video and the view-dependent geometry by using image-based rendering techniques, and it makes residual signals for each enhancement view. Finally, it encodes residual signals by the conventional video encoder as if they were regular video signals. We implement one encoder that employs this scheme by using a depth map as the view-dependent geometry and 3-D warping as the view generation method. In order to increase the coding efficiency, we adopt the following three modifications: (1) object-based interpolation on 3-D warping; (2) depth estimation with consideration of rate-distortion costs; and (3) quarter-pel accuracy depth representation. Experiments show that the proposed scheme offers about 30% higher compression efficiency than the conventional scheme, even though one depth map video is added to the original multiview video.

I. Introduction

Free-Viewpoint television (FTV) and three-dimensional (3-D) video are attracting a lot of interest as one of the new types of video. FTV allows viewers to change their viewpoints without concern for the camera position; current TV and DVD systems constrain all viewers to watch the video from the viewpoint of the producer [1]. Three-dimensional video gives depth effect to the observed scenery as if it really existed in front of viewers. It is said that these types of videos offer many advantages in many fields, including entertainment, education, and archiving. So, in the future, these videos may be offered in various applications including broadcast television, video storage, Internet streaming video, and mobile video. Because there are such a wide variety of applications, there is great advantage in enhancing the interoperability of their data.