Efficient Multiview Depth Coding Optimization Based on Allowable Depth Distortion in View Synthesis

Depth video is used as the geometrical information of 3D world scenes in 3D view synthesis. Due to the mismatch between the number of depth levels and disparity levels in the view synthesis, the relationship between depth distortion and rendering position error can be modeled as a many-to-one mapping function, in which different depth distortion values might be projected to the same geometrical distortion in the synthesized virtual view image. Based on this property, we present an allowable depth distortion (ADD) model for 3D depth map coding. Then, an ADD-based rate-distortion model is proposed for mode decision and motion/disparity estimation modules aiming at minimizing view synthesis distortion at a given bit rate constraint. In addition, an ADD-based depth bit reduction algorithm is proposed to further reduce the depth bit rate while maintaining the qualities of the synthesized images. Experimental results in intra depth coding show that the proposed overall algorithm achieves Bjontegaard delta peak signal-to-noise ratio gains of 1.58 and 2.68 dB on average for half and integer-pixel rendering precisions, respectively. In addition, the proposed algorithms are also highly efficient for inter depth coding when evaluated with different metrics.

I. Introduction

Three Dimensional Video (3DV) has been attracting more and more attention recently since it is able to provide immersive vision, real 3D depth perception and new visual enjoyments for new type of multimedia applications, such as Three Dimensional TeleVision (3DTV) and Free-viewpoint TeleVision (FTV). Multiview depth video is one of the most important data of 3DV [1], which provides the geometrical information of the 3D world scene and allows arbitrary view rendering with high quality and low complexity [2]. To lower the complexity of the video clients, the depth videos are generated, encoded at server and transmitted to the clients instead of being generated with the multiview video at the clients [1]. However, the depth videos possess large data volume and their amounts increase with the number of views. So in addition to efficient color encoder, high efficiency depth encoding algorithm is also highly desired to reduce the requirements on the storage space and transmission bandwidth.