A Novel Method of Minimizing View Synthesis Distortion Based on Its Non-Monotonicity in 3D Video

In depth-based 3D video, the view synthesis distortion (VSD), is generally measured by modeling the effect of texture and depth errors separately. With such a development, it has been referred that the VSD changes monotonically with respect to to both the texture and depth distortions. In this paper, we find that the VSD does not always change monotonically with them by both theoretical analysis and experimental test, when the effect of the texture and depth errors is considered together. Specifically, first, we prove that the VSD is non-monotonic with the texture distortion. That is, the VSD increases with the increasing texture distortion at higher distortion range but conversely decreases with it at lower range. It is different from the general scenario that only considering the effect of the texture errors. We also analytically depict their relationship with low computational cost and identify the turning point at which the change of the VSD is converted. Second, we confirm that the VSD is always monotonic with the depth distortion, which is consistent with the general scenario that only considering the effect of the depth errors. The non-monotonicity property of the VSD can be utilized to improve the viewing performance of 3D video in relevant applications, since a minimal value of the VSD exists at the turning point. We conduct two applications for this purpose. First, it is used to generate the synthesis view of minimal distortion, which achieves 0.51-dB gain of PSNR on average for the tested scenarios. Second, it is used for lossy compression of texture videos in 3D video, which reduces the coding rate by 24% on average for the tested scenarios, meanwhile, keeps the VSD not increased simultaneously.

I. Introduction

Over the last decade, 3D video technique has gained more and more attentions due to its capability in providing realistic 3D viewing experience. Depth-based 3D video [1], [24] is considered the most promising solution due to its efficient representation and flexibility in view generation compared with general stereo video [47] and multi-view video [46], which introduces depth map perception in addition to conventional flat texture video, such as multi-view-plus-depth video and free-viewpoint video. In this solution, texture videos of different views are captured together with the associated depth videos. At the terminals, desired views are generated with neighboring texture and depth videos by view synthesis technique like depth-image-based rendering (DIBR) [2], [3].