Fusion side information based on spatio-temporal correlation in distributed multi-view video coding

Generating accurate SI is crucial to the decoding performance and the quality of reconstructed frames in distributed multi-view video coding (DMVC). There have been some fusion methods between intra-view estimation and inter-view estimation to get accurate SI. In this paper, we present a new fusion scheme to get more accurate SI by exploiting the spatio-temporal correlation, i.e. the inherent spatial correlation of the inter-view estimation, and the temporal correlation obtained by the feature of Motion Vector (MV) during intra-view estimation. After getting the block's MV, the fusion scheme distinguishes the motion characteristics of this block so that it can select from triple pairs of KFs in the adjacent views to finish inter-view estimation. Then the fusion mechanism works on the temporal SI and the selected spatial SI to get the final SI. Experimental results show that up to 1.3dB of PSNR improvement can be obtained for “Ballroom” and about 1.0dB for “Exit” than the initial fusion method in [9].

I. Introduction

The abilities of memory and computation of the cameras in the new emerging applications, such as wireless video sensors for surveillance, wireless PC cameras, mobile cameraphones and disposable video cameras, are scarce. So they need encoders with low complexity, which is quite different from traditional video codec whose encoder bears most of computation. The theoretical foundations of distributed video coding (DVC) can meet with the requirements of these new applications due to separate encoding and joint decoding. [1] states that separate encoding and joint decoding can get the same performance with the scheme of joint encoding and joint decoding that is utilized in traditional video codec. Thus, DVC shifts the complexity from the encoder to the decoder and can get the same performance with the traditional video codec.