An Effective Epipolar Geometry Assisted Motion Estimation Technique for Multi-View Image and Video Coding

To efficiently encode data-intensive multi-view imaging content, conventional hybrid predictive coding methodologies choose to address the compression by exploiting temporal and inter-viewpoint redundancy. However, their key yet time-consuming component, motion estimation (ME), is usually not efficient in inter-viewpoint prediction because inter-viewpoint motion is quite different from temporal motion. In essence, inter-viewpoint correlation is subject to epipolar geometry, which provides constraints for multi-view image sequences. A fast inter-viewpoint ME technique is hence proposed in this paper to accelerate the encoding by employing epipolar geometry. Theoretical analysis and experimental results prove that the proposed ME algorithm can greatly reduce search region and effectively track large and irregular motion that is typical for convergent multi-view camera setups. As a result, compared with fast full search at large search size adopted in H.264, our proposed ME algorithm can obtain a similar coding efficiency while achieving a speedup ratio of 2.9.

1. INTRODUCTION

Multi-view video or free viewpoint video is an exciting application, because it enables users to interactively watch a static or dynamic scene from different viewing angles. As a brand new application, it has received increasing recent attention. Generally, to provide a smooth multi-perspective viewing experience, content producers need to capture a distinct scene with ideal quality from multiple camera positions, such as the convergent multi-view camera setup in Fig. 1, where the cameras are posed inward to capture the scene from different angles. Usually, the simultaneous multiple video streams from multi-view cameras are referred to as multi-view video. A multi-view video sequence can be naturally regarded as a temporal sequence of special visual effect snapshots, captured from different viewpoints at multiple times. Such a special snapshot is comprised of all the still images taken by multiple cameras at one certain time instance, so it is essentially a multi-view image sequence or a frozen moment sequence as named in [1].