Light Field Multi-View Video Coding With Two-Directional Parallel Inter-View Prediction

Light field (LF) technology has been popularly adopted by a wide range of conventional industries. However, one problem when dealing with LFs is the sheer size of data volume. There have been many multi-view video coding (MVC)-based LF video coding methods reported in the literature, aiming at finding the best prediction structure for LF video coding. It is clear that the number of possible prediction structures is unlimited, and it is also observed that the coding bit-rate can be reduced by increasing the number of bi-directionally encoded views in the prediction structure. However, none work has been conducted to analyze the relationship of the prediction structure with its coding performance. In light of this observation, we first design a new LF-MVC prediction structure by extending the inter-view prediction into a two-directional parallel structure. Analytical models for source coding rate and encoding time are developed to analyze their relationships with the prediction structure, and are proven to be well-matched to our experimental results. Experimental evaluation of two LF video sequences demonstrates that the proposed LF-MVC prediction structure can achieve a factor of 26% bit-rate reduction against the conventional MVC prediction structure for an LF video with 5 × 5 views, and a further 34% bit-rate reduction for an LF video with a larger 10 × 10 views. Compared with the state-of-the-art MVC-based LF video coding prediction structures in the literature, LF-MVC can achieve the best coding performance, and with its high encoding efficiency, is well suited for deployment in practical LF-based 3D systems.

I. Introduction

Recent advances in computational photography have enabled us to explore “light fields (LFs)” from a digital perspective [1]. LF photography can capture both spatial and angular information, and enables new possibilities for digital imaging [2]. LF data can be captured by an array of cameras, resulting an array of images representing the 4D LF. On the other hand, LF cameras use a microlens array to capture LF data, which provide a compact solution for LF photography and replace the cumbersome setup of the camera array. With the advent of commercial LF cameras, LF photography has been popularly adopted by professional photographers to create multi-focus and multi-perspective digital photos. However, one problem when dealing with LFs is the sheer size of data volume, which brings new challenges on how to efficiently store and transmit this massive amount of complex data [3], [4]. Compared to conventional 3D data, LF data exhibits a more complex and unique structure. Therefore, the conventional methods are not directly applicable to LF data compression [5].