Compact Representation of Light Field Data for Refocusing and Focal Stack Reconstruction Using Depth Adaptive Multi-CNN

A light field camera can record image information of the same scene from different viewpoints. Its 4D data allow post processing among which digital refocusing is popular due to its practical usability. However, in server-device applications, the transmission cost of the large volume of data from a server to a device is a major issue. In this paper, we propose a novel hardware-friendly refocusing representation and a depth adaptive parallel multi-CNN (DAPM) neural network to address this problem. At the server, an extended center sub-aperture image and depth map information are generated to be used in reconstructing focal stack instead of the whole light field data. At on-chip devices, the focal stack is reconstructed by a DAPM network using the novel low-size refocusing representation data. Compared to conventional representation methods, the proposed approach shows a 32% reduction of storage size and transmission cost while retaining the same PSNR focal stack image quality.