Depth Image Inpainting for RGB-D Camera Based on Light Field EPI

RGB-D camera is an active depth measurement device. Depth images collected by RGB-D camera usually have depth errors at objects edges and corners in image. Current researches of depth image inpainting methods were focusing on exploiting existed depth data and making use of corresponding RGB image. This paper proposes a method to inpaint depth image by using Light Field EPI (Epipolar Plane Image), which has a special linear structure. We present a software framework to extra objects edges depth data from light field EPI. Then we merge the depth image and light field EPI edges depth data to an integral depth image. Experimental results demonstrate the accurateness of the proposed depth image inpainting method, especially for thin objects and edges in image.

I. Introduction

In augmented reality and 3D reconstruction fields, one of the key steps is capturing scene depth image. Active depth measurement equipment which projects light to the scene can easily capture depth image. Laser range scanner techniques usually achieve high accuracy [1]. However, it causes lots of time because of its slice-by-slice scanning mode. RGB-D camera is a kind of active depth measurement equipment. It contains RGB and depth cameras to capture color and depth images. Time-of-flight (TOF) technology is an advance active depth measure method. It emits light to the space and calculates the depth according to the phase difference between emitted and reflected lights. With TOF, camera's measurement speed and accuracy have been improved. Depth camera generally has a safe distance. If the measurement distance is less or further than its safe distance, ghost area and holes will occur in depth image. Also, when objects obscure each other in the scene, it will cause more holes at the edges in image. Figure 1.

RGB and depth images of kinect V2