A Signal-Processing Framework for Occlusion of 3D Scene to Improve the Rendering Quality of Views

Occlusions will reduce the performance of systems in many computer vision applications with discontinuous surfaces of 3D scenes. We explore a signal-processing framework of occlusions based on the light ray visibility to improve the rendering quality of views. An occlusion field (OCF) theory is derived by calculating the relationship between the occluded light rays and the nonoccluded light rays to quantify the occlusion degree (OCD). The OCF framework can describe the various in-scene information captured by the changes in the camera configuration (i.e., position and direction) through a quantitative description of the occlusion information. From a spectral analysis of the OCF, we mathematically derive analytical functions to determine the changing relationship between the scene and the camera configuration. A reconstruction filter can be designed to achieve interference cancellation and compensate for the missing information caused by the occlusions. Our measurements of different occlusions using this OCF framework included both synthetic and actual scenes. The experimental results show that the proposed OCF framework can improves the rendering quality of views and outperforms other known occlusion quantization schemes in a complex scene.

I. Introduction

Occlusions are one of the most frequently observed object surface features in 3D scenes [1]. Occlusion can cause random and discontinuous changes in the scene signal. Obviously, the random and strong discontinuity caused by occlusions will induce strong interference and noise in information acquisition, resulting in local information loss. Information loss can greatly reduce the performance of computational vision systems, such as by increasing the amount of captured information or reducing the rendering quality of novel views. Computational vision algorithms may even fail in conditions of extensive occlusion. For example, the plenoptic sampling theory is derived from the nonocclusion condition for scene surfaces [2]–[7]; these sampling theories are not fully applicable in practice, as some aliasing is always present and affects the rendered output images [8], [9]. The following question thus becomes very important for computer vision algorithms: How is it possible to mathematically quantify the occlusion phenomenon to achieve high-quality reconstruction and display of a 3D scene?