Graph-Based Depth Denoising & Dequantization for Point Cloud Enhancement

A 3D point cloud is typically constructed from depth measurements acquired by sensors at one or more viewpoints. The measurements suffer from both quantization and noise corruption. To improve quality, previous works denoise a point cloud a posteriori after projecting the imperfect depth data onto 3D space. Instead, we enhance depth measurements directly on the sensed images a priori, before synthesizing a 3D point cloud. By enhancing near the physical sensing process, we tailor our optimization to our depth formation model before subsequent processing steps that obscure measurement errors. Specifically, we model depth formation as a combined process of signal-dependent noise addition and non-uniform log-based quantization. The designed model is validated (with parameters fitted) using collected empirical data from a representative depth sensor. To enhance each pixel row in a depth image, we first encode intra-view similarities between available row pixels as edge weights via feature graph learning. We next establish inter-view similarities with another rectified depth image via viewpoint mapping and sparse linear interpolation. This leads to a maximum a posteriori (MAP) graph filtering objective that is convex and differentiable. We minimize the objective efficiently using accelerated gradient descent (AGD), where the optimal step size is approximated via Gershgorin circle theorem (GCT). Experiments show that our method significantly outperformed recent point cloud denoising schemes and state-of-the-art image denoising schemes in two established point cloud quality metrics.

I. Introduction

Point Cloud (PC) is a collection of discrete geometric samples of the surface of a physical object in 3D space, useful for a range of imaging applications such as immersive communication and virtual/augmented reality (AR/VR) [1], [2], [3], [4]. With the ubiquity of inexpensive active sensors like Microsoft Kinect and Intel RealSense, one common method to generate a PC is to deploy one or more sensors at multiple viewpoints to capture depth measurements (in the form of images) of an object, then project these measurements to 3D space to synthesize a PC [5], [6]. However, limitations in the depth acquisition process mean that the acquired depth measurements suffer from both imprecision (due to quantization) and additive noise. This results in a noisy synthesized PC, and previous works focus on denoising PCs using a variety of methods: low-rank prior, low-dimensional manifold model (LDMM), surface smoothness priors expressed as graph total variation (GTV), graph Laplacian regularizer (GLR) and feature graph Laplacian regularizer (FGLR), Moving Robust Principal Components Analysis (MRPCA), data-driven learning, etc [7], [8], [9], [10], [11], [12].