Progressive Point Cloud Compression with the Fusion of Symmetry Based Convolutional Neural Pyramid and Vector Quantization

In this paper, we present a novel 3D structure-awareness image-based point cloud compression scheme, which applies the proposed Symmetry based Convolutional Neural Pyramid (SCNP) to compress colored point clouds view-by-view for 3D model transmission. Input a 3D model to the system, a preprocessing step is first applied to represent the input point cloud as a sequence of view-specific six-dimensional (6D) images, where each pixel is characterized by an RGB color vector and a XYZ 3D point. The transformed 6D images preserve the regular grid structure and thus the redundant information is easy to be removed by conventional image/video compression techniques. Our SCNP first represents each 6D image as a multiple-level pyramid structure for progressively compressing and transmission. The lowest resolution image at the highest level of the pyramid is then decomposed into multiple patches with each of them being coded as the index of a small dictionary through vector quantization. The residual images at other levels are also represented by the vector quantization codes with different patch sizes for progressively reconstructing the input colored point cloud. This process results in a multiple description coding scheme for 3D point cloud compression. With the pre-learned set of dictionaries, the projected view-specific 6D images of the input 3D model are encoded one-by-one to obtain the compressed results for 3D model transmission. In the receiver end, the 3D model is reconstructed by merging all the reconstructed point clouds where each of them is decoded from the corresponding view-specific image. Finally, the conventional 3D reconstruction approach has been applied to remove redundant 3D points for reconstructing the 3D model. Experiments demonstrate the effectiveness of our approach which attains better performance than the current state-of-the-art point cloud compression methods.

I. Introduction

Since the point clouds allow free-view point rendering, it makes the model computationally efficient to represent objects which has complex topology [1]. Especially, it benefits from the rendering techniques [2] and computing devices. However, a substantial issue with 3D point cloud processing is the unstructured representation has a sparse, disruptive influence in the grid structure, and thus is often difficult to be compressed. To deal with these difficulties, to represent the input 3D model as a point cloud, the 3D point cloud is first transformed into a set of view-specific images by projecting them into 2D planes [3]. The preprocessing converts the input 3D point cloud into a set of view-specific 6D images which share a common regular 2D grid structure for further processing. Each pixel of a view-specific image is then characterized by a feature vector (p,c), where p=(x,y,z) and c=(r,g,b) are the 3D position and color information of the pixel, respectively. The transformed 6D image consisting of a color sub-image and a structure sub-image preserves the regular grid structure. We can apply the complex intra-frame prediction of High Efficiency Video Coding (HEVC) [4] to compress each sub-image in 2D image space for the purpose of improving rate-distortion (RD) performance of the implicit point cloud compression. However, the goal is obviously limited by the complexity of HECV