Rate Control for Video-Based Point Cloud Compression

Rate control is a necessary tool for video-based point cloud compression (V-PCC). However, there is no solution specified on this topic yet. In this paper, we propose the first rate control algorithm for V-PCC. Generally, a rate control algorithm is divided into two processes: bit allocation and bitrate control. In V-PCC, the total bits are composed of three parts: the header information including the auxiliary information and occupancy map, the geometry video, and the attribute video. The bit allocation aims to assign the total bits to these three parts. Since the auxiliary information and occupancy map are encoded losslessly, the bit cost of the header information is fixed. Therefore, we only need to assign bits between the geometry and attribute videos. Our first key contribution is the proposed video-level bit allocation algorithm between the geometry and attribute videos to optimize the overall reconstructed point cloud quality. Then we assign geometry and attribute video bits to each group of pictures (GOP), each frame, and each basic unit (BU). Our second key contribution is that we assign zero bits to the BUs with only unoccupied pixels. The unoccupied pixels are useless for the reconstructed quality of the point cloud and therefore should be assigned zero bits. In the bitrate control process, the encoding parameters are determined, and the model parameters are updated for each frame and BU to achieve the target bits. Our third key contribution is that we propose a BU-level model updating scheme to handle the case where various patches may be placed in different positions in neighboring frames. We use the auxiliary information to find the corresponding BU in the previous frame and apply its model parameters to the current BU. The proposed algorithms are implemented in the V-PCC and High Efficiency Video Coding (HEVC) reference software. The experimental results show that the proposed rate control algorithm can achieve significant bitrate savings compared with the state-of-the-art method.

I. Introduction

A point cloud is a set of points in 3D space that can be used to represent a 3D surface. Not only the geometry information but also each point contains some specific attributes, such as colors and material reflection. The capability of the point cloud for recovering 3D objects makes it promising for extended virtual-reality applications such as 3D immersive telepresence [1] and virtual-reality viewing with interactive parallax [2]. However, the high data rate of the point cloud prevents the adoption of this media format. For example, for a typical dynamic point cloud (DPC) captured by 8i with 30 frames per second [3], each frame usually has approximately one million points. If 30 and 24 bits are used to represent the geometry and attribute of the point cloud, the bitrate of the DPC can be as high as per second without compression. The Moving Pictures Experts Group Immersive media working group (MPEG-I) is currently working on a video-based point cloud compression (V-PCC) standard [4] utilizing the existing video coding technologies to solve this problem.