Rate-distortion analysis for H.264/AVC video coding and its application to rate control

In this paper, an efficient rate-control scheme for H.264/AVC video encoding is proposed. The redesign of the quantization scheme in H.264/AVC results in that the relationship between the quantization parameter and the true quantization stepsize is no longer linear. Based on this observation, we propose a new rate-distortion (R-D) model by utilizing the true quantization stepsize and then develop an improved rate-control scheme for the H.264/AVC encoder based on this new R-D model. In general, the current R-D optimization (RDO) mode-selection scheme in H.264/AVC test model is difficult for rate control, because rate control usually requires a predetermined set of motion vectors and coding modes to select the quantization parameter, whereas the RDO does in the different order and requires a predetermined quantization parameter to select motion vectors and coding modes. To tackle this problem, we develop a complexity-adjustable rate-control scheme based on the proposed R-D model. Briefly, the proposed scheme is a one-pass process at frame level and a partial two-pass process at macroblock level. Since the number of macroblocks with the two-pass processing can be controlled by an encoder parameter, the fully one-pass implementation is a subset of the proposed algorithm. An additional topic discussed in this paper is about video buffering. Since a hypothetical reference decoder (HRD) has been defined in H.264/AVC to guarantee that the buffers never overflow or underflow, the more accurate rate-allocation schemes are proposed to satisfy these requirements of HRD.

I. Introduction

Rate control plays an important role in all standard-compliant video encoders. Without rate control, the underflow and overflow of the client buffer may occur due to the mismatching between the source bit rate and the available channel bandwidth for delivering a compressed bitstream. In other words, without rate control, any video coding encoder would be practically hard to use. Therefore, the video coding standards usually recommend their own nonnormative rate-control schemes during the standardization process, such as TM5 for MPEG-2 [1], TMN8 for H.263 [2], and VM8 for MPEG-4 [3], which have been developed with regards to these video coding standards.