Compression Artifact Removal with Ensemble Learning of Neural Networks

We propose to improve the reconstruction quality of DLVC intra coding based on an ensemble of deep restoration neural networks. Different ways are proposed to generate diversity models, and based on these models, the behavior of different integration methods for model ensemble is explored. The experimental results show that model ensemble can bring additional performance gains to post-processing on the basis that deep neural networks have shown great performance improvements. Besides, we observe that both averaging and selection approaches for model ensemble can bring performance gains, and they can be used in combination to pursue better results.

1. Introduction

The amount of image/video data has grown rapidly in the past decade, which brings great challenges to both transmission and storage. To meet these requirements, most of the existing image/video coding schemes perform lossy compression. However, the quantization process in the lossy compression pipeline causes loss of information, leading to artifacts such as blocking, ringing and blurring. As a response to these artifacts, post-processing has been proposed in video compression standards, such as Deblocking Filters (DF) and Sample Adaptive Offset (SAO) in HEVC [10]. In recent years, witnessing the success of deep learning in computer vision tasks, such as super-resolution [3], [5] and denoising [14], [11], researchers have tried to employ deep learning tools to perform post-processing, and have achieved remarkable progress [2], [14], [11], [1], [9], [4], [6].