Enhancing Image Quality by Reducing Compression Artifacts Using Dynamic Window Swin Transformer

Video/image compression codecs utilize the characteristics of the human visual system and its varying sensitivity to certain frequencies, brightness, contrast, and colors to achieve high compression. Inevitably, compression introduces undesirable visual artifacts. As compression standards improve, restoring image quality becomes more challenging. Recently, deep learning based models, especially transformer-based image restoration models, have emerged as a promising approach for reducing compression artifacts, demonstrating very good restoration performance. However, all the proposed transformer based restoration methods use a same fixed window size, confining pixel dependencies in fixed areas. In this paper, we propose a new and unique image restoration method that addresses the shortcoming of existing methods by first introducing a content adaptive dynamic window that is applied to self-attention layers which in turn are weighted by our channel and spatial attention module utilized in Swin Transformer to mainly capture long and medium range pixel dependencies. In addition, local dependencies are further enhanced by integrating a CNN based network inside the Swin Transformer Block to process the image augmented by our self-attention module. Performance evaluations using images compressed by one of the latest compression standards, namely the Versatile Video Coding (VVC), when measured in Peak Signal-to-Noise Ratio (PSNR), our proposed approach achieves an average gain of 1.32dB on three different benchmark datasets for VVC compression artifacts reduction. Additionally, our proposed approach improves the visual quality of compressed images by an average of 2.7% in terms of Video Multimethod Assessment Fusion (VMAF).

I. Introduction

Image restoration is the process of enhancing the visual quality of an image that has been degraded by some kind of noise, distortion or compression. This can include removing noise, blur, or other artifacts from a low quality image to improve the overall visual quality. Image restoration is an important area of image processing that has many practical applications in fields such as medical imaging [1], [2], image super-resolution [3], [4] and compression artifacts reduction [5]. For example, Triantafyllidis et al. [6] and Liu and Bovik [7] utilized frequency-domain techniques to detect and remove the blocking artifacts of compressed images. Different from general image restoration tasks, compression artifacts reduction could take advantage of the prior knowledge of compression standards. For example, block partitioning and Discrete Cosine Transform (DCT) within JPEG result in noticeable blocking artifacts [8]. To address this problem, various blocking artifacts reduction methods have been proposed to improve the quality of JPEG images [9], [10]. Improving the quality of images and videos that have some type of artifacts caused by noise of compression is a hot research topic for both academia and industry. Advances of compression standards such as JPEG and MPEG make quality restoration a more challenging task, as the resulting artifacts are more difficult to locate compared to the old compression approaches [11], [12]. There have been many efforts that focus on image quality restoration.