Recaptured Screen Image Demoiréing in Raw Domain

Capturing screen content by smart-phone cameras has become a daily routine to record or share instant information from display screens for convenience. However, these recaptured screen images are often degraded by moiré patterns and usually present color cast against the original screen source. We observe that performing demoiréing in raw domain before feeding into the image signal processor (ISP) is more effective than demoiréing in the sRGB domain as done in recent demoiréing works. In this paper, we investigate the demoiréing of raw images through a class-specific learning approach. To this end, we build the first well-aligned raw moiré image dataset by pixel-wise alignment between the recaptured images and source ones. Noting that document images occupy a large portion of screen contents and have different properties from generic images, we propose a class-specific learning strategy for textual images and natural color images. In addition, to deal with moiré patterns with various scales, a multi-scale encoder with multi-level feature fusion is proposed. The shared encoder enables us to extract rich representations for the two kinds of contents and the class-specific decoders benefit the specific content reconstruction by focusing on targeted representations. Experiment results demonstrate that our method achieves state-of-the-art demoiréing performance. We have released the code and dataset in https://github.com/tju-chengyijia/RDNet

I. Introduction

Nowadays, various digital screens and high resolution phone cameras are ubiquitous in our daily life. It becomes more and more often to capture screens via phone cameras to note or share useful information for convenience. However, recaptured screen images usually suffer from quality degradation by moiré patterns, which are mainly due to the aliasing among the grids of display screen, color filter array (CFA) in camera sensor, and phone screen. Due to the varying scales and stripes of moiré patterns, it is quite challenging to remove them while keeping image details.