Replacing Averaging with More Powerful Self-Attention Mechanism for Multi-Image Super-Resolution

In the field of multi-image super-resolution, most advanced models adopt a strategy of calculating an increment and then adding it to a baseline image. However, most existing work focuses on obtaining the increment by modeling the correlation between input images using deep learning techniques, while little attention is paid to the computation of the baseline, which is typically obtained by simply averaging the input images. This paper proposes an improved model that replaces averaging with self-attention mechanism in the existing PIUNet model, which makes the baseline computation phase more powerful. The experimental results show that compared to the original model, our improved model not only shows improvements over the state of the art models on a subset of the PROBA-V dataset but also reduces the required training time.

I. Introduction

Image super-resolution technology is one of the key technologies in computer vision, which aims to convert low-resolution images into high-resolution images. There are two sub-directions worth studying in image super-resolution technology: single-image super-resolution (SISR) and multi-image super-resolution (MISR). MISR aims to extract information from multiple low-resolution images with slight differences captured from the same scene, and then merge them into a high-resolution image, while SISR extracts information from only a single low-resolution image. Due to the complementary information carried by these slight differences, if multiple low-resolution images can be captured and appropriate methods can be used to merge this information, MISR can significantly improve the quality of the resulting image compared to SISR.