Blind Superresolution of Satellite Videos by Ghost Module-Based Convolutional Networks

Deep learning (DL)-based video satellite superresolution (SR) methods have recently yielded superior performance over traditional model-based methods by using an end-to-end manner. Existing DL-based methods usually assume that the blur kernels are known and, thus, do not model the blur kernels during restoration. However, this assumption is rarely held for real satellite videos and leads to oversmoothed results. In this article, we propose a Ghost module-based convolution network model for blind SR of satellite videos. The proposed Ghost module-based video SR (GVSR) method, which assumes that the blur kernel is unknown, consists of two main modules, i.e., the preliminary image generation module and the SR results’ reconstruction module. First, the motion information from adjacent video frames and the wrapped images are explored by an optical flow estimation network, the blur kernel is flexibly obtained by a blur kernel estimation network, and the preliminary high-resolution image is generated by feeding both blur kernel and wrapped images. Second, a reconstruction network consisting of three paths with attention-based Ghost (AG) bottlenecks is designed to remove artifacts in the preliminary image and obtain the final high-quality SR results. Experiments conducted on Jilin-1 and OVS-1 satellite videos demonstrate that the qualitative and quantitative performance of our proposed method is superior to current state-of-the-art methods.

I. Introduction

Video satellite imagery [1], [2], [3], which is a newly developed frontier Earth observation technology, has become one of the most active research fields due to its ability to provide dynamic information. Different from traditional satellites, satellite videos can capture video image sequences and, therefore, provide rich information for moving objects, such as vehicle monitoring [4], ship detection [5], and object tracking [6]. However, the spatial resolution is limited because of the increased temporal resolution and the degradation factors in the imaging process [4]. Superresolution (SR) [7], [8], [9], which aims to restore a high-resolution (HR) image (or sequence) from an observed low-resolution (LR) image (or sequence) of the same scene, is a highly ill-posed yet challenging problem to improve the quality of satellite videos [10], [11].