Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites

Modern Earth observation satellites capture multi-exposure bursts of push-frame images that can be super-resolved via computational means. In this work, we propose a super-resolution method for such multi-exposure sequences, a problem that has received very little attention in the literature. The proposed method can handle the signal-dependent noise in the inputs, process sequences of any length, and be robust to inaccuracies in the exposure times. Furthermore, it can be trained end-to-end with self-supervision, without requiring ground truth high resolution frames, which makes it especially suited to handle real data. Central to our method are three key contributions: i) a base-detail decomposition for handling errors in the exposure times, ii) a noise-level-aware feature encoding for improved fusion of frames with varying signal-to-noise ratio and iii) a permutation invariant fusion strategy by temporal pooling operators. We evaluate the proposed method on synthetic and real data and show that it outperforms by a significant margin existing single-exposure approaches that we adapted to the multi-exposure case.

1. Introduction

High resolution (HR) satellite imagery is a key element in a broad range of tasks, including human activity mon-itoring and disaster relief. Super-resolution by computational methods has recently been adopted [7], [41] by the re-mote sensing industry (Planet SkySat, Satellogic Aleph-1). By leveraging high frame rate low-resolution (LR) acquisitions, low-cost constellations can be effective competitors to more traditional high-cost satellites. In order to capture the full dynamic range of the scene, some satellites use ex-posure bracketing, resulting in sequences with varying ex-posures. While several works have addressed multi-image super-resolution (MISR) of single-exposure sequences, al-most no previous work considers the multi-exposure case.