MSDF-Net: A Multi-Scale Deep Fusion Network with Dilated Convolutions for Cloud Removal from Sentinel-2 Imagery

Advancements in satellite imaging catalysed exploratory leaps in Earth observation activities. The high spatial and temporal resolution of remotely sensed imagery aid land use land cover identification , cropland monitoring, change detection and disaster management. However, keeping the quality of these observations is still a challenge. The presence of obstacles, such as clouds and aerosols, usually impairs observations from space, and this is not desirable for computer-based applications as they rely hugely on the available data. The cloud removal problem has been studied for decades, but the randomness in the presence and type of cloud made reconstruction tasks challenging. Fusing Synthetic Aperture Radar (SAR) auxiliary information with optical imagery guided the cloud reconstruction problem. The proposed work introduces a Multi-Scale Deep Fusion Network (MSDF-Net) that learns the spatial and temporal dependency from seen samples and removes clouds from multispectral Sentinel-2 observations. The network extracts physical properties from the SAR auxiliary data and aids the cloud-free reconstruction. The proposed method shows its capability to remove both thin and thick clouds. The network is trained, tested, and validated on the SEN12MS-CR dataset, a global real cloud-removal dataset consisting of triplets of Sentinel-2 cloudy, Sentinel-2 cloud-free and Sentinel-1 SAR data. The network is compared against state-of-the-art techniques and found to exhibit superior performance.