Multiscale Attention Spatiotemporal Fusion Model Based on Pyramidal Network Constraints

Spatiotemporal fusion (STF) technology effectively addresses the difficulty in obtaining high spatial and temporal resolution images due to compromises in satellite design. Deep learning algorithms are extensively applied in this field, but their performance is significantly impacted by hardware factors, such as the vast differences in sensor resolutions. To address this issue, we have fully considered the scale differences among data from various sources and proposed a two-stage coarse-to-fine STF approach, named SIFnet. SIFnet comprises two stages; the first stage concentrates on learning the scale differences between the data, while the second stage integrates feature maps that represent different spatial resolutions to impose constraints on the network’s learning. This approach enables the preservation of detailed information by reusing feature maps at different scales during the learning process. As a result, the mapping relationship with the real image is closer. In this letter, three datasets containing different ground feature characteristics are used for STF experiments. The results demonstrate that SIFnet can effectively utilize the concept of scale transformation to improve feature extraction and enhance the accuracy of reconstructed images. Compared with various advanced STF algorithms, SIFnet achieves an average improvement of 2% in structural similarity, proving the effectiveness of this method.