Balancing Spectral and Spatial Quality in CNN-Based Unsupervised Pansharpening

In the last years it has been observed a growing interest toward deep leaning techniques for the pansharpening of multiresolution images. Due to the lack of data with ground truth, most deep learning solutions exploit synthetic reduced-resolution data to carry out supervised training. Such an approach, though granting an easy way to step over the lack of labeled data, has shown its limitations due to the statistical mismatch between real full-resolution data and synthetic reduced-resolution data, which eventually impacts on the generalization capacity of the trained models. This has motivated a recent paradigm shift from supervised to unsupervised learning frameworks for pansharpening. Unsupervised schemes, however, involve the definition of more sophisticated loss functions which comprise, at least, two fundamental terms: one responsible for the spectral quality, meant as consistency between the pansharpened image and the input multispectral component; the other accounting for the spatial quality, read as consistency between the output and the panchromatic input. Despite the very good results shown by many such unsupervised solutions, a minor attention has been devoted to the investigation of the interaction between the above mentioned loss terms and to their proper balance to grant stability while pursuing accuracy. This work aims to explore to what extent unsupervised spatial and spectral consistency losses can be reliably combined without impairing quality.