Hyperspectral-Multispectral Image Fusion Using Nndiffuse: Performance Assessment Using A Pixel Classification Task

The spatial resolution of a hyperspectral image (HSI) can be enhanced using a higher-resolution co-registered multi-spectral image and an image fusion algorithm. Assessing the quality of the fused image is traditionally done by way of Wald’s protocol using image-wide quality metrics but these indices alone may not be enough to predict whether or not the enhanced image results in better performance on a task such as target detection or pixel classification. In this paper we demonstrate hyperspectral-multispectral image fusion using NNDIFFUSE (nearest neighbor-based diffusion algorithm) and assessment of fusion performance via pixel classification. Three classifiers are used (SVM and two 3D deep learning-based classifiers) and their performance on the reference HSI and fused imagery are compared using three post-classification metrics: overall accuracy, mean per-class F1 score, and kappa coefficient. Results show that among fusion algorithms compared, NNDIFFUSE provides the best image fusion product for a pixel classification task.