Masked Spectral Bands Modeling With Shifted Windows: An Excellent Self-Supervised Learner for Classification of Medical Hyperspectral Images

Hyperspectral imaging has become a popular imaging technique in the medical field, and the development of algorithms for computer-aided diagnosis (CAD) is urgently required. Traditional deep learning techniques require a lot of annotated data, which is a burden on doctors. Self-supervised learning (SSL) is a solution for extracting feature representations from unlabeled data. However, traditional CNN-based SSL algorithms cannot explore relations between neighboring and long-range spectral bands, which limits classification performance. In this letter, the proposed solution is a novel SSL method using a transformer-based technique called masked spectral bands modeling with shifted windows (MSBMSW). This method predicts masked spectral bands as the pretext task and uses a self-attention mechanism with shifted windows to capture the divergence of neighboring spectral bands and enhance information exchange between long-range spectral bands. Experimental results demonstrate that MSBMSW achieves better classification results than many state-of-the-art methods and has potential clinical value for CAD of MHSIs.

I. Introduction

Hyperspectral imaging has advantages for acquiring two-dimensional images across a wide range of electromagnetic spectrums, providing more discriminative information. There are various domains where hyperspectral imaging has been used [1], [2], [3], [4], [5]. Due to the absorption of different wavelength bands by different biological tissues, hyperspectral imaging has tremendous promise for noninvasive illness detection and surgical guiding [6]. Because medical hyperspectral images (MHSIs) have hundreds of bands to examine, computer-aided algorithms are necessary to alleviate the load of diagnosis on physicians. More recently, deep learning-based algorithms have shown promise in fields as CAD of MHSIs [7], [8], [9], and 3DCNN is the most frequently used deep learning method for hyperspectral data due to its ability to utilize both spatial and spectral domains simultaneously [10], [11]. However, the performances of these data-hungry algorithms are limited for the difficulty to realize large-scale annotation of MHSIs.