Masked Spectral Bands Modeling With Shifted Windows: An Excellent Self-Supervised Learner for Classification of Medical Hyperspectral Images

Hyperspectral imaging has become a popular imaging technique in the medical field, and the development of algorithms for computer-aided diagnosis (CAD) is urgently required. Traditional deep learning techniques require a lot of annotated data, which is a burden on doctors. Self-supervised learning (SSL) is a solution for extracting feature representations from unlabeled data. However, traditional CNN-based SSL algorithms cannot explore relations between neighboring and long-range spectral bands, which limits classification performance. In this letter, the proposed solution is a novel SSL method using a transformer-based technique called masked spectral bands modeling with shifted windows (MSBMSW). This method predicts masked spectral bands as the pretext task and uses a self-attention mechanism with shifted windows to capture the divergence of neighboring spectral bands and enhance information exchange between long-range spectral bands. Experimental results demonstrate that MSBMSW achieves better classification results than many state-of-the-art methods and has potential clinical value for CAD of MHSIs.