Parameter-Free Similarity-Aware Attention Module for Medical Image Classification and Segmentation

Automatic classification and segmentation of medical images play essential roles in computer-aided diagnosis. Deep convolutional neural networks (DCNNs) have shown their advantages on image classification and segmentation. However, they have not achieved the same success on medical images as they have done on natural images. In this paper, two challenges are exploited for DCNNs on medical images, including 1) lack of feature diversity; 2) neglect of small lesions. These two issues heavily influence the classification and segmentation performances. To improve the performance of DCNN on medical images, similarity-aware attention (simi-attention) module is proposed, including a Feature-similarity-aware Channel Attention (FCA) and a Region-similarity-aware Spatial Attention (RSA). Our simi-attention provides three advantages: 1) higher accuracy can be achieved since it extracts both diverse and discriminant features from medical images via our FCA and RSA; 2) the lesions can be exactly focused and located by it even for the data with low intensity contrast and small lesions; 3) it does not increase the complexity of backbone models due to NO trainable parameters in its module. The experimental results are conducted on both classification and segmentation tasks under four public medical classification datasets and two public medical segmentation datasets. The visualization results show that our simi-attention can accurately focuses on the lesions for classification and generate fine segmentation results even for small objects. The overall performances show that our simi-attention can significantly improve the performances of backbone models and outperforms compared attention models on most of datasets for both classification and segmentation.

I. Introduction

Medical image classification aims to distinguish medical images based on clinical pathologies or imaging modalities [1], and medical image segmentation can locate the position of lesions and assist clinicians to quantitatively evaluate the effect in preoperative and postoperative examination [2]. In recent years, deep learning methods, especially deep convolutional neural networks (DCNNs), have gained significant achievements in medical image classification and segmentation tasks. However, they have not achieved the same success on medical images as they have done on natural images [3], [4]. In this work, two challenges in medical image classification and segmentation are exploited and considered. 1)

Lack of feature diversity: Different from natural color images, most medical images are gray-scale images and the lesion area usually has low intensity contrast with the neighboring normal tissues [5], [6]. As shown in Fig. 1(a) of one malignant ultrasound image for breast, the lesion area is blocked with yellow dotted line, which is shown to have very low intensity contrast with the surrounding normal tissues. Hence, it is difficult to extract as diverse features as that extracted from natural color images through convolution operation only [7];

2)

Neglect of small lesions: The key information such as the lesion area in medical images often occupies much fewer pixels than normal tissue and the lesion area is different from normal tissue on particulars [8], [9]. In other words, the particulars in medical images are not so unimportant as in natural images. As shown in Fig. 1(b) of one AMD (Age-related Macular Degeneration) fundus images. The lesion areas are blocked with yellow dotted line, which are too small and inconspicuous when compared to normal tissues. Hence, it is difficult to learn discriminant features from the small and inconspicuous lesions through convolution operation only due to the neglect of small lesions in high level feature maps [10].

Fig. 1.

Illustration of medical image examples: (a) a malignant ultrasound image; and (b) an AMD fundus image.