Capturing Local and Global Features in Medical Images by Using Ensemble CNN-Transformer

This paper introduces a groundbreaking classification model called the Controllable Ensemble Transformer and CNN (CETC) for the analysis of medical images. The CETC model combines the powerful capabilities of convolutional neural networks (CNNs) and transformers to effectively capture both high and low frequency features present in medical images. The model architecture comprises three main components: a transformer classification block (TCB), a transposed-convolutional decoder block (TDB), and a convolutional encoder block (CEB). The CEB is responsible for capturing multi-local features at different scales and draws upon components from VGGNet, ResNet, and MobileNet as backbones. By leveraging this combination, the CEB is able to effectively detect and encode local features. The TDB, on the other hand, consists of sub-decoders that decode and sum the captured features using ensemble coefficients. This enables the model to efficiently integrate the information from multiple scales. Finally, the TCB utilizes the SwT backbone and a specially designed prediction head to capture global features, ensuring a comprehensive understanding of the entire image. The paper provides detailed information on the experimental setup and implementation, including the use of transfer learning, data preprocessing techniques, and training settings. The CETC model is trained and evaluated using two publicly available COVID-19 datasets. Moreover, the model is able to outperform existing state-of-the-art models across a variety of evaluation metrics, which is encouraging. Experimental results clearly demonstrate that the CETC model performs better than other deep learning models, emphasizing its potential for accurate and efficient image analysis in the field of medicine.