Compact Convolutional Transformer based on Sharpness-Aware Minimization for Image Classification

Compact Convolutional Transformer (CCT) is a model that mainly combines Convolution Neural Network (CNN) and Vision Transformer (ViT). However, can CCT model achieve a very low minimum loss when performing the training and increase the model performance during the training so that the curation increases? In this research, we examine the application of Sharpness-Aware Minimization (SAM) and Gradient Centralization (GC) to the CCT model for rice classification. SAM is used as a technique that improves CCT generalization while minimizing loss value and loss sharpness, while GC works directly with gradients by centralizing the gradient vectors to have zero mean. It can both speed up the training process and improve the final generalization performance of CCT. Based on our investigation results, the original CCT model can produce a loss value of 0.100 and an accuracy of 0.9690. However, these results can be improved by using SAM and GC. Through our experiments, the loss decreased, and the accuracy values increased to 0.9988 after applying SAM and GC algorithms.