SaRa: A Novel Activation Function with Application to Melanoma Image Classification

Non-linear activation functions play a crucial role during the training and classification phase in deep neural networks. In the past few years, many non-linear activation functions have been proposed, but they have several shortcomings, making deep neural network models inefficient. The drawback of these functions can be in the form of vanishing gradient, dying neuron and gradient explosion, etc. To handle all these problems, this research work has proposed a novel activation function called SaRa (Swish and ReLU Activation), a hybrid function based on Swish and ReLU function. The proposed activation function's performance is validated on the ISIC archive dataset using a transfer learning-based Convolutional Neural Network (CNN) model for melanoma skin cancer classification. It is shown that the proposed function outperforms other activation functions in terms of training validation and classification performance.