Design of CNNs with adaptive activation function and application in fault detection

Activation function plays an important role in deep convolutional neural network. It realizes the nonlinear mapping of eigenspace. The traditional activation function is a fixed form in the network model. In this paper, the methods of linear combination activation functions and gated adaptive activation functions are proposed to learn the activation function. By combining basic activation functions and using gating ideas to allow activation functions to adapt to the input. Parameters are learned through back propagation. Using the learning parameters, a nonlinear mapping suitable for the specific data distribution is established. Compared with the traditional activation function, more complex nonlinear representation or linear representation can be obtained, which can help a network model more suitable for input data. Moreover, the proposed gated activation function can adjust the form of the activation function according to the specific input, which can increase the ability of feature extraction and generalization of network model. Experiments show that the proposed adaptive CNN model has better performance in fault detection.