Design of the Codewords for Performing the Pattern Recognitions via a Set of Perceptrons with the Domains of These Activation Functions Have More Than Two Pieces

This paper proposes a method for designing the codewords for performing the pattern recognitions via a set of perceptrons with the domains of these activation functions have more than two pieces. First, perform a clustering algorithm on a set of input feature vectors. Second, initialize the total number of the perceptrons as the dimensions of the input feature vectors. Third, for each perceptron and each cluster obtained in the above, estimate the weight vector as well as both the upper and the lower projection boundaries for all the input feature vectors in this particular cluster via an optimization approach. Fourth, determine the total number of the regions generated by the activation functions such that it is larger than or equal to the total number of the clusters. Fifth, for each perceptron, perform a clustering algorithm on the sets of both the upper and the lower projection boundaries obtained in the above. Sixth, for each perceptron and each piece of the activation function, find the weight vector and both the upper and the lower projection boundaries for all the input feature vectors in this piece of the activation function via a linear programming approach. Seventh, assign the codewords to the regions generated by the perceptrons. Eighth, if any two different regions generated by the perceptrons with the same codewords correspond to different classes of objects, then increment the total number of the perceptrons. Repeat these procedures until any two different regions generated by the perceptrons with the same codewords correspond to the same classes of objects. Finally, employ these perceptrons for performing the pattern recognitions.

I. Introduction

A traditional single layer perceptron performs the inner products of the weight vector to the input feature vectors. Then, assign the input feature vectors to one of two classes of objects based on the signs of the inner products. This is the simplest neural network [1]. However, it is limited for performing the two classes linearly separable pattern recognitions. To address this issue, a single layer perceptron with the domain of its activation function has more than two pieces is proposed for performing some two classes nonlinear separable pattern recognitions. Here, after performing the inner products of the weight vector to the input feature vectors, a different activation function is applied on the inner products. In particular, the domain of activation function of this new perceptron has more than two pieces where that of the conventional perceptron only has two pieces. In other words, the activation function of this new perceptron looks like a pulse train function where that of the conventional perceptron looks like a two level quantization function. Similarly, a training algorithm is employed for finding the weight vector. However, the training algorithm for this new perceptron [2]–[5] in general does not guarantee to be converged. To address this issue, the optimization approach is employed for finding the weight vector [6]. Nevertheless, it is required to determine the total number of these new perceptrons and design the codewords for these new perceptrons before formulating the design problem as an optimization problem. Therefore, the total number of the perceptrons and the design of the codewords play the important roles for using these new perceptrons for performing the pattern recognitions.