Genetic selection of non-linear product terms in the inputs to a linear classifier for handwritten digit recognition

The purpose of the study is to compare neural network and linear system based models in 2-D pattern recognition tasks. Using a linear classifier, non-linear inputs are generated based on the linear inputs using different forms of generating products. These nonlinear inputs form a candidate set from which nonlinear inputs are selected to improve classification performance. A genetic search is performed to find an appropriate set of non-linear inputs. The method is applied to the handwritten digit recognition problem. Results show that the linear model with linear inputs reaches a classification performance on the testing database of 79.3%. However, when nonlinear inputs selected by the genetic algorithm were used, and included as new inputs, the classification performance increased up to 92.8%. These results are compared with those of three non-linear neural network models widely used in classification tasks using the same database. A single-layer perceptron with linear inputs reached 81.0% of correct classification. Perceptron models having a single-hidden layer and two-hidden layers reached classification results of 90.1% and 92.5%, respectively. Therefore, these results show that a linear classifier with an appropriate set of non-linear inputs reached a classification performance similar or better than those obtained by nonlinear neural network classifiers with one and two-hidden layer and with linear inputs.