Face Recognition in Video Using a What-and-Where Fusion Neural Network

A what-and-where fusion neural network is applied to the recognition of human faces from video sequences. The spatio-temporal information contained in successive video frames allows to effectively accumulate a classifier's predictions for each person being tracked in an environment. In a particular realization of this network, a fuzzy ARTMAP neural network is used to classify faces detected in each frame, while a bank of Kalman filters is used to track blobs that contain the extracted faces moving in the environment. Performance of the what-and-where fusion neural network is compared to that of the fuzzy ARTMAP and k-nearest-neighbor (k-NN) classifiers in terms of classification rate, convergence time and compression. In this paper, the impact on performance of setting different region of interest (ROI), of optimizing fuzzy ARTMAP parameters, and of selecting different training subset sizes, is assessed. Simulation results on real-world video sequences indicate that this network can achieve a classification rate that is significantly higher (by approximately 50% in some cases) than that of fuzzy ARTMAP alone, and than that of the k-NN.