Application of Convolutional Neural Networks for Multimodal Identification Task

Currently, biometric identification systems are often used in mobile applications, banking systems, access control and management systems as well as for the management of mobile robots. In this paper, we consider the problem of personality recognition using facial images and audio signals with speech recordings. The results of the research will be used to create a system of multimodal biometric identification. Since convolutional neural networks demonstrate the highest results regarding the problems of detection, segmentation and classification of objects, this paper also proposes an approach to person identification based on convolutional neural networks. The research was carried out using modern audiovisual database VoxCeleb1. To decrease the computational capability of the experiment, the researchers reduced the number of classes from 1,251 to 200. The development results showed the possibility of using the proposed algorithm as a part of a multimodal identity identification system.