Gauging Customer Interest Using Skeletal Tracking and Convolutional Neural Network

Retail stores and fashion chains have taken-advantage of data analysis to understand their customers preferences and better serve them. However, not many retail stores have been able to take advantage of technology to collect and analyze their customers preferences in real time. This paper attempts to present a solution to this by making use of depth cameras and image processing using Kinect, and an Emotion recognition deep learning model to gauge customer interest. Cohn-Kanade, an open dataset of facial expression images was used to train the deep learning model. A ResNet-18 based CNN model was employed for emotion recognition and a validation accuracy of 92.19% was achieved for the same. When deployed on a shop window, the approach presented in this paper can be used to detect the number of people that walk by in close proximity of the window and look at the mannequin on display, and then use the emotion recognition model to determine the customers emotions. This data can then be logged and analyzed to find the level of interest that customers have in products displayed on a particular shop window.