Development Of Video Based Action Recognition System For Item Taking From Shelf

This paper explores the development of video-based action recognition system for taking/returning item from/to the shelf that is fast enough to be used in real-time. For action classification, transfer learning is used on the Inflated 3D model with the ResNet50v1 backbone. Meanwhile, for object recognition, transfer learning is used with the YOLOv4 model because it includes components to determine the position of objects in the image. The system also adds a detector component so that the two classification components do not run continuously. Evaluation of the system produces an accuracy of 80% for the action classification component and 70% for the object recognition component. The entire system is capable of running on real-time video at speeds of up to 25 FPS.