Auto-Tracking Camera System for Remote Learning Using Face Detection and Hand Gesture Recognition Based on Convolutional Neural Network

The purpose of this study is to develop an auto-tracking camera used for remote learning with a direct recording classroom video type, which aims to focus on reducing the model's size so that the model can run on devices that have low computational capabilities, such as the use of the Central Processing Unit (CPU). This study used face detection FaceBoxes to detect faces so that auto-tracking can be applied. This study also used hand gesture Unified Gesture and Fingertip Detection, which can be used for doing specific commands towards the camera. The idea to implement this multi-modal is to help the lecturers create a seamless transition during the lecturing process thus the lecturers do not need to worry the their movement that could make the camera pointing to the wrong direction due to the static movement. This study conducted experiments on the number of batch sizes, CNN architecture, and optimizer as well as the IoT protocols. While the model inferences are running on CPU, we used Raspberry Pi for controlling the servo motor to follow the face in the frame by defined protocols. The expected result is that an auto-tracking camera can enhance learning more effectively and study a more accurate model and inference time to do face detection and recognize hand gestures in real-time on CPU devices.