Acceleration of DNN-based Hand Pose Estimation with Monocular Camera Using TVM

We propose a method to accelerate hand detection and 3D hand landmark estimation using the MediaPipe Hands. MediaPipe Hands performs in real time by switching neural networks. However, we focused on speeding up the neural network for smoother detection. We used Tensor Virtual Machine (TVM) and AutoScheduler to optimize the neural network model and maximize CPU and GPU execution performance. In our experiment, we compared the output of three metrics: total inference time, total joint coordinate error, and total program time, in which we constructed a hand pose estimation system. Experimental results showed that the palm detection model and the hand landmark estimation were 3.9-7.5 times faster on the Ryzen9 3950X and RTX3070. Based on the number of frames per second from the total program time, the hand pose estimation system ran 1.3-1.4 times faster on the CPU but 0.8 times slower on the GPU. Our proposed method effectively speeds up neural networks based on the experimental results. On the other hand, we conclude that further investigation is needed to determine how our proposed method can contribute to the original MediaPipe Hands system.