Development of a real-time pen-holding gesture recognition system based on improved YOLOv8

There is a significant correlation between pen-holding gestures and the visual acuity and skeletal development of school-aged children. However, this issue has not received sufficient attention from teachers or parents, resulting in a high error rate in pen-holding gestures among school-aged children. To address this problem, researchers have proposed various pen-holding gesture recognition methods based on computer vision (CV) technology. However, these methods suffer from a lack of fine-grained analysis. Therefore, this study conducted research using the YOLOv8 model and employed a pose estimation method with higher granularity. To fully capture the key point features in pen-holding gestures that possess stronger semantic information, researchers selectively added a status branch to each key point detection branch. This improvement achieved an accuracy of 99.50%, which is a 0.70% improvement over the initial model. Comparative experiments also demonstrated that the proposed method outperformed advanced models such as VIT and DETR by 5.16% and 1.20%, respectively, indicating the advantages of the proposed model. Additionally, researchers developed a corresponding real-time pen-holding gesture recognition system to facilitate the practical application of this model in handwriting practice for school-aged children.