Automatic Foot Fault Detection in Tennis Using Shoe Keypoints

A foot fault in tennis is an illegal move performed by a server that is subject to the judgement of the line umpire. We propose a computer vision-based foot fault detection algorithm. Our aim is to detect if the player’s foot crosses the baseline during service. As a first step, we crop the person from the video frame that is being analyzed. Following this, we obtain the cropped shoe region and obtain the keypoint for the front of the shoe using a custom algorithm implemented in this paper using a variant of the LeNet architecture. We make use of Hough line transforms to detect the court boundary lines. The keypoint position of the tip of the server’s shoe is compared with the equation of the boundary line that is calculated in order to obtain a result of whether there was a foot fault violation. We make use of a pre-trained YOLOv8 model for the detecting the person and make use of a custom annotated dataset to train the YOLOv8 network for localizing the shoe. We obtain an accuracy of 97.49% on our keypoint detection algorithm while making use of tennis service videos found on the internet. This simple system would enable faster analysis of game footage and would provide an objective assessment of fouls in tennis as well as other sports.