Soccer Pitch Areas Segmentation with Hierarchical U-Net on the SoccerNet Dataset

Soccer video analysis is a challenging area of research in computer vision. Several soccer video analysis systems exist for tasks such as player detection and tracking, player performance analysis or team behaviour analysis. These systems are composed of several building blocks, such as image classifiers for shot classification, object detectors for player and ball detection, or object trackers for player and ball tracking. Another useful building block is an image semantic segmentation module, which may be used to segment different elements in the frame. In our setting, it is used to segment the different areas of the soccer field. This paper describes a semantic segmentation network, that segments the 10 different areas of the soccer pitch. This work builds on our previous work for soccer field line segmentation. Our current method directly segments the areas in the image, using a Deep Learning Convolutional Neural Network, based on U-Net, with hierarchical outputs, and balanced or unbalanced loss weights. The hierarchical output contains four outputs with segmentation masks for different segmentation tasks, arranged in a hierarchical tree. Balanced or unbalanced loss weights allow the system's training to be more influenced or less influenced by the accuracy of a particular output. Our best model produces visually convincing results, and is able to achieve an Average Precision of 80.4%, Average Recall of 73.1%, Average F-score of 76.1%, Average Accuracy of 95.1%, and Average IoU of 62.4%.