Object Detection under Finger Occlusion in AR Geography Assisted Teaching System by Using PCCNet

Augmented reality(AR) plays an important role in geography teaching for creating interactive and immersive experiences. Combining object detection algorithms with AR can identify the specified content quickly and thus overlay digital content to the real world. However, finger occlusion in AR interactions has a bad influence on object detection, which will affect the users' experience. In this paper, we focus on the detection of country regions on a globe, and aim to improve the performance of object detection in practical AR development. Firstly, we propose a geographic region recognition approach based on region missing-completion. Specifically, we design a supplementary algorithm PCCNet to infer the obscured country by utilizing the invariance of relative position between countries. Moreover, to reduce manual annotation and enrich the virtual dataset, we design a scalable automatic annotation system based on the Unreal Engine and construct a virtual globe dataset named DGAR. Finally, we build an AR geography-assisted teaching system to recognize the area pointed and play multi-media materials. Experiment results show that our proposed approach effectively improves the recognition accuracy from 88.5% to 94%. The practical significance and value of the proposed recognition approach have been confirmed based on the positive user experience with the AR system, highlighting its efficacy in real world scenarios.