Enhancement of Visual Place Recognition for Robot Localization Subject to Pedestrian Occlusion

Visual detection methods have been vastly applied for place recognition and localization of autonomous mobile robots (AMRs). As most of the AMRs are deployed in human-centric environments, encountering dynamic changes such as passing-by pedestrians is inevitable. Pedestrian occlusion may greatly reduce the performance of autonomous localization; however, so far there has not been a method developed to address this problem. In this article, we proposed an online image inpainting process to reduce the adverse influence of the pedestrian on visual localization. Specifically, the proposed scheme integrates a deep neural network (DNN)-based pedestrian detector to find and remove the pedestrian pixels on the image. We then repair the image using another DNN that has exhibited excellent image inpainting performance. To verify the proposed scheme, series of field tests were carried out on indoor corridors of an office building. The results showed that the pedestrian appearing in the surveillance camera of the AMR may reduce the accuracy of the topological localization system by 10%. The proposed scheme successfully corrected more than 50% of the predictions that were erroneously made previously.