Indoor Place Category Recognition for a Cleaning Robot by Fusing a Probabilistic Approach and Deep Learning

Indoor place category recognition for a cleaning robot is a problem in which a cleaning robot predicts the category of the indoor place using images captured by it. This is similar to scene recognition in computer vision as well as semantic mapping in robotics. Compared with scene recognition, the indoor place category recognition considered in this article differs as follows: 1) the indoor places include typical home objects; 2) a sequence of images instead of an isolated image is provided because the images are captured successively by a cleaning robot; and 3) the camera of the cleaning robot has a different view compared with those of cameras typically used by human beings. Compared with semantic mapping, indoor place category recognition can be considered as a component in semantic SLAM. In this article, a new method based on the combination of a probabilistic approach and deep learning is proposed to address indoor place category recognition for a cleaning robot. Concerning the probabilistic approach, a new place–object fusion method is proposed based on Bayesian inference. For deep learning, the proposed place–object fusion method is trained using a convolutional neural network in an end-to-end framework. Furthermore, a new recurrent neural network, called the Bayesian filtering network (BFN), is proposed to conduct time-domain fusion. Finally, the proposed method is applied to a benchmark dataset and a new dataset developed in this article, and its validity is demonstrated experimentally.