DNN Based Camera Attitude Estimation Using Aggregated Information from Camera and Depth Images

In this paper, we propose a camera attitude estimation network that features aggregated information extracted from camera and depth images. When robots estimate attitude, the estimation method using IMU or gyro-sensors like robots widely used is affected by noise generated from the ground, which makes the attitude estimation difficult. Although there are several methods for estimating the attitude of ground robots, a camera image-based estimation method using deep learning has been studied in recent years. Previous studies have improved the accuracy of the attitude estimation in a known environment, but that in an unknown environment remains low. When making an attitude estimation in an unknown environment, it is important to have an assumption of the terrain, such as that walls are approximately vertical to the ground. Our research uses as input two types of images: a camera image for landscape information and a depth image for terrain information. We propose a network that incorporates a feature extractor that uses cross-reference information of different modalities obtained from these two types of images and a classification type output layer. This network aims to improve the accuracy of attitude estimation in unknown environments. Source code of proposed method is available at https://github.com/Hibiki1020/camera_and_depth_image_attitude_estimator