Camera pose estimation using voxel-based features for autonomous vehicle localization tracking

In an autonomous driving environment, the interactive platform technology inside the vehicle is important. To this end, interactive technology that allows multiple users to interact naturally in real-time through transparent display devices and technology that combines external real-world and augmented reality based on multiple scenarios have been developed. In such a fusion technology, position/orientation information of a vehicle for sensing an outdoor environment is important. In this paper, we propose camera pose estimation using voxel-based features for vehicle position and orientation tracking. Our method is a camera pose estimation method based on a 2D-3D matching scheme using multi-task learning that utilize CNNs(Convolutional Neural Networks) for performance improvement. The scene is divided into voxels, and the 3D points of each voxel are projected back onto the image using ground truth camera pose, and the polygonal area is identified using convex hull. We designed a training dataset for 2D-3D matching, which consists of internal 3D points, correspondence between pairs of images, and voxel regions in the image. We train the learning structure with three multi-task CNNs to obtain voxel region detection and the location and description of the points of interest in it. Thereafter, a 2D-3D matching correspondence is formed, and based on this, the camera pose is estimated by solving the PnP(Perspective-n-Point) problem.

I. Introduction

Due to the commercialization of 5G and autonomous driving, realistic cultural experience service technology using vehicle space is becoming important, and interaction platforms that are linked around transparent display windows are emerging. In particular, the market size of the Augmented Reality (AR) interaction-based content service industry is growing. AR -based services (ex. information, entertainment, digital signage, immersive content, etc.) such as object-based AR advertisements and driving information shown in Figure 1 require registration technologies of external real-world objects [1]–[3].