Urban Localization with Street Views using a Convolutional Neural Network for End-to-End Camera Pose Regression

This paper presents an end-to-end real-time monocular absolute localization approach that uses Google Street View panoramas as a prior source of information to train a Convolutional Neural Network (CNN). We propose an adaptation of the PoseNet architecture [8] to a sparse database of panoramas. We show that we can expand the latter by synthesizing new images and consequently improve the accuracy of the pose regressor. The main advantage of our method is that it does not require a first passage of an equipped vehicle to build a map. Moreover, the offline data generation and CNN training are automatic and does not require the input of an operator. In the online phase, the approach only uses one camera for localization and regresses poses in a global frame. The conducted experiments show that augmenting the training set as presented in this paper drastically improves the accuracy of the CNN. The results, when compared to a handcrafted-feature-based approach, are less accurate (around 7.5 to 8 m against 2.5 to 3 m) but also less dependent on the position of the camera inside the vehicle. Furthermore, our CNN-based method computes the pose approximately 40 times faster (75 ms per image instead of 3 s) than the handcrafted approach.

I. Introduction

The localization of a vehicle is a task that has raised a lot of attention lately, especially in urban environments. For autonomous driving or simply navigation, positioning a vehicle in cities has proved to be a challenging task due to urban canyons and non-line-of-sight propagation of GNSS signals. As such, many methods rely on the detection of distinctive environment features to localize a vehicle. Simultaneous Localization and Mapping (SLAM) is the privileged method due to its ability to incrementally build a map of the surroundings while localizing the vehicle inside it. However, the application of such methods at a worldwide scale can be problematic, as detailed below.