Enabling spatio-temporal aggregation in Birds-Eye-View Vehicle Estimation

Constructing Birds-Eye-View (BEV) maps from monocular images is typically a complex multi-stage process involving the separate vision tasks of ground plane estimation, road segmentation and 3D object detection. However, recent approaches have adopted end-to-end solutions which warp image-based features from the image-plane to BEV while implicitly taking account of camera geometry. In this work, we show how such instantaneous BEV estimation of a scene can be learnt, and a better state estimation of the world can be achieved by incorporating temporal information. Our model learns a representation from monocular video through factorised 3D convolutions and uses this to estimate a BEV occupancy grid of the final frame. We achieve state-of-the-art results for BEV estimation from monocular images, and establish a new benchmark for single-scene BEV estimation from monocular video.

I. Introduction

Autonomous vehicles require spatially and semantically-rich representations of their environment and doing this from cameras alone is challenging. While semantic segmentation in the image-plane is a good initial step, it lacks the spatial layout that would make it directly useful for downstream tasks such as trajectory forecasting and path planning. A semantically segmented birds-eye-view (BEV) map provides a compact method of capturing the spatial configuration of a scene and the agents within it.