MonoGAE: Roadside Monocular 3D Object Detection With Ground-Aware Embeddings

Although the majority of recent autonomous driving systems concentrate on developing perception methods based on ego-vehicle sensors, there is an overlooked alternative approach that involves leveraging intelligent roadside cameras to help extend the ego-vehicle perception ability beyond the visual range. We discover that most existing monocular 3D object detectors rely on the ego-vehicle prior assumption that the optical axis of the camera is parallel to the ground. However, the roadside camera is installed on a pole with a pitched angle, which makes the existing methods not optimal for roadside scenes. In this paper, we introduce a novel framework for Roadside Monocular 3D object detection with ground-aware embeddings, named MonoGAE. Specifically, the ground plane is a stable and strong prior knowledge due to the fixed installation of cameras in roadside scenarios. In order to reduce the domain gap between the ground geometry information and high-dimensional image features, we employ a supervised training paradigm with a ground plane to predict high-dimensional ground-aware embeddings. These embeddings are subsequently integrated with image features through cross-attention mechanisms. Furthermore, to improve the detector’s robustness to the divergences in cameras’ installation poses, we replace the ground plane depth map with a novel pixel-level refined ground plane equation map. Our approach demonstrates a substantial performance advantage over all previous monocular 3D object detectors on widely recognized 3D detection benchmarks for roadside cameras. The code and pre-trained models will be released soon.

I. Introduction

Monocular 3D object detection is the task of estimating three-dimensional information solely from a single 2D image, offering extensive applications in real-world scenarios, including autonomous driving and robotics. Due to its low cost and closer proximity to mass production, it has attracted increasing attention from researchers in academia and industry. However, existing research has mainly focused on ego-vehicle applications [5], [32], [34], [37], [38], [54], [55], [56], [57], where the camera’s position is close to the ground and obstacles can be easily occluded by other vehicles. This greatly limits the ego-vehicle perception capabilities and further leads to potential safety hazards in autonomous driving. Therefore, researchers have begun studying roadside perception systems using higher-mounted intelligent sensors, such as cameras, to solve this occlusion problem, expand the perception range, increase the reaction time for autonomous driving in dangerous situations through cooperative techniques [11], [16], [36], [48], [50], and thereby improve safety. In order to promote future research, some large-scale roadside datasets [9], [13], [46], [47], [49] containing images collected from roadside view and corresponding 3D annotations, have been released to provide an important basis for training and evaluating roadside monocular 3D object detection methods.