Infrastructure-based Perception with Cameras and Radars for Cooperative Driving Scenarios

Roadside infrastructure has enjoyed widespread adoption for various tasks such as traffic surveillance, traffic monitoring, control of traffic flow, and prioritization of public transit and emergency vehicles. As automated driving functions and vehicle communications continue to be researched, cooperative and connected driving scenarios can now be realized. Cooperative driving, however, imposes stringent environmental perception and model requirements. In particular, road users, including pedestrians and cyclists, must be reliably detected and accurately localized. Furthermore, the perception framework must have low latency to provide up-to-date information. In this work, we present a refined, camera-based reference point detector design that does not rely on annotated infrastructure datasets and incorporates fusion with cost-effective radar sensor data to increase system reliability, if available. The reference point detector design is realized with box and instance segmentation object detector models to extract object ground points. In parallel, objects are extracted from radar target data through a clustering pipeline and fused with camera object detections. To demonstrate the real-world applicability of our approaches for cooperative driving scenarios, we provide an extensive evaluation of data from a real test site.

I. Introduction

Roadside infrastructure units (RSUs) have become a vital component of many road networks around the world and an active topic of research [1], [2], mainly due to the widespread availability of inexpensive camera sensors. Early utilization of roadside infrastructure included poles and gantries equipped with camera sensors, which are used to monitor the traffic on the observed road. With the addition of different sensor types, the emergence of vehicle-to-everything (V2X) communication, and intelligent traffic lights and signage, more applications can be explored. These applications include traffic flow control, traffic warnings and congestion detection, traffic rule enforcement, road condition detection, obstacle and construction site warnings, and prioritization of public transport and emergency vehicles. At the same time, with the increasing availability of automated driving functions, trajectory prediction [3] and cooperative maneuver planning can now be realized, especially with assistance from RSUs, which is schematically shown in Fig. 1.