A Joint Perception Scheme For Connected Vehicles

Currently visual sensing systems, used in autonomous vehicle's research, typically perceive the surrounding environment up to 250m ahead of the vehicle. However, the detection reliability drops when the object's position is more than 50m, due to objects being sparse or unclear for the detection model to make a confident detection. Cooperative perception extends the visual horizon of the onboard sensing system, by expanding the sensing range which improves the detection precision. This paper explores early distributed visual data fusion by creating a multi-vehicle dataset using the Carla simulator to create a shared driving scenario, equipping every spawned vehicle with LiDAR, GNSS, and IMU sensors to emulate a real-driving scenario. Furthermore, we investigate the usage of ZeroMQ-based communication system to distribute visual and meta- data across relevant neighboring vehicles. Since the proposed method distributes raw LiDAR data, we utilize point cloud compression to reduce the size of the published data between relevant connected vehicles to satisfy communication bandwidth requirements. Subsequently, we transform and fuse the received data, and apply a deep learning object detection model to detect the objects in the scene. Our experiments prove that our proposed framework improves the detection average precision while satisfying bandwidth requirements.