Addressing Open-set Object Detection for Autonomous Driving perception: A focus on road objects

Autonomous Vehicles (AVs) are expected to take safe and efficient decisions. Hence, AVs need to be robust to real-world situations and especially to cope with open world setting i.e. the ability to handle novelties such as unseen objects. Classical object detection models are trained to recognize a predefined set of classes but struggle to generalize well to novel classes at inference stage. Open-Set Object Detection (OSOD) aims to address the challenge of correctly detecting objects from unknown classes. However, autonomous driving systems possess specific open-set characteristics that are not yet covered by OSOD methods. Indeed, a detection error could lead to catastrophic events, emphasizing the importance of prioritizing the quality of box detection over quantity. Also, the specific characteristics of objects encountered in road scenes could be leveraged to improve their detection in the open-world setting. In this vein, we introduce a new definition of objects of interest for autonomous driving perception, enabling the proposition of an AV specialized open-set object detector coined ADOS. The proposed model uses a new score, learnt with the background ground truth of the semantic segmentation. This On Road Object score measures whether the object is on drivable areas, enhancing the selection of unknown detection. Experimental evaluations are conducted on simulated and real world datasets and reveal that our method outperforms the baseline approaches in unknown object detection settings with the same detection performance on known objects as the closed-set object detector.