Quadsight® Vision System in Adverse Weather Maximizing the benefits of visible and thermal cameras

Autonomous vehicles are currently one of the most popular research topics in computer vision. United Nations Economic Commission for Europe recently proposed a regulation for SAE level 3 automated driving systems. The current Operational Design Domains (ODD) are highway, slow speed (i.e. traffic jam), and clear weather conditions. Research is steadily creeping towards a focus on harsh weather conditions. There are now two major issues to investigate: (1) knowing how to characterize ODD and (2) extending ODD to include ‘new’ conditions. This investigation is being carried out within the framework of the AWARD project at Cerema's PAVIN platform. Foresight Automotive's QuadSight® vision system was tested under a range of artificially reproduced weather conditions. The novelty of this work is to present results of a 3D object detection ODD characterization: (a) on a commercially ready system, (b) using visible and thermal wavelengths, and (c) in controlled fog and rain conditions. The use of dual visible and long-wave infrared thermal sensors in stereo is essential to the all-weather detection of pedestrians and vehicles. The thermal sensor is essential in challenging conditions such as nighttime or adverse weather conditions. Rain and low lighting conditions pose no problem for the QuadSight system. The system also performs well in foggy conditions, with the only exception of compromised performance in very dense fog.

I. Introduction

Vision sensors are now commonly used to perceive the environment, detect surrounding objects, and make driving decisions. United Nations Economic Commission for Europe (UNECE) [1] recently proposed a regulation for SAE level 3 automated driving systems [2]. The regulation defines the Operational Design Domains (ODD) that correspond to the use cases which are validated, and in which the vehicle is able to drive in autonomous mode. The current research is focused on unusual or difficult environmental conditions. These include harsh weather conditions, dense traffic scenarios, urban areas, poorly surfaced or damaged roads, abnormal behaviors of other road users, and edge-cases. Two major issues remain concerning the use of vehicle perception systems (sensors and associated algorithms): (i) knowing how to characterize and verify the ODD, and (ii) to extend it to include new conditions.