Audio-Vision Emergency Vehicle Detection

Emergency vehicles (EVs) are permitted to travel at high speed to quickly reach the destination with the aid of audible and visual warning signals, and other road users are required to clear the path for EVs. However, car drivers may sometimes be unaware of nearby EVs, leading to delay in response or even traffic collisions. This work proposes audio-based and vision-based EV detection systems (EVD) that can detect EVs and alert car drivers to respond appropriately. First, we propose a modified YOLO model tailored to the EVD problem, namely YOLO-EVD, and develop a novel image dataset for vision-based EVD (V-EVD). We utilize cross-stage partial connections at the YOLO-EVD’s neck to enhance the detection performance, in which YOLO-EVD achieves 95.5% mean average precision that is better than those of the other single-stage object detectors. Second, we propose WaveResNet, an end-to-end convolutional neural network, for audio-based EVD (A-EVD) based on the classification of siren sound and traffic noise. With raw waveform input of at least one second, the WaveResNet attains high accuracies, at above 98% in normal traffic, and is robust to noise. Both YOLO-EVD and WaveResNet meet the real-time operation requirement. Also, we integrate YOLO-EVD and WaveResNet to develop a prototype of the audio-vision EVD system (AV-EVD) that is a novel approach in the literature of the EVD problem. Our experiments show the promising results of the AV-EVD system as it produces a low misdetection rate of 1.54%. The proposed A-EVD, V-EVD, and AV-EVD systems can be applied to provide safety functions for private cars, self-driving cars, or smart road infrastructure.