mmWave-YOLO: A mmWave Imaging Radar-Based Real-Time Multiclass Object Recognition System for ADAS Applications

This article presents a millimeter wave (mmWave) imaging radar-based real-time multiclass object recognition system for advanced driver-assistance system (ADAS) applications related to construction machinery. While mmWave radar has the advantage of being able to detect signals even in the optically harsh, dark, and dusty environments in which construction machinery is often used, the radar has two orders of magnitude lower resolution than cameras. Since the distance from the radar increases, object features vary, resulting in making it difficult to classify and detect the location of multiple objects. To address this issue, an mmWave-you only look once (YOLO) architecture is proposed that enables highly accurate object classification and location recognition by applying different detectors to each distance data. To provide precise labels for the radar data semiautomatically, a camera-radar cooperative data annotator is also developed. Using the radar only, a real-time (46.6 ms) object classification and location detection of six class objects is achieved. The accuracy of our system is 84% [mean average precision (mAP)], which is slightly higher than that of the red, blue, green (RGB)-camera-based system (78%). In addition, since the radar can acquire similar data of target objects against any background, mmWave-YOLO can detect objects in a variety of scenes with only a small variation of the training dataset, resulting in a lower training cost. Experiments confirmed that it can detect objects in outdoor scenes even when it is trained only with indoor scene data.

I. Introduction

In order to reduce the number of fatal accidents in the construction field, the installation of advanced driver-assistance systems (ADASs) in construction machinery, such as excavators, is strongly recommended. Accidents occurring at construction sites account for 33% of all accidents in all industries, which represents the largest proportion among them [1]. In particular, the number of accidents involving contact with construction machinery occupies a large percentage. To reduce fatal accidents, ADAS that can detect people and objects is increasingly being used in construction machinery [2].