Evaluation of Alternative Pre-trained Convolutional Neural Networks for Winter Road Surface Condition Monitoring

Real-time winter road surface condition (RSC) monitoring is of critical importance for both winter road maintenance operators and the travelling public. Accurate and timely RSC information during snow events can help maintenance operators to deliver better maintenance services, such as plowing and salting, for reduced costs and salt usage and improved level of service. With this information, the traveling public can make more informed decisions on whether or not to travel, where to go, and which highways to drive on. In our previous effort we have shown the potential of applying a pre-trained convolutional neural network (CNN) for automatically detecting winter road surface conditions based on images from fixed traffic/weather cameras or in-vehicle devices. This paper focuses on comparing the performance of four most successful CNN models available from the leaders of this technology, namely, VGG16 (Oxford University), ResNet50 (Microsoft), Inception-V3 (Google) and Xception (Google), for solving the RSC classification problem. The models were first customized with additional fully-connected layers of neurons for learning the specific features of the RSC images. The extended models were then trained with a low learning rate for fine-tuning by using a small set of RSC images. The models were tested using a hold-out set of images from cameras installed at different locations, showing highly encouraging results.

I. Introduction

Winter road surface condition monitoring is commonly used by road maintenance operators during decision making. The data’s timeliness and reliability can enable winter maintenance personnel to deploy the right type of maintenance treatments, such as de-icing materials and plowing, at the right time, leading to significant savings in costs and reducing car accidents. Real-time RSC data also benefits the general public by allowing them use data on the current state of the road network when making travelling decisions. RSC is traditionally done using road weather information system (RWIS) stations or highway maintenance contractors that rely on manual patrolling and human observation [1 – 2] . This process is subjective, labor-intensive, time-consuming, and produces low-accuracy results that severely affect the quality the real-time information received by maintenance contractors and the public. In recent years, new technologies such as in-vehicle video recorders and smartphone-based systems have been developed to automate RSC monitoring [3 – 7] . However, limitations surrounding the conditions they work in and classification accuracy have also been observed in these systems. In recent years, researchers have started using machine learning models for solving this problem, such as artificial neural networks (ANN), support vector machine (SVM) and random forests (RF); showing that they can be effective in different road conditions. However, issues surrounding these models have often been raised, including a lack of generality once trained, and a lack of judgement that would typically come from an expert’s opinion [8 – 9] . Deep learning (DL), also known as deep neural network (DNN), is a novel machine learning technique that has been widely explored and successfully applied on a variety of problems, such as image and voice recognition and games [10 – 12] . This technique has been successfully applied to RSC problems, including classifying road surface conditions based on image data, and previous work [22] has showed high promising results.