A System for Real-Time Display and Interactive Training of Predictive Structural Defect Models Deployed on UAV

Periodic inspection of ancillary structures is an important practice for the infrastructure of a public highway system. Using Unmanned Aerial Vehicles (UAVs) can reduce the cost and time of these inspections due to their speed, convenience, and operational flexibility. However, commercially available UAV solutions often do not include all the following key features: real-time data collection, multispectral sensors, and defect detection model integration. In this paper, a novel system is proposed that accomplishes all these functions. This system includes visual and thermal sensors and a microcomputer capable of running multiple Convolutional Neural Network (CNN) models to detect structural defects. One such CNN was tested with two datasets of different defect types, resulting in accuracy rates over 90% for each dataset. The results indicated a high performance to aid operators in identifying structural defects. A Graphical User Interface (GUI) is designed to interact with the CNN models, allowing an operator to re-classify and re-train the models for continuous improvement. A live stream of the visual and thermal sensors allows the operator to quickly assess the structure and determine which regions need further evaluation. The payload was optimized for weight and power, allowing for long flight times and a variety of UAV platforms.