Unmanned Aerial Vehicle and Artificial Intelligence for Thermal Target Detection in Search and Rescue Applications

Recent developments in unmanned aerial vehicles (UAV), artificial intelligence and miniaturized thermal imaging systems represent a new opportunity for search and rescue (SAR) experts to survey relatively large areas. The system presented in this paper includes thermal image acquisition, the video processing pipeline that performs object detection and classification of people in need of SAR in outdoor environments. The system is tested on thermal video data from ground based and test flight footage and is found to be able to detect all the target people located in the surveyed area. The training dataset is a combination of gathered data and internet sourced data. The initial data procurement utilised online academic thermal databases and also the simulation of the UAV mounted camera environment. Ground based data was collected at Kangaroo Point cliffs in Brisbane, Australia, giving an approximate elevation of 26m. Airborne datasets were collected at South Bribie Island in Queensland, Australia, at a range of heights and vegetation density. These datasets where collected at different times of the day, allowing for a range of contrast levels between background and intended target. Once all data was collected, individual frames where extracted from each image and augmentation and annotation was completed. The images were gaussian blurred, lightened and darkened, once all annotation were completed. A total of 2751 original images were annotated, with the augmented dataset comprising of 10380 images. The YOLOV3 algorithm was selected as the neural network (NN) to be used for this experiment throughout training and classification. The ‘Experiencor’ GitHub pipeline was also used throughout the entirety of this project for data output and analysis purposes. The algorithm training was implemented on the Queensland University of Technology (QUT) High Performance Computing (HPC) cluster. Two main models were trained using different hyperparameters for comparison purposes. The first model had a loss of 3.81, AP of 98.6 after ˜88 hours of training, with model two having a loss of 4.73, AP of 97.7 after ˜184 hours of training. The comparison shows the importance of the chosen parameters when training detection algorithms of this nature, as minor changes can be the difference between an efficiently trained model and an inefficient failed training attempt. The prediction testing was completed on the test data sets that were not included in the training of the two models. This was done to remove any bias from the system, although it is noted that due to the shared environments of two of the test sets with the training sets, a small amount of bias will exist. Predictions made on the two data sets sharing environmental conditions with the training data, showed good prediction results for both trained models, with very limited false positive and missed detections. The system is flexible in that the user can readily define the types of objects to classify and the object characteristics that should be considered during training.