A Dataset for Autonomous Aircraft Refueling on the Ground (AGR)

Automatic aircraft ground refueling (AAGR) can improve the safety, efficiency, and cost-effectiveness of aircraft ground refueling (AGR), a critical and frequent operation on almost all aircraft. Recent AAGR relies on machine vision, artificial intelligence, and robotics to implement automation. An essential step for automation is AGR scene recognition, which can support further component detection, tracking, process monitoring, and environmental awareness. As in many practical and commercial applications, aircraft refueling data is usually confidential, and no standardized workflow or definition is available. These are the prerequisites and critical challenges to deploying and benefitting advanced data-driven AGR. This study presents a dataset (the AGR Dataset) for AGR scene recognition using image crawling, augmentation, and classification, which has been made available to the community. The AGR dataset crawled over 3k images from 13 databases (over 26k images after augmentation), and different aircraft, illumination, and environmental conditions were included. The ground-truth labeling is conducted manually using a proposed tree-formed decision workflow and six specific AGR tags. Various professionals have independently reviewed the AGR dataset to keep it no-bias. This study proposes the first aircraft refueling image dataset, and an image labeling software with a UI to automate the labeling workflow.