Reinforcement learning for taxi-out time prediction: An improved Q-learning approach

Flights undergo a large percentage of delay between scheduled departure of an aircraft and actual takeoff. This not only leads to passenger resentment but also, results in emission of harmful pollutants that adversely affect the environment. The major causes of delays in the on-time performance of flights are due to air traffic congestion, congestion at the departure terminal, gate push-back delay (taxi-out time delay), other reactionary delays, etc. Delays in taxi-out time have a noteworthy impact on the airline's economy and public health. Also, since the airport operations are dynamic in nature, prediction of taxi-out times for all flights can be challenging. The aim is to show that this method provides promising and potential features to tackle the airport departure problems. This research tests three approaches: Linear Regression, ANFIS & Q-learning, which fall under the realm of prediction to accurately predict the taxi-out times. Historic data of an airport is analyzed and utilized for validation. A novel Q-learning algorithm is proposed to predict the accurate taxi-out times at a specific airport. Operational data is analyzed using the Markov Decision Process (MDP) after which a reinforcement learning methodology for Q-learning is formalized for the estimation of taxi-out time of flights. The predicted taxi-out time result is then compared with the actual taxi-out time of the next set of data. This is done to reduce the taxi-out time error and thus by correctly predicting the taxi-out time harmful emissions and other reactionary delays are reduced. The mean square error obtained from Q-learning, Linear Regression and ANFIS are calculated to identify which of these methods have the least prediction error for taxi-out time.