Long-Term Tracking of Evasive Urban Target Based on Intention Inference and Deep Reinforcement Learning

Unmanned aerial vehicles (UAVs) have been widely used in urban target-tracking tasks, where long-term tracking of evasive targets is of great significance for public safety. However, the tracked targets are easily lost due to the evasive behavior of the targets and the unstructured characteristics of the urban environment. To address this issue, this article proposes a hybrid target-tracking approach based on target intention inference and deep reinforcement learning (DRL). First, a target intention inference model based on convolution neural networks (CNNs) is built to infer target intentions by fusing urban environment information and observed target trajectory. Then, the prediction of the target trajectory can be inspired by the inferred target intentions, which can further provide effective guidance to the target search process. In order to fully explore the policy space, the target search policy is developed under a DRL framework, where the search policy is modeled as a deep neural network (DNN) and trained by interacting with the task environment. The simulation results show that the inference of the target intentions can effectively guide the UAV to search for the target and significantly improve the target-tracking performance. Meanwhile, the generalization results indicate that the proposed DRL-based search policy has high robustness to the uncertainty of the target behavior.

I. Introduction

Evasive target tracking in an urban environment is an important issue for public safety, such as terrorist tracking, suspicious criminal tracking, and illegal vehicle tracking [1], [2], [3]. To monitor the behavior of the target in real-time, it is necessary to maintain long-term observation of the target. Generally, unmanned aerial vehicles (UAVs) are used to perform target-tracking missions in urban environments due to their deployment flexibility, low cost, and good maneuverability [4], [5], [6]. However, because of the urban environment’s complexity and the evasive target behavior’s uncertainty, the target is easily out of the observation of the UAV, resulting in the loss of the target. To address this problem, the objective of this study is to develop a target-tracking policy for the UAV to achieve long-term tracking of the evasive target in urban environments.