Proximal Policy Optimization based computations offloading for delay optimization in UAV-assisted mobile edge computing

UAVs have the potential to enhance wireless systems by improving range and quality, and this can be achieved by utilising the Mobile Edge Computing provided by the unmanned aerial vehicle. In this system, the MEC server mounted on the unmanned aerial vehicle can offer offload services to all the User Equipment in a given space. By offloading some proportion of its tasks to unmanned aerial vehicle for computation, the UE can perform the remaining tasks locally. The objective of this study is to minimize the maximum processing delay in the whole process by optimizing on four parameters, scheduling of the user equipment, portion of the task that has to be offloaded, angle of flight for the UAV, and speed of flight of the UAV, taking into account discrete variables and power constraints. However, due to the nonconvexity, the state space’s high dimension, and action’s space continuous nature of this problem, we are proposing a Proximal Policy optimization algorithm which is based on boosting the policy gradient. Further we will do a comparative analysis of PPO algorithm with other popular Reinforcement Learning algorithms, particularly the Deep Deterministic Policy Gradient algorithm. PPO algorithm can quickly achieve the optimal policy for offloading computation in a dynamic environment. The results obtained implies that the both PPO and DDPG algorithm converges quickly, and the processing delay is minimizd. However, our proposed PPO algorithm has shown significant improvement in minimizing the processing delay. Our model also performs way better compared to basic algorithm such as Deep Q Network (DQN).