Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

In large-scale mobile edge computing (MEC) systems, the task latency, and energy consumption are important for massive resource-consuming and delay-sensitive Internet of Things Devices (IoTDs). Against this background, we propose a distributed intelligent resource scheduling (DIRS) framework to minimize the sum of task latency and energy consumption for all IoTDs, which can be formulated as a mixed-integer nonlinear programming. The DIRS framework includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. Specifically, we first introduce a novel multiagent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Second, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel Levy flight search. Finally, an imitation acceleration scheme is presented to pretrain all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. The simulation results in three typical scenarios demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.

I. Introduction

Recently, with the rapid increase of resource-intensive tasks, e.g., augmented reality (AR), Internet of Things (IoT) applications, and autonomous driving, the quality of our life has the potential to be improved greatly. However, due to the limited size and battery life of IoT devices (IoTDs), these applications may be difficult to be implemented in practice. Fortunately, mobile edge computing (MEC) has been proposed recently as a promising technique to liberate IoTDs from computation-intensive tasks by allowing them to offload their high workloads to edge servers [1], [2].