Large-Scale Cooperative Task Offloading and Resource Allocation in Heterogeneous MEC Systems via Multiagent Reinforcement Learning

In multiaccess edge computing (MEC) systems, existing task offloading methods have provided ultrashort latency services for heterogeneous tasks on mobile devices (MDs). Nevertheless, the complexity of MEC systems grows exponentially with the number of MDs or edge servers (ESs), so learning a good offloading policy is a huge challenge when the number of MDs or ESs is large. Moreover, MDs are often unable to find optimal ESs for offloading since the restricted ESs infrastructures and the spatiotemporally imbalanced task offloading requirements. To solve these problems, we propose a curriculum spatiotemporal multiagent actor–critic (CSTMAAC)-based task offloading method. Each ES is regarded as an agent and the problem is formulated as a multiobjective optimization task. To adapt to the large-scale MEC systems, we first introduce an evolutionary curriculum learning by gradually raising the number of trained ES agents in a phased way. Second, to facilitate the coordination of the offloading policies among geographically distributed ESs, we design an attention-based centralized critic-network. Besides, a delayed access mechanism is introduced that uses information about future task processing competition to capture the impact of potential future task processing contention and help ES agents obtain a better offloading strategy. Finally, critic-network is expanded to multicritics and a dynamic weight mechanism is designed to adaptively optimize multiobjectives and obtain a good balance for multiple objectives. Real-world data sets used in experiments demonstrate that CSTMAAC raises task completion rates and total utility by 13.01%–15.21% and 16.89%–18.32% compared with the existing algorithms.