M3OFF: Module-Compositional Model-Free Computation Offloading in Multi-Environment MEC

Computation offloading is one of the key issues in mobile edge computing (MEC) that alleviates the tension between user equipment's limited capabilities and mobile application's high requirements. To achieve model-free computation offloading when reliable MEC dynamics are unavailable, deep reinforcement learning (DRL) has become a popular methodology. However, most existing DRL-based offloading approaches are developed for a single MEC environment, with invariant system bandwidth, edge capability, task types, etc., while realistic MEC scenarios tend to be of high diversity. Unfortunately, in multi-MEC environments, DRL-based offloading faces at least two challenges, learning inefficiency and interference of offloading experiences. To address the challenges, we propose a DRL-based Multi-environmental Module-compositional Modelfree computation OFFloading (M
³
OFF) framework. M
³
OFF generates offloading policies using module composition instead of a single DRL network so that learning efficiency could be improved by reusing the same modules and learning interference could be reduced by composing different modules. Furthermore, we design multiple module composition-specific training methods for M
³
OFF, including alternate modules-and-composer updates to improve training stability, loss-regularization to avoid module degeneration, and module-dropout to mitigate overfitting. Extensive experimental results on both simulation and testbed demonstrate that M
³
OFF outperforms the performances of most state-of-the-arts in multi-MEC and reaches close to single-MEC.