Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing

Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize long-term offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Finally, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies.

I. Introduction

Mobile edge computing (MEC) is a promising solution that allows mobile devices to run the highly demanding applications by providing computational resources. However, building trust among multiple parties (e.g., different mobile users and edge/cloud providers) in MEC is a challenge because these parties usually have conflicting interests. Fortunately, blockchain technologies taking advantage of decentralization, anonymity, and trust have begun to exert a significant influence on MEC [1], [2]. Nonetheless, running blockchain mining processes (e.g., performing Proof of Stake (PoS)) while supporting increasingly intelligent applications (processing/analyzing tasks) can require vast computing and storage resources [3]. Thus, the limited computing and storage capacities of mobile devices are restricting real-world blockchain-empowered mobile applications [4]. Therefore, it is vital to develop a computation offloading solution that can extend the capacities of mobile devices by offloading computation-intensive tasks (e.g., blockchain consensus processes) to the edge/cloud servers.