Adaptive Resource Scheduling in Permissionless Sharded-Blockchains: A Decentralized Multiagent Deep Reinforcement Learning Approach

Existing permissionless sharded-Blockchains come on the scene. However, there is a lack of systematic formulations and experiments regarding the behaviors of individual miners. In this article, we interpret block mining in a permissionless sharded-Blockchain as a repeated $M$ -player noncooperative game with finite actions, and propose a new multiagent deep reinforcement learning (MADRL) framework to allow the miners to maximize their profits in a decentralized fashion by scheduling their resources across the shards without centralized coordination. We formulate the rewards, and design a two-scale action space for each miner to reduce the action space and expedite convergence. We also propose a new MADRL model, named Rainbow-WoLF-PHC, which allows each miner to learn its resource allocation online and converge fast to a mixed strategy Nash equilibrium. Extensive experiments show the superiority of the Rainbow-WoLF-PHC to its alternatives in terms of convergence, stability, and profitable actions. This work provides a prosperous design of an end-user-friendly permissionless sharded-Blockchain.

I. Introduction

Sharding is an emerging technique to address the poor scalability of traditional Blockchains revealed during adoptions in many realms [1], [2], [3]. It partitions a network and allows parallel transactions to be handled in different parts of the network simultaneously [4]. Sharding offers remarkable horizontal scalability for Blockchains as a modern scale-out technology, boosting the total transaction throughput to about 10⁵ to 10⁶ (transactions per second) [4]. Proof-of-work (PoW)-based Blockchain sharding mechanisms are considered suitable in a permissionless environment. Monoxide is the first sharding mechanism that implements a PoW-based consensus algorithm as its intraconsensus process [5], where miners can allocate their computational resources based on their local strategies with no coordination of any central servers. The miners can mine in multiple shards simultaneously to increase their mining profits by multiple folds, as compared to a nonsharded Blockchain. Monoxide achieves high throughput with a large number of shards and a flexible shard size.