MDRL-IR: Incentive Routing for Blockchain Scalability With Memory-Based Deep Reinforcement Learning

Blockchain-based cryptocurrencies have developed rapidly in recent years, however, scalability is one of the biggest challenge. Payment channel networks (PCNs) are one of the important solutions to blockchain scalability and routing is the most critical problem in PCN. Routing algorithms in PCNs have evolved fast and achieved high throughput. However, most of these routing algorithms are designed from the perspective of technical feasibility, and few algorithms focus on the incentives of each off-chain participant, especially the economic incentives for intermediate routing nodes. Besides, due to the highly dynamic nature of off-chain channel deposits, existing routing algorithms rely heavily on channel deposit probing in order to ensure high throughput. In this article, we design routing algorithms from an incentive perspective to improve the profit of intermediate nodes and use deep learning to reduce the dependency of off-chain routing on channel deposit probing. Our experiments show that under the same model, MDRL-IR can increase the profit of intermediate nodes by up to 1.87x and increase the throughput by up to 2.0x compared to the state-of-the-art routing algorithm, while ensuring that the user routing cost per unit throughput remains unchanged. Moreover, approximate performance can be achieved when deposit probing is greatly reduced.

I. Introduction

Originating from Bitcoin [1], which emerged more than a decade ago, blockchain provides a distributed consensus mechanism through which mutually mistrusting entities can cooperate without a trusted third party. The advent of blockchain has spurred rapid and widespread innovation across different scientific disciplines, such as healthcare, finance, government operations, logistics, and many others. Cryptocurrency represented by Bitcoin is an important application of blockchain. However, currently cryptocurrencies such as Bitcoin still suffer from poor scalability [2], [3]. On-chain transactions have the disadvantages of high handling fees and high latency. Compared with a centralized system that processes a peak of tens of thousands of transactions per second [4], a decentralized system using the consensus mechanism can only process a peak of dozens of transactions per second [5], [6]. Since miners generally prioritize large transactions with high fees, it may take longer for micropayments to be confirmed.