LoRa-RL: Deep Reinforcement Learning for Resource Management in Hybrid Energy LoRa Wireless Networks

LoRa wireless networks are considered as a key enabling technology for next-generation Internet of Things (IoT) systems. New IoT deployments (e.g., smart city scenarios) can have thousands of devices per square kilometer leading to huge amount of power consumption to provide connectivity. In this article, we investigate green LoRa wireless networks powered by a hybrid of the grid and renewable energy sources, which can benefit from harvested energy while dealing with the intermittent supply. This article proposes resource management schemes of the limited number of channels and spreading factors (SFs) with the objective of improving the LoRa gateway energy efficiency. First, the problem of grid power consumption minimization while satisfying the system’s quality of service demands is formulated. Specifically, both scenarios the uncorrelated and time-correlated channels are investigated. The optimal resource management problem is solved by decoupling the formulated problem into two subproblems: 1) channel and SF assignment problem and 2) energy management problem. Since the optimal solution is obtained with high complexity, online resource management heuristic algorithms that minimize the grid energy consumption are proposed. Finally, taking into account the channel and energy correlation, adaptable resource management schemes based on reinforcement learning (RL) are developed. Simulation results show that the proposed resource management schemes offer efficient use of renewable energy in LoRa wireless networks.

I. Introduction

LoRa networks are considered as a key technology for next-generation of Internet of Things (IoT) wireless networks [1]. These systems are based on the deployment of a large number of low-powered connected devices. Indeed, an innovative wireless network, such as LoRa, enables the exponential growth connected devices, robust operations, wider coverage, and higher energy efficiency [1]. Hence, LoRa may provide sustainable connectivity to low-powered devices distributed over very large geographical areas [2], [3]. LoRa that operates in the unlicensed bands [4] provides also adaptive transmission rates and coverage for low-powered devices. LoRa enables long range transfer of information with a low transfer rate [6]. The chirp spreading modulation (CSM) was adopted as the modulation technique for LoRa transmission [7]. This scheme is based on coding the information in the frequency shift at the beginning of the symbol. The chirp is assumed to be a kind of carrier and the modulated signal is a chirp waveform, whose behavior depends on the spreading factor (SF). LoRa signals with different SFs are quasiorthogonal [7]. However, LoRa signals with the same SF exhibit cross-correlation properties that could make them vulnerable to interference. The performance of CSM was theoretically investigated in [7]. The performance analysis of this modulation scheme was extended by considering various fading channels in [8] and by considering interference in [9]. The scalability of LoRa networks was investigated in [10] by proposing a stochastic geometry framework. This framework supports the exponential growth of connected devices. Furthermore, adequate and intelligent resource management strategies may be adopted in LoRa networks to enhance the system performance.