Efficient Online Learning Based Cross-Tier Uplink Scheduling in HetNets

Heterogeneous cellular networks (HetNets), where low-power low-complexity base stations (Pico-BSs) are deployed inside the coverage of macro base stations (Macro-BSs), can significantly improve the spectrum efficiency by Pico- and Macro base station collaboration. Due to cross-tier interference, joint detection of uplink signals is widely adopted so that Pico-BS can either detect the uplink signals locally or forward them to Macro-BS for processing. The latter can achieve increased throughput at the cost of additional backhaul transmission. In this paper, we study the delay-optimal uplink scheduling problem in HetNets with limited backhaul capacity. Local signal detection or joint signal detection is scheduled in a unified delay-optimal framework. Specifically, we first prove that the problem is NP-hard and then formulate it as a Markov Decision Process. We propose an efficient algorithm, called
OLIUS
, that can deal with the exponentially growing state and action space. Furthermore,
OLIUS
is online learning-based which does not require any prior knowledge on user behavior or channel characteristics. We prove the convergence of
OLIUS
and derive an upper bound on its approximation error. Extensive experiments in various scenarios show our algorithm outperforms existing methods in reducing delay and power consumption.