Online Learning for Joint Beam Tracking and Pattern Optimization in Massive MIMO Systems

In this paper, we consider a joint beam tracking and pattern optimization problem for massive multiple input multiple output (MIMO) systems in which the base station (BS) selects a beamforming codebook and performs adaptive beam tracking taking into account the user mobility. A joint adaptation scheme is developed in a two-phase reinforcement learning framework which utilizes practical signaling and feedback information. In particular, an inner agent adjusts the transmission beam index for a given beamforming codebook based on short-term instantaneous signal-to-noise ratio (SNR) rewards. In addition, an outer agent selects the beamforming codebook based on long-term SNR rewards. Simulation results demonstrate that the proposed online learning outperforms conventional codebook-based beamforming schemes using the same number of feedback information. It is further shown that joint beam tracking and beam pattern adaptation provides a significant SNR gain compared to the beam tracking only schemes, especially as the user mobility increases.

I. Introduction

Next generation communication networks aim to satisfy multiple challenging requirements over various applications. Designing such a network becomes complexly challenging when the requirements are matched with fundamentally challenging environments, for example, supporting high data rate communication for fast mobile entities such as vehicles and unmanned aerial drones [1]–[3]. In 5G, underutilized mmWave bands are used to support high data rates which is coupled with massive multiple input multiple output (MIMO) and advanced beamforming techniques to cope with the heavy path and penetration losses in high frequency ranges. For high mobility applications, supporting pencil point sharp beam alignments becomes much more challenging since the communication channel is continuously changing.