Real-Time 3-D MIMO Antenna Tuning With Deep Reinforcement Learning

The 3D MIMO in 5G system requires adaptive and real-time adjustment of antenna azimuth angle, downtilt and beam combination according to the user distribution, which is a powerful technique to improve coverage and capacity. However, due to the complicated interactions between cells and complex environments, it is challengeable to jointly tune the antenna of multiple cells. Besides, the user distribution is complicated because of huge number of users and variable user locations. In this paper, a practical real-time 3D MIMO antenna tuning method with deep reinforcement learning (DRL) is proposed to jointly adjust the antenna configuration according to time-varying user distributions to improve the coverage and access performance of the system. Specifically, a deep neural network is trained to configure the antenna parameters and the Multi-agent reinforcement learning (MARL) is used to adapt various user distributions. The proposed method has been verified on a 5G simulation environment with real geographical features from the real network of China Mobile. Simulation results indicate that the proposed method can improve the coverage and access performance compared with the typical schemes used in practical networks.

I. Introduction

In 5G cellular networks, the BS will use an antenna array to implement 3D MIMO, which concentrates the transmit (and receive) power in both horizontal and vertical directions and has a downtilt [1], [2]. Furthermore, in order to expand the coverage of cellular networks, beam sweeping is also implemented [3] in 5G, which claims the beam combination in a time interval, as shown in Fig. 1. Because the beam of 3D MIMO tends to be narrower, improper configuration of antenna parameters may lead to the coverage holes, leading to lower access and coverage performance. Therefore, the antenna of multi cells need to be tuned jointly according to the user distribution to ensure the access and coverage performance. In real world, the user distribution has a “tidal effect”, which means that the user distribution is regular and predictable at a specific time [4]. Several studies have shown that human mobility is regular. Human often moves around several major areas, such as homes and workplaces, repeatedly gathering and dispersing [5], [6]. On the other hand, the user distribution may change unexpectedly, that is, users may suddenly gather at some areas. This requires that the antenna configuration is capable of meeting the coarse time granularity adjustment, and can adjust swiftly according to the unexpected user distribution as well. However, the number of users in the actual network tends to be huge, which will lead to a very complicated user distribution. Moreover, multi-cell joint adjustment of the cellular network will lead to an exponential increase in the complexity of the problem. Complex geographical features and uneven BS locations make the problem even more challengeable. In practice, the conventional network optimization method is to configure these parameters through expert experience. The antenna parameter configuration will only be changed after a long period of time (weeks or days). Obviously, real-time (minute or second) antenna parameter configuration can effectively improve network performance. However, traditional method cannot adapt to the real-time and the parameter adjustments under complex and dynamic environment are far from optimal. Meanwhile the global optimum solution of the joint optimization of antenna parameters is nondeterministic polynomia (NP) hard and non-convex. Fig. 1.

A 5G MIMO BS has many narrow beams while 4G BS has only one wide-beam.