On the Training of Reinforcement Learning-based Algorithms in 5G and Beyond Radio Access Networks

Reinforcement Learning (RL)-based algorithmic solutions have been profusely proposed in recent years for addressing multiple problems in the Radio Access Network (RAN). However, how RL algorithms have to be trained for a successful exploitation has not received sufficient attention. To address this limitation, which is particularly relevant given the peculiarities of wireless communications, this paper proposes a functional framework for training RL strategies in the RAN. The framework is aligned with the O-RAN Alliance machine learning workflow and introduces specific functionalities for RL, such as the way of specifying the training datasets, the mechanisms to monitor the performance of the trained policies during inference in the real network, and the capability to conduct a retraining if necessary. The proposed framework is illustrated with a relevant use case in 5G, namely RAN slicing, by considering a Deep Q-Network algorithm for capacity sharing. Finally, insights on other possible applicability examples of the proposed framework are provided.