A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement

Network Slice placement with the problem of allocation of resources from a virtualized substrate network is an optimization problem which can be formulated as a multi-objective Integer Linear Programming (ILP) problem. However, to cope with the complexity of such a continuous task and seeking for optimality and automation, the use of Machine Learning (ML) techniques appear as a promising approach. We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization heuristic based on the “Power of Two Choices” principle. The DRL algorithm uses the so-called Asynchronous Advantage Actor Critic (A3C) algorithm for fast learning, and Graph Convolutional Networks (GCN) to automate feature extraction from the physical substrate network. The proposed Heuristically-Assisted DRL (HA-DRL) allows for the acceleration of the learning process and substantial gain in resource usage when compared against other state-of-the-art approaches, as evidenced by evaluation results.

I. Introduction

Network Slicing is a major stake in 5G networks, which is notably enabled by virtualization techniques applied to Network Functions, a.k.a. Network Function Virtualization (NFV), and by Software Defined Network (SDN) techniques [1]. Thanks to these technical enablers now widely used in the telecom industry, a telecommunications network becomes a programmable platform, which can offer virtual networks enriched by Virtual Network Functions (VNFs) and IT resources, and can be tailored to the specific needs of certain customers (e.g., companies) or vertical markets (automotive, e-health, etc.). These augmented virtual networks give rise to the concept of Network Slicing, which is specified by standardization bodies [2], [3]. In this paper, we shall consider a Network Slice as a chain of VNFs interconnected by a transport network and with networking (bandwidth) and computing (CPU, RAM) requirements.