Near Optimal VNF Placement in Edge-Enabled 6G Networks

Edge Computing (EC) in the sixth generation (6G) of mobile networks requires efficient resource allocation mechanisms for the Virtual Network Functions (VNFs) placement. Machine learning (ML) methods, and more specifically, Reinforcement Learning (RL), are a promising approach to solve this problem via learning an appropriate policy. The main contributions of this work are twofold. First, we obtain the theoretical performance bound for VNF placement in EC-enabled networks by formulating the problem mathematically as a finite Markov Decision Process (MDP) and solving it using a dynamic programming (DP) method called Policy Iteration (PI). Second, we develop a practical solution to the problem using RL, where the problem is treated with Q-Learning that considers both computational and communication resources when placing VNFs in the network. The simulation results under different system parameters settings show that the performance of the Q-Learning approach is close to the optimal solution by the PI algorithm (without having its restrictive assumptions on service statistics). This is particularly interesting when the EC node resources are scarce and efficient management of these resources is required.