A Two-Stage GCN-Based Deep Reinforcement Learning Framework for SFC Embedding in Multi-Datacenter Networks

Network Function Virtualization (NFV), which decouples network functions from hardware and transforms them into Virtual Network Functions (VNFs), is a crucial technology for data center (DC) networks. A service function chain (SFC) is composed of an ordered set of VNFs and virtual links (VLs) connecting them. To optimize the resource allocation in DC networks, we need to efficiently map SFCs onto the physical network. Nevertheless, the dynamics and diversity of SFC requests in multi-datacenter (MDC) networks pose a significant challenge in embedding SFCs. To overcome this challenge, we design a two-stage graph convolutional network (GCN) assisted deep reinforcement learning (DRL) scheme. This framework aims to maximize the overall acceptance ratio of SFC requests while minimizing the total cost in an MDC network. In the first stage, we propose a GCN-based DRL algorithm as a coarse granularity solution to the SFC embedding problem from the macro perspective. This solution outlines a local observation scope (LOS) for each agent in the multi-agent system of the second stage, where all agents simultaneously handle SFC requests from their respective DCs using a multi-agent framework from the micro perspective. Numerical evaluations show that, compared to state-of-the-art methods, the proposed scheme improves the acceptance ratio by approximately 13% compared with the Kolin algorithm and 18% compared with the DQN algorithm and saves the cost by around 28% compared with the Kolin and the DQN.

I. Introduction

Nowadays, datacenters (DCs) have become a key component in supporting cloud computing, large-scale online services, and geographically distributed applications. To provide high-quality and non-disruptive network services to end-users, large enterprises such as Google, Facebook, and Amazon have created multiple DCs in different geographic regions and built inter-DC networks to interconnect them. A network service can be symbolized as a service function chain (SFC) or a virtual network functions forwarding graph (VNF-FG). An SFC is composed of an ordered set of VNFs and virtual links (VLs), realized by DC network resources (e.g., bandwidth (BW), central processing unit (CPU) cycles, and memory space) using network function virtualization (NFV). A successful embedding of an SFC requires that both the resource constraints of the infrastructure and the service level agreement (SLA) [1] of the network service be satisfied. To accommodate as many network services as possible with limited resources, efficient and automatic placement of SFCs becomes increasingly important.