DRL-Based Two-Stage SFC Deployment Approach Under Latency Constraints

With the continuous emergence of latency-sensitive services, such as IoT applications and augmented reality services, the delay constraint has increasingly become a crucial factor influencing the Quality of Service (QoS) in Service Function Chaining (SFC) provision. However, ensuring the acceptance ratio of SFC requests within deadline still faces several challenges, including path planning with sufficient resources and low latency, as well as sequentially deploying Virtual Network Functions (VNFs) on paths with varying resources. In this paper, we propose a Deep Reinforcement Learning (DRL)-based Two-Stage SFC deployment algorithm (DTS-SFC) to enhance the acceptance ratio of SFCs while meeting the deadlines. Specially, a Graph-based Resource Aggregation Routing method (GRAR) is proposed in the first stage, which jointly considers the computational and bandwidth resources to obtain candidate paths within delay constraints. Then, we propose a DRL-based algorithm that takes SFC demands and path resources into account to determine the placement of each VNF. In particular, the VNF movements are utilized as actions to fix the varying dimensions of the action space and enable DRL agent to output effective decisions. Comparative analysis with several DRL-based algorithms and greedy policies demonstrates that the DTS-SFC algorithm outperforms in SFC acceptance ratio, average node utilization, and achieves lower average edge utilization.