Cybertwin-Driven DRL-Based Adaptive Transmission Scheduling for Software Defined Vehicular Networks

Efficient transmission control is a challenging issue in vehicular networks due to the highly dynamic and unpredictable link status. In this paper, we propose a cybertwin-driven learning-based transmission scheduling mechanism for software-defined vehicular networks, which can adaptively select/adjust transmission control methods, i.e., loss-based, delay-based and hybrid ones, to suit to the time-varying network environment. In particular, we first analyze the dynamic network characteristics of three realistic vehicular network scenarios in terms of network throughput, round-trip time (RTT) and RTT jitter. Furthermore, we propose a novel transmission scheduling model and formulate the SDVN transmission scheduling issue as a linear programming problem. To obtain the optimized scheduling policies and guarantee the effectiveness of transmission control, we further propose a Cybertwin-driven and Deep Reinforcement Learning based transmission control solution (TcpCDRL). Specifically, TcpCDRL is featured with: (i) using deep reinforcement learning (DRL) to adaptively adjust transmission control policy, (ii) using cybertwin-driven transmission controlling to improve the policy-making effectiveness and timeliness. Simulation results show that the proposed TcpCDRL approach outperforms the single well-known transmission control approach (i.e., TcpWestwood, TcpBic, TcpVeno and TcpVegas) in terms of network throughput and RTT.

I. Introduction

Nowadays, leveraging the fast development of automobile industry, ever-growing vehicles expect to facilitate information sharing through vehicle-to-vehicle (V2V), vehicle-to-roadside-units (V2R), vehicle-to-people (V2P), vehicle-to-infrastructure (V2I), and vehicle to cloud (V2C) [1]. As a promising blueprint, the internet of vehicles (IoV) has been widely proposed to promote data sharing among connected cars as well as provide reliable connections to objects outside cars [2]. As shown in Fig. 1, the IoV is composed with a connected network of vehicles, which can communicate with each other and with roadside units (RSUs), pedestrians handheld devices, even more with satellite/drone assisted communication networks [3], [4]. Due to the powerful inter-connectivity, the IoV is believed to greatly alleviate traffic congestion, reduce accidents, and provide extraordinary entertainment experience to vehicular users. However, the heterogeneity of wireless networks and the inflexibility in protocol deployment greatly hinder the practical application of vehicular networks. There are still several challenging issues to overcome for its real deployment and applications due to high-dynamic peculiarity of vehicular networks [5]–[7].