Hybrid Sharing Scheme for Distributed Reinforcement Learning in Vehicular Networks

Distributed deep reinforcement learning (DRL) has emerged as a powerful tool for 6G applications in vehicular networks (VNs). To cope with the negative impact on training efficiency by the diversity in computing capability at each vehicle, we propose a hybrid sharing scheme, where experience sharing and gradient sharing modes coexist and are adaptively adopted in different computing and communication situations. Specifically, to enhance training efficiency at the cost of an acceptable energy consumption, we formulate a joint optimization of mode selection, channel assignment and computing resource allocation. To overcome the inherent difficulty in solving the problem, a novel learning algorithm is designed to integrate deep deterministic policy gradient with parameterized dueling double deep Q-learning networks (DDPG-PD3QN). Extensive simulations validate the advantages of our proposed DDPG-PD3QN-based hybrid sharing scheme in terms of training efficiency and energy saving.