Dual-layer Deep Reinforcement Learning for Joint Beam Management and Resource Allocation

The utilization of millimeter-wave in vehicle-to-vehicle (V2V) communications can ensure high system capacity. However, in dense high-mobility environment, V2V communications will encounter severe resource collisions and require fre-quent beam training resulting in substantial signaling overhead. To address the above issues, we study the joint optimization problem of beam management and resource allocation in the millimeter-wave V2V communication system. Specifically, we propose a dual-layer deep reinforcement learning (DRL) archi-tecture that combines beam management and resource allocation into two interconnected tasks. Leveraging this dual-layer DRL architecture, we obtain a solution that involves interactive work between a communication module and an adaptive learning module. This approach is able to collect channel state information in real time and adapt to the ever-changing environment sufficiently. Simulation results show that the joint optimization scheme exhibits fast convergence, improves the effective achievable rate, and reduces the signaling overhead.