A Deep Reinforcement Learning Approach for Integrated Automotive Radar Sensing and Communication

We present a deep reinforcement learning approach to design an automotive radar system with integrated sensing and communication. In the proposed system, sparse transmit arrays with quantized phase shifter are used to carry out transmit beamforming to enhance the performance of both radar sensing and communication. Through interaction with environment, the automotive radar learns a reward that reflects the difference between mainlobe peak and the peak sidelobe level in radar sensing mode or communication user feedback in communication mode, and intelligently adjust its beamforming vector. The Wolpertinger policy based action-critic network is introduced for beamforming vector learning, which solves the dimension curse due to huge beamforming action space.

I. Introduction

Millimeter Wave (mmWave) vehicle radar, which operates at 76–81 GHz, is one of the key technologies of autonomous driving system and can sense the environment under various weather conditions [1]. Multi-input multi-output (MIMO) radar is a widely used cost-effective and scalable solution for increasing antenna aperture size by exploiting the idea of virtual sum coarray [1]. The use of sparse arrays combined with MIMO radars can further reduce costs without losing angular resolution [2]. Autonomous vehicles need to exchange information with road infrastructure and other neighbor vehicles to achieve operation coordination, especially in vehicle platooning [3]. Traditionally, the automotive radar sensing and vehicle communication functions are implemented via separated hardware. A dual-function radar communication (DFRC), or integrated sensing and communication (ISAC) system utilizes the same hardware platform to send electromagnetic waves for both environment sensing and communication with neighboring devices [4]–[10], which has found applications in autonomous vehicles [11], [12].