Distributed Deep Reinforcement Learning-Based Gradient Quantization for Federated Learning Enabled Vehicle Edge Computing

Federated learning (FL) can protect the privacy of the vehicles in vehicle edge computing (VEC) to a certain extent through sharing the gradients of vehicles’ local models instead of the local data. The gradients of vehicles’ local models are usually large for the vehicular artificial intelligence (AI) applications, thus transmitting such large gradients would cause large per-round latency. Gradient quantization has been proposed as one effective approach to reduce the per-round latency in FL enabled VEC through compressing gradients and reducing the number of bits, i.e., the quantization level, to transmit gradients. The selection of quantization level and thresholds determines the quantization error (QE), which further affects the model accuracy and training time. To do so, the total training time and QE become two key metrics for the FL enabled VEC. It is critical to jointly optimize the total training time and QE for the FL enabled VEC. However, the time-varying channel condition causes more challenges to solve this problem. In this article, we propose a distributed deep reinforcement learning (DRL)-based quantization level allocation scheme to optimize the long-term reward in terms of the total training time and QE. Extensive simulations identify the optimal weighted factors between the total training time and QE, and demonstrate the feasibility and effectiveness of the proposed scheme.

I. Introduction

With the rapid development of autonomous driving technology, a large amount of data are generated by various sensors on vehicles, such as cameras, radar, lidar, as well as proximity and temperature sensors. For example, a self-driving car is expected to generate about 1 GB of data per second [1]. Vehicles need powerful computing capability to process and analyze the data to support the modeling training of the vehicular artificial intelligence (AI) applications, such as simultaneous localization and mapping (SLAM), augmented reality (AR) navigation, object tracking, and high-definition (HD) map generation [2]. However, the computing capability of vehicles is limited. In this situation, vehicle edge computing (VEC) becomes a promising technology to facilitate these applications, where a base station (BS) connected with an edge server can collect and utilize the vehicles’ data for the model training [3]. However, the raw data generated by a vehicle often contains personal information, thus there may be a risk of data leakage in data privacy in VEC [4].