PreM-FedIoV: A Novel Federated Reinforcement Learning Framework for Predictive Maintenance in IoV

The Internet of Vehicles (IoV) enhances data availability by equipping a plethora of sensors, driving the automotive industry towards data-driven Predictive Maintenance (PreM) models. However, traditional centralized PreM solutions, requiring complete access to training data, raise concerns about data privacy. PreM in the automotive domain is more challenging than in many other fields, partly due to the varying distribution nature of data samples and the limited network connectivity time caused by vehicle mobility. To address these challenges, we propose the PreM-FedIoV framework, extending single-agent Double Deep Q-Network (DDQN) to Multi-Agent Double Deep Q-Network (MADDQN). In each round, each vehicle client uploads a data packet to the server based on the current contention window, containing its local model, local test Mean Absolute Error (MAE), and a timestamp. The server initially performs federated aggregation on the received local models. The MADDQN module then dynamically adjusts the contention window of each vehicle for the next round based on the local test MAE and communication statistical state, aiming to optimize communication costs and predictive performance. Additionally, we utilize NS-3 to create IoV simulations and deploy the PreM-FedIoV framework within NS3-gym. We choose Federated Averaging (FedAvg) and FedAdam following the IEEE 802.11p standard as baselines. The experiments demonstrate significant improvements in our framework compared to state-of-the-art algorithms. On the C-MAPSS dataset, we achieve reductions of up to 10.2% in MAE, 26.31% in average communication clock time per round, and 65.6% in the number of participating clients per round. For the Random Battery Usage dataset, with up to 4.55%, 24.44%, and 36.58% improvements in the respective metrics.

I. Introduction

Connected fleets of vehicles have been extensively integrated into the Industrial Internet of Things (IoT) field globally [1], [2]. Through efficient collaboration, they achieve seamless data collection and extraction, thereby injecting new momentum into the optimization of operational models and the simplification of complex production processes [3]. In this context, vehicle malfunctions pose not only a significant threat to the safety of vehicle owners but also result in substantial labor costs for enterprises. Due to the exponential increase in vehicle-generated data and the significant resurgence of Artificial Intelligence (AI), data-driven Predictive Maintenance (PreM) has emerged as the most efficient solution to address the aforementioned issues [4]. Based on features extracted from state monitoring data, data-driven PreM methods can forecast the Remaining Useful Life (RUL) of components, estimating the time remaining before operational failure and guiding maintenance strategies [5]. The existing methods are primarily implemented by uploading data captured by on-board sensors to a central server for model training. However, transmitting these data wirelessly to the central server incurs significant wireless resource costs and communication delays, especially in the presence of a large number of moving vehicles. On the other hand, data captured by onboard sensors often includes sensitive privacy information, such as vehicle location, driver behavior, and onboard camera data. In recent years, there has been increasing interest in applying Federated Learning (FL) to the IoV [6], [7].