Reinforcement Learning for Charging Scheduling in a Renewable Powered Battery Swapping Station

Battery Swap technology represents a promising solution to overcome the main obstacles to a widespread adoption of electric vehicles (EVs) in urban environment, like the limited range of EVs and the long battery charging time. Furthermore, with respect to traditional charging stations, it offers higher flexibility in dynamically managing the EV electricity demand to prevent the risk of power grid overload. Nevertheless, proper scheduling of the battery charge process is crucial to offer effective e-mobility services, trading off cost, Quality of Service (QoS) and feasibility constraints. We consider a renewable powered multi-socket Battery Swapping Station (BSS) and design two algorithms based on Approximate Dynamic Programming (ADP) and Reinforcement Learning (RL) to dynamically adapt the battery charging scheduling to the stochastic nature of the system. Both approaches result effective in remarkably increasing the capability to satisfy the customer demand for EV battery charging, at a lower cost with respect to benchmark approaches, with RL outperforming ADP under any budget constraint. In particular, under RL the probability of not satisfying the EV demand is decreased by up to more than 40% with respect to benchmark approaches, and cost can be significantly reduced by almost 20%, jointly with a greener system operation. Furthermore, our results show that a fine tuning of hyper-parameters is fundamental to properly trade off cost and QoS constraints according to varying business needs. Finally, we analyse how the proposed strategies may affect the battery health due to their impact on battery degradation, hence influencing the BSS management cost.