Reinforcement Learning for Charging Scheduling in a Renewable Powered Battery Swapping Station

Battery Swap technology represents a promising solution to overcome the main obstacles to a widespread adoption of electric vehicles (EVs) in urban environment, like the limited range of EVs and the long battery charging time. Furthermore, with respect to traditional charging stations, it offers higher flexibility in dynamically managing the EV electricity demand to prevent the risk of power grid overload. Nevertheless, proper scheduling of the battery charge process is crucial to offer effective e-mobility services, trading off cost, Quality of Service (QoS) and feasibility constraints. We consider a renewable powered multi-socket Battery Swapping Station (BSS) and design two algorithms based on Approximate Dynamic Programming (ADP) and Reinforcement Learning (RL) to dynamically adapt the battery charging scheduling to the stochastic nature of the system. Both approaches result effective in remarkably increasing the capability to satisfy the customer demand for EV battery charging, at a lower cost with respect to benchmark approaches, with RL outperforming ADP under any budget constraint. In particular, under RL the probability of not satisfying the EV demand is decreased by up to more than 40% with respect to benchmark approaches, and cost can be significantly reduced by almost 20%, jointly with a greener system operation. Furthermore, our results show that a fine tuning of hyper-parameters is fundamental to properly trade off cost and QoS constraints according to varying business needs. Finally, we analyse how the proposed strategies may affect the battery health due to their impact on battery degradation, hence influencing the BSS management cost.

I. Introduction

Nowadays the transportation sector still relies mainly on oil as its main energy source, capable to satisfy more than 90% of the overall demand so that road transportation alone is responsible of the half the total oil consumption among all sectors [1]. Interestingly, road transportation is the only sector that has been characterized by a considerable growth of the total final oil consumption in the past years. Indeed, achieving an amount of 2000 Mtoe per year, this consumption has become three-fold larger than five decades ago, determining a raising trend that poses remarkable concerns in terms of sustainability [1]. In addition, air pollution represents a relevant critical issue related to traditional transportation, especially in urban environments. In a similar scenario, the adoption of Electric vehicles (EVs) is gradually emerging as a promising solution to address all the discussed concerns raised by road transportation, as long as EV charging is provided by energy supply systems which rely on Renewable Energy Sources (RES) [2]. Nevertheless, several barriers still prevent a wider diffusion of EVs. First, a more extensive penetration of EVs is hindered by the lack of adequate charging infrastructure [3], [4]. Second, the cost per kWh reduction is not yet sufficient to guarantee a purchasing cost for an EV comparable to that of Internal Combustion Engine (ICE) vehicles [5], [6]. Furthermore, the travel range per charge still results rather limited with respect to ICE vehicles, hence leading drivers to experience a significant range anxiety (fear that a vehicle has insufficient range to reach the destination) [7], [8]. Finally, an additional relevant drawback that slows down the diffusion of EVs on a larger scale is represented by the long charging time of EV batteries, since fast chargers represents only about 13% of the newly installed chargers [2], also due to the huge and unpredictable load that may pose on the electricity distribution infrastructure [9], [10].