RLC: A Reinforcement Learning Based Charging Scheme for Battery Swap Stations

Over the past decade, the Electric Vehicle (EV) market has witnessed remarkable expansion. Nevertheless, apprehensions regarding prolonged charging durations frequently contribute to range anxiety. Battery Swap Station (BSS) is a promising solution to the range anxiety problem. Typically, when an EV arrives at a BSS, the depleted battery in the EV is replaced with a fully charged one, then the depleted battery is charged at the maximum speed. In this paper, we propose a novel battery swapping/charging scheme for BSS, Reinforcement Learning based Charging (RLC), to serve as many EVs as possible and minimize the total electricity cost. Specifically, with RLC, for the EVs that do not need full batteries, partially charged ones will be provided. To reduce the electricity cost whenever possible, RLC tries to shift the charging time of a battery to a low-electricity-price period. Technically, RLC uses Deep Deterministic Policy Gradient (DDPG), a Deep Reinforcement Learning (DRL) algorithm, to optimize the charging strategy for the batteries in BSS. Our experimental results indicate that RLC outperforms the existing charging/swapping schemes in terms of battery service rate and total electricity cost.

I. Introduction

Over the past decade, the Electric Vehicle (EV) market has witnessed remarkable expansion. In 2021 alone, global EV sales reached a record high of 6.6 million units, resulting in a twofold increase from the previous year. This is a significant leap from the modest sales of 120,000 units in 2012 [1]. Such a robust surge in EV adoption has been primarily catalyzed by various government initiatives endorsing the transition towards cleaner, more sustainable modes of transportation [2]. Nevertheless, despite the amazing progress, concerns over extended charging periods, often leading to range anxiety, still pose a serious barrier for many prospective EV owners [3], [4].