Deep Reinforcement Learning-Based Charging Pricing Strategy for Charging Station Operators and Charging Navigation for EVs

With the rapid development of electric vehicles (EVs), the number of charging stations (CSs) has increased charging demands. In general, charging station operators (CSOs) in urban areas optimize pricing strategies to maximize their profits, while EV users aim to minimize traveling and charging costs by routing and choosing optimal CSs which can be formulated as a charging navigation problem. In this paper, a bi-level Nash-Stackelberg game framework is established to investigate the competitive pricing strategies for CSOs and the interactions between CSOs and the transportation system operator (TSO) which is introduced as a non-profit third-party to navigate EV users. Specifically, at the upper level of the framework, the pricing competition forms a Nash equilibrium with incomplete information due to privacy constraints. At the lower level, the TSO recommends the optimal CSs and related travel routes for EV users to reduce costs. The interactions between two levels, which are the charging load response to the price signals, are represented by a non-cooperative Stackelberg game. Meanwhile, a soft actor-critic algorithm based on a multi-agent deep reinforcement learning method is developed to determine the optimal pricing strategies for CSOs while considering privacy protection and spatial-temporal characteristics. A double deep Q-network is used to learn the optimal charging station choice for TSO. The case studies on Shenzhen are conducted to verify the effectiveness of the proposed method.