Loading [a11y]/accessibility-menu.js
Deep Reinforcement Learning-Based Controller for SOC Management of Multi-Electrical Energy Storage System | IEEE Journals & Magazine | IEEE Xplore

Deep Reinforcement Learning-Based Controller for SOC Management of Multi-Electrical Energy Storage System


Abstract:

The ongoing reduction of the total rotational inertia in modern power systems brings about faster frequency dynamics that must be limited to maintain a secure and economi...Show More

Abstract:

The ongoing reduction of the total rotational inertia in modern power systems brings about faster frequency dynamics that must be limited to maintain a secure and economical operation. Electrical energy storage systems (EESSs) have become increasingly attractive to provide fast frequency response services due to their response times. However, proper management of their finite energy reserves is required to ensure timely and secure operation. This paper proposes a deep reinforcement learning (DRL) based controller to manage the state of charge (SOC) of a Multi-EESS (M-EESS), providing frequency response services to the power grid. The proposed DRL agent is trained using an actor-critic method called Deep Deterministic Policy Gradients (DDPG) that allows for continuous action and smoother SOC control of the M-EESS. Deep neural networks (DNNs) are used to represent the actor and critic policies. The proposed strategy comprises granting the agent a constant reward for each time step that the SOC is within a specific band of its target value combined with a substantial penalty if the SOC reaches its minimum or maximum allowable values. The proposed controller is compared to benchmark DRL methods and other control techniques, i.e., Fuzzy Logic and a traditional PID control. Simulation results show the effectiveness of the proposed approach.
Published in: IEEE Transactions on Smart Grid ( Volume: 11, Issue: 6, November 2020)
Page(s): 5039 - 5050
Date of Publication: 21 May 2020

ISSN Information:

Author image of Francisco Sanchez Gorostiza
Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University, Loughborough, U.K.
Francisco Sanchez Gorostiza (Member, IEEE) received the B.Sc. degree in electrical engineering from Simón Bolívar University, Caracas, Venezuela, in 2011, and the master’s degree in renewable energy technologies from Polytechnic University of Madrid, Spain, in 2013. He is currently pursuing the Ph.D. degree in electrical engineering with Loughborough University, U.K. His research is focused on the development of artificia...Show More
Francisco Sanchez Gorostiza (Member, IEEE) received the B.Sc. degree in electrical engineering from Simón Bolívar University, Caracas, Venezuela, in 2011, and the master’s degree in renewable energy technologies from Polytechnic University of Madrid, Spain, in 2013. He is currently pursuing the Ph.D. degree in electrical engineering with Loughborough University, U.K. His research is focused on the development of artificia...View more
Author image of Francisco M. Gonzalez-Longatt
Department of Electrical Engineering, Information Technology and Cybernetics, University of South-Eastern Norway, Porsgrunn, Norway
Francisco M. Gonzalez-Longatt (Senior Member, IEEE) received the Electrical Engineering degree (First Class) from the Instituto Universitario Politécnico de la Fuerza Armada Nacional, Venezuela, in 1994, the Master of Business Administration degree (Hons.) from the Universidad Bicentenaria de Aragua, Venezuela, in 1999, and the Ph.D. degree in electrical power engineering from the Universidad Central de Venezuela in 2008....Show More
Francisco M. Gonzalez-Longatt (Senior Member, IEEE) received the Electrical Engineering degree (First Class) from the Instituto Universitario Politécnico de la Fuerza Armada Nacional, Venezuela, in 1994, the Master of Business Administration degree (Hons.) from the Universidad Bicentenaria de Aragua, Venezuela, in 1999, and the Ph.D. degree in electrical power engineering from the Universidad Central de Venezuela in 2008....View more

Author image of Francisco Sanchez Gorostiza
Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University, Loughborough, U.K.
Francisco Sanchez Gorostiza (Member, IEEE) received the B.Sc. degree in electrical engineering from Simón Bolívar University, Caracas, Venezuela, in 2011, and the master’s degree in renewable energy technologies from Polytechnic University of Madrid, Spain, in 2013. He is currently pursuing the Ph.D. degree in electrical engineering with Loughborough University, U.K. His research is focused on the development of artificial intelligence techniques for power system analysis and energy management applications.
Francisco Sanchez Gorostiza (Member, IEEE) received the B.Sc. degree in electrical engineering from Simón Bolívar University, Caracas, Venezuela, in 2011, and the master’s degree in renewable energy technologies from Polytechnic University of Madrid, Spain, in 2013. He is currently pursuing the Ph.D. degree in electrical engineering with Loughborough University, U.K. His research is focused on the development of artificial intelligence techniques for power system analysis and energy management applications.View more
Author image of Francisco M. Gonzalez-Longatt
Department of Electrical Engineering, Information Technology and Cybernetics, University of South-Eastern Norway, Porsgrunn, Norway
Francisco M. Gonzalez-Longatt (Senior Member, IEEE) received the Electrical Engineering degree (First Class) from the Instituto Universitario Politécnico de la Fuerza Armada Nacional, Venezuela, in 1994, the Master of Business Administration degree (Hons.) from the Universidad Bicentenaria de Aragua, Venezuela, in 1999, and the Ph.D. degree in electrical power engineering from the Universidad Central de Venezuela in 2008. He is currently a Full Professor of electrical power engineering with the Institutt for elektro, IT og kybernetikk, Universitetet i Sørøst-Norge, Norway. He has prolific research productivity including several industrial research projects and consultancy worldwide. Also, he had authored or editor several books (Spanish and English), an Associated Editor in several journals with an impressive track record on scientific publications. His research interest includes innovative (operation/control) schemes to optimize the performance of future energy systems.
He is the Vice-President of Venezuelan Wind Energy Association, a member of the Institution of Engineering and Technology, U.K., and a member of the International Council on Large Electric Systems—CIGRE. He received the Professional Recognition as a fellow of the Higher Education Academy in January 2014.
Francisco M. Gonzalez-Longatt (Senior Member, IEEE) received the Electrical Engineering degree (First Class) from the Instituto Universitario Politécnico de la Fuerza Armada Nacional, Venezuela, in 1994, the Master of Business Administration degree (Hons.) from the Universidad Bicentenaria de Aragua, Venezuela, in 1999, and the Ph.D. degree in electrical power engineering from the Universidad Central de Venezuela in 2008. He is currently a Full Professor of electrical power engineering with the Institutt for elektro, IT og kybernetikk, Universitetet i Sørøst-Norge, Norway. He has prolific research productivity including several industrial research projects and consultancy worldwide. Also, he had authored or editor several books (Spanish and English), an Associated Editor in several journals with an impressive track record on scientific publications. His research interest includes innovative (operation/control) schemes to optimize the performance of future energy systems.
He is the Vice-President of Venezuelan Wind Energy Association, a member of the Institution of Engineering and Technology, U.K., and a member of the International Council on Large Electric Systems—CIGRE. He received the Professional Recognition as a fellow of the Higher Education Academy in January 2014.View more

References

References is not available for this document.