Multi-Agent Reinforcement Learning-Based Maximum Power Point Tracking Approach to Fortify PMSG-Based WECSs

Traditional maximum power point tracking (MPPT) schemes for wind energy conversion system (WECS) suffer from slow tracking speed, poor accuracy, and hypersensitivity to alterations in wind speed. In order to address the shortcomings of the permanent magnet synchronous generator (PMSG) based WECS, a multi-agent reinforcement learning (MARL) technique is developed in this paper. The MARL wind MPPT methodology has a multitude of advantages over traditional methods. A multiple agents would collaborate together to optimize power generation owing to the controller's decentralized capability, rather than only one agent. This concept ensures superior scalability, improved communication bandwidth, and more accurate operation. The capacity to rapidly respond to changes in wind speed is another advantage of the MARL method, which raises energy output and sustainability. In the pursuit of choosing the optimum discount factor (DF), which is a crucial hyperparameter in RL process, the proposed MARL algorithm is further improved through using meta-learnt DF within the strategy. The wind MPPT system's efficiency is boosted by the meta-learnt DF, which also accelerates convergence rate, and shortens overall training time. Moreover, it diminishes the sensitivity to hyperparameter, resulting in a more stable solution, featuring improved long-term performance and energy yield. In order to demonstrate the proposed concept's potential relevance to real-world wind MPPT issues, a thorough performance evaluation of the proposed model-free MARL technique is offered through a number of simulation results.

I. Introduction

The urge for effective renewable energy alternatives has intensified due to the rising costs and negative consequences of fossil fuels on the environment. In the past decades, the incorporation of wind generators in contemporary power systems has increased significantly. Wind energy can be a magnificent source of environmentally friendly and dependable energy. Through the end of 2022, 840 GW of wind energy is anticipated to be produced worldwide [1], [2]. Due to the fundamentally variable nature of wind energy, tracking the maximum power point (MPP) to harvest foremost energy at constantly varying wind speeds is of considerable importance. Approaches for maximum power point tracking (MPPT) provide the optimum power extraction from wind energy conversion systems (WECS). Direct power control (DPC), indirect power control (IPC), smart or artificial intelligent (AI) based, and hybrid algorithms which combine conventional and intelligent techniques are the four primary categories of MPPT methodologies for WECS. The categorization of several MPPT techniques for wind energy appears in Fig. 1. Tip Speed Ratio (TSR) serves as the most widely used IPC-based traditional MPPT method. TSR is straightforward to navigate and provides swift responses when modifying the rotor speed in various environmental circumstances. However, the disadvantages include high maintenance costs, poor efficiency, and a shortage of reliability [3]. The optimum torque (OT) approach, on the other hand, controls the generator torque through an ideal torque curve for various wind speeds. Even though the method has higher degree of control complexity, but provides higher performance efficiency than TSR in ideal circumstances, it is highly sensitive to the weather and the properties of the wind turbines (WTs) [4], [5]. Any discrepancy between the genuine climate and WT specifications and the optimal torque curve may result in severe performance deficits.