The Automation of Quality Control for Large Irradiance Datasets

The automation of quality control for large irradi-ance datasets is presented in this article. Data-driven techniques in photovoltaic systems have increased significantly and the need for sufficient and time-efficient quality control procedures has increased. The methodology usually includes a process in which the user has to manually review the erroneous data that has been flagged by the quality control procedure. As datasets increase, the time to review these flags increases substantially. A three-step quality control procedure is presented that includes an automatic elimination process of erroneous irradiance measurements and a novel empirical formulation for automatically identifying mea-surement tracking errors. The tracking error was introduced as other formulations were not adequate in removing these data points automatically. The first step is a time-series visualisation, the second step is the automatic removal of missing and du-plicate values, and the third step is an automatic flagging and elimination step to identify erroneous data. Two South African case studies were used to illustrate the practical application of the automatic quality control procedure. The automatic quality control procedure successfully removes the night-time data points as well as the tracking errors. An initial flagging procedure for reviewing data was also included as part of the quality control procedure; however, these data points were found to consist of closure test flags. Upon further investigation, the flagged data did not indicate that the measurements were faulty, and therefore the automated elimination process shows adequate performance in removing data without this manual review process. The automatic process proves to be a time-efficient method to remove erroneous data and is therefore recommended as a minimum quality control procedure for large irradiance datasets.