A Novel Clustering Index to Find Optimal Clusters Size With Application to Segmentation of Energy Consumers

Increased deployment of residential smart meters has made it possible to record energy consumption data on short intervals. These data, if used efficiently, carry valuable information for managing power demand and increasing energy consumption efficiency. An efficient way to analyze these data is to first identify clusters of energy consumers, and then focus on analyzing these clusters. However deciding on the optimal number of clusters is a challenging task. In this article, we propose a clustering index that effectively finds the optimal number of clusters. The proposed index is an entropy-based measure that is obtained from eigenvalue analysis of the correlation matrix of time series of consumption data. A genetic algorithm based feature selection is used to reduce the number of features, which are then fed into clustering algorithms. We apply the proposed clustering index on two ground truth synthetic data sets and two real world energy consumption data set. The numerical simulations reveal the effectiveness of the proposed method and its superiority to a number of existing clustering indices.