Prediction of Household Energy Usage via Multi-Head Attention

The analysis of spatial and temporal information has been a task commonly accomplished through traditional statistics and signal processing, however with the information typically possessing nonlinear characteristics, the usage of machine learning based approaches has increased due to their ability to analysis data with high nonlinearity. Long Short Term Memory (LSTM), a type of Recurrent Neural Networks (RNNs), have shown to be favored due to their internal memory allowing the network to process data in a sequence of steps due to its feedback connections propagating predicted information back multiple times. We propose a neural network algorithm utilizing a modified Convolutional Neural Network (CNN) with incorporated LSTM layers and multi-head attention to predict spatio-temporal data with higher metric performance compared to current state of the art networks, when each network is trained and tested on energy data of a single household.