Temporal Fusion Transformer with Non-Intrusive Attention for Data-Driven Electricity Load Forecasting

Accurate and timely load prediction can provide reference for grid operation control, facilitating energy saving, emission reduction and fine management of grid dispatch. In this paper, we propose a data-driven electric load forecasting method that incorporates Transformer and Auto Regression (AR), where the Transformer model is responsible for predicting the nonlinear high-frequency components of load data, while the AR model predicts the linear low-frequency components, thus jointly improving the forecasting results. The encoder in the classical Transformer model is improved by adopting a hierarchical design. Through the way of “layer-by-layer distillation and multi-layer fusion”, the improved encoder can extract the cross-periodic features from the temporal data in a more focused manner. Furthermore, this paper proposes a non-intrusive attention mechanism, which allows the model to incorporate auxiliary information such as weather and holiday in a non-intrusive way. Under the premise of ensuring the relative independence of the primary information representation space corresponding to the load, using this attention mechanism, more reliable temporal dependence can be uncovered with the help of auxiliary information. The experimental results on real transformer bus load data show that the proposed method outperforms the existing methods in both Mean Square Error and Mean Absolute Error, which verifies the effectiveness of the proposed method.

I. Introduction

With the increasing proportion of renewable energy such as wind power, and the access of a large number of power electronic devices, the safe operation and stable control of the power grid have been greatly impacted. Accurate and timely load prediction of power grid bus can provide reference for power grid operation control and help realize the fine management of power grid dispatch. Transformer bus load prediction mainly refers to the bus load prediction in the next day, a week or even a few weeks. By predicting the distribution of power load in the future, which can improve the refinement of dispatching management, the safe and stable operation of power grids can be promoted, thus achieving orderly power consumption and energy saving.