RecVAE-GBRT: Memory-Fused XGBoost for Time-Series Forecasting

Time series forecasting is a crucial task for control and decision in various fields. Recent efforts focus on integrating complex deep learning techniques, such as RNN or Transformer, into sequential models. However, these solutions are often criticized due to their excessive complexity. Inspired by the effectiveness of Gradient Boosted Regression Trees (GBRT) methods (such as XGBoost) on tabular datasets, this study proposes a hybrid method for time series forecasting. In this method, we design a memory mechanism for GBRT, namely, Recursive Variational AutoEncoder (RecVAE), which can generate compressed representations of historical sequences by recursively summarizing a section of input time series and preceding internal outputs into current internal outputs. This compensates for the limitation of the GBRT in incorporating long historical sequences for time series forecasting. The resulting memory-fused forecasting model, namely, RecVAE-GBRT, is tested on 4 real-world time series datasets. The results indicate that it generates competitive results compared to Transformer-based time series forecasting methods, all happening at the same level of computation efficiency or better.