ST2T: A Spatio-Temporal Transformer for Cellular Traffic Prediction in Digital Twin Systems

Cellular traffic prediction is vital in network optimization, traffic management, and anomaly detection within Digital Twin (DT) systems. However, the task poses significant challenges due to the complexity of spatio-temporal data and its high dynamic range, especially when integrated within a DT framework. Traditional methodologies often interpret cellular traffic data as images, utilizing Convolutional Neural Networks (CNNs) or Transformers. Still, such techniques can be compromised by weak semantic information and limited spatial correlation in the data. This paper introduces a Spatio-temporal Transformer for Digital Twins (ST2T), a groundbreaking model designed to tackle these challenges within the DT context. The ST2T combines the strengths of convolutional and self-attention mechanisms in a unique hybrid encoder and employs a two-stage decoder specifically devised to handle the high dynamic range of spatio-temporal data. We also utilize tubelet patching for an intuitive capture of adjacent space-time relationships. Our experimental results show that the ST2T model substantially advances the state-of-the-art in cellular traffic prediction within DT systems. It provides a promising avenue towards more accurate and efficient cellular traffic prediction, showcasing the potential of Vision Transformers for handling complex spatio-temporal data in Digital Twins systems.