DIFA-GAN: Differential Attention-Guided Generative Adversarial Network for Unsupervised Video Forecasting

Unsupervised video forecasting, which aims to generate future frames by observing only the beginning few frames, has promising applications in autonomous driving, robot navigation, and video surveillance systems. However, realistic video forecasting is very challenging due to both complex content and diverse dynamic motions of video. We propose a Differential Attention-Guided Generative Adversarial Network (DIFA-GAN) for unsupervised video forecasting, where an attention mechanism is incorporated with a frame differential algorithm to exploit both spatial and temporal information. The model learns to focus selectively on parts related to motions and to offer different weights to them. Extensive experiments on two advanced datasets — KTH and KITTI show the advancement of our method.