Adversarial Attention Networks for Early Action Recognition

Early action recognition endeavors to deduce the ongoing action by observing partial video, presenting a formidable challenge due to limited information available in the initial stages. To tackle this challenge, we introduce an innovative adversarial attention network based on generative adversarial networks. This network leverages the characteristics of both the generator and discriminator to generate unobserved action information from partial video input. The proposed method comprises a cross attention generator, self Attention discriminator, and feature fusion module. The cross attention generator captures temporal relationships in input action sequences, generating discriminative unobserved action information. The self attention discriminator adds global attention to the input sequence, capturing global context information for accurate evaluation of consistency in generated unobserved feature from cross attention generator. Finally, the feature fusion module helps the model obtain richer and more comprehensive feature representations. The proposed method is evaluated through experiments on the HMDB51, UCF101 and Something-Something v2 datasets. Experimental results demonstrate that the proposed approach outperforms existing methods across different observation ratios. Detailed ablation studies confirm the effectiveness of each component in the proposed method.