Learning Dual-Routing Capsule Graph Neural Network for Few-Shot Video Classification

Few-shot video classification (video FSL), which learns classifiers for novel concepts, has gained increasing attention in the last few years from only a few samples. The existing methods rarely consider the local-global relation for video feature learning, which would ultimately result in low discriminative ability. Recently, the capsule network (CapsNet) has shown considerable potential in local-global relation learning in the image analysis field. However, CapsNet cannot be directly applied in video FSL since it ignores the interaction between videos and has high computational complexity. In this paper, a dual-routing capsule graph neural network (DR-CapsGNN) is proposed to solve the above issues. The DR-CapsGNN leverages CapsNet and a graph neural network (GNN) to explore local-global relations and to preserve the detailed properties. Specifically, the CapsGNN is used to learn video relations and structural information to generate high-quality hierarchical capsules. Furthermore, a novel dual-routing mechanism is designed to filter low-discriminative capsules from a holistic perspective and achieves high efficiency, which consists of inter-video and intra-video routing. Extensive experimental results demonstrate that our proposed approach performs favorably compared to state-of-the-art methods on two popular benchmarks.