Recognizing A Complex Human Behaviour with Zero Video Training Sample Base on Human Montion Data Encoding

Complex human behaviour (CHB) recognition is more meaningful, but also more challenging than human action recognition (HAR). Understanding the semantic information included in CHB video is helpful to develop advanced human-machine interaction applications. However, utilizing the existing action recognition models can hardly gain satisfactory accuracy because of the CHBs’ long duration and unenumerable characteristics. In this paper, we break through this bottleneck from the semantic perspective via motion data encoding and semantic similarity judgment. The contributions of our method are mainly reflected in three aspects. First, a CHB is innovatively defined as the combination of the human basic actions which are composed in time order. Thus, the unenumerable CHBs can be represented by limited human actions. Second, to automatically construct the action combination of a CHB, a skeleton-based multi-scale CNN model and an efficient one-stage temporal action localization model are put forward to exactly extract action and its duration from a CHB video. Third, the word-vector in the nature language processing field is creatively imported into our method. Therefore, the action words can be encoded by a set of float-point numbers. Moreover, the word-vector is helpful to judge the similarities of different action combinations. The proposed method is evaluated on a self-collected dataset. Through extensive experiments, the results prove the method’s feasibility and universality. Moreover, its use of only human skeleton motion data provides privacy assurances to users.