Versatile Model for Activity Recognition: Sequencelet Corpus Model

In this paper, we propose a Sequencelet Corpus Model (SCM) that is quite versatile for human activity recognition in video. A 'sequencelet', a sequence of actional words, is automatically learned using a data mining approach without any prior knowledge about activity structure and describes a representative partial structure of an activity. We model an activity as a combination of sequencelets in the 'sequencelet corpus' that is a set of learned sequencelets. The SCM does not depend on the type of low-level feature as well as can perform various activity recognition tasks. We evaluate our model on the UT-Interaction dataset and the ActivityNet dataset for activity classification from both fully- and partially-observed video, untrimmed video classification and measuring semantic similarity between activities. The SCM achieved state-of-theart results in activity classification even though the activity is partially observed and promising results in untrimmed video classification, Moreover, by constructing the hierarchy of activities from the similarity computed using the SCM, we demonstrate that the SCM can be used a metric for measuring semantic similarity between activities; these results indicate that the SCM has potential to be an all-around activityrecognition model.