I. Introduction
The paradigm for surveillance video summarization changed after the concept of video synopsis [1]–[3] was introduced. Earlier approaches before the video synopsis include fast forward [4] and video skimming [5], which suffered from several problems when time duration of the original video was long; for example, low frame condensation ratio (FR) or missing information. Video synopsis resolves the problems by extracting moving objects, called object tubes, from the scene and rearranging them in the temporal domain to make a short and condensed video.