Video frame rate up conversion using region based motion compensation

Video frame rate conversion is often recognized as video temporal interpolation. In this paper, a region-based approach to automated video frame rate conversion is presented. The region extraction is performed by spatially dividing the reference video frames. Since this step involves image segmentation, the resulting regions are under or over segmented. Consistent frame to frame region extraction is key to region based motion compensation. We propose a novel inter- and intra-frame region merging technique to compensate for inconsistent frame segmentation problem. The video frame rate is then up converted using the motion compensation and region interpolation between every pair of matching regions. The deformation parameters of the match region pair are estimated based on a non-linear estimation model. Heuristic techniques are used to eliminate overlapped and uncovered areas in the generated frames. Experimental results are presented to evaluate its strength over the popular block matching algorithm (BMA).

1. Introduction

The progressive development of telecommunication, computer, and TV/film industries, in the recent years, has resulted in widespread research activities in the field of efficient and effective transmission and coding of video sequences. Video sequences contain a high degree of spatiotemporal redundancy that can be dramatically reduced to decrease the cost of transmission and storage. However, restoring the compact video sequence to its original length is an interesting problem. Video frame rate up conversion or temporal interpolation. referring to reconstructing of the missing or skipped frames that are ignored while the video sequence is being transmitted or captured, is the concern of this paper.