Kinect depth stream pre-processing for hand gesture recognition

Over the recent years there has been growing interest to propose a robust and efficient hand gesture recognition (HGR) system, using real-time depth sensors like Microsoft Kinect. The performance of such HGR systems have been affected by the low resolution, noise and quantization error in the depth stream. In this paper, we propose a method to pre-process Kinect depth stream in order to overcome some of these limitations. The design approach utilizes the hand tracker from OpenNI SDK to perform distance invariant segmentation of hand region depth stream. This is followed by the construction of three different projections of hand in XY, ZX and ZY planes. These projections are then further enhanced using a combination of morphological closing and simple averaging based interpolation. The evaluation results show above 80% similarity with ground truth, and 1.45-5.35% increase in accuracy for gestures with recognition accuracy less than 90%.

1. INTRODUCTION

With the recent introduction of Microsoft Kinect, there has been increasing interest in research community to use this inexpensive and powerful sensor in almost every domain in the field of Computer Vision. The main attraction of Kinect sensor in this field is the realtime provision of depth data. Combining this with the functionality of different Computer Vision libraries, a whole new category of applications has started to emerge [1]–[3].