Real-Time Head and Hand Tracking Based on 2.5D Data

A novel real-time algorithm for head and hand tracking is proposed in this paper. This approach is based on data from a range camera, which is exploited to resolve ambiguities and overlaps. The position of the head is estimated with a depth-based template matching, its robustness being reinforced with an adaptive search zone. Hands are detected in a bounding box attached to the head estimate, so that the user may move freely in the scene. A simple method to decide whether the hands are open or closed is also included in the proposal. Experimental results show high robustness against partial occlusions and fast movements. Accurate hand trajectories may be extracted from the estimated hand positions, and may be used for interactive applications as well as for gesture classification purposes.

I. Introduction

Gesture recognition technologies are being widely applied to countless applications. One may notice a global tendency to replace traditional control devices with vision-based Natural Human Interaction (NHI) solutions. In a first step towards this goal, tactile interfaces are already in the market and allow the suppression of traditional input devices such as keypads and mouse. Indeed, the mid-term objective is to obtain marker-less, gesture recognition systems which allow users to interact as naturally as possible, providing a truly immersive experience.