Hand Shape Recognition Using Fingertips

Recently the importance of studies to recognize the shape of human hands in realtime is being more stressed to implement the user-friendly user interface. However, human hands have a large degree of freedom and thus it is very difficult to accurately recognize the hand shape, especially in a complicated background of the color same as the skin color. In this study, we detect the fingertip, without any special sensors, by using the hand shape based on the skin color information detected by a camera. This paper describes an algorithm to show directions using the detected fingertip. This algorithm is used to detect the fingertip through template matching and then detect forward, backward, left and right directions based on the detected fingertip and wrist center information. We applied the algorithm to navigation in 3D virtual-reality space, and achieved a very good result in an experiment on the change of directions to forward, backward, left and right. It is expected that this algorithm can provide more natural interaction between man and computer by allowing the user to instruct change of directions intuitively without the aid of a mouse, a keyboard or a joystick.