Sign language image dataset with a hand pose type attribute

We present hand image datasets for training convolutional neural networks (CNNs) for sign language recognition and translation. Technologies to translate sign language into another language will facilitate the communication between deaf and hearing people. Recently, deep learning has been considered to be promising for analyzing sign language videos and estimating a signer's message. If a CNN is trained to discriminate hand poses, it would work to estimate sign language. For the training of CNN, it is required to have hand pose dataset. We constructed hand pose datasets by automatically classifying images based on two types of features: 1) the shape of five fingers as bent/stretched and aligned/separated, and 2) palm orientation and rotation. We devised a method to calculate feature parameters for the finger shape and palm orientation and rotation. Hand images were clustered using these feature parameters. We classified more than 200,000 images from sign language videos showing nine different signers expressing more than 1000 different signs. A CNN trained on our dataset successfully discriminated hand poses. The trained CNN further worked to estimate sign language messages. The results show that, compared with a baseline network, the estimation performance was improved.