Enhancing the Understanding of Distance Related Uncertainties of Vocal Navigational Commands Using Fusion of Hand Gesture Information

Itis a prevalent trend nowadays in most places that the elderly population keeps increasing. Also, there is a community that has physical disabilities, especially mobility. Due to the lack of trustworthy caretakers, the busy lives of family members, and psychological issues attributed to loneliness, intelligent service robotic devices have been developed. Vocal navigational commands consist of uncertain terms. There are methods introduced to enhance these uncertainties using spatial parameters such as obstacle distances, previous robot movements, robot localization, etc. However, they have only considered the vocal information provided by the user and deduced the rest of the information from external spatial information. In reality, users include other partial information from complementary modalities such as hand gestures and there is a significant probability that the partial information carried by the hand gestures might change the interpretation completely. Therefore, this paper presents an intelligent system that would enhance the understanding of distance-related uncertainties of vocal navigational commands using multimodal fusion of hand gesture information. Partial gesture information extracted from hand gestures has been used to interpret the distance-related uncertainties using a fuzzy logic-based approach. Experiments were conducted to validate the intelligent system and the user ratings given were used to validate the system.