Hand-Gesture Based Emoji Identification and Chat Integration

Every aspect of human life is slowly getting computerized and gradually turning electronic. This has made a rangeof tasks a lot easier and accessible for people from all walks of life. Unfortunately, this rapid computerization and the ever changing landscape of electronics has given birth to a new classof computer - illiterate individuals. Such individuals exist due to the lack of accessibility to latest electronic gadgets, or humansocial problems like old age. There is a need to make technology fun and easier to use for individuals belonging to such classes. The very primary use of internet-based networking is text-based communications namely: e-mails, texts chats, blogs, etc. The text based interaction is mainly facilitated using a keyboard which makes it the most focal human-computer interaction involved. The modern usage of this text-based communication involves extensive usage of emoticons. While most handheld devices feature a customised emoji-keyboard, the emoji keyboard is difficult to access in personal computers. This can adversely affect the interactions the aforementioned computer-illiterates partake in. This project aims to deal with this problem by creating a hand-detection based emoji detection algorithm in python through OpenCV and Mediapipe for detection of hand landmarks and implementing it in one such communication channel. The algorithm consists of two modules: a Hand-Tracking Moduleand an emoji detector capable of matching the tracked hand to existing emoji database. The resulting application is consistent, allow frequent users to use shortcuts and make the user feel more in control. The work also strives to integrate error prevention and simple error handling in existing emoji detection technologies along with managing short term memory load.