Discovering Bengali Consonant-Linked Vowels in Facial Images using Neural Architecture Search

Visual Speech Recognition (VSR) plays a crucial role in cognitive functions, particularly for individuals with hearing impairment. This research delves into the realm of Bengali VSR, addressing the scarcity of works in this domain. Our focus is on consonant-associated vowel identification. Leveraging the efficiency of Neural Architecture Search (NAS), we aim to discover the most effective Neural Network (NN) model for enhancing consonant-associated vowel recognition rates. Face images emerge as superior features compared to lip images, and our work builds upon this insight. We created a dataset with 2030 consonant and 345 vowel-pronouncing images from four speakers. For benchmarking, we applied 17 different state-of-the-art (SOAT) models. The results show that our proposed methods achieved maximum 70% accuracy, surpassing all the benchmarks. Notable contributions include: the introduction of consonant-associated vowel identification in Bangla and the application of NAS to identify optimal NN architectures. This research strives to fill critical gaps in Bangla VSR, ultimately contributing to improved recognition rates and fostering inclusivity in communication technologies.