An Offline Assistance Tool for Visually Impaired People Based on Image Captioning

Eye defects of visually impaired people bring great inconvenience to daily lives. With the rapid development of computers and artificial intelligence, some assistance tools and wearable devices have provided convenience to a certain extent, but they still possess many shortcomings, such as high prices, limited functionality, poor real-time performance, the inability to be used offline and so on. This paper designs an offline assistance tool specifically for visually impaired people that can help them perceive their surrounding environments by taking photos and reading the descriptions output by an image captioning model. Considering that the photos taken by visually impaired people may be incomplete, an improved image stitching method based on a genetic algorithm is proposed to obtain high-quality stitched images as the inputs of the image captioning model. An improved model pruning algorithm is also designed to compress and accelerate the image captioning model to meet the offline and real-time deployment requirements of portable devices. Experimental results and a practical tool show that the developed system based on an image captioning model and pruning can quickly and accurately describe environments, thus assisting visually impaired people anytime and anywhere.