Identifying universal safety signs using computer vision for an assistive feedback mobile application

When walking on the road or navigating unfamiliar areas, the elderly, visually handicapped, and persons with hearing loss encounter challenges. The information on numerous safety signs such as road signs, traffic signs, workplace safety signs or industrial hazard signs is usually unhelpful to them. This paper proposes a solution that can detect a comprehensive set of safety signs in real-time. Our app runs a deep learning model pre-trained on a custom-built dataset. The deep learning model we have used is explicitly built for object detection to find regions of interest, create appropriate bounding boxes, and classify the signs with three different levels of severity - Danger, Caution, and Prohibitory. The camera-equipped smartphone relays haptic or audio feedback upon the successful detection of a safety sign.