An Analytical Approach for Enhancing the Automatic Detection and Recognition of Skewed Bangla License Plates

Although there has been a huge body of work on Bangla license plate detection and recognition, the successes of these works have largely been limited to correct detection and recognition of undistorted license plates whose images are taken chiefly from the front or the back of vehicles with slight angular variations. As a result, most Bangla automatic license plate recognition (ALPR) systems in practice struggle when the license plates are skewed on the viewing or the image planes of the license plates. In this paper, we address this issue by proposing an analytical approach that can enhance the ALPR of both normal and skewed license plates and can be incorporated into existing Bangla ALPR systems without modifying their internal structures. Specifically, we demonstrate how existing ALPR systems can be treated as black boxes and analyzed to understand what sort of license plate images they work best on and introduce a novel pipeline that combines deep learning and an algorithmic procedure for transforming images of both normal and skewed license plates into formats that are best suited for the ALPR systems. We note that our proposed method can be easily generalized and applied to non-Bangla license plates as well.

I. Introduction

Correct recognition of vehicle license plates has numerous use cases that include penalizing irresponsible driving and parking, keeping track of vehicles coming in and going out of parking lots, identifying vehicle ownership and so on. While a lot of work on automatic detection and recognition of license plates has been done for decades, there is still a lot of progress to be made on correct Bangla automatic license plate recognition (ALPR). The variance that exists in Bangla license plates owing to existence of different metro and vehicle types, and the scarcity of adequate samples to catch all of that variety makes Bangla ALPR specially challenging. A Bangladeshi company named HeadBlocks is working with the Dhaka Metropolitan Police (DMP) to address this and has developed a commercial Bangla ALPR system with a four stage pipeline that includes license plate detection, character segmentation, character detection, and character recognition. Although the ALPR system developed by HeadBlocks (referred to as HB- ALPR henceforth) has over 96% accuracy as determined by experiments on their confidential test set, HB-ALPR finds it difficult to correctly identify license plates that are more than 30° skewed on the image plane (ip-skewed) or the viewing plane (vp-skewed). Examples of such ip-skewed and vp-skewed license plates are shown in Fig. 1.