Handwritten English Script Recognition System Using CNN and LSTM

In this digital era, computers and phones are more important than ever, but many people still prefer handwritten documents to digital formats. Sometimes, storing, carrying, and reading through all handwritten documents at once is difficult, so there is a need to convert handwritten documents into digital versions. This paper presents two classification algorithms to convert into digital form: one is classifying words directly using CNN, and the other is character segmentation. The proposed model is based on a convolutional neural network, which was trained using word images from the IAM dataset. The number of layers and dataset size are kept smaller so that training and testing can be done on the CPU instead of the GPU. The proposed model consists of CNN layers for character recognition and LSTM layers for word and syntax correction. The recognition system takes an image as input and displays the digital version of a handwritten document as output after recognition. The CTC loss function and CTC decoder were used after LSTM to align sequences. This paper covers detailed intuition about the architecture and implementation of the proposed system, and its performance was evaluated based on the correct prediction of characters.