Text Spotting of Electrical Equipment Nameplate Based on CTPN and Transformer

Given the complex image background, non-uniform text format, and diverse character types of electrical equipment nameplates, we propose a text spotting method for electrical equipment nameplates based on CTPN and Transformer. First, to overcome the disadvantage that the original CTPN cannot detect the tilted text lines, we add a nameplate localization and correction network based Xception to locate the nameplate region and correct it before text detection. Second, we use ResNet50 to replace the feature extraction network of CTPN, adjust the width of the text proposed region, and add a L2 regularization term during the training process to better adapt to the text detection of electrical equipment nameplate. Third, a text recognition network based on Transformer using ResNet50 to form an encoder and Transformer as a decoder is employed to improve the recognition accuracy of multiple kinds of characters. Evaluated on a self-built nameplate dataset consisting of 550 equipment nameplate images and a public nameplate dataset consisting of 3,949 text instance images, the F1 score of the nameplate text detection model is 88.7
^%
, and the character recognition accuracy and text line recognition accuracy of the nameplate text recognition model are 92.3% and 85.6%, respectively. Experiments demonstrate that the proposed approach effectively reduces the effects caused by complex backgrounds and skewed captured images, and is better adapt to the text spotting of electrical equipment nameplate.