Text localization, extraction and inpainting in color images

In this paper, we propose a new algorithm for subtitle text detection, extraction and text inpainting in color images. The proposed algorithm includes three stages. In the first stage, we localize text blocks to extract subtitles. We then extract the text characters precisely and in the third stage, we use an inpainting algorithm to recover the original texture of the image. To localize text blocks, we use stroke filter and new segmentation and verification algorithms based on the image profiles. To extract text character precisely, we estimate background and text color in the candidate blocks using the color histogram measured in different sub-blocks of the candidate text blocks. Finally, the inpainting algorithm based on matching algorithm is utilized to reconstruct the initial image contents in text areas. Our inpainting algorithm considers priority for inpainting of pixels, which results in perfect reconstruction in areas with texture variation. We tested the proposed algorithm with different image and video frames and compared the results with those of other methods. Experimental results showed the efficiency of the proposed algorithm.