A study on object detection method from manga images using CNN

Japanese comics (manga) are popular content worldwide. In order to acquire metadata from manga images, techniques automatic recognition of manga content have been studied. Recently, Convolutional Neural Network (CNN) has been applied to object detection in manga images. R-CNN and Fast R-CNN generate region proposals by Selective Search. Faster R-CNN generates them using CNN layers called Region Proposal Network (RPN). Single Shot MultiBox Detector (SSD), the latest detection method, performs object classification and box adjustment for small regions in an image. These methods are effective to natural images. However, it is unclear whether such methods work properly to manga images or not, since those image features are different from natural images. In this paper, we examine the effectiveness of manga object detection by comparing Fast R-CNN, Faster R-CNN, and SSD. Here, manga objects are panel layout, speech balloon, character face, and text. Experimental results show that Fast R-CNN is effective for panel layout and speech balloon, whereas Faster R-CNN is effective for character face and text.

I. Introduction

Electronic comic is an important content which accounts for about 80% of sales in e-book market in Japan. Many of existing manga images are obtained by scanning paper mediums. By recognizing content in such manga images, it is possible to obtain useful metadata for services such as searching and image processing. Since handwritten manga objects have big change in shape than general objects, it is necessary to construct a dedicated detection method. Attempts to detect objects from manga images have been studied. Arai et al. proposed panel layout and speech balloon extraction using blob detection [1]. Ho et al. proposed to extract them based on region merging and mathematical morphology [2].