High Throughput Hardware/Software Heterogeneous System for RRPN-Based Scene Text Detection

Rotation Region Proposal Networks (RRPN) are used to generate rotated proposals with the information of text angle for arbitrary oriented scene text detection (STD). However, the computational complexity of RRPN inference is relatively high compared with other methods, which makes it difficult for massive deployment. In this paper, the first full-stack FPGA-CPU heterogeneous system design of RRPN-based STD algorithm is proposed. A hardware/software partition method is presented to analyze and split the tasks to enhance the computation efficiency of hardware. The fast 2D Winograd algorithm and block floating point are utilized to reduce computation complexity while maintaining a relatively high precision. The implementation results show that the peak performance of MAC arrays in the proposed architecture reaches 655.4 GOPS and the energy efficiency achieves 64.9 GOPS/W. By fully exploiting the parallel and pipelined merits in the algorithms, the first hardware architectures for skew non-maximum suppression (S-NMS) layer and rotation region-of-interest (RRoI) polling layer are proposed. The throughput of the proposed hardware/software heterogeneous system achieves 40 times and 1.4 times improvements compared with CPU and GPU, respectively. Moreover, the comprehensive operating expense ratio of pure CPU, GPU, and the proposed system is 80.7:2.5:1, which indicates that it is suitable for massive deployment.

1 Introduction

Scene text detection (STD) and recognition from natural scene images are important research topics in computer vision [1]. Current text detection and recognition techniques have been deeply applied in many industries such as finance, insurance, medical care, transportation, education, etc. The scenarios involving pictures or videos include e-commerce text translation, user-made content review, content/advertising recommendation distribution, and so on. While these business scenarios need to process tens of billions of data every day, the number of requests for these algorithms is still increasing significantly.