Towards Efficient Convolutional Neural Network for Domain-Specific Applications on FPGA

FPGA becomes a popular technology for implementing Convolutional Neural Network (CNN) in recent years. Most CNN applications on FPGA are domain-specific, e.g., detecting objects from specific categories, in which commonly-used CNN models pre-trained on general datasets may not be efficient enough. This paper presents TuRF, an end-to-end CNN acceleration framework to efficiently deploy domain-specific applications on FPGA by transfer learning that adapts pre-trained models to specific domains, replacing standard convolution layers with efficient convolution blocks, and applying layer fusion to enhance hardware design performance. We evaluate TuRF by deploying a pre-trained VGG-16 model for a domain-specific image recognition task onto a Stratix V FPGA. Results show that designs generated by TuRF achieve better performance than prior methods for the original VGG-16 and ResNet-50 models, while for the optimised VGG-16 model TuRF designs are more accurate and easier to process. and layer fusion to enhance hardware design performance. We evaluate TuRF by end-to-end deploying a pre-trained VGG-16 model for a domain-specific image recognition task onto Stratix V FPGA. Results show that designs generated by our framework achieve better performance than prior works, regarding the original VGG-16 and ResNet-50, and the optimised VGG-16 model is more accurate and easier to process.

I. Introduction

There has been much recent work on developing FPGA implementations of Convolutional Neural Networks (CNNs). While significant progress has been made in optimising the inference process of general CNN models on FPGAs, training and optimising CNNs for various domain-specific applications remain a demanding task. CNN models for domain-specific applications only need to detect or classify objects from a narrow range of classes. Recent discovery in transfer learning [1] - a research topic focusing on exploiting features reusable from one task to another - shows that CNN models that are pre-trained on general datasets can be efficiently fine-tuned [2] for specific domains. This approach works well for medical image analysis: a pre-trained CNN with adequate fine-tuning can outperform or perform as well as training from scratch [3].