HeteroEML: Heterogeneous Design Methodology of Edge Machine Learning on CPU+FPGA Platform

The diverse applications with a wide variety of machine learning (ML) models have made fast design and deployment of ML computing systems an imperative task. The integration of CPU and FPGA have become a suitable ML computing platform to concurrently support programmability on CPU as well as high performance processing on the logic of FPGA. However, deploying ML models on CPU+FPGA platforms is challenging due to increasing model complexity and the need for cross-layer optimization. This paper proposes HeteroML, a heterogeneous design methodology of edge ML on CPU+FPGA platforms. We developed a customized end-to-end compilation process of ML models. The proposed methodology is based on TVM compilation framework, and enables seamless SW/HW integration and fast and effective optimization flow. When compared to conventional CPU-based edge systems, the proposed design can attain 13.78x and 6.47x performance enhancement on VGG and YOLOv2 respectively.