Extending Data Flow Architectures for Convolutional Neural Networks to Multiple FPGAs

Convolutional Neural Networks (CNNs) are the state-of-the-art Artificial Intelligence frameworks that are used for a variety of applications including autonomous driving, surveillance and object detection. Since CNNs are very compute intensive, Field-Programmable Gate Array (FPGA)-based accelerators such as HPIPE have been used to speed up CNN computations. Although such accelerators show significant improvements in CNN performance, large CNNs such as ResNet152 still pose a challenge due to their high compute and memory requirements. This article discusses how we augment and extend HPIPE, the state-of-the-art CNN inference accelerator for FPGAs. We propose an automated CAD flow to partition large CNNs across multiple FPGAs that communicate via 100 Gb Ethernet. We utilize this CAD flow to estimate the performance improvements when scaling HPIPE to multiple FPGAs on six different CNNs. Moreover, we demonstrate through a prototype system that doubling the number of FPGAs results in 2× performance improvement on three CNNs: MobileNet-V1, MobileNet-V2, and ResNet-50.