Fully-Fusible Convolutional Neural Networks for End-to-End Fused Architecture with FPGA Implementation

The inference of artificial intelligence algorithms, and especially Convolutional Neural Networks (CNNs) in edge devices has gained attention in recent years. However, low on-chip memory and restricted off-chip memory bandwidth have hindered its advancement. The conventional bottom-up and top-down approaches in design, which start from either a network structure or a hardware architecture and go to the other one, have not been adequate to rectify memory barriers. In this paper, with a network-hardware co-design approach, we revisit the architecture and network several times, looking for opportunities to shrink the hardware of CNN while achieving acceptable accuracy. In this line, we introduce Fully-Fusible CNNs (FFCNNs) that can be implemented with the layer fusion technique to build fully-fused architectures that omit memory components between layers. We showcase fully-fused architectures by implementing an end-to-end (including the fully-connected layer) fused CNN on FPGA. We also design an FFCNN for CIFAR-10 as a proof of concept for the suitability of our approach for edge devices. The FFCNN achieved 78.79% with only 95522 trainable parameters and 97660 FLOPs. The results show a promising opportunity for efficient edge implementation of CNNs using the techniques in this paper.