Optimization and Deployment of DNNs for RISC-V-based Edge AI

Deploying Deep Neural Networks (DNNs) on edge devices to handle artificial intelligence (AI) tasks is increasingly important, but this is often limited by the computational and energy constraints of typical edge devices. In this context, RISC-V-based internet of things edge computing engine, GAP8, provides a platform for edge AI. The engine features an 8-core cluster and ultra-low power consumption. Most importantly, GAP8 supports the deployment and acceleration of DNNs. However, there are limitations on memory size. In this research, we propose a hardware-adaptive optimization algorithm EN-HSPG to ensure high-speed operation of DNNs in edge AI. Specifically, the algorithm imposes sparsity-induced penalties to the weights and biases in batch normalization layers and selects layer-by-layer sparsity. Based on the optimized DNN, we further explored the performance of GAP8 for DNNs. Experimental results show that after deploying DNNs, the parameter count and floating point operations pruning rate of the model optimized with the EN-HSPG algorithm reach as high as 98.82% and 91.28% respectively, using only 54% of L2 memory. Additionally, the inference time of the EN-HSPG optimized model is just 115.26ms on the GAP8. Thus, the research enables the operation of complex deep learning networks on edge devices with limited computing capabilities.