A Multi-mode Convolution Coprocessor Based on RISC-V Instruction Set Architecture

Different modes of convolution, as the main operations of convolutional neural networks, determine the efficiency of the entire network. This paper proposes a multi-mode convolution coprocessor based on the RISC-V instruction set in order to accelerate convolution operation. In the coprocessor, a round-based convolution unit and dataflows for different convolution modes are designed to reduce memory access frequency. A custom instruction subset based on RISC-V ISA is designed, and the common convolution algorithm is implemented on the coprocessor. Finally, performance analysis and resource consumption evaluation of the coprocessor are completed on a Xilinx FPGA. Implementing different modes of convolution based on the designed instruction subset takes fewer cycles than using the standard instruction set. And the acceleration ratio of pointwise convolution is 8.74 times that of the standard instruction set.