Enhancing FPGA-Based YOLO Object Detection: Multi-Bank Storage Optimization and Model Refinement for Real-Time Applications

Object detection algorithms are widely used in image classification and detection tasks recently. Among the other algorithms, You Only Look Once (YOLO) performs better in real-time detection. Convolutional neural networks (CNNs) in the algorithm require a large amount of computation and memory access. However, researchers tend to access parameters through the external memory in accelerator design, which leads to slow processing speed and high power consumption. Therefore, we propose a Multi-Bank storage structure that flexibly combines URAM and BRAM in storing feature maps, which significantly improves the throughput. We also improve the YOLOv3-Tiny model by distillation, quantization and pruning and make it more suitable for deployment in FPGAs. These methods can significantly reduce the number of parameters in the model while ensuring almost no decrease in detection accuracy. Finally, tested on VOC 2007, our model achieves a frame rate of 67.6 FPS with a reduction in mAP of 0.38%.