A Real-Time 1280 × 720 Object Detection Chip With 585 MB/s Memory Traffic

Memory bandwidth has become the real-time bottleneck of current deep learning accelerators (DLAs), particularly for high definition (HD) object detection. Under resource constraints, this article proposes a low memory traffic DLA chip with joint hardware and software optimization. To maximize hardware utilization under memory bandwidth, we morph and fuse the object detection model into a group fusion-ready model to reduce intermediate data access. This reduces the YOLOv2’s feature memory traffic from 2.9 to 0.15 GB/s. To support group fusion, our previous DLA-based hardware employees a unified buffer with write-masking for simple layer-by-layer processing in a fusion group. When compared to our previous DLA with the same processing element (PE) numbers, the chip implemented in a 40-nm process supports
$1280\times 720$
at 30 frames per second (FPS) object detection and consumes
$7.9\times $
less external dynamic random access memory (DRAM) access energy, from 2607 to 327.6 mJ.