A 593nJ/Inference DVS Hand Gesture Recognition Processor Embedded With Reconfigurable Multiple Constant Multiplication Technique

Hand gesture recognition (HGR) is a popular technique for edge-based human-computer interaction. Dynamic vision sensors (DVS) are often used in HGR systems due to their low latency, high dynamic range, low energy consumption, and asynchronous event triggering. While Spiking Neural Networks (SNNs) are commonly thought to consume less energy than Convolutional Neural Networks (CNNs) in DVS-based HGR systems, this work demonstrates that a DVS-based HGR system on chip (SoC) incorporating CNN can achieve lower power consumption through algorithm and hardware co-design. The proposed edge-side processor for DVS-based HGR integrates a median filter processing core and an AI accelerator core for preprocessing and CNN inference on the DVS output data. To reduce hardware costs without sacrificing accuracy, the median filtering core uses a simplified median filtering function tailored to the specific application scenario. The paper suggests using reconfigurable multiple constant multiplication (RMCM) techniques for the AI accelerator core to share computational resources among processing element (PE) arrays, thereby reducing computational costs and power consumption. The entire DVS gesture processor was implemented in a 65nm technology, achieving an energy requirement of 593.4nJ per inference on-chip with a guaranteed accuracy of 92.4%.