A Near-Eye DVS-Based End-to-End Eye Tracking Processor for AR/VR Applications

This paper presents a near-eye DVS-based end-to-end eye tracking processor for augmented reality/virtual reality (AR/VR) devices, addressing the need for low-power, high frame rate and accurate eye tracking deployment on wearable devices with extended battery life. The processor features a dedicated hardware implementation for energy-efficient event-driven pre-processing and acceleration, and delivers accurate gaze estimation utilizing a customized neural network (NN) with constant-event-count sample partitioning method. It is capable of performing end-to-end eye tracking exclusively with the Dynamic Vision Sensor (DVS), achieving an average accuracy of 0.91° in a 96°×64° FoV. The processor is implemented and simulated using UMC 40-nm LP CMOS technology, with a core area of 1.88 mm
²
. Operating at a clock frequency of 200 MHz, it achieves a dynamic frame rate of up to 1250 Hz with a power dissipation of 8.09 mW, corresponding to a 20.82 µJ energy per gaze estimation. With the integration of DVS, this processor enables real-time, low-power and accurate eye tracking for an immersive interaction experience on AR/VR devices.