NimbleAI: 3D-Integrated Neuromorphic Vision Sensor-Processor to go where our Eyes can’t

AI paves the way for intelligent space agents that make autonomous decisions without the need for Earth-based interaction. Unfortunately, mainstream AI frameworks and algorithms as well as the COTS processors and AI accelerators on which these run are driven by the consumer market, and hence prioritize productivity over efficiency. The NimbleAI Horizon Europe project is aimed at redefining the basis of AI-based visual sensing and processing to boost efficiency by leveraging key biological principles in eyes and brains. The project will deliver novel neuromorphic hardware designs and IP to sustain event-driven vision with the following target KPIs: (1) 100x performance per mW gains compared to COTS processors (i.e., CPU/GPUs processing frame-based video); (2) 50x processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm
²
. This paper seeks liaison of the project with the space community and stakeholders to get feedback early in the design cycle, and ultimately increase adoption opportunities of NimbleAI technology in next-generation vision payloads.