Sensor-Health Aware Resilient Fusion With Application to Multi-Vehicle Tracking Using Infrastructure Sensors and Edge Compute

Multi-agent tracking of multiple moving objects is a well-researched problem in support of many applications. The core sensor fusion algorithms evolve from the Kalman Filter, with different variants on fusion algorithms when multiple sensors are providing similar information. Such fusion is often driven by the uncertainty associated with these sensors; however, such uncertainty is characterized a priori based on the sensor characteristics and potentially a model of the system dynamics. When sensors fail (or even deviate from their presumed characterization), such algorithms also fail or degrade. In this work, we propose to characterize the “health” of individual sensors using a probabilistic health monitor (previously proposed for general cyber-physical systems), as well as the health of the collection of sensors providing observations, and use this characterization to appropriately weight the different sensors when performing sensor fusion. The resultant “Sensor-Health-Aware Resilient Fusion” (SHARF) provides unbiased, consistent, and optimal state estimates. SHARF is applied to the “Infrastructure Enabled Autonomy” (IEA) problem, where sensors mounted on the road-side are used to localize the vehicles on the road. The localization is performed using SHARF on the edge. The proposed SHARF is implemented and evaluated in field experiments with cameras and a RADAR on the infrastructure and with two vehicles on the road driving in real-world scenarios. We demonstrate that SHARF can detect faulty sensory data and incorporate the health status of sensors leading to successful tracking of the target vehicles.