STARNet: Sensor Trustworthiness and Anomaly Recognition via Lightweight Likelihood Regret for Robust Edge Autonomy

Complex sensors such as LiDAR, RADAR, and event cameras have proliferated in autonomous robotics to enhance perception and understanding of the environment. Meanwhile, these sensors are also vulnerable to diverse failure mechanisms that can intricately interact with their operational environment. In parallel, the limited availability of training data on complex sensors affects the reliability of their deep learning-based prediction flow, when their prediction models fail to generalize to environments not adequately captured in the training set. To address these reliability concerns, this paper introduces STARNet, a Sensor Trustworthiness and Anomaly Recognition Network designed to detect untrustworthy sensor streams that may arise from sensor malfunctions and/or challenging environments. STARNet employs the concept of likelihood regret (LR) for continuous evaluation of the trustworthiness of sensor streams. We tailor the framework to resource-constrained edge devices with two settings: a gradient-free framework suit- able for low-complexity hardware with fixed-point precision capabilities, and a low-rank tunability of underlying models for LR extraction, reducing the extraction workload. Through extensive simulations, we demonstrate the efficacy of STARNet in detecting untrustworthy sensor streams in unimodal and multimodal settings. In particular, the network shows superior performance in addressing internal sensor failures, such as cross-sensor interference and crosstalk. In diverse test scenarios involving adverse weather and sensor malfunctions, we show that STARNet enhances prediction accuracy by approximately 15% by filtering out untrustworthy sensor streams. STARNet is publicly available at https://github.com/nstrndrbi/STARNet.